Wednesday 29 October 2014

Using webPA, by Michele Bourne



Using WebPA

Group projects offer many well documented benefits to students (Astin, 1997; Tinto, 1998; National Survey of Student Engagement, 2006). However, in group projects, members of the group don’t all make the same contribution to the project, and so marking the group project with a single mark for all students is not a fair way to mark the work. A single mark can lead to some students freeloading on the work of others, feelings of resentment from those doing the work and can lead to the potential benefits of group work becoming lost.

In group projects in the Foundation year, we originally allowed the whole group to submit one percentage grade for each member of the group which would be used to adjust their mark. However, this offered no real incentive to the students to do anything other than give each person 100%

We therefore changed this system, giving each student 100 points which were to be shared among the other students in their group. This method meant there was much more reason for students to be fair in the marks that they allocated. It was quite complex to administer this system using paper forms and fairly blunt.

WebPA is an open source, online system which can be used by students to allocate individual marks to members of their group. It is installed and run on a server, and students and staff access it via a simple web interface. Originally, Keele University agreed to install and support WebPA for a year.

The teacher interface for WebPA was quite simple and straightforward to use. It is accessed through Blackboard, where a WebPA assignment can be set up. The University of Loughborough who created WebPA had a test set up, so it was possible to try things out first. As a tutor, when you first log on, you see three options, my forms, my groups and my assessments.

My forms

This area allows you to create the forms that students will fill in. There is the choice of using a Likert scale or to split 100 points. We always choose to split 100 points, as when the students use a Likert scale there is not real reason for them not to allocate the top marks to everyone in the group, which ends up with the final scores not being as fair.

You can also create the assessment criteria that you will use. You can have as many of these as you like. We chose to have five criteria and made these available to the students at the start of their group project with the assignment instructions and the mark sheets, so that they would know how they would be assessed by their peers. The criteria that we used are given in Appendix 1. We also warned them at this stage that those who didn’t complete the peer assessment would lose 10% of their mark for the assignment.

We really liked being able to have a range of criteria, as it encouraged the students to think about the different types of contributions that each of the group members had made and allowed them to be fair in assigning marks to people who they may be friends with, but who hadn’t done so much work.

My Groups

Clicking on syncdata on the left of the screen will populate WebPA with any students in the module. When we first used WebPA, although WebPA was able to pick up our students from the module lists on Blackboard, it couldn’t pick up the groups that we had already created, and so we had to make these manually in WebPA. However, in a later version, it picked up groups from Blackboard, which was great. On looking at it just now, I am not sure if this automatic group creation is currently working. However, it is quite straightforward to create the groups manually.

My assessments

This is where you actually say the rules for how particular students will use a particular form. The assessments area has four areas, pending, open, closed and marked. When you first create an assessment, it will be in the pending tab, and will move from one location to the next, depending on its status (so don’t panic and think it has vanished if you no longer see it in the pending tab! Just look in the other tabs).

When you create an assessment, you can say which dates it will be open and give it a name and an introduction. You also have advanced options on these tabs, so make sure that you check those. We chose not to allow students to give comments. We also chose that the student would assess their peers but not themselves.

When the assessment is open to students, you can see who has completed WebPA, although you can’t at this point see the marks that were given. We did e-mail students to remind them that if they didn’t complete the assignment by the given time, then they would lose 10% of their assignment mark. We also put a statement in the introduction to the WebPA assignment to warn them that they should not collude when assigning marks and that we reserved the right to adjust marks if necessary.

Once an assessment moved into the closed tab, you will see an icon for a ‘new mark sheet’. Here, you can choose what weighting WebPA will have (what percentage of their group project mark will be affected by the peer review). We chose 100%. You can also choose to have a penalty for any student that doesn’t complete the WebPA assessment. We chose 10%.

Once you have created your mark sheet and input your own marks, one for each group projects, the assessment will move into the marked tab, and you will be able to see various reports showing individual marks for each student. There are lots of different reports which show you the information in different formats, so you can look at marks for individual criteria, who gave who which marks, what time they completed the WebPA and several other formats. We do reserve the right to alter these marks if there is a problem, and you can use your normal academic misconduct channels if this happens, but we have never had a problem. On the whole, students seem to be very fair minded.

After the first year of WebPA, the University decided to permanently implement WebPA. We were very glad not to have to go back to our old paper based systems! Using WebPA was much more professional looking. The students were able to be peer assessed using several criteria, rather than just on their overall contribution. We could let them know these criteria in advance. The interface was simple for both students and staff to use and didn’t require us to work out complicated formulae on spreadsheets to account for students who hadn’t participated in the peer review – WebPA did that all for us. Students could do the review in private, in their own time. We could easily look over the data to pick out any incongruities in the peer marking. Students were much happier doing the group projects and felt much more that their final marks were a fair assessment of the effort that they had put in. Students who had worked really hard on projects could get a high mark, which is particularly important for students who need high marks for progression. This encouraged students to have a higher involvement in group projects and discouraged the idea of some students getting a free ride. WebPA has become an essential part of our teaching.

References

Astin, A. (1993). What matters in college? Four critical years revisited. San Francisco: Jossey-Bass.

National Survey of Student Engagement Report. (2006). http://nsse.iub.edu/NSSE_2006_Annual_Report/docs/NSSE_2006_Annual_Report.pdf.

Tinto, V. (1987). Leaving college: Rethinking the causes and cures of student attrition. Chicago: University of Chicago Press.

Appendix 1

Criteria that you will use for peer assessment of the contribution of others in your group towards the group project assignment.

During week 12, each member of the group will mark every other group member on their contribution to the project using set criteria. These marks will be used to determine each individual student’s mark for the group project report. You can see the criteria that will be used for this peer assessment in the assignment 2 folder on Blackboard. Any student who doesn’t carry out the peer assessment will lose 10% from their individual mark.

Communication

How well did each group member keep in touch? Did they let everyone else know what was happening? Did they make their work available for proof reading and editing?

Planning and Organisation

Did each group member help to create a work plan or Gantt chart for the project? Did they help to organise meetings, by booking rooms or by chairing the meeting? Did they help to plan who should do each part of the project?

Written Material

Did their writing tie in with the overall aim of the project? Did they use high quality sources, such as journal articles in their research and did they reference correctly? Did they produce a good volume of written work? Was spelling and grammar good and did the written work read well?

Reliability

Was the group member on time for meetings? Was their attendance good? Did they meet deadlines and do jobs that they said they would do?

Contribution to the group

Did the group member support others in the group? Did they have a positive attitude and listen well to others? Did they do extra jobs within the group, such as putting the final report together, acting as team leader or helping with proofreading? Did they give positive contributions to meetings?




Creative Commons License
Using webPA by Michele Bourne, from Keele University is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Based on a work at http://lpdcsolutions.blogspot.co.uk.

Monday 27 October 2014

Ask-a-Techie episode 3

The Ask-a-Techie's 3rd episode is now available to watch at http://youtu.be/HDB46qQE9F8
Episode 4 will be announced shortly and will be recorded towards the end of October.


Topics covered (click time to jump to them)
00:00 Welcome and Introduction to Panel (Episode 3! not 4 - sorry)
00:45 When you set up an assignment submission folder on the KLE you have to allocate it points...what do these points mean and what do they contribute to?
01:43 Can you talk us through how to use the self and peer assessment function on the KLE?
07:13 If you append a feedback file to a student assignment which has been submitted via the KLE - can you show us the student view to see how the students retrieve and access their feedback? (43:22 Part 2)
11:29-31:53 GMail - Google Email
-- 11:40 Is there a way that I can visibly see that I have responded to an email in google mail without checking sent folder?
-- 17:10 Can I change the default search on gmail? I search for a name, but by default it will search​ for all emails that 'include the' name. I'd like to change the default so it searches for emails 'from' that name only.
-- 18:48 How can I work on my computer with my Keele Google account and my private google account - I keep getting logged in to the wrong account
-- 21:40 [what are your] top tips for managing emails?
31:53 Can I suggest an addition to Google Glass - Smart watches?! A colleague has one of those and the camera on it is so unobtrusive that I can easily see students using this to record teaching sessions.
35:20 Management of my students consultation slots (some call them ‘surgery hours’, or ‘office hours’) using google calendars (or another google app?)
45:05 Please discuss: Data security for teaching staff working with laptops [and other portable tech]
49:20 Can I somehow protect the calendar icon (only) on the homescreen of my ipad/I phone from the fingers of a five year old?
52:49 How can you present from an iPad?
56:02 Tech news (Google Drive, CPDA Copyright on image use in presentations, Samsung Laptops in Europe and Apple Watch)

Resources in relation to the tools discussed at 01.43 can be found at these links, the group peer assessment tool webPA and the self and peer assessment tool in the KLE

Thanks to:
Tim Smale - http://www.keele.ac.uk/health/staff/e-learningstaff/timsmale/
Jon Knight - http://www.keele.ac.uk/it/
Dan Harding - http://www.keele.ac.uk/hss/facultycontacts/danharding/
Matt Street - http://www.keele.ac.uk/lpdc/ourstaff/mattstreet/
Pete Lonsdale - http://www.keele.ac.uk/nursingandmidwifery/staff/staffa-z/petelonsdale/

Saturday 25 October 2014

Using Twitter in Sociology Teaching and Learning, Dr Emma Head and Dr Ala Sirriyeh


Using Twitter in Sociology Teaching and Learning

Recently launched, a new research project titled ‘Using Twitter in Sociology Teaching and Learning'.

The project is being led by Dr Emma Head and Dr Ala Sirriyeh, and is funded by Keele's Teaching Innovation Project Scheme. The objectives of this investigation are to firstly assess the use of Twitter as a tool to promote enhanced student learning and engagement on a sociology module and, secondly, to explore the role of social media in the education and wider lives of undergraduate students. The research will evaluate the use of Twitter as a teaching and learning tool in a new second year undergraduate module 'Race', Racism and Resistance, which will run this semester. This is an elective module in sociology, led by Ala who will be using Twitter as a teaching and learning tool within the module. Emma will evaluate this use of Twitter on the module as well as exploring this within the wider context of student engagement with various forms of social media. As part of this evaluation, students will be invited to complete two questionnaires and to take part in two focus group discussions.

Research aims

In this project we will be exploring three main research aims:
  • To explore whether Twitter contributions and exchanges can help enable students to feel a greater sense of connection to their studies and whether a more collaborative approach to learning will develop as students exchange ideas about their reading and seminar preparation. 
  • To evaluate whether, and to what extent, the use of Twitter on this module encourages students to think of different forms of communication, to reflect on how to act in an online environment, and to consider their self-presentation whilst using social media. 
  • To examine how contributions online relate to contributions in seminars and preparation for seminars to explore the possible connections between student engagement both online and in the face-to-face settings.
We are interested in talking to students about these issues and considering whether existing social media networks might offer any advantages over the Keele Learning Environment (KLE) for student learning. This is particularly significant in a subject like sociology where we encourage students to relate the observations of scholars to the contemporary social world and wider social issues.

Undergraduates are often heavily embedded in social media networks and this project will allow us to explore the experiences students have of issues around privacy and self-presentation online. We will be able to ask students to reflect on their ‘digital footprint’ and to consider related issues at a time when they will soon be making the transition from student to potential employee in a graduate labour market.

Using Twitter on the module

Students on this module will be encouraged to participate in a series of tasks that connect their seminar preparation with the weekly key readings and lecture topic. Students might tweet questions for a forthcoming seminar discussion or tweet their favourite quote from the week’s key reading. Students will be encourage to use a shared hashtag in their tweets. In the seminars the tweets from that week’s online discussion will be displayed via a hashtag search, or by using storify to facilitate discussion on themes raised. Ala will also tweet and retweet links (from @AlaSirriyeh) to resources that debate contemporary issues around ‘race’ and racism, including journal articles, news reports, and documentaries. Students will also be encouraged to share resources that they find in the same way. One of the assessments on this module asks students to present an analysis of a relevant contemporary debate. These tweets will collate a set of materials that will be useful for this task. Ala's twitter feed is displayed within the KLE for any students who don't wish to access Twitter, or to set up their own account.

We are looking forward to finding out student opinions about the role of social media in teaching and learning. Updates on our project will be posted on our School blog.

Creative Commons License
Using Twitter in Sociology Teaching and Learning by Dr Emma Head and Dr Ala Sirriyeh, from Keele University is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Based on a work at http://lpdcsolutions.blogspot.co.uk.

Wednesday 22 October 2014

Using WebPA for Peer Assessment of Laboratory-Based Group Work, by Laura Hancock



Using WebPA for Peer Assessment of Laboratory-Based Group Work
Laura Hancock


Group work can be unpopular with students. The main reason for this seems to be the presence of ‘free-riders’ who are content for other members of their group to complete the majority of the work. Whilst it may be considered to be a more authentic experience if all group members receive the same mark for their work, at university level (especially in first year) it is important to ensure that the mark each student receives is a fair reflection of their contribution to the work, and this can be incredibly difficult.

WebPA is an online peer-moderated marking system designed to be used where a group of students receives a single mark for a piece of assessment. Students assess both their peers and their own contribution to the group work (either on a Likert scale or as a percentage) and these marks are combined with the overall group mark to generate an individual grade for each student. A worked example of the scoring algorithm can be found here*

First year chemistry and medicinal chemistry students at Keele undertake a laboratory based project in groups of 4 or 5. Each group is provided with a project title, list of objectives and the laboratory resources (both chemicals and equipment) available to them. The groups are required to produce a project proposal outlining how they will achieve the project objectives, undertake 8 hours of laboratory work and produce a poster to summarise their results. They receive separate group marks for the proposal and the poster.

We started to use WebPA for peer assessment of these group projects after using a stand-alone peer-assessment mark (generated from a number of questions students were asked to answer on a Likert scale using a Google docs form) which comprised a small percentage of the total mark (~17%). This method proved to be unsatisfactory as the difference the peer-assessment mark made to final grade was minimal, and it was incredibly time consuming to collect the individual data for each student. There were also comments from students that they did not feel the mark they had received was a fair reflection of their contribution to the work. WebPA goes a large way to addressing all of these concerns.

As part of our group projects, students are now asked to complete two forms, one to moderate the proposal mark and the other to moderate the poster mark. The latter includes assessment of contribution to the laboratory work. We have found WebPA to be exceptionally user friendly, and very quick to use. It is embedded within the KLE so it is straightforward to assign groups, and once the assessments are completed, the adjusted marks can easily be downloaded. In addition, it is easy and quick for students to use – there were no reports of anyone struggling to access or use WebPA. Importantly, 96% of students surveyed felt that the use of WebPA to moderate their marks was a fair way to assess their projects. Staff members leading the projects also believed that those who made substantial contribution had been adequately rewarded whilst any ‘free-riders’ had been penalised for their lack of effort. Following some student feedback, in the future we may expand the assessment to allow students to define their own criteria by which the group projects are assessed.

WebPA is a fantastic tool; it saves staff time and provides students with a fair and transparent method for assessing group work.

*Weblink: http://webpaproject.lboro.ac.uk/academic-guidance/a-worked-example-of-the-scoring-algorithm/



Creative Commons License
Using WebPA for Peer Assessment of Laboratory-Based Group Work by Laura Hancock, from Keele University is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Based on a work at http://lpdcsolutions.blogspot.co.uk.

Friday 17 October 2014

Using Google+ to support students undertaking a research project, by Claire Stapleton

A Google plus community was set up for one year group who were about to embark on their research project journey. In Physiotherapy this starts in semester two of their second year where they are introduced to a selection of research titles. Students have a couple of weeks to view the titles and submit their preferences (we ask for the top three preferred titles, their preferred topic area and their preferred study design). The google plus community was set up to support students through this decision making process. They could ask any questions to aid their selection. All questions about title selection had to go through the google plus community discussion board in order to receive a response, which allowed all students to benefit from the answer and spark further queries. The benefit for the module leader was that there was no repetitive emails to respond to and sometimes students answered the questions between themselves. The advantages of using this forum instead of the KLE's discussion board was the ease of use on smart phones and tablets. At the time of releasing the titles to the students, they were on clinical placements sometimes with little access to internet connected PC's. You can also change the settings so you receive a notification when a comment or question has been posted. This feature allowed the module leader to respond to each query in a timely fashion. The discussion board also allows the members as well as the moderators to post information, files and video clips. The list of titles, a google form to submit their preferences and a screencast to explain how to use the google form were all posted on the google plus community page. The moderator can use a feature to create events which was employed to remind the students when they should be submitting work to their supervisor for formative feedback and for submission deadlines. Following the selection process the google community was of continued use when the students started to prepare their research proposals for submission, many queries were answered and students also posted interesting and useful information they had found to share with the rest of the cohort. The module leader as the moderator of the group has the capacity to remove anything deemed inappropriate however this issue did not arise. The google plus community for this cohort will continue to run into their third year and support them through the research project module. The use of this method of communication within the module was deemed a success with high levels of student engagement. This judgement was echoed by students who rated the following items on the module evaluation report for that year as 4.6 out of a possible 5, 'staff created an environment to ask questions and seek help' and 'the team were helpful dealing with queries'). A second google community will be created for the next cohort who embark on their research project journey.

Creative Commons License
Using a Google+ community to support students undertaking a research project by Claire Stapleton from Keele University is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Based on a work at http://lpdcsolutions.blogspot.co.uk/.

Thursday 2 October 2014

StudyWrite, interactive resources to support students with Referencing and Understanding Plagiarism, By Dan Harding


StudyWrite was an innovation project funded by Keele University in 2012/13 a link to the project proposal and final report can be found here.  The project has continued to develop and has produced some really valuable interactive resources, an update on developments can be seen below

During the past 12 months, the StudyWrite project has been piloting interactive resources that aim to improve student’s understanding of academic scholarship. Following the conclusion of this pilot, staff and student feedback has been collated and used to develop StudyWrite into a resource which has shown to be of significant benefit to students.

Amongst the feedback received, some of the stand out statistics include:

StudyWrite Plagiarism (191 respondents)
91% of students feel that it has improved their understanding of plagiarism.
97% felt that the content was clearly explained.
81% reported that interactive activities helped them to understand the material.
90% now know where to go if requiring further help regarding plagiarism.

StudyWrite Referencing (129 respondents)
88% of students feel that it has improved their understanding of referencing.
92% felt that the content was clearly explained.
89% reported that interactive activities helped them to understand the material.
90% now know where to go if requiring further help regarding referencing.

During the past few months, StudyWrite has undergone a major re-design, along with the addition of a further resource that deals with note taking and assignment planning. These resources are now freely available for all Keele University staff and students to use and can be accessed at:

https://sites.google.com/a/keele.ac.uk/studywrite/


If you have any questions or comments, please do not hesitate to get in touch.
By Dan Harding
Faculty of Humanities and Social Sciences Learning Technologist

What are Learning Technologists sharing?

Blackboard tests- Item analysis,
This blog post from February 2014 by Tim Smale an e-learning Fellow covers the Blackboard tests - Item Analysis, demonstrating how statistics from tests can be accessed.