Resources from the Sharing of Good Practice event 15th January 2014.

Keynote

The day started with introductions from Professor Fiona Cownie Pro Vice Chancellor for Education and the student experience and a lively keynote talk from Professor Sally Brown.  The talk challenged us to think about amongst other aspects of assessment, assessment design, the purpose of assessment, assessment criteria, the impact of assessment on students and assessment for learning.  Sally has shared her presentation on her website we have also produced a short video of the main points from her keynote below

All video recordings were produced in High Definition, to alter the settings when they begin to play select the cog from the bottom right of the player and select 720p this will give you a better quality picture

Presentations

Following a short break we began a series of short 15 minute presentations details and slides are below where available.  Short videos of the presentations are also present where are available.  Please do keep checking back to the blog post as more of these videos will be released when available.

All video recordings were produced in High Definition, to alter the settings when they begin to play select the cog from the bottom right of the player and select 720p this will give you a better quality picture

The passages below are extracts from the Sharing Good Practice event programme located at this link 

Getting Wise with PeerWise, Dr Katherine Haxton and Dr Dave McGarvey,

PeerWise is a free online tool designed to support students in creating and answering multiple choice questions. Based on the basic notion that 'if you teach it, you understand it', it allows students to write multiple choice questions with feedback on the correct and incorrect answer. It also provides a means for students to discuss the correct answer and the users frequently correct any errors that have arisen in the questions. PeerWise allows the creation of a large question bank through the contribution of the class that is searchable by topic and level of difficulty. It also provides a means of assessing effort through various metrics. PeerWise is a web-based interface that is intuitive to use and is available at: http://peerwise.cs.auckland.ac.nz .

In this presentation we will outline the use of PeerWise in 1st Year Chemistry classes as a formative means of supporting student learning and revision for module assessments. We will provide a brief description of how to use PeerWise, particularly how to introduce it to a class, and discuss the metrics generated should it be required for summative assessment.

School of Medicine, preparing students better: Giving rich, timely and personalised feedback after assessment, Adrian Molyneux

This session will describe the work we have done to improve the provision of feedback following formative and especially summative assessment in the School of Medicine, and the associated evolution of our assessment procedures. In particular, it will look at: (1) Why we believe students should receive feedback after summative assessments; (2) Our evidence showing how students receive and make use of feedback after summative assessments; and (3) How everything works on a practical level.

It will demonstrate the innovative developments we have put in place with the aim of enriching and personalising the feedback available whilst greatly decreasing the time taken to deliver this to students, including the ways we have incorporated verbal, written and score-based feedback into a single convenient web-accessible resource for multiple examination types. It will also look both at students’ views on the feedback provision and the patterns of usage for different attainment groups, and staff perception of the new assessment processes.

Concluding with a brief discussion on the next steps we plan to take, it will look at how we are working to further improve and embed our new assessment and feedback processes into the different areas of the course.

Enhancing first year students' engagement with feedback: Exploring the use of the Essay Feedback Checklist (EFC), Dr Hala Mansour

The research sought to help students to be more engaged in feedback by using the Essay Feedback Checklist (EFC) as an interactive two-way communication tool between tutors and students. The EFC may perhaps help students to be aware of the assessment criteria used in marking their essays and it potentially can help them to self-evaluate their effort in each criterion by ranking the level of confidence they feel when doing the work. The research also aimed to help tutors to establish an environment that encourages students to engage in their feedback and to think about their teaching style based on this.

The findings indicate that first year students are looking for timely, personal, accurate, developmental and constructive feedback. Students in the study found the use of the EFC useful in providing a detailed feedback and they appreciated following up their feedback with tutors after receiving it. They also suggested that the EFC satisfies some of their interests in receiving effective feedback which may possibly help them to develop their work further. Students claimed that the EFC created an interactive environment in providing feedback and it helped them to open a dialogue about the feedback with tutors.

A link to the presentation Dr Hala Mansour can be found here

Perils of Praise Dr Yvonne Skipper

According to Dweck, praise can be delivered using person (‘you are good at psychology') or process terms (‘you worked hard on this').

The aim of the present research was to examine the effects of person and process praise compared to a control group who received no praise. Participants were 114 students who read scenarios where they succeeded and received either; person, process or no praise. Participants then read two scenarios where they failed. Following each scenario participants evaluated their performance, affect, and persistence.

Following success, participants who received person, process or even no praise showed equally positive outcomes.

However, after one failure, participants who received person praise reacted most negatively. Person praise attributes outcomes to inherent abilities which cannot be easily changed. Therefore if you do not possess the ability to succeed now; you are unlikely to succeed in the future. However, process praise attributes outcomes to effort levels, which can be easily changed. Thus even if you have failed in the current task, greater effort in future could lead to success.

This suggests the importance of using feedback to encourage students to focus on their effort levels and techniques over their abilities

Using Online tests for feedback, Dr Jim O’Neill

Using the Test Function on Blackboard to Improve Assessment and reduce turn-around time.
With paper –based class tests in two accounting modules , I was faced with the daunting prospect of setting, organising, invigilating, marking and giving feedback to over 300 students in Semesters 1 and 2.I decided to use the test function on Blackboard to overcome these seemingly impossible tasks.
I used a combination of multiple–choice, true/false and calculation questions, uploaded them on the Test area of Blackboard and carried out the tests.
All students performed the test over the two semesters, level three in the first semester and level two in the second semester. There were a number of teething problems but overall the process was successful.
Student attainment improved and there was no difference in attainment regarding gender, home or overseas students. Students were overwhelming in favour of the method in particularly enjoying the instant communication of their result. Additionally they liked receiving feedback on their answers once all students had completed the test.
From my point of view, it has been an extremely enjoyable process. Devising the questions and using the Blackboard in new areas for me had developed me as a teacher. It has also stimulated an interest for me in the reasons why some students do better in computer based rather than paper based tests. At the moment I am investigating student learning styles and their connection with attainment in tests.
Now I want to spread the word to colleagues! A link to the presentation can be found here

Food for Thought - Engaging Students with Their Feedback, Dr Sheila Hope

We are striving to improve our feedback provision, but feedback can only be effective if students actually engage with the process, reflect upon the advice they have been given and act upon it. This session will introduce the Food For Thought project currently being piloted in the School of Life Sciences, which aims to promote student engagement with feedback. A Pebble Pad template has been produced and students have been asked to complete the template whenever they receive feedback on their work. The template gives the students the opportunity to reflect upon and respond to their feedback. At the end of the year, students will be asked to complete a second template, giving a final reflection on all the feedback they have received and any changes that they have made based on the feedback.

If successful, it is hoped that the template could be incorporated as one of the ILM activities

World Cafe

Following lunch we began a more interactive session led by Sally Brown.  We sat around tables in groups of 5-6, each table had on it a statement for discussion, we were asked to discuss the statements and write of any interesting comments or questions on the tablecloth after 10-15mins we were asked to move tables to the next statement and repeat the process.  This built up a rich picture of our collective thoughts and questions.  The facilitators for each table were then asked and the end of the rotations to share the main points from their tables conversations.  The points and the questions and comments raised can be seen in the videos below.

All video recordings were produced in High Definition, to alter the settings when they begin to play select the cog from the bottom right of the player and select 720p this will give you a better quality picture

Don't forget you have the pause button, feel free to pause and read, rewind fast forward as you need to

This table discussed the following, exams vs coursework or something else? What opportunities present themselves to rethink assessment?

This table discussed, formative vs summative, how do you strike a balance and make it worthwhile?



This table discussed approaches to engaging staff and students in conversations around assessment criteria



This table discussed electronic feedback, whats working for you, tips and tricks



This table discussed, sharing ideas and approaches to peer and self assessment, concerns and solutions to challenges

Key messages from delegates shared

Following the world cafe event delegates were asked to reflect for 30 seconds on what they had been discussing during the day and share it with everyone, these were captured in the video below.

All video recordings were produced in High Definition, to alter the settings when they begin to play select the cog from the bottom right of the player and select 720p this will give you a better quality picture

Diamond nines evaluation and close

The event drew to a close with a Diamond nine activity led by Dr Hala Mansour.  This activity looked at a range of statements focusing on assessment and related to the topics discussed in the World Cafe activity, staff were then asked to rank these statements.  The results can be seen in the images to the left.  If you want to replicate this activity or repurpose it Dr Hala Mansour has made the original files available for you to download.  This link takes you to an outline of the activity and statements, this link takes you a document detailing each of the statements so you can replicate the activity if you wish

Video soapbox

During the day delegates had the opportunity to share their own practice, ask questions or share thoughts about the day via a video soapbox.

All video recordings were produced in High Definition, to alter the settings when they begin to play select the cog from the bottom right of the player and select 720p this will give you a better quality picture

Twitter

Throughout the day twitter was used to share and capture delegates views and links to other resources shared.  Tweets from the day drawn together using Storify.  Storify is a useful tool for drawing together different tweets about an event and adding a narrative around them.  This link takes you to the Sharing of Good practice Storify

A take away message for delegates and readers of the blog, share this resource and what you learned from the day with one colleague who couldn't attend.

Using Screencasting and Grademark to provide feedback, Dr Sheila Hope

In this recording (video file) Dr Sheila Hope outlines her experiences of providing feedback using Grademark and screen capture software Camtasia.  More specifically she describes the practical issues faced and compares the use of jing (freely available screen capture software) with Camtasia a more complex programme, and details how else she has been using Camtasia to support student learning.  In this project the general process followed was students would submit their assignment to a Turnitin drop box in the VLE where the work would be marked using Camtasia to record the comments and screen actions.

Using audio to provide general feedback on exams, Dr Bill Dixon

Dr Bill Dixon (School of Sociology and Criminology) provides a description of his experiences of using audio to provide general feedback to all students on a module in relation to a multiple choice exam.

1.    In the project what technology and /or process did you use to provide feedback to students?

I wanted to provide generic feedback in an accessible format to first year students on an MCQ exam.  I recorded a sound file in MP3 format using a very simple recorder provided by the STAF project and uploaded it to the module section (CRI-10010 Understanding Crime) on the KLE.  

2.    Prior to the project how would you normally provide feedback to students?

In previous years I had provided written generic feedback in the form of a Word document posted on the KLE.  I was disappointed by the number of students accessing this feedback and given up doing so in 2009/10.

3.     Did you find providing feedback was more or less efficient than the process normally used?

Yes.  Although the process was new to me and I made several false starts with the recording, it took me no more than an hour to record and upload a 17 minute sound file.  Writing a Word document covering the same ground has taken me up to 2 hours in the past.

4.     Did using the technology allow you to provide richer or more detailed feedback to students when compared with the ‘normal’ process used?

It is hard to say whether the feedback provided in the sound file was richer or more detailed but it may well have seemed to be more immediate and personal.  But I have no hard evidence to support this hunch.

5.     Do you think that the students benefited from the change in feedback process? (Why? What is your data?)

With roughly similar numbers of students registered on the module the total number of visits to the audio file in 2010/11 from the date of release to 27th May was 52.  The corresponding number of visits to the Word document made available in 2008/9 was 29 over the same period.  

6.    From your perspective, what were the main advantages of using this process to produce feedback?

From my perspective the strengths of this process are the sense of immediacy (perhaps even intimacy) that a sound file provides and the time saved in creating such a file rather than a written document.

7.    From your perspective, were there any disadvantages to using this process to deliver feedback?

None.

8.    How do you think the process used could be improved to make it easier to use?

No - like most things, practice will make the process (more) perfect.

9.    Will you continue to use the intervention in your teaching practice? If not, what would need to change for you to adopt the intervention as current practice?

I would certainly use a sound file to provide generic exam feedback again.  Since using this technology to record feedback I have also used it to record an entire lecture which was then uploaded to the KLE.  

Using Screencasting to provide timely general feedback on assessments, Dr Katherine Haxton

Screencasting, also known as vodcasting, offers a means of providing mini-lectures working through answers to assessments that students can view at a time and place of their choosing, rather than being forced to listen in a timetabled class.  Crucially, screencasts of model answers and problem solving allow teachers to reveal their problem solving strategy, something not easily conveyed with written answers. 

We use a Toshiba tablet PC running Windows 7 and Camtasia Studio 7 and a digital voice recorder was used in place of the internal microphone to record high quality audio which was then edited into the screencast. The screencasts were edited before production of the video to remove any awkward pauses or stumbles.  This enabled the screencasts to be recorded in one attempt. Custom production settings were used within Camtasia Studio 7 to enable a table of contents to be produced in the finished video. This allows students to navigate to a specific part of the screencast, for example a specific PowerPoint slide title, enabling them to revisit content easily. Screencasts were approximately 5 minutes in length (the maximum length for online production) and took around 20 minutes to prepare, record and edit each after the initial learning curve. 

Screencasts were used to provide model answers to a number of class tests in the 2010/11 academic year including those involving spectroscopic data interpretation and main group inorganic chemistry. For first year Chemistry, screencast model answers and feedback were provided in addition to marked class test scripts, but the level of feedback offered on the marked scripts was reduced to only the key points and uncommon errors, and students were directed to consult the screencast model answer feedback.  

Screencasts were well received by the students with most rating them equal to or better than written, individual feedback. 

In particular, the opportunity to watch and see the problem solving strategy was identified as particularly useful:

“I found the video feedback to be very helpful.  It allowed me to look at my answers in the comfort of my own home, without using up valuable problem class / lecture time.  [The] commentary highlighted some points that would not have been easily conveyed in writing alone.” – First Year Student

Please see below for a more video of Dr Katherine Haxton explaining more  

http://tinyurl.com/projectstaf-eval-Katherine 

Providing video feedback to Music Technology Students, Dr Diego Garro

In this recording Dr Diego Garro discusses his experiences of providing video feedback to students, the recording has been separated in to three parts, part 1, part 2, part 3.  The students submitted a portfolio of video clips on video-DVD.  Feedback was recorded by playing  the video-DVD back on the computer and recording, with a screen-capture application, the computer output and sound/video from a webcam. A talking head will appear, in the video feedback, in a little inset window inside the DVD player window. Comments will be provided on parts of the videoclips.  Feedback was then delivered to the students through the KLE.

This approach was taken on a level 2 module in music technology consisting of 35 students.

Using audio for feedback on assessments

Supported by project STAF Dr Dave McGarvey has been using audio to provide feedback on a range of assessments. A detailed explanation of the approach take can be found here using audio feedback on assessments, a video outlining his experience of providing feedback in this way can be found here, part 1 and part 2

Moving from paper to online marking using Grademark, Dr Mark Featherstone

In this project Dr Mark Featherstone moved from paper based assignment marking and feedback to electronic marking and feedback using Grademark.  This was initially tried on a small piece of written work.  It worked well so Mark decided to use Grademark on a larger piece of written work (circa 2500 words for 25 students).  Outlined below are Marks findings, to summarise he found that using Grademark changed the way he marked and allowed him to provide richer more detailed feedback to students.  

Comments from the students included 

“The electronic feedback was very helpful and made my research proposal have a better direction and conclusion”

“Electronic feedback was fantastic, best and most helpful way I’ve had work marked so far at uni”

“Electronic feedback was positive and encouraging. Provided new links to further improve quality of word project”

Responses to Questions

Further details of his experience can be found here