Friday, December 10, 2010

Grading with my voice

A few people have asked me for some more detail about how I use screencasting to grade papers so I thought I'd post this.

As a physics professor I don't grade nearly as many writing assignments as my colleagues in other disciplines but I do take seriously my job to help all my students improve their writing. When I first started grading such papers, I would write comments on each draft and provide a grade via some sort of rubric.  The rubrics started really as requirements that could be met in various ways and have evolved to fully fleshed out rubrics with careful descriptions of what it takes to get a check mark in columns like "meets basic expectations."

I quickly realized that giving the students the type of feedback I really thought they could use took a lot of time and effort, and I found that meeting with the students about their writing was one of the best ways to do this.  However, those appointments are hard to schedule, especially in a semester like I have now where all three of my lecture courses have heavy writing requirements (my First Year Seminar is writing intensive, my junior-level advanced lab course has writing a grant as 80% of the points, and my fully online course for teachers has a lesson plan as a major assignment).

I realized that the screencasting I was doing for my lectures could also be used to simulate the office experience with students.  I use my pen tablet mouse to mark up the documents digitally (using either Jarnal or Adobe Acrobat, or FoxIt reader) and I use Jing to record my voice while explaining my concerns with the paper.  Here's an example.   In that example you can see how I mix in discussion of content, style, and how it meets the expectations.  You'll also see that the 5 minute limit that Jing holds me to wasn't enough (so I just did a second one).  Typically I get my comments done in under the five minute limit (though the reading of a typical paper still takes me something like 20 minutes).

I ride the bus a lot and I like to use that time to grade.  What I've taken to doing is marking up the papers in regular ink with little notations to myself about what to say.  When I'm back in my office I then open the digital document, grab my pen tablet, and begin.  I transfer the marks that are necessary all the while holding a pseudo-conversation with the student.

I've gotten a lot of good feedback about this from my students.  They really like having the screencast at their disposal for pausing and rewinding.  One student told me that he opens his paper on his computer before playing the screencast and he makes changes immediately, while pausing the playback.  Others have said that they really feel they understand what I'm looking for after watching.  There's also been an interesting study on the preferences of students for the type of feedback they'd like, comparing regular comments, track-changes in Word, track changes with audio, and what I do.   The upshot is that my way was strongly preferred in their survey.

I'd love to hear how others use a similar system.  Feel free to drop me a line here in the comments or on twitter @arundquist.

7 comments:

  1. Andy/Superfly - Thanks so much for posting the example of your simulated student conference/markup process. Teaching 100% online I am really interested in anything that will increase the sense of contact experienced by my students, many of whom are new to online education.

    I have two reservations about this process aside from the cost of the pen tablet - I'd have to be better convinced of the utility of the method before spending $400 on this!

    One is that a colleague of my husband's, teaching freshman comp, tried it for a full semester, and found that it took more time to preplan, record, and upload the recording than expected, even after many sessions.

    A second question is how succinct records of your assessment are maintained. Do you keep your own marked-up paper copy as a record, plus an actual rubric? I saw no mention (unless I missed that part) of a summation/rubric, although I agree that your comments are very good in a formative manner. The process seems incomplete in that way.

    I am teaching (in a library program) database structure and information retrieval, and am extremely interested in using Jing for demonstration and feedback purposes. There's a sort of Rubicon to be crossed as students learn to think differently about information queries and their translation into a search engine syntax, and I look for ways to help ease a passage between confusion and clarity

    ReplyDelete
  2. Thanks for the comment Carol.

    As to the cost, my pen tablet from Bamboo costs $70 and works on every computer I've tried (it's essentially a USB mouse as far as the computer is concerned). If you have Adobe Acrobat, great, but Jarnal is free and works great too. I like Adobe Acrobat better because I can just hit save and the pdf is updated immediately. In Jarnal you have to export the pdf.

    Time: Once I've marked it up on paper (on the bus, typically), I get back to my office, load the digital copy (that I've added the rubric to at the end, though you could also have the rubric document up separately), hit the Jing button and 5 minutes later I hit the upload button. In Jing, the upload starts immediately and puts the link into your clipboard when it's done. I then post the updated pdf to the student along with the link. I also save the pdf for myself. When I get a second draft I'll often look at just the rubric part of the pdf to remind me of things but sometimes I'll just watch the screencast.

    ReplyDelete
  3. OK - now I know what I want for Christmas! Thanks for your followup comments. I'll go back to your example and look for the rubric part.

    ReplyDelete
  4. Unfortunately that one doesn't have the rubric part (it's one where I went a little over 5 minutes). I'm using an in-house rubric that I'm not supposed to distribute so that's why I picked one. Sorry about that.

    ReplyDelete
  5. No problem. I was interested to know if you did also incorporate a rubric. Do you use this method to give final assessment comments or is it mostly used for draft docs?

    You might think about creating a video to share about this method!

    ReplyDelete
  6. I use this method for all drafts but the final. For the final I only provide a rubric score (which shouldn't come as a surprise at that point :)

    ReplyDelete
  7. Andy,

    Thanks for writing about this. Anything that helps students to use the feedback is a good thing. Perhaps it will wear off as students become more familiar with audio feedback...

    I do electronic marking I use a combination of typed comments (using Word comments), voice comments and pasted reusable comments. You can see an video example about 1 minute into the video at http://emarkingassistant.com/emarking_movie.htm

    Best wishes with all your teaching (and your no TV resolution)

    Peter Evans

    ReplyDelete