I've been teaching with Standards Based Grading (sbg or #sbar on twitter) in my advanced-level Theoretical Mechanics course (sometimes called Classical Mechanics) for a few weeks now and I wanted to post some of my impressions early on in this pedagogy experiment.
A quick sbg primer first: I determine a set of standards that I want students to know. For this class you can see them the same way my students do. Each standard is assessed multiple times for each student with the last assessment acting as the current score for the grade book. Students can initiate reassessment at (nearly) any time and I can also initiate (re)assessments. That's the nuts and bolts. The philosophy is much broader and I'm greatly indebted to a passionate online community to help me wrap my brain around it. A recent collection of blog posts called the SBG Gala #5 is a great place to get a flavor of that great community.
My implementation: I put my content online ahead of time (sometimes called a flipped classroom or teaching naked) and use class time for problem solving practice and occasionally assessment. I've been teaching "flipped" like that for a while and you can check out a lot of my older posts to see some of how I do that. For assessment I've decided to focus on students' voices and so I've required all assessment to include some version of their voice. That means students are doing pencasts (using LiveScribe smartpens), screencasts (typically using Jing on their own computers), and in-person office visits. I'd like to also see online office visits and oral "exams" in class. I've changed my grade book so that students can see their progress with a Google Annotated Timeline flash plot of every assessment of every standard on one graph (note, seems not to work in Safari for some reason).
How it's going: So far I've had one student (of only 9 total - the reason I was willing to do this experiment on such short notice) who has really embraced the system. He's turned in at least one assessment for almost every standard (that is available - bold on the standards list) and many standards have seen multiple reassessments. With one exception his reassessments have done as well or better on the rubric than the last and he has told me privately that he really likes the system. A few middle-of-the-roaders have turned in a few assessments and are promising more and a few not-quite-sure-what-to-make-of-it-ers haven't really turned in anything yet. I required them all to turn in one screencast early on to make sure that they can get Jing and screencast.com to work so the technical details have been mostly solved. With the flipped classroom there is time for discussing how it's going with them and it seems that they're still giving it the benefit of the doubt. When I asked if anyone had any successful strategies for approaching the class a couple talked about when they watch my screencasts and when they do theirs. When I asked about not-so-successful strategies mostly I just heard the usual "I haven't gotten around to it yet" type comments.
Changes we had to make already: On my syllabus I had said that we'd also do group quizzes where the answer to a question had to be on the board after 10 or 15 minutes and then I'd ask follow up questions to random students. Then we'd all work together to determine the rubric assessment for the relevant standards for each individual. I've always loved those sorts of group quizzes in the past and was happy to include them in this system. We did one on the second day and it was fun to see the students interact with each other in working to solve a problem. At the end of the day, though, they didn't feel that they should each be assessed individually because they didn't get to show their own understanding on every step, rather they were just asked a few questions about a random step. So we've gone away from those for the moment.
I've also had to change my approach to my screencasts. I'm trying much harder now to make sure that everything I do is important for them to learn to help them do well on standards assessment. A recent example is some tangential info that I would have covered in the past only to say come test time that it wouldn't be on the test. I got away with it in the past because I typically wouldn't specifically say what was on a particular test until close to test day. Now the standards ARE the test and the students know all about them all the time. That doesn't mean I skip all tangential material but I think about whether it's a fair follow up type question for the in-person assessments. If it is, I leave it in.
What I like: I love hearing their voices! I started having students turn in screencasts/pencasts last semester in general physics and I love how you can judge their confidence and understanding when you hear them talk, especially when they're simultaneously pointing out the relevant equation or part of a figure. A lot of my standards require them to do a calculation in Mathematica and it's quite telling to watch them have to type their code in character by character while discussing the meaning/usefulness/physics of a particular portion. Having to watch them type code from scratch is a little tedious but I've told them that eventually I'll approve their Mathematica skills and they can start just doing screencasts of finished code, just walking me through it. I am starting out forcing them to start with a blank canvas, so to speak, because I need to make sure they're not just cutting and pasting code from someone else.
For non-Mathematica standards I like having them write it on a sheet of paper and then scanning it to do a screencast walking me through it. I also let them check out one of my LiveScribe smartpens and tablets to do it. To see and hear a student walk through the derivation of the Euler formula for the calculus of variations is just fabulous, especially compared to just looking at the written work, since often that's just copying from the book.
I'm really happy, too, about using screencasting to show them how to do Mathematica calculations. In the past I would just give them the .nb files and let them build on them but now all they have is my video so they have to type it themselves. That is a huge deal!
What I don't like (so far?): I don't like giving the students so much rope to hang themselves. By requiring their voice I can't do assessments in class without canceling the plan for that day. In a couple of weeks we've set aside days to do just that but by then they'll be pretty far behind. We'll see.
I also don't like not being able to grade on the bus. I ride my bike in to work and then take it on the bus home. In the past the bus was where I got almost all my grading done. Now I need a computer with internet to do it (although apparently you can watch screencast.com material on an Android phone so who knows.)
What I'm hopeful for: I really hope the students learn the material better this way. I've never been overly proud of my students' retention of material and I'm hoping that'll get much better this semester. I also hope they can realize that the practice we do in class and the reassessments are all about learning and that as long as they eventually "get it" the grade will come.
What I'd love: I'd love to get some more great feedback from you. This is a grand experiment of mine and, while it feels like it's going well now, it's really too early to tell if it's worth the effort.
Subscribe to:
Post Comments (Atom)
Andy, fantastic from-the-trenches post. I have tons of additional questions. You clarified a ton of things for me, which of course opened the floodgates for an entire second level of questions.
ReplyDeleteIn your flipped courses, what fraction of the students do you think come to class having gone through all the screencasts and doing any assigned reading? Between this and other posts of yours that I have read it sounds like you have done a great job of fully flipping your course and students can expect to be fully lost if they come in unprepared. Right now I use weekly JITT-style reading assignments in my intro courses and last term averaged 78% of the students completing the reading assignments having shown sufficient effort to receive marks. I still end up re-presenting most of the material in mini-lecture form and would love to move to only having to discuss the points that people had trouble with.
What would you estimate is their weekly total “outside-of-class-time” time spent on this course? Have you taught this course before? If so, how would you compare their “outside-of-class-time” pre/post using the flipped class?
Have you had any drastic drops in a given when re-assessing? With your assessment methods I would guess that this would most likely occur when you are able to give “oral exam” type assessments that are able to better get at some of the things which they might be able to gloss over (or get help preparing for) when screencasting.
When assessing their screencasts, how do you provide the feedback that goes along with the numerical assessment?
And I completely, entirely, whole-heartedly and just plain old really agree that I hate giving my students too much rope to hang themselves. A large part of the marks that I hand out in my courses are participation or other easy marks meant to make “those that would be most likely to hang themselves” do the minimum amount of work to stay caught up. Fortunately the students that are not of the hanging themselves variety tend to like the easy marks as well. But with the open-ended due dates of reassessment by screen cast or office visit I imagine you will experience what I have, with students accepting frighteningly low marks because they can’t find the time to do a few little things that will greatly boost their marks.
Joss
Thanks for the great questions, Joss. I'll try to answer them here:
ReplyDeleteI'm never entirely sure how many of my students are prepared for class. I can see how often my screencasts have been watched but not by whom. I would say it's usually about the same percentage you've seen based on their ability to engage in class. I'm very careful to only elaborate on things they ask about, though, using either my online summary/question page or, this semester, using groupme where we all text each other. In the past I've also been sure to put things on exams that no one ever asked about but with sbg it's easier, they know what they need to know and if they don't ask about it they know it's their problem.
I've asked this group how much time they're putting in outside of class and the answer is 2 hours per class period. I don't know if I believe that but it's what they claim.
By drops in reassessing do you mean less times reassessing or lower scores? If the former, I would say it's still just getting started. If the latter, no, not really, not yet.
I provide feedback with written comments in the grade book and, when it seems needed, a screencast of my own pointing out the things they can improve. I also talk to them in person about them the next time I see them. In class I also will discuss with the whole class how I assessed a particular person's screencast.
I'm looking forward to the three consecutive days coming up with open assessment time to catch everyone up (or at least to show them that they don't really understand things after all). With only 9 people I figure every student will have to reassess a couple standards during that period.
"I'm looking forward to the three consecutive days coming up with open assessment time to catch everyone up"
ReplyDeleteHow are you planning to do the open assessments? Oral exams without specific tasks in mind, redoing their original assessments and basically doing the screencast process live, other?
"By drops in reassessing do you mean less times reassessing or lower scores?"
I was trying to say drops in scores upon reassessment. I'm still really trying to develop a mental picture of SBG in action, especially at the college level, and was wondering how often or in what circumstances a student gets a regular master score (3/4) on a standard and then upon a reassessment gets a very low score.
"I've asked this group how much time they're putting in outside of class and the answer is 2 hours per class period."
2 hours per class period (6h/week) is usually about my target for non-intro courses, even though in practice it usually ends up being more like 6-9h/week. Sounds like you are right on target! On occasions when my students end up working more than 9h per week they usually have a small and justifiable collective fit aimed in my direction. In these cases I have had to readjust the size of the assignments or even the scope of course a bit.
"I'm very careful to only elaborate on things they ask about..."
I would like to move in that direction. It's one of those classroom culture things. If you establish this as the norm right off the bat, they will usually play along. Not that I want to, but I always give in to the unprepared students and provide a bare minimum of coverage of each topic so they aren't completely lost. I need to get past this. I declare that I will fully flip a course! It will happen.
The 3 day open assessment will work like this (remember, only 9 students):
ReplyDeleteevery 10 or 15 minutes I'll randomly select a student and a standard. They'll go up to do it in front of the class where everyone can ask follow up questions. My questions will likely have them incorporate other standards as well. We'll likely come to a group consensus on the rubric score.