Online Course

Immersed as I am lately in the world of participatory assessment, I go through cycles of forgetting and then remembering and then forgetting again that not everybody in educational research thinks assessment is sexy.

I was reminded of this again recently while reading Lorrie Shepard's excellent 2005 paper, "Formative Assessment: Caveat Emptor." The piece argues that the notion of "formative assessment" has been twisted in unfortunate ways as a result of the excessive hammering kids get from high-stakes standardized tests.

I helpfully plugged the entire paper into the wordle machine for you and got this:


In theory, then, assessment should be easy to understand: All of the most frequently used words in Shepard's paper are fairly common and comprehensible. In practice, though, assessment research is complicated by the impulse to put a fine point on things. Here's a sample paragraph from Shepard's piece, which starts out okay but descends into chaos before the end:

“Everyone knows that formative assessment improves learning,” said one anonymous test maker, hence the rush to provide and advertise “formative assessment” products. But are these claims genuine? Dylan Wiliam (personal communication, 2005) has suggested that prevalent interim and benchmark assessments are better thought of as “early-warning summative” assessments rather than as true formative assessments. Commercial item banks may come closer to meeting the timing requirements for effective formative assessment, but they typically lack sufficient ties to curriculum and instruction to make it possible to provide feedback that leads to improvement.

I'm not saying the language is unnecessary; I'm not saying that assessment types are putting too fine a point on things. What I will argue here is that assessment research has, for lots of good and not-so-good reasons, been divorced so thoroughly from other aspects of educational research that it's decontextualized itself right into asexuality. It's like that guy in the corner booth at the bar on Friday night who wants to talk about Marxism when everybody else just wants to make sure everybody gets the same amount of beer before closing time.

Think about that guy for a second. Let's call him Jeff. Jeff has been single for a long time now, and he's spent a lot of that time reading. Maybe he's grown nostalgic for the early days before his girlfriend cheated on him and then moved in with some guy she met in her Econ class. His friends miss those days, too, mainly because he was so much goddamn fun back then. They're nice enough; they want to take him out and help him snap out of it. But the minute the beers come he's back on the Marxism soapbox again and NOBODY. FREAKING. CARES. It's Friday night, late July, and everybody just wants to get stupid drunk. They drop him some hints. Sully slaps him on the back and asks him to tell that one joke he told last week.

"In a minute," Jeff says. "I'm explaining where Marxism went wrong."

Eventually his friends will tell him to either cut it out or go home. If he wants to keep hanging out with these guys, he'll shut up. Or maybe he'll tell that one joke he executes so well. If the girls around him laugh, he might tell another one. Girls like funny guys, he'll suddenly remember. They don't necessarily like Marxists.

All of this is what we might call "formative assessment." This guy wants to be accepted by his friends, which means he needs to pay attention to his behavior. He learns (or re-learns) how to act at the bar on Friday night by paying attention to the feedback he gets from his friends, from other people at the bar, from his memories of having a social life all those years ago.

If we wanted to, we could spend some time talking about better ways to help Jeff learn the social skills he needs. For example, his friends could have sat him down before they went out and explained that his primary goal was to be the funniest guy in the room. "Because girls like funny guys," his buddy Rufus might remind him. They might also set deadlines: By 11:30 you better have told at least three jokes. Then, over the course of the evening, they could check in with him and get a joke-count.

The point is that everybody's on board with the evening's goals. Everybody--Jeff, his friends--wants Jeff to have a good time, and they want to have a good time with him.

Haha! I tricked you into caring about formative assessment.

This is what assessment is, even if it doesn't always feel that way to students, teachers, or researchers. There is an end goal, an objective, and formative assessment is a way of getting everyone on board with this goal and keeping them on board. When it works right, everybody involved actually wants to achieve the objective and the assessment is valuable because it helps them get where they want to go.

But as Shepard's piece points out, too often the insanity of NCLB substitutes test scores for real, intrinsic motivation. Too often and too easily, students learn skills it takes to attain high test scores without actually learning anything. Though "(the) idea of being able to do well on a test without really understanding the concepts is difficult to grasp," Shepard writes, she gives as evidence a 1984 study performed by M.L. Koczor, which focused on two groups of children learning about Roman numerals:

One group learned and practiced translating Roman to Arabic numerals. The other group learned and practiced Arabic to Roman translations. At the end of the study each group was randomly subdivided again (now there were four groups). Half of the subjects in each original group got assessments in the same format as they had practiced. The other half got the reverse. Within each instructional group, the drop off in performance, when participants got the assessment that was not what they had practiced, was dramatic. Moreover, the amount of drop-off depended on whether participants were low, middle, or high achieving. For low-achieving students, the loss was more than a standard deviation. Students who were drilled on one way of translation appeared to know the material, but only so long as they were not asked to translate in the other direction.

Because NCLB and other insane policies that mandate high-stakes testing for accountability have pushed assessment out of its natural home--as Jim Gee explains it, "in human action"--assessment researchers have themselves been backed into a separate corner of the room.

This is not okay. It doesn't help anybody to take the sexy out of assessment by tossing it into a corner. What we need, more than anything, is to push assessment back where it belongs: inside of the participation structures that support authentic learning.

Participatory assessment is, at its core, about social justice, about narrowing the participation gap that keeps our society stratified by race and class, about motivating learners to achieve real goals and overcome real obstacles to their own learning. Participatory assessment, if we do it right, can make almost anything possible for almost anyone.