The more the conventional wisdom denounces standardized testing, the more money there is to be made in standardized testing. For example, the whole world decided about a decade ago that filling in ovals with a Number 2 pencil couldn't possibly be a good test of Critical Thinking skills, so essays were added to many tests, such as the SAT and GRE.
This increased the cost (and slowed down the grading) considerably. Today, thousands of people, typically temps, are employed each year grading the essay portion of standardized tests. This article by Jessica Lussenhop of the Minneapolis City Pages takes you behind the scenes of essay grading. It's not a particularly edifying scene (although not all that dismal).
The take home lesson I get is that you should make sure your first sentence is sterling. Graders don't have time to read your essay carefully, so you want to make a good first impression.
so essays are not a way for graders to "fudge" things and give NAMs a boost right?
ReplyDeletegood. I thought so.
I actually like to assign essays for pure comedic value.
ReplyDeleteI've noticed a strong correlation between essay length and score in my academic experiences. The more senior get, the more affordable the luxury of word economy becomes.
ReplyDeleteI am extremely familiar with the ACT and SAT essay grading processes--although I'm not a reader, I grade my own students' essays on their practice tests and benchmark my kids' practice essays against their actual scores. I've done this for a couple hundred or more kids and tracked four classes of students all the way through. I'm satisfied that they are accurately assessing students based on the criteria they ask for.
ReplyDeleteEssays are not a great way to do standardized tests--they aren't granular enough, and the writing quality can never be the top consideration (and the guy who is deliberately making it the top consideration should be fired). Most standardized essay tests follow the following priorities:
1. Did you answer the question? (The Martin Luther King kid did not appear to meet this criteria directly).
2. Did you support your answer in the manner required by the test? (The most common reason for a ding).
3. Organization is only an issue if the essay is spectacularly unorganized, which is rare.
4. Mechanics/style considerations (can never be more than a point).
Failing to finish the essay is not grounds to give it a lower score.
While the grade is "holistic", and a good opening sentence is important, it's really not that simple, at least with the SAT and ACT. Since most kids can answer the question and most kids can organize, support in the proper form (for the SAT, it's still safest to use specific examples, despite the CB's protests) and writing quality are the primary differentiators.
The people interviewed all sound like idiots who are arguing with the rubric. Certainly, the one guy should be fired. Most of them should, though. The kid who wrote a creative essay about peeing on his burnt town should have received a low score. Sorry, that's how it goes.
What I like about the SAT/ACT essays is that low ability kids can get motivated enough to write a decent essay that gets them a good score (3-4 out of 6). What I don't like about the SAT/ACT essays is that kids who can write far better than low ability kids but aren't brilliant writers will get the same grades as the low ability kids. It's not great, but it's not a scandal.
Essay exams are fine when given to a smallish class and graded by the professor (or a competent TA). In fact, for a philosophy or history or literature class, you pretty much have to use essay exams. But it's ridiculous to think you can give a precise, accurate numerical grade for a written response to the kind of bullshit questions asked on the essay portion of standarized exams like the SAT. I suppose it might be useful to, say, admissions committees if they can glance at your SAT "essay" to see if can write coherently and grammatically under pressure. But beyond that....
ReplyDeleteThere are two styles of testing (at least). The first type are those who like to delay their work till the end and the second type do their testing work up front.
ReplyDeleteEssay questions are easy to create but hard to grade. For example, "Compare and contrast Jacksonian democracy with Jeffersonian democracy". I could make up a test with questions like this with no prep at all. I'd just go to the blackboard and write four or five such questions off the top of my head. Of course I would have to spend a lot more effort and time in grading. Figure zero prep and three hours grading.
Or you could prepared a well crafted multiple choice test ahead of time. That might take you a couple hours but if you keep good security you can use that test over and over again. By the second semester you have nearly no prep and no grading.
I always graded multiple choice tests in the class by the class. No home work for me!
My experience is that those instructors who give essay tests were constantly turned in to the administration for being unfair. I never had a grade complaint. Everything was objective, everything was factual and verifiable. If a question was ambiguous, I threw it out.
I had the highest flunk out rates in the school and I never had an official complaint from any of the flunkees.
You would think that if you got a degree in education you would be proficient in testing. Not so. Classroom teachers tend to be pretty innumerate.
Albertosaurus
Using my psychic powers, I will make a prediction: score on the essays will correlate with IQ, and won't noticeably narrow the black/white gap in scores.
ReplyDelete"so essays are not a way for graders to "fudge" things and give NAMs a boost right? "
ReplyDeleteIt was done for girls.
http://ezinearticles.com/?Money-Makes-the-World-Go-Round---A-Brief-History-of-the-PSAT-NMSQT&id=5119851
http://bit.ly/fXTIqo
handwriting plays a fair part too.
Essays graded that way are pathetically easy to produce. They're basically vocabulary tests.
ReplyDeleteThen again, very few people can follow logical argument in textual form. If it sounds right, they think it's right.
Another nail in the coffin lid for the concept of democracy and other forms of utilitarianism being a good idea.