October 30, 2013

The view from the trenches on David Coleman's plan to revamp the SAT

Two comments from readers of Inside Higher Ed on the new College Board president David Coleman's plan to introduce an all new Common Core-ized SAT college admission test in 2015:
john • a month ago − 
This article is wrong. It's misguided in its premise and fallacious in its reasoning. 
Oh where to begin. At the core there's the lament from the progressive cognoscenti that the SAT/ACT is unfair and unnecessary. The later assertion is demonstrably untrue. Rampant grade inflation and the unevenness in the quality of public and private schools makes GPA's at best an imprecise indicator of a student's scholasticism. To the former 'unfair' quality of these tests -I say yes. Of course, no test can can measure a student's heart, his resourcefulness in the face of terrible adversity, his ability to carry the day by inspiring others. In other words, test’s are imperfect. As Churchill opined, “Democracy is the worst form of government, except for all those other forms that have been tried from time to time.” The same can be said for these standardized tests. 
Now to Coleman’s specific criticisms that the essay is meaningless. First it accounts for only about 200 of the 800 points on the writing so this seems to be more of a tempest in a teapot than anything else, or more likely a sop to throw to those within the educational establishment who constantly hector others over this or that perceived inequity in testing. I’m not going to get into an epistemological debate over what is thinking and learning, nor will I point out how doltish the assertion is that students can simply fictionalize their examples. The link w/in the article

to MIT students who make up plausible-sounding hoax versions of American history to use in essays.
demonstrates my point in that anyone who can spin such a well written alternate US history thesis is demonstrating greater skill than any bare recitation of the actual facts. 

Kids, don't try making up alternate versions of American history if you're not an MIT prankster.
The differences between the SAT/ACT are small and, except in one area, insignificant. All my students who’ve prepared for one comment on the tests' similarities. ... 
What rankles me is when Coleman says this, “He noted that when someone says a person has just used "an SAT word," the idea isn't that the person has shown eloquence or clarity but that "they have used a word they would never use again." The false assumptions there are many. The SAT has moved away from vocabulary in the last 25 years, and it has been wrong to do so. Through the years, educators have been left disconsolate over the sight of students sitting in test prep classes or in lonely school hallways trying to force feed themselves a diet of 2,000 vocabulary words. These folk lamented that students weren’t ‘learning’ anything, and therefore this test was bad. This mode of thinking is emblematic of the failure of much of our educational system today. Educators with soft hearts are under the mistaken belief that everything can be taught in a fun and engaging manner. This is pie in the sky thinking that points to people like Jamie Escalante (the famous L.A. teacher that was chronicled in the film Stand and Deliver) as a teacher who can make even the most erudite subjects accessible. The failure there is confusing the outlier with the majority. The brilliance and exceptionalism Escalante demonstrates is the very reason we make movies of these individuals. We don’t make movies of the thousands upon thousand of mediocre teachers, nor do we exalt those sports participants who are just okay. 
Yet, the education establishment posits much of its curriculum around just such teachers. Analogously, designing and teaching a high school basketball team under the notion that they are all fledgling Michael Jordans could only lead to disastrous results. Learning words is hard. For most students it’s tedious. There is no magic, no epiphanies to be had where like Dorothy’s yellow brick road the whole journey is clear and stretches out plainly for the student to see. The same could be said for memorizing times tables, the Periodic table, the capitols of the States and so on. The point is that life is full of these moments and the sooner you learn this the sooner you can stop fretting over the supposed inequity and get the job done and hopefully move on to something more fulfilling. 
Second, Coleman’s assertion that SAT words will never be used again is truly abhorrent. See there’s one right there. Forbes did a survey of major companies that asked what they looked for in new hires, and other than what college they attended, being articulate was deemed essential for success. Coleman’s attempt to dumb down the SAT or any test by removing this element follows the specious assumption that testprep is somehow undemocratic, elitist, unfair to the majority. 
In the area of vocabulary he couldn’t be more wrong. For no cost a student can jump on the internet and download the 2,000 or so tough words that may be on the test. Where’s the elitism there? Maybe the student hasn’t been in the best english program but that’s irrelevant if she’s just willing to work hard at memorization. Hard work, what is more egalitarian than that? 
Furthermore, learning vocabulary, as previously cited, has positive consequences in the real world. ... 

Zam • a month ago − 
This guy does not make very much sense to me... 
1) Aligning the test more closely with curriculum will make it MORE coach-able, not less. We already have tens of thousands (maybe 100k+) straight A students graduating from high school each year. High school curriculum just isn't that difficult. With hard work and organization (either directly or by parental fiat), you don't need to be particularly bright to get all the answers correct. The purpose of the SAT and ACT, regardless of the political incorrectness of the observation, is to isolate reasoning ability independent of things like hard work and organization skills. What would be the purpose of a test that simply mimics the GPA? How would that be useful to admissions officers? 
2) Furthermore, the College Board itself has conducted a fair amount of research on the efficacy of test prep and they have shown that in the aggregate, it just doesn't improve scores very much. So what's with the diatribe against test prep?

I think we need a major independent study of the effectiveness of massive test prepping.
... I've got a news flash for Mr. Coleman... the test prep industry will get MUCH bigger, not smaller, if the test becomes more aligned with curriculum! 
3) The essay is not a writing test, it's a written reasoning test. Surely it can be improved. But my experience is that essay scores correlate pretty well with the rest of the test, which tells me that in a broad sense it accomplishes its goal.

As Robert Gordon has suggested, everything is an IQ test, in the sense that we see positive correlations between almost all tests of anything.
4) This is actually the strangest part. "they have used a word they would never use again." ??? Seriously, has he ever looked at the Critical Reading portion of the test? Pedantic, Wry, Disingenuous, Mitigate, Sedentary, Soporific, Antithetical, Charlatan, Pragmatic, Didactic, Ubiquitous, Insidious, Allusion, Paradigm...
Oh the humanity, when will I ever use these words again?? How about, 'just about every day in your adult life'! These are all words that appear regularly in level 4 and 5 difficulty questions. The number of truly obscure words on the SAT is very small. This test does a great job of including words in its difficult questions that most college freshman should know, but don't. To put someone in charge who does not agree with this statement seems utterly bizarre to me. 
In conclusion, it looks to me that the College Board is shooting itself in the foot. No college admissions officer needs a test score that is simply going to mimic a student's GPA, and that seem to be the direction Mr Coleman is going in.

9 comments:

Anonymous said...

Of course, the best way to learn vocabulary is by doing a lot of reading. Students who read for pleasure are likely to do well on this aspect of the test, and they'll probably do well in college. I had no trouble with the vocabulary on the SAT or GRE, and I barely prepped for that at all - I've just read a lot of books and picked up those words along the way.

Education Realist said...

Those comments are dead on, except I believe the essay is a third, not a quarter, of the writing score. It was originally going to be half, if I remember my Kaplan training, but they killed that after the field test.

There's actually one SAT prep place that gives, for $800, a list of 200 words that are "guaranteed" to be on the SAT. They claim they do it by analyzing the previous tests, although I wonder if they are paying off a College Board official. You used to be able to see the kids in the college confidential forum talk about how accurate it was. Most of the times it was extremely accurate but every so often something went wrong (maybe the payoff didn't work!).

It's absolutely true that the SAT's vocabulary has been de-emphasized, and it does worry me that it's now much easier to memorize even 1000 words that have no meaning and then regurgitate. Never used to think it possible, but I'm getting a bit more concerned (again, I've been reading a whole lot lately). I hope I'm too doom and gloomy.

I still think standardized tests are better than anything else. I'm just worried that just as the Chinese broke the GRE, that the Chinese, Pakistanis, and Indians broke or constantly try to break the techie certification tests, that our other standardized tests are under assault in ways we can't fathom.

IN other news, Korea's SAT is under investigation again, a little fact that Amanda Ripley managed to utterly ignore in her recent paean.

Anonymous said...

The writing scores also correlate highly with college grades, more than other subtests. Sounds about right for american academia.

Anonymous said...

Uh, you guys?

I just had a look at the 2013 SAT vocab list. If words like "sermon" or "scrutinise" (ethnic pride forbids me from using Amurkin spelling) or "ornate" or (god help us all) "taxing" are considered words you'll never use again then, well, you weren't going to do well in the test to begin with. (That's the most charitable, least profanity laden way of putting it that I could come up with. And this is coming from a certified moron who just likes to read.)

jody said...

i read the dictionary. dead serious. got a dictionary, began literally reading it. webster's collegiate, 10th edition. 1300 pages. started reading it in the early 90s, made it all the way through in a year.

part of this way spurred by reading my dad's books when i was a kid, and hitting tons of words where i had no idea. when i was a kid i read neuromancer in 1985, only a year after it came out. 'read' would probably be a better description because gibson's vocabulary was enough to stop me dead in my tracks every couple pages. hard to follow the story until i learned a lot more words.

it took a couple tries to read it, but after i got enough new words, i realized the book was awesome.

i was surprised to hear the verbal analogies were whacked from the new SAT - that was definitely the hardest part of the old SAT. those last couple questions were the real deal, pushed you to the limit. had that feel of an IQ test and those number sequence questions.

you could sit and stare at those last questions for a while and just blank out, no ideas coming at all. even the hardest couple SAT math questions you got glimpses of a slight, furtive idea in your mind, flashing briefly before blowing apart into unarticulated mind fragments as your brain couldn't put the pieces together on how to solve that one.

jody said...

best way to check whether a word is something any particular person should know is to run a google check on it. number of occurrences should be the guide.

i've seen some of these lists of '100 words any college graduate should know' and their google occurrence rates vary wildly, with a few of the words dipping to 1000 occurrences. meaning ALMOST NOBODY uses them.

i'm not sure what a good rule of thumb should be but i'm guessing if a word occurs less than 20000 times or something along those lines, most people probably don't need to know it.

occurrences for all words may have gone up in the last 5 years or so since i did that experiment, as google inexorably grows, so don't quote me on absolute levels. ratios is what matters. for instance i just ran a check on pecuniary, an uncommon word, and it returns 3.8 million results. scintilla, 3.1 million. bifurcate, 841000. vitiate, 629000. exiguous, 177000. logomachy, 57000.

just grab one of those lists off the web and run a google check on them all. you'll quickly see a few of the words are almost never used. looking over my updated, 2013 results, anything below 1 million results and the average guy probably won't know, anything below 500000 and you're into the obscure. 100000 seems to be the new 1000 as far as 'NOBODY uses THIS word'.

man, google has grown. 57000 results for some really obscure stuff. still remember when i was getting 1000 or 2000 results for some of the most obscure words.

good thesis project for a computer science/robotics/linguistics undergrad would be to run the dictionary through google this way and find the least commonly used words in the english language which are still recognized as words by the authorities (probably have to use scrabble rules here IE the authority will be some particular exact dictionary). results for the most commonly used words would be fuzzy. too much noise.

Anonymous said...

The distinction between aptitude and achievement seems less important in practice than in theory.

The smart people I know are not just intelligent, but they actually enjoy learning stuff.

People talk about "hard work" like it's a completely exogenous factor, but it's the naturally smart kids who are more likely to work hard, since they possess the most intellectual curiosity.

I'm sure there are brilliant people who are academically and intellectually lazy, and I'm sure there are dim people who are enthusiastic about learning, but I don't think there are very many of either.

Difference Maker said...

I scored 800 more than once on the writing and failed both high school and college English

I'm pretty great

Anthony said...

Kids, don't try making up alternate versions of American history if you're not an MIT prankster.

Or Harry Turtledove.