June 20, 2012

Test Prep gains by race

Chris Hayes argues in his new book Why Elites Fail the real reason that blacks and Hispanics are making so little progress over the generations at qualifying on their own merit for selective academic institutions is because rich whites are hogging all the test prep.

Education Realist, who is in the test prep business, has a post linking to various recent studies of the popularity and effectiveness of prepping for admissions test by race. Here's one table (with score increases out of 1600, not 2400):

Use of Test-Prep Courses and Gains, by Race and Ethnicity
Group% Taking
Test-Prep Course
Post-Course Gain
in Points on SAT  
East Asian Am.
30%
68.8
Other Asian
15%
23.8
White
10%
12.3
Black
16%
14.9
Hispanic
11%
24.6
So, unsurprisingly, East Asians try the hardest at and get the most benefit from test prep, while whites, who are more likely to have heard and believed ETS's propaganda that test prepping is insignificant, try the least hard and get the least benefit. In the middle, blacks and Hispanics benefit from all the racial uplift programs for them.

This doesn't disprove Chris Hayes' assumption that Upper East Side whites are benefitting from test prep. My guess is that the big losers in this game are naive flyover folks. 

35 comments:

Anonymous said...

OT, but for future posts you might find this short YouTube clip of 1950s Los Angeles comes in handy, Steve.

How well behaved we all were! Wonder why.

http://www.youtube.com/watch?v=n77NxU0CHPw&feature=colike

Anonymous said...

You HBD folks are a queer bunch--so obsessed with test-prep and race.

V said...

I'd be interested in seeing a verbal / math breakdown. (More broadly, seeing distributions rather than just means would be helpful.) Moving from a 400 to a 500 is very different from moving from a 680 to a 700.

Anonymous said...

I think your misinterpreting the data.

The 10% of whites who take test prep are the ones who LEAST NEED it, their scores are already high.

If you expanded Test Prep amongst the white population to those who aren't already hitting their intellectual limit, I'm sure the numbers would be higher.

SFG said...

They're not just naive. One of the things I realized when living in a red state is that you really don't need an Ivy League degree in these places. Go to Ohio State, go to their med school, and go become a big Cleveland orthopedic surgeon. No need to fly to Cambridge. So why kill yourself scratching out those extra points?

anony-mouse said...

The smart flyover people who have been successes have been people like the Wrights and the Edisons.

How well did they do on tests? Who knows? Who cares?

I suspect truly practical people don't bother as much with written tests.

sunbeam said...

I'm surprised that with intensive preparation the increase was only 68 points for asians.

It's been about 30 years for the Sat, but my recollection now is it would have been easy to game if the importance of it was clear to me.

It was a different world even 30 years ago. The current environment with the prestige schools didn't exist to the same extent. Or maybe everyone wasn't aware of it, as they are now.

Still I think I could have done very, very well if I were 17 again, and taking this with a serious mind to do as well as I could.

There were only certain kinds of questions they were going to ask you. Prepare with things like memorizing latin and greek prefixes and suffixes, practicing taking these kinds of tests. I guess it is old hat.

The math would have been easier to do well on. I had a higher score on verbal than math, but in retrospect I think I could have scored higher on the math portion than the verbal with preparation.

My high school was pretty bad academically, but I had a great English teacher, and I think he made most of the difference.

Bostonian said...

Buying a book such as "Dr. John Chung's SAT Math" and working through
the 20 practice tests should make you better prepared than most. Many of the favorable Amazon reviews are by East Asians.

Truth said...

So; studying has no effect upon exam performance? Yes, well of course. What idiot would have ever argued the other way?

Anonymous said...

I had an Askenazic girlfriend from a lower middle-class Brooklyn family. She took the SAT without prep and got a 1250. Her mother found that unacceptable, she took a prep class, took it again and got 1420. That's a big jump, maybe more from taking it the second time, although when I was a kid you took the PSAT, then the SAT, and maybe that's what she was talking about. Prep makes a big difference in tests, it makes a big difference in *everything*. As the Boy Scouts say, Be Prepared.

I took the SAT once with no prep and got 1370. My military test score was 146, which matches.

TGGP said...

Anonymous, Steve isn't obsessed with prep. Chris Hayes brought up the subject and Steve is responding.

NOTA said...

I wonder how much the differences reflect the deficits on the tests in different groups that are amenable to cramming to fix. At one extreme, there are test taking techniques and tricks for handling specific kinds of question, which are probably relatively easy to teach in a test prep environment, and which boost your score a lot if you don't know them. People who already know them won't benefit much. At the other end, there are questions that, given a certain level of background knowledge, are more or less IQ test kinds of questions, and are really hard to cram for. If that's where you are falling down, I would expect test prep not to help much.

But this is speculation far outside my expertise. Does anyone have data that bears onthis question?

Pincher Martin said...

"Prep makes a big difference in tests, it makes a big difference in *everything*. As the Boy Scouts say, Be Prepared. "

And yet most of the evidence suggests otherwise for testing. A not insignificant percentage of test takers get lower scores after taking a test prep course and most see no significant improvement (30 points or more).

East Asians and your former Ashkenazi girlfriend are outliers, not proof that the exception of test prep helping to dramatically improve scores ought to be the rule.

Rather than assume that everyone ought to be test prepping more based on the East Asian experience, perhaps we first should figure out why they are outliers.

Anonymous said...

"My guess is that the big losers in this game are naive flyover folks."

Interestingly enough the same people are the most discriminated against in the college selection process as well.

So what we should expect to see as a result is a growing number of very bright White electricians in flyover country.

If you think America as a unified country is heading for an iceberg then that's probably a good thing for the people in those areas.

.
"Rather than assume that everyone ought to be test prepping more based on the East Asian experience, perhaps we first should figure out why they are outliers."

I'm beginning to think this article relates to the first glimmerings of concern that the immigration policies supported by people like Mr Hayes might have some unintended consequences they didn't expect.

lol

Anonymous said...

The two big names in American test prep are Kaplan and Princeton Review. The story of their beginnings is pretty interesting:

http://www.culturewars.com/2008/BrooklynExistentialism.html

"In the early 1950s Stanley H. Kaplan, a graduate of City College of New York, who in spite of his good grades couldn’t get a job, set up a small tutoring operation in a modest building in the Flatbush section of Brooklyn. Kaplan capitalized on Jewish educational aspirations at a time when the SAT had firmly established itself as the official rite of passage for entry into the colleges that granted access to the top positions in the American meritocracy. The WASP ruling class under Henry Chauncey’s direction had created what it thought was an uncoachable test that measured pure mental ability. The Jew Kaplan was smart enough to see through WASP pretentions and come up with a system that guaranteed better test results. The system was so simple that it hardly qualifies as a system at all. In the days when the blueprint for building the atomic bomb was an open secret compared to the questions on the SAT test, Kaplan came up with a simple but ingenious way to subvert the system. After each class graduated from Kaplan’s school and took the test, he would invite them back to celebrate with hot dogs and root beer; admission to the party was gained by having each student tell Kaplan one question he remembered from taking the test. The net result of Kaplan’s parties was a list of the questions that his students would face when taking the SATs. If Kaplan tutored five classes of fifty students in one year, at the end of that year he had 250 questions. By the time Kaplan sold his test-prep business to the Washington Post company in the ‘70s, for $50 million he had over 30 years experience in gathering questions, which meant he could tell his students with increasing accuracy the answers to those questions as well. Jewish scores on the SATs rose accordingly, as did Jewish admission to the prestigious colleges that had established quotas to keep them out in the early 20th century."

"As the ‘70s progressed the subversion continued, but because ETS refused to acknowledge what was going on, they played into the hands of Jews like Kaplan and his successor, the founder of Princeton Review, John Katzman. By refusing to acknowledge that people like Kaplan and Katzman could in fact raise test scores by their coaching, ETS unwittingly allowed a generation of cheaters into the meritocracy."

"In 1977, David Halperin, a state senator from Brooklyn introduced the New York State truth-in-testing bill because, as “one of the striving Jewish boys tutored for the SAT by Stanley Kaplan,” he felt that it was unfair that he not only recognized many of the questions but also knew the answers to them when he took the SATs. Halperin’s solution was to make the questions public, something which happened when the bill he co-sponsored was passed by the New York legislature. In doing that, Halperin effectively kicked over the secret ladder that had been used to subvert the tests and insured that no one else who took the prep courses after 1977 would have the advantages that the Jews had had up until that time."

Mr Lomez said...

"I'd be interested in seeing a verbal / math breakdown."

This is the missing info here. How much of test prep's effectiveness is teaching non Standard-White-English speakers how to speak the test's preferred dialect? Much, I suspect.

A lot of our language's grammar (which is now a part of the SATs and other major aptitude tests) is totally arbitrary. Take prepositional idioms for example: is it "different FROM" or "different THAN?" Knowing which, especially for a teenager, is not a question of g. One learns these rules by rote.

Someone growing up in a well-educated English speaking (see: white) family has the least to gain from this kind of test prep because they've already learned most of the rules merely by listening to their parents speak.

American blacks, though they also speak English, speak a different dialect than the one on standardized tests (though similar enough) and therefore have SOME to gain from test prep.

Children of immigrants and non-English speakers, unsurprisingly, have the most to gain. I.e - Hispanics and Asians.

Ultimately, though, Steve is right. The real losers are flyover whites, who neither speak Standard White English, nor qualify for test-prep outreach.

Pat Boyle said...

I'm not surprised that Asians do more test prep or that they benefit from it more.

First The Asian brain is obviously different. In a sense it is more masculine. All the ability tests were normed on Whites so that the quantitative and the qualitative scores were set to be equal. But Asians do better on the quantitive side and worse on the qualitative side. This is a very stable finding. It is found in all studies that I know of. It also corresponds to job market patterns. Asians are dominant in the computer sciences but much less so in law.

White men's brains also differ from those of women in much the same way on the math versus verbal dimension.

It's not clear why the Asian brain if different from the white brain but it is certainly possible that it responds to certain training regimens differently. I wonder if men benefit more from test prep than women?

Asians are more likely to desire test prep because testing has long been a part of their culture. As far back as the Sui Dynasty (I looked it up) China has had civil service exams. The Vietnamese and Koreans soon followed.

At about that time in Europe Johns Scotus and the Pope were the only two men who could read Latin. Literacy was at a low point in the West.

Asians have been doing test prep more more than a millenium. The planet's biggest civil war in the last century - the Taiping Revolt - revolved around a young man who failed the civil service tests twice. He then had a dream that he was the younger brother of Jesus Christ - a more Western style approach to prominence.

Asians take testing seriously.

Albertosaurus

Anonymous said...

"The smart flyover people who have been successes have been people like the Wrights and the Edisons." - citing examples from a 90% white America 100 years ago merely showcases how right Sailer is here. Conditions have changed and we have to adapt to that.

Anonymous said...

"Ultimately, though, Steve is right. The real losers are flyover whites, who neither speak Standard White English, nor qualify for test-prep outreach."

The flyover whites who go on to college do speak standard white English. Where did you get the idea that they don't?

Anonymous said...

A little math from the original paper from which these numbers are drawn.

E. Asian SAT average 1138 (out of 1600)
White average 1045

Average E. Asian boost from test prep: 68 points
Fraction of E. Asians participating in test prep: 30%

Does anyone here understand what this means in terms of the W-EA intelligence differential? (Hint: 30% of 68 is only about 20 points.)

(For the real experts, also consider that a larger fraction of the entire E. Asian population takes the SAT than in the white population. In the white population SAT-taking is biased toward elites.)

Hurrah HBD!

Anonymous said...

Kaplan was smart enough to see through WASP pretentions and come up with a system that guaranteed better test results. The system was so simple that it hardly qualifies as a system at all. In the days when the blueprint for building the atomic bomb was an open secret compared to the questions on the SAT test, Kaplan came up with a simple but ingenious way to subvert the system

Anon, that's really interesting. Imagine, New York's Scots-Irish showing so little gratitude that they'd try to break it to climb! Those wily Presbyterians!

Non-WASPs marched through our institutions, now trust in American institutions is about the lowest it's ever been, probably bcz the new elite are looting and breaking the country. Now can we finally mention some non-IQ [there ya go, whoever u are who babbles about the IQ obsession] reasons the WASP elite was better than the current crop of strivers and subverters?

One quality of WASP institutions that's sadly lacking in Mozillos and Kaplans is fairness: every system will be gamed by someone. WASPs tend to think that doing that is bad. Kaplans and such are all we cheated fair and square!

Test prep is pretty much zero sum and red queen: everyone's gotta do it to keep up, so everyone in society ends up worse off. WASPs made a better system
Problem is, a system made up of people who are all trying to subvert the system is more a dark comedy.

Soviet of Washington said...

V is correct. Michele Hernandez in her book 'A for Admission' states (from her days in Dartmouth admissions), that most of the test prep gain comes in the low-to-mid 600 range, i.e. from learning how to 'game' the test. Improving 700 and above scores really didn't happen often.

rob said...

Just want to mention that Asian gains may be so high for the same reason that Asians in Asia were rocking the TOEFL, GRE, and maybe others: they cheat, and 'prep' was cheating practice. Because Asians had such outsized gains, cheating is a pretty reasonable hypothesis.

'Course, the apples to oranges 10 hours vs whole childhood grinding is also reasonable.

Anonymous said...

There's prepping and then there's prepping. I'd like to know the details of the study: what exactly were the prepping methods, how much and how long were they practiced? There are little tricks for dealing with multiple-choice exams that can be taught in a few hours and can boost your score slightly. But if you want to make big gains, you have to commit a couple of hours a day to prepping, every day, for 4 or 5 months straight. Asians usually have no problem doing that.

Svigor said...

Just my anecdote again, but if memory serves, my combined SAT score (on the old test) went up almost a hundred points (from just below 1300 to just below 1400) after I took a cheesy (non-intensive) SAT prep course, read a couple of "SAT Vocab" and "SAT math" books, etc. I was an indifferent student in high school to say the least, and I think I was the kind of student that will typically see the biggest gains from test prep.

Svigor said...

Actually, I think my gain was slightly more than a hundred points.

Mr Lomez said...

"Ultimately, though, Steve is right. The real losers are flyover whites, who neither speak Standard White English, nor qualify for test-prep outreach."

"The flyover whites who go on to college do speak standard white English. Where did you get the idea that they don't?"


You're stacking the deck. The losers are the flyover whites who aren't on the college track, but should be -- it's the kid in the Missouri Ozarks with a 115 IQ but who can't string a proper sentence together. These are the whites with the most to gain from test-prep but who fail to get it for the reasons stated above, thus the white deficit in these statistics.

Bostonian said...

The SAT is supposed to predict college grades. If Asian prepping for the SAT is distorting their scores, you would expect their SAT scores to overpredict their college grades.

A 1994 College Board study "Student Group Differences in Predicting College Grades: Sex, Language, and Ethnic Groups" found that a model using high school grade point average, SAT score, and Test of Standard Written English (TSWE) score underpredicted the course grades of Asians by 0.10 and overpredicted that of Blacks and Hispanics by 0.11 and 0.08, while neither under- or overpredicting the grades of whites. The numbers are from the last row of Table 8 of the paper, which is available online.

Truth said...

That's pretty impressive Svigor, DOUBLING your SAT score through test prep!

Anonymous said...

The SAT is supposed to predict college grades. If Asian prepping for the SAT is distorting their scores, you would expect their SAT scores to overpredict their college grades.

If you're the kind of person who busts his ass prepping for the SAT, you're probably also the kind of person who busts his ass doing homework and studying for midterms & finals. And we know that busting your ass on those things is even more effective than busting it on standardized "aptitude" tests.

JSM said...

"That's pretty impressive Svigor, DOUBLING your SAT score through test prep!"

Wtf? How is going from just under 1300 to 1400 a doubling?

Anonymous said...

Saying top schools are not meritocratic because there are fewer blacks is like saying NBA isn't meritocratic because it has no Mexicans.

M. Simon said...

My test prep was taking the test every time it was offered in my jr and sr year. I got roughly the 70 point increase reported by the East Asians. (May have been 50 - it was a very long time ago)

M. Simon said...

Jewish scores on the SATs rose accordingly, as did Jewish admission to the prestigious colleges that had established quotas to keep them out in the early 20th century.

Keeping Jews out with quotas hardly qualifies as a meritocracy.

So Jews fought back by figuring out how to raise their scores. Good. It ruined the value of testing? Well every subversion of "intent" to get results does that.

Slavery had a similar effect on our political system.

Evidently no elite gives way willingly for the betterment of the human condition. So be it.

Kurt said...

SFG wrote: They're not just naive. One of the things I realized when living in a red state is that you really don't need an Ivy League degree in these places. Go to Ohio State, go to their med school, and go become a big Cleveland orthopedic surgeon. No need to fly to Cambridge. So why kill yourself scratching out those extra points?

This is very true. Although I have an Ivy undergraduate degree and graduate degrees from an elite public institution, I now live in "flyover" country, and one of the things that is quite evident to me is how few of the most successful and influential folks in my state have degrees from elite institutions. The more time I spend here, the more I realize that elite degrees are really most valuable to those who worry about prestige in the large coastal metropolises and in elite (or aspiring) academic institutions.