Showing posts with label SAT. Show all posts
Showing posts with label SAT. Show all posts

September 19, 2011

Asians, aptitude, and achievement: a positive sum reform proposal

The traditional concept of college admissions was that the goal was to predict applicants' future achievement (which could be measured in terms of first year in college grades or money donated 50 years later or whatever). The most obvious way to predict future achievement was past achievement: e.g., high school grades. Presumably, past achievement had two main components: hard work and aptitude.

But there were some obvious problems with relying solely on high school grades, such as different levels  of grade inflation at high schools. If you were getting most of your applicants from St. Paul's and Dalton, well, you could keep in mind the differences, augmented with letters of recommendation from headmasters you had known for years. 

But once the Ivy Leagues started trying to find the most promising non-upper class kids from the rest of the country, they needed something more objective about individuals than just grades. Another issue is that high school grades have certain inherent shortcomings. The future Nobelist in physics might not care about his social studies class and thus wind up with a lower overall GPA than the well-adjusted grind. Plus, grades have a ceiling. Even an A+ in physics doesn't really tell you that much. Moreover, lots of future successes are alienated in high school. Some people who get all As in high school might not have the upside to continue to do so in college. Incentives toward grade inflation at the high school are built in. And so forth ... Top colleges kept asking for recommendation letters, but their value (and, thus, importance) dropped as they increasingly came in from random teachers in random places like Burbank, CA.

So, for various reasons like this, the Scholastic Aptitude Test was created and spread. The idea was to have an objective, national test of academic intelligence. Overall, the SAT would appear to have been a huge success. American colleges are the most fashionable and richest in the world today. 

However, there have long been complaints about the SAT. The most fashionable involved The Gap. Whites averaged higher scores than blacks. This posed a major PR problem for the academic establishment. The SAT (and ACT) is essential for their continued thriving, but saying that blacks are less intelligent than whites on average is The Worst Thing in the Whole World. But that's what the SAT says. And the SAT is the cornerstone of academic elitism, which has made American academia globally the envy of the academic world.

Thus, over the last half century or so, there have been anguished discussions between the front men for the academic world and the psychometricians at ETS about how to Close the Gap, without throwing the baby of predictive power out with the bathwater.

One change was purely PR: the SAT doesn't stand for Scholastic Aptitude Test anymore. It just stands for SAT these days. Under the hood, there have been a host of tweaks intended to narrow the gap without trashing the predictive powers of the SAT too much.

For example, the upper range of the Verbal (now Critical Reading) test has been capped. Before 1995, it was very, very hard to get an 800 on the Verbal test. I came fairly close the first time I took the test in 1975, so I gave it another try, got a little closer, but gave up and didn't take it a third time because the two scores seemed quite accurate: I'm very good at verbal logic, and have a certain gift for insights that other people wouldn't come up with, but I'm not a meticulous thinker. I make lots of mistakes. I'm more of a let's run it up the flagpole and see if anybody salutes thinker. In contrast, say, Charles Murray's brain works like a BMW V-12: powerful and precise. Mine's a jalopy that might surprise you and win the race or might break down on the starting line and go nowhere. So, there didn't seem like much point in me doing a lot of test prep to try to score 800 on the verbal -- I'd still make a mistake or two or they throw a really hard question at me.

But now, an 800 is well within reach of a lot of well-drilled students.

So, before scores were inflated in 1995, the SAT-V was an excellent test of high end verbal brainpower. In contrast, the SAT-M was widely recognized to need more headroom. It wasn't uncommon at Rice in the 1970s to hear good but not great Sci-Eng majors say, "Well, sure, I got an 800, but I'm not a real 800 like Joe is."

An obvious reform would have been to make scoring of SAT-M more like scoring of SAT-V. Instead, College Board - ETS did the opposite in 1995. One reason was that all that headroom on the Verbal modestly increased The Gap. The V test was made much easier to score 800 upon in 1995. A 730 old style became an 800 new style.

Lots of other tweaks were made, but as far as we can tell, anything that raises black average scores just encourages harder scraping of the bottom of the barrel by society, so the white-black The Gap remains remarkably stable over the generations. For example, The Gap on SAT-Critical Reading dropped about a half decade ago, perhaps because of the changes on the test, such as deep-sixing analogies. But that apparently just encouraged the College Board to troll for more black test-takers with free tests, so The Gap is now even bigger. (I may be overinterpreting a few squiggles on the trend graph.)

But white parents still tend to assume that SAT stands for Scholastic Aptitude Test. It's not an achievement test in their heads. The College Board says there is no point in studying extra hard for the SAT, and why would a prestigious not-for-profit institution spin the truth? If you can't trust the College Board, who can you trust? And signing your child up for intensive test prepping would be unfair to poor blacks who can't afford all that tutoring and drilling. Plus, prepping for years would be a lot of work for little Taylor, so just let him have his fun.

Meanwhile, lots of people from Fujian are showing up in America whose merchant ancestors ascended to mandarin status by spending their mercantile profits at Confucian literature cram schools for their sons. The assumptions about the SAT flitting around in white people's heads would never occur to them. "Test prep is unfair to poor blacks? Huh? You crack me up! I like you! You are very funny!"

Not surprisingly, we see vast amounts of white upper middle class rage directed at Amy Chua. 

Now, is devoting hundreds of hours to prepping for the SAT a Good Thing or a Bad Thing? Well, let me try to reframe that question more productively and think about SAT test prep's opportunity cost.

It could be that SAT test prep has long term benefits other than getting into a fancier college. Could be ... I dunno. I haven't seen any evidence one way or another. But, it seems more like a zero sum game. SAT test prep seems kind of a sterile form of studying compared to studying an actual subject like Physics or French or Music Theory or World History or Microeconomics.

The return on investment for the colossal number of hours devoted in recent decades to SAT cram schooling is modest. The test is designed to be hard to prep for, so it's taken gigantic efforts for gains measured in fractions of a standard deviation. 

It seems to me that it would be better for everybody if more test prepping energy was invested instead into positive sum games, such as studying for achievement tests rather than for aptitude tests. Fortunately, we currently have a quite good set of national achievement tests: the College Board's Advanced Placement tests. They are not subject to the incentives for high schools to inflate their grades: the AP tests provide objective national grades that do a good job of predicting what the high school student would score on a 101-102 level course as a freshman in college.

In contrast, AP tests are intended to be ones you can study for, so the ROI on test prep effort tends to be quite a bit higher than on the SAT. Far more students are passing AP tests than a decade ago, and that's a good thing. Why have young people waste their time studying for something that's built to be hard to study for when they can instead study subjects that are intrinsically worth studying?

Unfortunately, the current college admissions system gives little weight to AP test scores. Instead, perversely, it gives too much weight to taking AP classes in high school, even if you then do bad on the AP test. The University of California, for example, in calculating high school GPA adds a full point to classes designated Advanced Placement. Thus the average high school GPA of UC Berkeley freshmen is a wacky 4.39. An internal study by UC showed that cutting the bonus for an AP course down to 0.5 would better predict freshmen grades. I don't believe the UC system counts AP test scores  in the admission's process, or doesn't count them much.

As Mitch has pointed out, this system is doubly rigged in favor of the more goody-two-shoes high school students. Typically, you need high grades in earlier classes to get into high school AP classes, where you are then given a full extra point for your GPA -- even if your AP score shows you didn't actually learn much.

So, the ideal system would be for college admissions to be retooled from mostly a two legged stool of grades and SAT scores to a three legged stool of grades, AP Test scores, and SAT scores. The SAT could then be redesigned to be more purely an aptitude test that would be less easy to game. Moreover, test takers would have less incentive to devote hundreds of hours to gaming the SAT because they were being encouraged to spend hundreds of hours mastering AP Chemistry or AP European History. 

The AP tests should be reformed to make them better for college admissions. They are currently scored on a 1 to 5 scale with a 5 equating to an A in the average college's freshman year introductory course in that subject, a 4 equal to a B, and so forth. A weird aspect of this is that all 5s are not created equal. For example, to get a 5 on the AP Chemistry test, you have to get a little over 60% right. So, people getting 98% right don't get a higher score than people getting 68% right. Test scoring should be kept the same at the lower levels -- a 3 would still be a C -- but the maximum score extended from 5 out to 7, which would be like an A at Caltech. Meanwhile, the GPA boost from taking an AP course would be eliminated, at least before senior year.

The initial winners from this changeover would, of course, tend to be Asians, who currently take a lot of AP tests. But good for them. Whites in heavily Asian areas, who have already started to adapt to the Asian challenge, would do okay. Whites in flyover regions would be challenged to get on the ball with AP. My guess is that it would be good for them and that they would eventually respond well to the challenge.

Overall, my plan looks like it would be better overall for society. There's a huge amount of energy out there looking to get an advantage in the college admissions process, so why not direct it in some positive sum direction?

Yes, sure, obviously it's a win-win, but, does it solve America's most overwhelming problem: Closing The Gap? Will blacks come closer to whites in scores under my system?

I dunno. I haven't thought about it. In fact, not worrying about Closing The Gap has allowed me to put forward a novel reform suggestion that might be better overall, which is not something you see too often these days.

In America today, 98% of the thinking devoted to college admissions goes to figuring out how your own kid can claw his way to the top, and the other 2% goes to airy handwaving theorizing about Closing The Gap. That leaves 0% devoted to thinking about improving the system overall.

Now, if I were truly, fanatically public-spirited, I would devote a lot of energy to dreaming up some bogus but persuasive-sounding theory about how my reform would Close The Gap, which would make it a lot more likely to be adapted. But, I'm not saintly enough to make up an elaborate lie.

September 16, 2011

Mitch explains SAT trends

Respected commenter Mitch explains what is going on with rising Asian SAT scores:
It's a mix of things. 
First, the 2005 changes made the math subject material easier, but an actual high score became harder because there were fewer hard problems and the impact of "unforced errors" became higher. Kids who were bright but careless could get a high score because the occasional unforced error was wiped out and more with their performance on the high difficulty questions. That last sentence describes whites more than Asians.  
So since 2005, the ability to nail every question and not make unforced errors--something that drill does, indeed, help--has been rewarded, whereas the number of creatively difficult problems is 0 or 1 per test. This hurts white students, on average, more than Asian students just by personality trait, and then the Asian tendency to drill for this test gives even more of an advantage.  
The reading test has been made unequivocally easier. I'm not sure what you mean about some reading questions being moved to the writing section. This is not true. The writing test is a near-exact replica of the old English Composition Achievement test, or the English Writing Subject test. There were no changes to it at all from a content perspective--they just changed the type of essay prompt and reduced the number of questions. 
Certainly, the easier reading test makes it easier for Asians to get high scores. The writing test rewards attention to detail above all 
So the test changes play a part, both in how they reward the traits more likely to be in asians over whites, and then in the Asian prep ritual--which really has to be seen to be believed. I teach in these schools, and the kids are in prep taking tests for 2 years. Even the ones who aren't getting super high scores are getting better scores, and that's bumping the average up. 
Then there's the fact that Koreans, Chinese, and Indians are immigrating here in huge numbers, which is presumably offsetting the once larger percentage of Filipinos and Tongans.  
I'm assuming you were only looking at US students, right? [I don't know -- it's not obvious from the College Board documents.] Koreans in Korea are taking the test as well, and there they study 40-60 hours a week, instead of school. The prep schools there buy copies of the most recent tests from students and use the tests to prep their students (something that's frowned upon here, although not technically illegal). The kids learn how to write essays by rote, and have whole essays memorized (use this essay for "change", this one for "education", and so on).

There's a general lesson here. Our society is constantly changing and bending procedures in hopes of Closing the Gap between the more feckless and the more feckful racial groups. Yet, perhaps unsurprisingly, the more feckful tend to better exploit these changes for their own benefit. 

For example, the changes in federal regulator attitudes toward zero-down and low-doc mortgages that George W. Bush announced in 2002 in the name of fighting racist redlining at his White House Conference on Increasing Minority Homeownership  poured hundreds of millions of dollars into Angelo Mozilo's net worth, most of which the SEC allowed him to keep even after fining him. But the net worth of the median Hispanic and black is now lower after mortgage follies.

The changes in the SAT in the last decade were mostly due to a single man, University of California chancellor Richard C. Atkinson. The UC system is the College Board's biggest client, and the UC was told by California voters in 1996 that racial preferences were now a violation of the state constitution.

Atkinson held an inflated notion of his expertise in the field of psychometrics. "When students asked me about IQ testing, I frequently referred them to Stephen Jay Gould’s book The Mismeasure of Man, published in 1981; it is a remarkable piece of scholarship that documented the widespread misuse of IQ tests," he wrote in an essay explaining part of his motivations, although he disingenuously left out all mention of the 800 pound gorilla in the room for UC, Proposition 209. The state legislature's Latino Caucus had threatened to cut the university system's budget unless they could figure out a way to cheat on Prop. 209 and get more Latinos admitted.

Now, Atkinson wasn't a complete cynic. He was more of a fool. He thought he could kill a whole bunch of birds with one stone. For example, he wanted the SAT to be more of an achievement test than an aptitude test because of his quasi-Gouldian views on IQ testing:
My views are similar to those of Alfred Binet, the French psychologist who, in the early years of the last century, devised the first IQ tests.  Binet was very clear that these tests could be useful in a clinical setting, but rejected the idea that they provided a meaningful measure of mental ability that could be used to rank order individuals.  Unfortunately, his perspective was soon forgotten as the IQ testing industry burst onto the American scene. 

This is like saying:
My views are similar to those of Nicolaus Copernicus, the Polish astronomer who, in the middle of the 16th Century, devised the first heliocentric system. Copernicus was very clear that heliocentrism reduced the number of crystalline spheres and epicycles necessary, but rejected the idea that we could explain the motions of the planets without the Music of the Spheres. Unfortunately, his perspective was soon forgotten as Kepler, Newton, and Einstein burst onto the astronomical scene.

Atkinson's goal was to make the SAT less of an aptitude test and more like the SAT II Subject Tests, which are more achievement tests. (UC long demanded applicants take both the SAT and three SAT Subject Tests, a high degree of overkill.) Since we've all know about how blacks and Hispanics are obviously oppressed by the white man's IQ pseudo-test, the thinking went, this would ameliorate UC's little problem with the Latino Caucus by Closing the Gap.

Except, it didn't work out that way because blacks and Latinos tend to not only have less aptitude, but they tend to be lazier about achieving academically. Also, it made the new three-part SAT 3.75 hours long, which may tend to mentally exhaust blacks and Latinos. The winners from these "reforms" turned out to be Asians, who, on average, work hardest. 

Of course, the UC schools already had plenty of Asians. So, a few years ago, UC, having turned the SAT into something like the SAT Subject Tests, announced it wanted to stop requiring the SAT Subject Tests as now being redundant. This sensible reform, however, outraged the Asian-Pacific Islander caucus in the California legislature. The more tests parents have to remember to sign their kids up for and pay for tutoring in how to beat the tests, the better Asians do versus everybody else.

One thing to keep in mind is that elite private colleges don't really seem to want more Asians (at least if they're not going to pay list price. Foreign Asians, who don't qualify for financial assistance, are increasingly fashionable with colleges.). The more Asians score high on the SAT and get inflated GPAs by taking a lot of AP courses, the more the super-prestigious private colleges, which can and do use quotas for admitting blacks and Latinos, appear to discriminate against Asians.

Why they discriminate against Asians has been speculated about, but not, so far as I know, studied in any truly illuminating fashion.

Do they not donate as much money to their alma maters? My vague impression is that Indians, with their ancient tradition of alms-giving, are pretty generous when they have a chance to get their name put on an academic building. (I recently took enthusiastic part in a three minute standing ovation for an Indian gentleman who was the chief donor for the new library at my son's high school. It's a really nice library and it was all paid for and build by the week before my son started school there, so it didn't cost me a penny, so clapping my hands sore was the least I could do.) Chinese benefactors? Maybe not so much ... I don't know. This is the kind of thing that colleges have no doubt studied in intense detail, but their findings are Top Secret.

Or maybe too many Asians is considered uncool by high school students. For example, UC Irvine has long been heavily Asian, but it never seems to climb in coolness with kids. 

Or maybe the elite colleges just don't believe the high SAT scores being recently recorded by Asians. Who knows? Obviously, elite private colleges know what their motivations are for requiring higher paper credential from Asians, but they aren't telling, and nobody seems to be asking.

SAT score trends

The Unsilenced Silence blog has a good graph of Asian v. white SAT scores from 1996 to 2010 in terms of gaps in standard deviations:
So, like most things involving test scores, the gap was stable in the  later 1990s. However, Asians improved through much of the 2000s, especially on the Writing test, a new section of the SAT that strikes me as easier to game because it uses subjective (but very quick) grading. 

One possibility is that globalization of American colleges has led to more Asian-in-Asia elites having their kids take the SAT, which could be increasing the pool of smart Asians taking the test. I'm also concerned about test security, however, when the test is administered in Asia. The Graduate Record Exam has been plagued by instances of cheating in Asia. Also, the new Writing test is rather like the tests that the Chinese Empire conducted for millennia to pick mandarins. 

Another possibility is that the legal immigration system is working in bringing in smart Asians, who are becoming a larger fraction of the Asian-American population over time. But what about regression toward the mean? Amy Chua is very smart, for example, but she's nowhere near as smart as her father, the conceptualizer of the memristor.

The take home lesson for Americans would seem to me to be that Americans shouldn't credulously trust assertions by the College Board / ETS / ACT that their tests can't really be gamed through test prep or other means. The SAT and ACT are, relatively speaking, very good tests, but we live in a highly competitive, globalized world full of people who are smart, hard-working, and less trusting and less believing in Fair Play than naive Americans.

Now, here's his graph of the white-black SAT gap, which has been widening:

The sharp decline on The Gap in  2006 on the Verbal test (renamed Critical Reading) may have something to do with changes in that test, perhaps intended to make it easier on blacks. But, the general trend has been for The Gap to get worse, especially since the early 1990s. Yet, since the early 1990s (the Crack Years) were not a halcyon period for young blacks, the widening of the gap probably reflects concerted efforts since then by the Great and the Good to get more blacks to take the SAT (scraping the bottom of the barrel harder). 

The process might work like this: the University of California, prevented from using racial quotas by Prop. 209) demands changes in the SAT Verbal that it hopes will boost black and Hispanic scores. A small amount of initial success in narrowing The Gap, however, leads elites to then over-confidently subsidize more black test-takers. Scraping the bottom of the barrel harder drives The Gap in Verbal back up, and increases The Gap in the other two sections to new heights. (Or I may be over-interpreting this.)

Also, it's likely that as test prep and gaming of the SAT has increased, that whites have benefited more from this than blacks. My mental model of who is doing more and more effectual test prep is Asians > whites > blacks and/or Mexicans > Other Hispanics (many of whom just arrived from Guatemala and are pretty clueless about the kinds of things that lead to higher scores that seem obvious to, say, a Hong Kong Chinese family in San Marino, CA).

Anyway, the White-Black gap graph shows the stability in gaps that is the overwhelming main lesson of generations of test score studies. Despite intense efforts by society since the mid-1960s, we still see about a one standard deviation gap between whites and blacks on a host of tests of cognitive ability (La Griffe du Lion's Fundamental Constant of American Sociology). But, that makes the recent change in Asian v. white scores even more interesting and more deserving of study.

Unsilenced Science also has informative graphs showing trends by year in SAT scores (i.e., not standard deviations).

September 15, 2011

SAT score changes by race since 1996

The oldest SAT score report on the College Board website is from 1996, right after the "recentering" in 1995 that raised scores about 100 points on a 400 to 1600 scale. Over the last 15 years, the average overall score on the original two-part Verbal + Math SAT (i.e., ignoring the new-fangled Writing section of the test introduced in the last decade) fell a grand total of two points, from 1013 to 1011. (See what I mean about baseball statistics being more volatile?)

1996 v. 2011 College-Bound Seniors Avg SAT Scores










Total (V+M)
Verbal

Math

1996 2011 Chg 1996 2011 Chg 1996 2011 Chg
ALL 1013 1011 (2) 505 497 (8) 508 514 6
Female 995 995 0 503 495 (8) 492 500 8
Male 1034 1031 (3) 507 500 (7) 527 531 4
Asian 1054 1112 58 496 517 21 558 595 37
White 1049 1063 14 526 528 2 523 535 12
Black 856 855 (1) 434 428 (6) 422 427 5
AmerIndian 960 972 12 483 484 1 477 488 11
Mexican 914 917 3 455 451 (4) 459 466 7
PR 897 904 7 452 452 0 445 452 7
Other Hisp 931 913 (18) 465 451 (14) 466 462 (4)

You'll note that the average white score went up 14 points form 1049 to 1063. Did white people get smarter over that period? I don't know. The SAT changed a lot over those 15 years, with analogies being dropped and some Verbal multiple choice questions being exiled to Writing. Also, kids appear to have cared more about prepping in 2011, although the College Board doesn't like to talk about this.

Asians went up 58 points, which is pretty striking. Everybody else fell farther behind whites, which wasn't supposed to happen.

Now, it could be that scores actually did pretty well over this 15 year stretch, because the College Board scraped the bottom of the barrel harder. In 1996, 1,085,000 college-bound seniors took the SAT. In 2011, there were 1,647,000 senior SAT-takers. Just between 2006 and 2011, the College Board let an incremental 150,000 students take the SAT free or at reduced cost.

On the other hand, my impression is that it became a lot more common for students to take both the SAT and ACT over that 15-year stretch, so some of the increase in the number of test-takers comes from people who would only have taken the ACT in 1996. It used to be that East and West Coasters took the SAT and Midwesterners the ACT, but by 2011, lots of students try both to see which one they'll do better on. These kids who take both tests probably tend to be fairly ambitious ones who are looking to game the system by taking both tests, then submitting only the test score they did better upon. So, double-dippers likely scored reasonably well (although, of course, not so 2400 / 36 outstanding that they wouldn't bother taking any test again).

(Has anybody recently done an authoritative study of the trend in overall SAT scores considering all the factors driving scores up or down?)

Between 1996 and 2011, everybody except Other Hispanics got a little better in Math. (I suspect that Other Hispanics used to be mostly Cubans and random fairly well-to-do South Americans, but now it includes a lot of Central Americans.) But Asians got a lot better: 37 points, from 558 to 595.

Verbal scores stagnated or declined slightly, except for Asians, who went up 21 points from 496 to 517. 

Now, let's look at scores relative to the white scores in 1996 and 2011. A decade and a half ago, the overall score for everybody was 36 points lower than the white score. Today, it's 52 points lower. Most of that 16 points of relative decline is due to the demographic composition of America's SAT-takers changing for the worse.

Difference v. whites


Total (V+M)

1996 2011 Chg
ALL 36 52 (16)
Female 54 68 (14)
Male 15 32 (17)
Asian 5 49 44
White 0 0 0
Black 193 208 (15)
AmerIndian 89 91 (2)
Mexican 135 146 (11)
PR 152 159 (7)
Other Hisp 118 150 (32)

The Gap got worse for most of the minority groups that the press gets worked up over. Blacks fell from 193 points behind whites to 208 points (a 15 point relative decline, or a point per year). Mexicans fell from 135 lower to 146 lower. Other Hispanics fell the most, from 118 behind to 150 behind.

These declines are probably mostly due to society (especially the College Board) scraping the bottom of the barrel harder in 2011. What with the recession and all, everybody is convinced that they must go to college, so they try the SAT. The number of people who scored below 400 on Verbal grew from 179,000 to 302,000 and on Math from 172,000 to 251,000.

The number who scored 700 or higher also shot upwards, but that might be due in part to kids taking the SAT more times or taking both the SAT and ACT to see if they can shoot the moon. The number scoring 700 or higher on Verbal went up from 47,000 to 77,000 and on Math from 58,000 to 112,000. High scorers are presumably the most likely to do a lot of test prep and otherwise try to game the system.

In contrast to all other ethnic groups, who fell farther behind whites over the last 15 years, Asians had a 5 point advantage over whites in 1996, which blossomed to a 49 point lead by 2011, a relative change of 44 points.That's a big change, relative to the near-stasis on everything else.

September 14, 2011

Asians pulling away in SAT scores

From FairTest:

2011 College-Bound Seniors Avg SAT Scores

W/score changes from 2006





READING MATH WRITING TOTAL
ALL 497 (-6) 514 (-4) 489 (-8) 1500 (-18)
Female 495 (-7) 500 (-2) 496 (-6) 1491 (-15)
Male 500 (-5) 531 (-5) 482 (-9) 1513 (-19)
Asian 517 (+7) 595 (+17) 528 (+16) 1640 (+40)
White 528 (+1) 535 (-1) 516 (-3) 1579 (-3)
Black 428 (-6) 427 (-2) 417 (-11) 1272 (-19)
AmerIndian 484 (-3) 488 (-6) 465 (-9) 1437 (-18)
Mexican 451 (-3) 466 (+1) 445 (-7) 1362 (-9)
PR 452 (-7) 452 (-4) 442 (-6) 1346 (-17)
Other Hisp 451 (-7) 462 (-1) 444 (-6) 1357 (-14)

I've been following baseball statistics since 1965 and educational test scores since 1972. Test scores are vastly more important for understanding how the world works, but they aren't as diverting because they seldom change. For example, in the above, whites are just treading water, down 1% of a standard deviation over half a decade. Boring. NAM scores are down, probably mostly because the College Board has been subsidizing more NAMs to take the SAT for free as a publicity move. In other words, it's probably not a real change.

But, wow, as I pointed out last year in writing about PSAT National Merit Semifinalists, Asians have just been pulling away from everybody else in the last few years.

Is the same trend true on low stakes tests, like most school achievement tests that are used to grade schools rather than students?

This is a big story and it deserves more research. Is the innate intelligence of Asians going up? Or does this prove that Tiger Mothering works? Is the SAT being unfairly gamed? There are a lot of questions here.

SAT reading scores down

From CBS:
SAT reading scores for the high school class of 2011 were the lowest on record, and combined reading and math scores fell to their lowest point since 1995. 
The College Board, which released the scores Wednesday, said the results reflect the record size and diversity of the pool of test-takers. ... 
Still, it’s just the second time in the last two decades reading scores have fallen as much in a single year. And reading scores are now notably lower than as recently as 2005, when the average was 508. 

In contrast, scores on state exams mandated by the No Child Left Behind act have gone up, UP, UP! I wonder why? Perhaps it has something to do with the fact that the College Board and ETS don't have large material incentives to get scores up by hook or by crook. There's this concept called "conflict of interest" that you may have heard of, although evidently George W. Bush and Ted Kennedy hadn't.
Average math scores for the class of 2011 fell one point to 514 and scores on the critical reading section fell two points to 489.

My guess is that math is more amenable to schoolroom instruction, since kids seldom do math for fun on their own. Since 1983, America has invested a lot in math teaching (often at the expense of history, arts, music, etc.), and we've gotten a (quite) modest but positive ROI. Reading in the classroom and homework, however, is merely a fraction of the total reading kids do (or ought to do), so spending less time on other subjects and more on reading has a more marginal effect.
College Board officials pointed to a range of indicators that the test-taking pool has expanded, particularly among Hispanics, which is a good sign that more students are aspiring to college. For instance, roughly 27 percent of the 1.65 million test-takers last year came from a home where English was not the only language, up from 19 percent just a decade ago.

Up through 2007, it was common for Hispanics to drop out of high school to take construction jobs, or the like. Since unemployment went up, Hispanics have been flooding into community colleges to give themselves something to do. Presumably, more marginal students are taking the SAT, depressing scores.

Also, they've been letting more and more students take the SAT for free since 2007, so the number of minority test takers increased from 553,000 to 720,000 from 2007 to 2011. (Here's the College Board's report.)
But the increasingly diverse group of test-takers is clearly having more trouble with reading and writing than with math. Wayne Camara, College Board vice president of research, said recent curriculum reforms that pushed math instruction may be coming at the expense of reading and writing — especially in an era when students are reading less and less at home. 
“We’re looking and wondering if (more) efforts in English and reading and writing would benefit” students, Camara said. ../
Based on research at 100 colleges, the College Board calculated that scoring 1550 (equivalent to about a 930 on the SAT M+V before 1995) or above on the three sections of the test indicated a 65-percent likelihood of attaining at least a B-minus average in the freshman year of college.

I once took a detailed look at what percentage of students in LA County public high schools score 1000 out of 1600 (M+V) on the SAT, which is a little below that 1550 out of 2400 cutoff that the College Board suggests. Among LAUSD seniors, only 14% scored at least 1000/1600.

The CB report says that this score of 517 / 800 is very close to the minimum Proficient score on the fed's NAEP test of 12th graders. 517 is about the 57th percentile, while 500 is exactly the 50th percentile this year.
Overall, 43 percent of test-takers reached that benchmark. The College Board emphasized the tool is for policymakers, and shouldn’t be used by college admissions officers to evaluate individual candidates. 
The main message from the College Board was the importance of a rigorous curriculum, which is a strong and perhaps growing predictor of SAT scores. 
For instance, nearly one in five students takes less than four years of high school English. That’s about the same percentage as a decade ago, but it now makes a much bigger difference on SAT scores: The reading scores of those students have fallen from 500 to 462. Students who took AP and honors classes, meanwhile, score significantly higher across the board.

Causality!
A decline in average scores isn’t necessarily good news for top students who were applying to competitive colleges. The number of high scores is also increasing. For instance, the number of students with math scores of at least 700 is up 22 percent since 2007.

Some of that may be ACT takers also taking the SAT. But, I think it's pretty clear that the SAT is more and more being successfully gamed by the upper middle class's Tiger Mothers. College Board / ETS don't have strong incentives to lift average scores, but they also don't have strong incentives to crack down on whatever it is that is allowing elite offspring to separate themselves from the masses.

For example, when I went to Rice, the average SAT score was about 1300. Now, it's about 1440, which makes me 140 points awesomer. What incentive do I have to complain?

July 15, 2007

How to score higher on the SAT

A reader writes:

On the SAT (at least when I was coaching it in the late 1980s for The Princeton Review), there were six reading comprehension passages. One of these was the "diversity" passage -- always about a woman or minority. Without even reading the passage itself, a smart person who understood how the test is designed and understood that the answers would never suggest anything derogatory about women/minorities, could get most of the questions correct. I used to amuse my father and older brothers by demonstrating how this worked. The principles I learned at TPR work, more or less, on any standardized college/professional school entrance test.


My published articles are archived at iSteve.com -- Steve Sailer

April 26, 2007

Rudy Giuliani's 1073 SAT score

A reader writes:


Rudy enrolled at Bishop Loughlin Memorial High School in 1957, an exclusive Catholic prep school. They accepted two kids from each parish.

From "Rudy: An Investigative Biography of Rudolph Giuliani" by Wayne Barrett on page 34:


"After seven semesters at Bishop Loughlin, Rudy's grade average of 84.8 earned him a ranking of 130, putting him in the class's second quintile. His report cards for those years show columns of mostly B's and C's, a few A's and one D. He scored a 65 in chemistry, a 74 in Latin and a 92 in American history. His combined College Board scores, 569 in verbal and 504 in math, were twenty-seven points shy of 1100, and quite ordinary."


Wayne Barrett is a writer at the Village Voice and professor at the Columbia School of Journalism, if you want to check with the source.


Here's his academic history: "He attended Bishop Loughlin Memorial High School, Manhattan College in the Bronx, and New York University Law School, graduating magna cum laude."

In comparison, George W. Bush scored 1206 and Al Gore 1330 on the SAT. All these scores are under the tougher pre-1995 scoring system. Add 70 or 80 points to get the equivalent under the current scores. Does anybody know what John McCain scored to get into Annapolis as the son and grandson of admirals?

It's striking that more than a few men considered Presidential Timber wouldn't have gotten a callback if they had applied to join, say, the Navy SEALs. It's not that a fairly high IQ is so utterly crucial to being a good SEAL, but it does improve the odds. There are many men who want to be SEALs, and plenty of them have reasonably high IQs, so it's no-brainer for the Navy to weigh IQ in the mix of qualifications.

Being Presdient, in contrast, does not generally require the physical ability to infiltrate an enemy harbor and silently kill sentries, so one might expect that IQ would be even more important in the Chief Executive job than in being a scuba commando.

Certainly, "intangibles" can make up for a modest IQ in a President, but are we so sure we are good at evaluating the intangibles of politicians? How good a job did we do with George W. Bush? And he wasn't some nobody from nowhere. He was the son of a President. Many important people had met him during the twelve years his father had held the two highest offices in the land, and few had thought him a worthy successor. We knew that his own parents considered him inferior to his own brother Jeb. And yet, the Republican Establishment got behind him in 1999, drinking Karl Rove's Kool-Aid that his intangibles would somehow make up for Bush's tangible deficiencies.


My published articles are archived at iSteve.com -- Steve Sailer