From the Telegraph of London, an account of a student in Australia: "who attended an elite private girls' school in Sydney, said she had a wrist problem, suffered discrimination and her mark should have been 100. Her result, a university entrance score, meant she beat 99.95 per cent of other students – but she believed she would have received the top mark if treated fairly."
Showing posts with label testing. Show all posts
Showing posts with label testing. Show all posts
July 4, 2012
June 30, 2012
Chinese Cheating
The NYT has an article on the gaokao, the national college admissions exam in China:
Each year, cheating scandals become the talk of China. One common tactic was for students to give their identification cards to look-alikes hired to take the test; later, many provinces installed fingerprint scanners at test centers.
Do even the Chinese find that other Chinese rather look alike?
In 2008, three girls in Jiangsu Province were caught with mini-cameras inside their bras; their aim was to transmit images of the exam to people outside the classroom who would then provide answers. This year, the big scandal involved students in Huanggang, Hubei Province, famous in the past decade for churning out students with high scores; several dozen students were caught there last month for using small monitors costing nearly $2,500 that resembled erasers and that allowed the students to receive electronic messages with test answers.
June 20, 2012
Test Prep gains by race
Chris Hayes argues in his new book Why Elites Fail the real reason that blacks and Hispanics are making so little progress over the generations at qualifying on their own merit for selective academic institutions is because rich whites are hogging all the test prep.
Education Realist, who is in the test prep business, has a post linking to various recent studies of the popularity and effectiveness of prepping for admissions test by race. Here's one table (with score increases out of 1600, not 2400):
Use of Test-Prep Courses and Gains, by Race and Ethnicity
| Group | % Taking Test-Prep Course | Post-Course Gain in Points on SAT |
East Asian Am.
|
30%
|
68.8
|
Other Asian
|
15%
|
23.8
|
White
|
10%
|
12.3
|
Black
|
16%
|
14.9
|
Hispanic
|
11%
|
24.6
|
So, unsurprisingly, East Asians try the hardest at and get the most benefit from test prep, while whites, who are more likely to have heard and believed ETS's propaganda that test prepping is insignificant, try the least hard and get the least benefit. In the middle, blacks and Hispanics benefit from all the racial uplift programs for them.
This doesn't disprove Chris Hayes' assumption that Upper East Side whites are benefitting from test prep. My guess is that the big losers in this game are naive flyover folks.
June 18, 2012
"Why Elites Fail"
From The Nation
Why Elites Fail
Christopher Hayes | June 6, 2012
This article is adapted from Twilight of the Elites: America After Meritocracy [1], © 2012 by Christopher Hayes and published by Crown Publishers, a division of Random House Inc.
In 1990, at the age of 11, I stood in a line of sixth graders outside an imposing converted armory on Manhattan’s Upper East Side, nervously anticipating a test that would change my life. I was hoping to gain entrance to Hunter College High School, a public magnet school that runs from grades seven through twelve and admits students from all five boroughs. Each year, between 3,000 and 4,000 students citywide score high enough on their fifth-grade standardized tests to qualify to take Hunter’s entrance exam in the sixth grade; ultimately, only 185 will be offered admission. ...
But the problem with my alma mater is that over time, the mechanisms of meritocracy have broken down. In 1995, when I was a student at Hunter, the student body was 12 percent black and 6 percent Hispanic. Not coincidentally, there was no test-prep industry for the Hunter entrance exam. That’s no longer the case. Now, so-called cram schools like Elite Academy in Queens can charge thousands of dollars for after-school and weekend courses where sixth graders memorize vocabulary words and learn advanced math. Meanwhile, in the wealthier precincts of Manhattan, parents can hire $90-an-hour private tutors for one-on-one sessions with their children.
By 2009, Hunter’s demographics were radically different—just 3 percent black and 1 percent Hispanic, according to the New York Times. With the rise of a sophisticated and expensive test-preparation industry, the means of selecting entrants to Hunter has grown less independent of the social and economic hierarchies in New York at large. The pyramid of merit has come to mirror the pyramid of wealth and cultural capital.
... But this ideal, appealing as it may be, runs up against the reality of what I’ll call the Iron Law of Meritocracy. The Iron Law of Meritocracy states that eventually the inequality produced by a meritocratic system will grow large enough to subvert the mechanisms of mobility. Unequal outcomes make equal opportunity impossible. The Principle of Difference will come to overwhelm the Principle of Mobility. Those who are able to climb up the ladder will find ways to pull it up after them, or to selectively lower it down to allow their friends, allies and kin to scramble up. In other words: “Who says meritocracy says oligarchy.”
Consider, for example, the next “meritocracy” that graduates of Hunter encounter. American universities are the central institution of the modern meritocracy, and yet, as Daniel Golden documents in his devastating book The Price of Admission, atop the ostensibly meritocratic architecture of SATs and high school grades is built an entire tower of preference and subsidy for the privileged:
At least one third of the students at elite universities, and at least half at liberal arts colleges, are flagged for preferential treatment in the admissions process. While minorities make up 10 to 15 percent of a typical student body, affluent whites dominate other preferred groups: recruited athletes (10 to 25 percent of students); alumni children, also known as “legacies” (10 to 25 percent); development cases (2 to 5 percent); children of celebrities and politicians (1 to 2 percent); and children of faculty members (1 to 3 percent).
This doesn’t even count the advantages that wealthy children have in terms of private tutors, test prep, and access to expensive private high schools and college counselors. All together, this layered system of preferences for the children of the privileged amounts to, in Golden’s words, “affirmative action for rich white people.”
Okay, but shouldn't Hayes' article mention the single most important word in the demographic transition of Hunter College High School over the last 20 years: "Asians"? Here's a school on Manhattan's Upper East Side, the largest concentration of rich white people in America, and there are now more Asians than whites attending it. The New York Times article Hayes cites, but doesn't quote, makes this clear:
As has happened at other prestigious city high schools that use only a test for admission, the black and Hispanic population at Hunter has fallen in recent years. In 1995, the entering seventh-grade class was 12 percent black and 6 percent Hispanic, according to state data. This past year, it was 3 percent black and 1 percent Hispanic; the balance was 47 percent Asian and 41 percent white, with the other 8 percent of students identifying themselves as multiracial. The public school system as a whole is 70 percent black and Hispanic.
It's fun to talk about test prep as "affirmative action for rich white people," but the test prep freight train is being driven by Asians (who have been test prepping for over a thousand years, by the way), which is a fact that the media ought to get around to acknowledging.
June 8, 2012
Are the English better at English?
Greg Cochran brings up a topic that seems like it has disappeared over the last generation: reading speed. In the old days, the immense velocity at which Democratic Presidents like JFK and Jimmy Carter could read was part of political lore. Skeptics like Woody Allen joked that he had speed-read War and Peace: "It was about Russia."
Has reading faster simply failed? Or has America just lost interest?
Cochran also asks whether different languages are read faster and slower: e.g., Mandarin versus Spanish? When I was a kid there was still some remnant of interest in the early 20th Century movement to reform the English language to make it more efficient. A century ago, for example, George Bernard Shaw, the dominant cultural intellectual of the time, campaigned hard for radical spelling reform. (Henry Higgins in My Fair Lady reflects some of GBS's numerous concerns about the English language and social equality.) The first time I ever won a prize in a Speech tournament was around 1970 for an original oratory making fun of the complexities of English grammar. Is anybody still amused by that kind of thing?
In a comment, Education Realist brings up an interesting point: based on SAT and GRE scores, 21st Century, white Americans appear to be better at Math than at Verbal relative to mid-20th Century white Americans. When the SAT was started before WWII, it was normalized based on Eastern Seaboard preppies with 500 as average for both the Verbal and the Math tests. As it expanded to a broader market of students, average Math SAT scores dropped dropped only slightly, but Verbal scores fell substantially. In 1995, the SAT was renormed to make 500 the means again, but the same process is visible again, with Math scores now notably higher than Verbal scores. (The Asian impact obviously affects this gap, but this trend is visible just among whites.)
The Graduate Record Exam has never been renormed, and today white men average 593 on the quantitative part and 508 on the verbal part of the GRE. Education Realist, who is a teacher and test tutor, then raises a number of interesting points:
Why do we appear to have fewer high verbal achievers than math achievers? I think Murray and Herrnstein were correct when they wrote that “a politically compromised curriculum is less likely to sharpen the verbal skills of students than one that hews to standards of intellectual rigor and quality” annd that “when parents demanded higher standards, their schools introduced higher standards in the math curriculum that really were higher, and higher standards in the humanities and social sciences that really were not”. (Bell Curve, page 432-433) Without question, we have lost a couple generations of cognitively able students who weren’t given the opportunity to really achieve to their fullest capability, and we stand to lose a few more.
But I also wonder if verbal intelligence is less understood and consequently less valued. If one is “good at math”, there’s a logical progression of courses to take, problems to solve (or spend a lifetime trying to), and increasingly difficult subjects to tackle–and plenty of careers that want them. But if one has a high verbal intelligence without good spatial aptitude (which seems to be necessary for higher math) it is often described as “good at reading”, a woefully inaccurate characterization of high verbal intelligence—and then what? Apart from law, there aren’t nearly as many clearly defined career paths with a wide range of opportunities for all temperaments and interests. Most of the ones I can think of involve luck and driving ambition just to get started (journalism, tenured academia, political consultant).
For a good twenty years or so, people with high verbal skills who were indifferent at high-level math went into technology. It’s hard to remember now in the age of Google and after the heyday of corporate computing, but IBM and mainframe shops were filled with bright people who had degrees in history and English and humanities who just “didn’t like math” but were excellent programmers. I routinely worked in shops where all the expert techies making six figures came from non-STEM majors. But that time appears to be over.
Of course, doing anything about this lack of clearly defined career paths for smart folks with less spatial aptitude would involve acknowledging it’s a problem, and I might be the only one who thinks it’s a problem.
As Education Realist points out, we have lots of prestigious national science and math fairs for high school students (which are now dominated by Asians), but little of the same fame for the reading and writing set. Everybody who is anybody in America seems far more obsessed with cultivating Math and Science than with raising our verbal ability. Yet, a native command of English would appear to be a prime asset of Americans in a future globalized (and, thus, English-speaking) economy.
Presumably, it's easier to raise math test scores in school than reading test scores, since reading scores depend heavily on how much reading the students do out of school. Still, nobody seems all that interested in trying to figure out how to improve our children's advantages in English. It's almost like we think it's unfair to the rest of the world that we speak English, so we should have our children bash their heads in to compete with Asians on the culturally level playing field of math. That strikes me as a noble but stupid response.
Presumably, it's easier to raise math test scores in school than reading test scores, since reading scores depend heavily on how much reading the students do out of school. Still, nobody seems all that interested in trying to figure out how to improve our children's advantages in English. It's almost like we think it's unfair to the rest of the world that we speak English, so we should have our children bash their heads in to compete with Asians on the culturally level playing field of math. That strikes me as a noble but stupid response.
But I want to go in a different direction with this topic and ask if there is any objective test evidence to support this idea I've had ever since I took American Literature in high school: historically, Americans are not as good with words on average as the British. Somehow, the Brits seem to inculcate better command of English than we Americans do. Perhaps that's not true up and down the social scale, but it would seem to me that, traditionally, Oxbridge graduates, say, had better vocabularies and better prose styles than Ivy League grads.
This notion first dawned on me in the 1980s when I noticed a London-based firm called WPP, run by a former Saatchi & Saatchi executive named Martin Sorrell, started buying up advertising agencies and other marketing services firms. While Britain seemed economically down and out back then, it struck me that they still were better at English than we were, and that had to be worth something in an increasingly English-speaking world.. Today, WPP employs 158,000 white collar workers around the world and even owns a large fraction of all the lobbying firms in Washington D.C., Democrat and Republican.
This notion first dawned on me in the 1980s when I noticed a London-based firm called WPP, run by a former Saatchi & Saatchi executive named Martin Sorrell, started buying up advertising agencies and other marketing services firms. While Britain seemed economically down and out back then, it struck me that they still were better at English than we were, and that had to be worth something in an increasingly English-speaking world.. Today, WPP employs 158,000 white collar workers around the world and even owns a large fraction of all the lobbying firms in Washington D.C., Democrat and Republican.
Throughout the 18th and 19th Century, American writing just wasn't very good compared to what the Brits were doing at the same time. Compare, say, The Federalist Papers or the Autobiography of Benjamin Franklin to The Decline and Fall of the Roman Empire, The Life of Johnson, or even The Wealth of Nations. Compare American stinkers like Ralph Waldo Emerson to his British contemporaries. Granted, we had people who were geniuses in their own way, like Poe and Lincoln and Twain, but they didn't come from a culture that was as good with words as the Brits.
Even in the 20th Century, when Americans were catching up, the home team still seemed awkward compared to the visitors. For example, F. Scott Fitzgerald's The Great Gatsby is a fine book, but compared to the seemingly effortless clarity and fluency of Evelyn Waugh's early novels of just a few years later, its prose seems provincial and striving.
I recently re-read Great Contemporaries, a collection of articles for Sunday newspapers that Winston Churchill wrote (or, to be precise, dictated) in the 1930s about celebrities he’d known. Allow me to express in my own crude, tongue-tied American way my reaction to the command of the English language exhibited in Churchill's commercial journalism: Holy cow! For mastery of English, for vast and precise vocabulary, I can’t imagine any major American politician of the last century coming close. Teddy Roosevelt had comparable mental energy, but few read his books for fun these days. Henry Kissinger is a very smart man who writes well in his second language, but he is more functional in style.
Churchill was recognized as exceptional in his own day, but, still, other British politicians were pretty handy with words, too. In Britain, Churchill was the champ but compared to American politicians, he's in a league of his own. (By the way, I have a vague hunch that, from the perspective of the 21st Century, the 1930s was the peak era for English prose: it's not so far in the past that it's difficult to decipher, but it's far enough away that its superiority is noticeable.)
Another anecdote about the superiority of the English: A number of years ago, I dropped in on John Derbyshire and family in Long Island. We went to a Blockbuster to pick out a movie for everybody to watch that evening, so I suggested the documentary about the Scripps-Howard national spelling bee, Spellbound, which had been a big hit in my household.
Now, I'd always figured that while John is obviously my superior in math and computer programming, we're fairly equal in verbal skills. But, when I watched Spellbound for the second time (with the closed captions off), I discovered that John could not only outspell me on words I'd already seen the first time I watched the movie, but he also knew the definitions of almost all the absurd words in the competition.
I attribute this to his having the unfair advantage of being born English.
June 7, 2012
Don't worry, studies show test prep doesn't work!
From the New York Times, an article on the annual Chinese college admissions test madness, the gao kao, and how Chinese test culture is spilling over to the U.S.:
With more and more Chinese students applying to foreign universities, the emphasis on the rote memorization required for the gao kao has come under criticism from some U.S. educators. Another cause for concern, they say, are the methods being used to study for the Test of English as a Foreign Language, or Toefl, which most U.S. schools require for admission.
In a story done jointly by The Times and The Chronicle of Higher Education, Patricia J. Parker, assistant director of admissions at Iowa State University, which enrolls more than 1,200 Chinese undergraduates, said “students have proudly told her about memorizing thousands of vocabulary words, studying scripted responses to verbal questions and learning shortcuts that help them guess correct answers.”
The story’s reporters, Tom Bartlett and Karin Fischer, wrote that Ms. Parker “has seen conditionally admitted students increase their Toefl scores by 30 or 40 points, out of a possible 120, after a summer break, despite no significant improvement in their ability to speak English. Her students, she says, don’t see this intense test-prepping as problematic: ‘They think the goal is to pass the test. They’re studying for the test, not studying English.’ ”
Zinch China, a consulting company that advises American colleges and universities about China, published a report last year that found cheating on college applications to be “pervasive in China, driven by hyper-competitive parents and aggressive agents.’’
An excerpt from the Zinch report:
The result? Fake achievements, often concocted by agents. Based on our interviews, this happens on about 10 percent of applications. Sometimes a student’s silver medal is turned to gold, and sometimes a student lists an award for an activity he or she never completed. At a top Beijing high school this year, ten students claimed to be Class President!
Most Chinese parents now understand that American schools are looking for “well-rounded” students who combine strong test scores, transcripts, and extra-curricular achievement. The problem is that most Chinese students don’t have time to participate in many extra-curricular activities — they are too busy studying for and taking tests. In fact, many Chinese parents see extra-curricular activities as a dangerous distraction from studying.
May 22, 2012
PISA scores for immigrants
Anatoly Karlin, who is making himself the go-to guy on analyzing the investment implications of international school test scores (a potentially lucrative niche), has a long, fascinating write-up of PISA scores adjusted by immigration status:
One thing that immediately leaps out from above is that just as US scores leap upwards (from 496 to about 525, in line with Australia and Canada) once only whites are considered, so do scores in many European states when only natives are considered (e.g. Germany from 510 to 533; Switzerland from 517 to 542; the Netherlands from 519 to 533). In fact, the Germanic nations equalize with Japan’s 529, Taiwan’s 534, and South Korea’s 541 (the natives of these developed East Asian societies also score a lot higher than their immigrants, but the overall effect on the national average is modest because migrant children are such a small percentage of their school-age populations). In other words, in the worst affected European countries, immigrants are lowering the mean national IQ (converted from PISA scores) by as much as 3 points.
This might not seem like much, but it is highly significant when bearing in mind the extremely close correlation between national IQ and prosperity. Furthermore, since immigrant population tend to be highly variant – for instance, Britain has a lot of Poles, who are essentially equal to the natives in cognitive capacity (maybe even superior, once you adjust for the fact that it is better-educated Poles who tend to emigrate), and a lot of Pakistanis, who are far below them. This is a good explanation for the general sense of dereliction one sees (and the crime one is likely to experience) when entering Pakistani ghettos in the UK.
Also note from the graph that there is typically a very high degree of overlap between 1st and 2nd generation immigrant children. The 2nd generation children DO typically perform better, presumably because 1st generation immigrants may frequently have language difficulties and problems with adjusting to a new culture. But the degree of convergence of 2nd generation children to the native mean is modest, despite their transferal to typically far more advanced educational environments. Convergence is almost inconsequential in most European countries like Germany, France, Benelux, Norway, and actually negative in the US (i.e. American 2nd generation immigrant children do worse than the 1st generation).
April 15, 2012
MCAT changes: More NAMS or fewer Tiger Cubs?
From the NYT:
Pre-Med’s New Priorities: Heart and Soul and Social Science
By ELISABETH ROSENTHAL
... In addition to the hard-science and math questions that have for decades defined this rite of passage into the medical profession, nearly half of the new MCAT will focus on squishier topics in two new sections: one covering social and behavioral sciences and another on critical analysis and reading that will require students to analyze passages covering areas like ethics and cross-cultural studies.
The Medical College Admission Test is, of course, much more than a test. A good score is crucial for entry into a profession that is perennially oversubscribed. Last year, nearly 44,000 people applied for about 19,000 places at medical schools in the United States. So the overhaul of the test, which was announced last year and approved in February, could fundamentally change the kind of student who will succeed in that process. It alters the raw material that medical schools receive to mold into the nation’s future doctors.
Which is exactly what the A.A.M.C. has in mind. In surveys, “the public had great confidence in doctors’ knowledge but much less in their bedside manner,” said Darrell G. Kirch, president of the association, in announcing the change. “The goal is to improve the medical admissions process to find the people who you and I would want as our doctors. Being a good doctor isn’t just about understanding science, it’s about understanding people.”
The public are idiots. I want Dr. House to diagnose me. I almost died in the 1990s because my nice guy doctor told me that the lump in my armpit, my night sweats, and my loss of energy was probably just a muscle pull. The cancer doctor who saved my life had a lousy bedside manner, but he had access to Rituxan years before everybody else did because he knew more about non-Hodgkins lymphoma than anybody else in the upper Midwest.
The adoption of the new test, which will be first administered in 2015, is part of a decade-long effort by medical educators to restore a bit of good old-fashioned healing and bedside patient skills into a profession that has come to be dominated by technology and laboratory testing. More medical schools are requiring students to take classes on interviewing and communication techniques. To help create a more holistic admissions process, one that goes beyond scientific knowledge, admissions committees are presenting candidates with ethical dilemmas to see if their people skills match their A+ in organic chemistry. ...
Where will students find time to take in the extra material? How to prepare pre-med students long primed to answer questions like “Where are the serotonin receptors 5-HT2A and 5-HT2B mostly likely to be located in hepatocytes” to tackle more ambiguous challenges, like: “Which of the following explanations describes why the Identity vs. Role Confusion stage likely affects views about voting and being a voter?”
... “With the growth in scientific knowledge, we were focused on making sure doctors had a good foundation in hard science,” Dr. Kirch said. Indeed, from 1942 to 1976, the MCAT had included a broad-based knowledge section called “Understanding Modern Society.” Liberal arts questions were eliminated in 1977. ...
Some experts have long identified the MCAT as a stumbling block in the often-failed quest to produce more caring, attentive doctors. It is a test that selects more for calculation skills than empathy. ...
And so the Association of American Medical Colleges began three years ago to redesign the MCAT, surveying thousands of medical school faculty members and students to come up with a test tailored to the needs and desires of the 21st century. In addition to more emphasis on humanistic skills, the new test had to take into account important new values in medicine like diversity, with greater focus on health care for the underserved, Dr. McGaghie said.
As a result, there will be questions about gender and cultural influences on expression, poverty and social mobility, as well as how people process emotion and stress. ...
The mere fact that psychology, sociology and critical thinking will be on the MCAT is likely to change priorities, prompting science majors to think harder about topics like the perception of pain, informed consent, community awareness and the ethics of the Tuskegee Syphilis Experiment.
Okay, let me toss out a guess: The current American medical establishment wants more of their own children and grandchildren to make it as doctors, which, mathematically, means fewer Asians. So they are making it sound like they are changing the test to get more blacks and Hispanics, but they are small change: it's really a plan to cut down on the Asians.
And here's a prediction: it won't much work. Whatever they put on the test, within a few years, the Asians will memorize it and spit it back.
Another possibility is that this is all part of a plan to liberalize Asians, to turn them into Nice White People, before they completely take over the world. That's not necessarily a bad idea. But of course the Nice White People couldn't imagine directly confronting Asians over their various bad habits, like, say, the caste system. Instead, the NWPs are attempting another classic triple bankshot by telling the Asians that Yes, the Tuskegee Syphilis Experiment Will Be on the Test.
April 14, 2012
Manhattan IS Lake Wobegon
If you are ever feeling in the need for a laugh, just look up the latest news from New York City on the Kindergarten Admissions Wars. Year after year, it's pure comedy gold. Amazingly, this NYT reporter, Anna M. Phillips, appears to be starting to get the joke:
After Number of Gifted Soars, a Fight for Kindergarten Slots
By ANNA M. PHILLIPS
Nearly 5,000 children qualified for gifted and talented kindergarten seats in New York City public schools in the fall, 22 percent more than last year and more than double the number four years ago, setting off a fierce competition for the most sought-after programs in the system.
On their face, the results, released on Friday by the Education Department, paint a portrait of a city in which some neighborhoods appear to be entirely above average. In Districts 2 and 3, which encompass most of Manhattan below 110th Street, more students scored at or above the 90th percentile on the entrance exam, the cutoff point, than scored below it.
But experts pointed to several possible reasons for the large increase. For one, more middle-class and wealthy parents are staying in the city and choosing to send their children to public schools, rather than moving to the suburbs or pursuing increasingly expensive private schools. And the switch to a test-based admissions system four years ago has given rise to test-preparation services, from booklets costing a few dollars to courses costing hundreds or more, raising concerns that the test’s results were being skewed. ...
Of the children who scored high enough on the entrance exam to be eligible for a gifted program, more than half — 2,656 — qualified for the five most selective schools by scoring at or above the 97th percentile. But those schools — three in Manhattan and one each in Brooklyn and Queens — have only about 400 kindergarten seats. The rest of the 4,912 children qualified for one of the dozens of gifted programs spread throughout the five boroughs.
A.K.A., the Loser Gifted Programs for Loser Children of Loser Parents who Don't Love Their Children Enough to Figure Out How to Get Them into the Golden 400.
Gifted programs generally offer an accelerated curriculum, as well as the opportunity to be around other high-performing children.
Keep in mind, we're talking about high-performing kindergarteners here.
The city did not provide a racial breakdown of students who qualified, but as in years past, the more affluent districts — 2 and 3 in Manhattan, in neighborhoods west and south of Prospect Park in Brooklyn, and in northeastern Queens — had the most students qualify. In District 2, 949 children qualified for a gifted program, far more than in any other district.District 2 starts at about 96th St. on the Upper East Side and includes all of Manhattan south of Central Park, except, amazingly enough, Alphabet City on the Lower East Side. (And even that's gentrifying.)
In District 3, 505 children qualified. By contrast, in District 7, in the South Bronx, only six children qualified for gifted placements and none for the five most exclusive schools.
Two orders of magnitude difference.
Every year since 2008, when the city put the current testing program into effect and 2,230 students qualified for seats in gifted and talented kindergarten classes, the number of children scoring at or above the 90th percentile has steadily grown. The chancellor in 2008, Joel I. Klein, made the change to standardize the admissions process, replacing a system in which each district set its own standards for entry, a process that drew criticism from parents who said favoritism sometimes played a role.
When school supremo Joel Klein made the switch to pure test-based admissions, using tests would obviously have a huge disparate impact effect. But, Klein didn't know or didn't care, because kindergarten admissions is serious stuff where testing is too crucial to be sacrificed on the altar of racial equality. This isn't something trivial like saving people from burning skyscrapers, this is NYC kindergarten admissions, and don't you forget it. Different rules apply.
But the new process has come under scrutiny for its complete reliance on the test — actually two exams, the Otis-Lennon School Ability Test, or Olsat, a reasoning exam, and the Bracken School Readiness Assessment, a knowledge test.
In January, the city awarded Pearson a three-year contract for roughly $5.5 million to replace the Bracken exam with the Naglieri Nonverbal Ability Test, which city education officials contend will better measure ability.
Isn't it weird that this is a golden age for the psychometric industry? Standardized tests are constantly denounced, yet governments keep shoveling more money to testing firms to create new tests that will Finally Get It Right. These firms have achieved the perfect marketing equilibrium.
The contract places restrictions on Pearson’s ability to sell its test materials to anyone outside the Education Department, to make it harder for test-preparation companies to get their hands on them.
Oh, well, that will stop New York City test prep firms dead in their tracks.
... Always on the alert for changes to admissions policies, some tutoring companies, true to the nature of their profession, are prepared for it.
One of the companies, Aristotle Circle, already offers a $300 “test preparation and enrichment kit” designed for the Naglieri and similar exams.
“You can build a better mousetrap, it doesn’t matter,” said Suzanne Rheault, one of Aristotle’s founders. “There’s no way you can stop it because now the idea of preparing for the kindergarten test is totally the norm. The stakes are so high.”
April 4, 2012
PISA scores by region in Russia and Italy
(2) Moscow pupils performed very well [546], at the level of the highest scoring OECD countries like Finland, Taiwan, and Korea. This is especially impressive considering the significant numbers of immigrants in that city from the North Caucasus and Central Asia, who come from poorly-scoring countries and rarely have good Russian.
(3) St.-Petersburg and Tyumen oblast [western Siberia] performed above the OECD average, while a few other regions performed at or only slightly below the OECD average.
(4) Among ethnic Russian republics, Siberian regions performed well, while the Urals and southern regions performed badly.
(5) Performance in ethnic minority republics differs dramatically. Many of the Turkic and Finno-Ugric regions, such as Tatarstan, Komi, Chuvashia, and Karelia did well; however, Mari El is a big exception. The Buddhist peoples of Asia, such as Chita oblast (now merged into Zabaykalsky Krai) and the Sakha Republic, performed relatively poorly, as did the Muslim North Caucasus region of Dagestan. Extrapolating from Dagestan, Chechnya would probably score around 400, i.e. like Brazil.
Bear these figures in mind when considering long-term investments into Russia alongside with their business climate, corruption levels, etc.
Western Russia doesn't do that well, and neither do the ex-Soviet Republics to the west of contemporary Russia. I wonder what the dysgenic effects of Leninism, Stalinism and Hitlerism were, especially on Ukraine, Belarus, Moldova, and western Russia? Many of the wealthy and middle class fled the Bolsheviks, then Stalin starved the kulaks, then murdered or imprisoned many people of above average talents, then the SS came through and shot the local leaders loyal to Stalin.
Also, here is a graph of Italian provinces with PISA scores on the horizontal axis and per capita GDP on the vertical axis.
The positive outlier above the line is Rome, the capital. The other positive outlier at the top of the chart is South Tyrol, which is a heavily German speaking area that Italy got carved out of Austria at the end of the Great War. The negative outlier is Apulia, in the southeastern heel of Italy, home to Brindisi and Lecce. I spent a day in those beat-up looking towns in 1980 and I recall a vague impression of the locals as seeming clever but anti-social, as cut-off-your-nose-to-spite-your-face types.
February 26, 2012
Rudi Can't Fail (in fact, Rudi Can Yale)
The NYT has a long article, To Be Black at Stuyvesant High, on a young heroine of diversity, Rudi-Ann Miller, who practically singlehandedly has kept multiculturalism alive at Stuyvesant H.S. by being one of only 40 African-American students out of 3200 at NYC's premiere exam-only public science and math school.She has also had enough of the grumbling at Stuyvesant that black students do better in the college-admissions game because of their skin color.
Rudi will have to assuage her hurt feelings next year at Yale.
Hey, wait a minute, what kind of African-American girl born in the 1990s is named Rudi anyway? Isn't there some foreign country where "Rudie" is close to being the national nickname?
If you followed the complaints of Henry Louis Gates and Lani Guinier about how Harvard's affirmative action slots tend to go to students who are not the descendants of American slaves (i.e., to Barack rather than Michelle Obama), you won't be surprised to find out from later on in the article that Rudi attended through seventh grade Campion College in Jamaica, a Jesuit school that her father, a Jamaican accounting executive recently relocated to the New York area, calls the finest in that country. (Campion College's website boasts that 14 of its graduates have gone on to win Rhodes Scholarships.)
A commenter notes:
As a Stuy alum who had many Black friends, I find it disappointing that the article didn't inquire further into the community of Black students who do make it to Stuy. While there is of course diversity within the Black community, I can testify that most are either the children of immigrants or products of inter-racial relationships. This is relevant because it shows that many are either of higher socio-economic status, or similar to the potpourri of second-generation immigrants who dominate the school. The real issue is why there are so few from the entrenched black communities, in Brooklyn, Queens and the Bronx who aspire to attend Stuy.
One reason is offered in the article:
Sometimes, Mr. Blumm said, blacks and Latinos who do well enough on the entrance exam to get into Stuyvesant are lured away by prestigious private high schools, which offer them full scholarships and none of the issues that even elite public schools have to contend with, like tight budgets and overcrowding.
By the way, is this the first statistical graph to be published in this century where blacks are represented by the color black? I thought there was some sort of Rule of Randomizing colors where one one graph blacks are, say, white, and whites are brown, and Chinese are green, and Mexicans are red, and then on the next graph blacks are blue, whites yellow, Asians purple, and Latinos white? I dunno, this graph could be setting a dangerous precedent by making it easier for readers to make sense out of racial data.
December 18, 2011
PISA: What about the rest of China?
The news that two states in India took the PISA test of 15-year-olds' school achievement in 2009 and bombed raises the question once again of China. As everybody remembers from a year ago, 2009 scores from Shanghai were released and they were higher than any country in the world. But what about the rest of China? Obviously, Shanghai is a dazzling place, but a lot of China is still stuck knee-deep in rice paddies. What about them?
I stumbled upon this year-old blog post by Anatoly Karlin of Sublime Oblivion, which relays a big hint:
As regular blog readers know, I think that educational capital and more broadly average IQ levels are one of the key – and frequently under-appreciated due to political correctness – determinants of economic development and whether or not convergence to developed country levels is even possible. Its much higher educational capital is one of the key reasons why I think China will continue doing much better than India in development, regardless of its “democratic deficit.” However, many people argue that China’s human capital must actually be quite low, because it doesn’t spend much on education, resources are bare in the provinces, statistical fudging under unaccountable governors, etc.
The recent results from the international standardized PISA tests in math, reading and science will make this an increasingly untenable position. Shanghai got by far the best results out of all the OECD countries (never mind the developing ones). . Now while you might (rightly) argue Shanghai draws much of the elite of the Yangtze river delta, the Financial Times has more: “Citing further, as-yet unpublished OECD research, Mr Schleicher said: “We have actually done Pisa in 12 of the provinces in China. Even in some of the very poor areas you get performance close to the OECD average.””
Since countries like the US and France get scores “close to the OECD average”, this means that the workforces soon to be entering China’s economy, even from its poorest regions, will be no less skilled than those of leading Western economies (note too that the numbers of Chinese university graduates are soaring). And with China’s massive population, four times bigger than America’s, its road to superpowerdom must be all but guaranteed.
Okay, there are a few leaps of faith there, but that's still news worth knowing. At minimum, it reduces the chances that the Shanghai numbers were a con job. At median, it suggests that we check twice before reflexively equating China and India. At maximum, it suggests, as Karlin says, that "resource constraints" are going to be perhaps the big issue of the 21st Century. It's a little hard to be certain what "Even in some of the very poor areas you get performance close to the OECD average" means, but it sounds pretty good.
The logic of using international test scores to predict future wealth is not that the causation runs only in one direction, from high test scores to wealth. Obviously, it runs in both directions. (For example, affluent Chinese have traditionally hired tutors to raise their children's test results.) But, if there are a whole bunch of poor farm kids in inland China who are scoring like kids in Europe and North America right now, well, that's worth knowing.
November 9, 2011
"A New Book Argues Against the SAT"
From the NYT:
A New Book Argues Against the SAT
By REBECCA R. RUIZ
When Wake Forest University announced three years ago that it would make the SAT optional for its undergraduate applicants, among those cheering was Joseph Soares, a sociology professor at the university. Mr. Soares has channeled his enthusiasm for Wake Forest’s decision — as well as for similar policies at several hundred other colleges — into a new book, “SAT Wars,” that argues for looking beyond standardized test scores in college admissions. (The book was published last month by Teachers College Press.)
“The SAT and ACT are fundamentally discriminatory,” Mr. Soares said in a phone interview last week.
Through his own essays in the book, as well as those of contributors that he edited, Mr. Soares seeks to build a case against the SAT. He characterizes it as a test that tends to favor white, male, upper income students with the means to prepare for it.
Because Asians do so badly on the SAT.
September 28, 2011
College admissions testing in Britain and France
It's interesting to compare college admissions in the U.S. to other countries. Here's an NYT article on Britain:
For over half a century, academically inclined students in Britain or other countries who hoped to study at British universities spent their final two years at school studying for A-levels, widely regarded as the gold standard of British education. These single-subject tests are generally considered more rigorous than the French baccalaureate and roughly comparable to the Advanced Placement exams in the United States.
Originally offered only in traditional academic subjects like English language and literature, mathematics, foreign languages and the sciences, in recent years the range has broadened to include media studies, health and social care, business studies, and travel and tourism. The grading ranges from A* and A down to E, and results are announced in early August.
Typically, a student will take three or four A-levels, which are administered in May and June of a student’s senior year. Since students currently submit university applications between September and January of their senior year — before they even take their A-levels — most British universities admit candidates with conditional offers, based on the A-level grades students are predicted to get by their teachers. For example, a student hoping to study medicine at Bristol, which last year admitted only 216 candidates out of over 3,100 applications, would need predicted grades of at least 2 A’s (including an A in chemistry) and one B.
However, according to the University & Colleges Admissions Service, or UCAS, the private organization that manages university applications in Britain, A-level predictions are only accurate about 45 percent of the time. Students who fail to make their predicted grades face a last-minute scramble for university places and often end up in courses based on the availability of places rather than their own interests or aptitudes. Critics of the system also argue that teenagers from low-income homes often do not believe themselves capable of being admitted to the best universities; by the time they receive the grades that might have prompted them to aim higher, it is too late.
Mary Curnock Cook, chief executive of UCAS, said at a meeting of university heads in London last week, according to news reports, that starting in 2016 students should wait to receive their actual A-level grades before applying to universities. Under one set of proposals, students would take their exams before what is now the Easter break, which would mean a five-month summer vacation. Results would be available in July, with applications due in August.
It goes on to talk about the French system as well.
In the second half of the 20th Century in the U.S., the two main pillars of the admissions process were high school grades and aptitude tests (SAT/ACT).
Those were traditionally viewed as something of polar opposites: High school GPA was a pretty good predictor of college GPA for the same reason that a baseball pitcher's ERA in the minors is a pretty good predictor of his ERA in the majors. GPA measured not only aptitude but also work ethic and effectualness. But, it had the problem that GPAs weren't nationally calibrated (e.g., which high school's grading is tougher: a 3.65 at Poly in Pasadena or a 3.65 at Poly in Sun Valley)?
Further, grades could be gamed in lots of ways -- e.g., tutors could be hired to do projects, students could cheat off other students during final exams, parents can complain to teachers to get grades raised, and so forth.
So, the SAT was envisioned as a national test of aptitude that couldn't be studied for.
It's a big national news story this week that seven students at a fancy high school in Long Island were caught paying a college student $1500-$2500 to take their SATs for them, but that's a bit of a man bites dog story. The SAT really does have better security than many GPA related activities.
So, the SAT was envisioned as a national test of aptitude that couldn't be studied for.
It's a big national news story this week that seven students at a fancy high school in Long Island were caught paying a college student $1500-$2500 to take their SATs for them, but that's a bit of a man bites dog story. The SAT really does have better security than many GPA related activities.
Over the years, standardized achievement tests have arisen within the American system that attempt to garner the advantages of GPA with national comparability of SAT/ACT: the SAT Subject tests, which are one hour multiple choice tests on specific subjects like American History, and the Advanced Placement tests. AP tests are rather like the British A-levels. They have a lot of prima facie validity at predicting college grades because they are similar to a comprehensive final exam in a college freshman level course
The AP tests aren't much used at present, in part for the same reasons that the British are dissatisfied with the A-levels: they are mostly given after college admissions decisions are made and they take a long time to grade.
About a half decade ago, the SAT was made more like the SAT Subject tests by incorporating the Subject test's writing essay and various other changes. But, are we losing the value of the differences between the SAT aptitude test and the SAT Subject achievement tests?
My general recommendations for designing college admissions would be to be aware of the ever-intensifying efforts to game the process by Tiger Mothers of all stripes and sexes. Institutional responses could come in two forms:
- Make some parts of the process harder to game
- Channel gaming efforts into actual learning. For example, putting more weight on the compromise measures (Subject tests and AP tests) might actually get students to, say, learn more chemistry or history as a byproduct of their frenzy to game the admissions process.
September 19, 2011
Asians, aptitude, and achievement: a positive sum reform proposal
The traditional concept of college admissions was that the goal was to predict applicants' future achievement (which could be measured in terms of first year in college grades or money donated 50 years later or whatever). The most obvious way to predict future achievement was past achievement: e.g., high school grades. Presumably, past achievement had two main components: hard work and aptitude.
But there were some obvious problems with relying solely on high school grades, such as different levels of grade inflation at high schools. If you were getting most of your applicants from St. Paul's and Dalton, well, you could keep in mind the differences, augmented with letters of recommendation from headmasters you had known for years.
But once the Ivy Leagues started trying to find the most promising non-upper class kids from the rest of the country, they needed something more objective about individuals than just grades. Another issue is that high school grades have certain inherent shortcomings. The future Nobelist in physics might not care about his social studies class and thus wind up with a lower overall GPA than the well-adjusted grind. Plus, grades have a ceiling. Even an A+ in physics doesn't really tell you that much. Moreover, lots of future successes are alienated in high school. Some people who get all As in high school might not have the upside to continue to do so in college. Incentives toward grade inflation at the high school are built in. And so forth ... Top colleges kept asking for recommendation letters, but their value (and, thus, importance) dropped as they increasingly came in from random teachers in random places like Burbank, CA.
So, for various reasons like this, the Scholastic Aptitude Test was created and spread. The idea was to have an objective, national test of academic intelligence. Overall, the SAT would appear to have been a huge success. American colleges are the most fashionable and richest in the world today.
However, there have long been complaints about the SAT. The most fashionable involved The Gap. Whites averaged higher scores than blacks. This posed a major PR problem for the academic establishment. The SAT (and ACT) is essential for their continued thriving, but saying that blacks are less intelligent than whites on average is The Worst Thing in the Whole World. But that's what the SAT says. And the SAT is the cornerstone of academic elitism, which has made American academia globally the envy of the academic world.
Thus, over the last half century or so, there have been anguished discussions between the front men for the academic world and the psychometricians at ETS about how to Close the Gap, without throwing the baby of predictive power out with the bathwater.
One change was purely PR: the SAT doesn't stand for Scholastic Aptitude Test anymore. It just stands for SAT these days. Under the hood, there have been a host of tweaks intended to narrow the gap without trashing the predictive powers of the SAT too much.
For example, the upper range of the Verbal (now Critical Reading) test has been capped. Before 1995, it was very, very hard to get an 800 on the Verbal test. I came fairly close the first time I took the test in 1975, so I gave it another try, got a little closer, but gave up and didn't take it a third time because the two scores seemed quite accurate: I'm very good at verbal logic, and have a certain gift for insights that other people wouldn't come up with, but I'm not a meticulous thinker. I make lots of mistakes. I'm more of a let's run it up the flagpole and see if anybody salutes thinker. In contrast, say, Charles Murray's brain works like a BMW V-12: powerful and precise. Mine's a jalopy that might surprise you and win the race or might break down on the starting line and go nowhere. So, there didn't seem like much point in me doing a lot of test prep to try to score 800 on the verbal -- I'd still make a mistake or two or they throw a really hard question at me.
But now, an 800 is well within reach of a lot of well-drilled students.
One change was purely PR: the SAT doesn't stand for Scholastic Aptitude Test anymore. It just stands for SAT these days. Under the hood, there have been a host of tweaks intended to narrow the gap without trashing the predictive powers of the SAT too much.
For example, the upper range of the Verbal (now Critical Reading) test has been capped. Before 1995, it was very, very hard to get an 800 on the Verbal test. I came fairly close the first time I took the test in 1975, so I gave it another try, got a little closer, but gave up and didn't take it a third time because the two scores seemed quite accurate: I'm very good at verbal logic, and have a certain gift for insights that other people wouldn't come up with, but I'm not a meticulous thinker. I make lots of mistakes. I'm more of a let's run it up the flagpole and see if anybody salutes thinker. In contrast, say, Charles Murray's brain works like a BMW V-12: powerful and precise. Mine's a jalopy that might surprise you and win the race or might break down on the starting line and go nowhere. So, there didn't seem like much point in me doing a lot of test prep to try to score 800 on the verbal -- I'd still make a mistake or two or they throw a really hard question at me.
But now, an 800 is well within reach of a lot of well-drilled students.
So, before scores were inflated in 1995, the SAT-V was an excellent test of high end verbal brainpower. In contrast, the SAT-M was widely recognized to need more headroom. It wasn't uncommon at Rice in the 1970s to hear good but not great Sci-Eng majors say, "Well, sure, I got an 800, but I'm not a real 800 like Joe is."
An obvious reform would have been to make scoring of SAT-M more like scoring of SAT-V. Instead, College Board - ETS did the opposite in 1995. One reason was that all that headroom on the Verbal modestly increased The Gap. The V test was made much easier to score 800 upon in 1995. A 730 old style became an 800 new style.
An obvious reform would have been to make scoring of SAT-M more like scoring of SAT-V. Instead, College Board - ETS did the opposite in 1995. One reason was that all that headroom on the Verbal modestly increased The Gap. The V test was made much easier to score 800 upon in 1995. A 730 old style became an 800 new style.
Lots of other tweaks were made, but as far as we can tell, anything that raises black average scores just encourages harder scraping of the bottom of the barrel by society, so the white-black The Gap remains remarkably stable over the generations. For example, The Gap on SAT-Critical Reading dropped about a half decade ago, perhaps because of the changes on the test, such as deep-sixing analogies. But that apparently just encouraged the College Board to troll for more black test-takers with free tests, so The Gap is now even bigger. (I may be overinterpreting a few squiggles on the trend graph.)
But white parents still tend to assume that SAT stands for Scholastic Aptitude Test. It's not an achievement test in their heads. The College Board says there is no point in studying extra hard for the SAT, and why would a prestigious not-for-profit institution spin the truth? If you can't trust the College Board, who can you trust? And signing your child up for intensive test prepping would be unfair to poor blacks who can't afford all that tutoring and drilling. Plus, prepping for years would be a lot of work for little Taylor, so just let him have his fun.
Meanwhile, lots of people from Fujian are showing up in America whose merchant ancestors ascended to mandarin status by spending their mercantile profits at Confucian literature cram schools for their sons. The assumptions about the SAT flitting around in white people's heads would never occur to them. "Test prep is unfair to poor blacks? Huh? You crack me up! I like you! You are very funny!"
Not surprisingly, we see vast amounts of white upper middle class rage directed at Amy Chua.
Now, is devoting hundreds of hours to prepping for the SAT a Good Thing or a Bad Thing? Well, let me try to reframe that question more productively and think about SAT test prep's opportunity cost.
It could be that SAT test prep has long term benefits other than getting into a fancier college. Could be ... I dunno. I haven't seen any evidence one way or another. But, it seems more like a zero sum game. SAT test prep seems kind of a sterile form of studying compared to studying an actual subject like Physics or French or Music Theory or World History or Microeconomics.
The return on investment for the colossal number of hours devoted in recent decades to SAT cram schooling is modest. The test is designed to be hard to prep for, so it's taken gigantic efforts for gains measured in fractions of a standard deviation.
The return on investment for the colossal number of hours devoted in recent decades to SAT cram schooling is modest. The test is designed to be hard to prep for, so it's taken gigantic efforts for gains measured in fractions of a standard deviation.
It seems to me that it would be better for everybody if more test prepping energy was invested instead into positive sum games, such as studying for achievement tests rather than for aptitude tests. Fortunately, we currently have a quite good set of national achievement tests: the College Board's Advanced Placement tests. They are not subject to the incentives for high schools to inflate their grades: the AP tests provide objective national grades that do a good job of predicting what the high school student would score on a 101-102 level course as a freshman in college.
In contrast, AP tests are intended to be ones you can study for, so the ROI on test prep effort tends to be quite a bit higher than on the SAT. Far more students are passing AP tests than a decade ago, and that's a good thing. Why have young people waste their time studying for something that's built to be hard to study for when they can instead study subjects that are intrinsically worth studying?
In contrast, AP tests are intended to be ones you can study for, so the ROI on test prep effort tends to be quite a bit higher than on the SAT. Far more students are passing AP tests than a decade ago, and that's a good thing. Why have young people waste their time studying for something that's built to be hard to study for when they can instead study subjects that are intrinsically worth studying?
Unfortunately, the current college admissions system gives little weight to AP test scores. Instead, perversely, it gives too much weight to taking AP classes in high school, even if you then do bad on the AP test. The University of California, for example, in calculating high school GPA adds a full point to classes designated Advanced Placement. Thus the average high school GPA of UC Berkeley freshmen is a wacky 4.39. An internal study by UC showed that cutting the bonus for an AP course down to 0.5 would better predict freshmen grades. I don't believe the UC system counts AP test scores in the admission's process, or doesn't count them much.
As Mitch has pointed out, this system is doubly rigged in favor of the more goody-two-shoes high school students. Typically, you need high grades in earlier classes to get into high school AP classes, where you are then given a full extra point for your GPA -- even if your AP score shows you didn't actually learn much.
As Mitch has pointed out, this system is doubly rigged in favor of the more goody-two-shoes high school students. Typically, you need high grades in earlier classes to get into high school AP classes, where you are then given a full extra point for your GPA -- even if your AP score shows you didn't actually learn much.
So, the ideal system would be for college admissions to be retooled from mostly a two legged stool of grades and SAT scores to a three legged stool of grades, AP Test scores, and SAT scores. The SAT could then be redesigned to be more purely an aptitude test that would be less easy to game. Moreover, test takers would have less incentive to devote hundreds of hours to gaming the SAT because they were being encouraged to spend hundreds of hours mastering AP Chemistry or AP European History.
The AP tests should be reformed to make them better for college admissions. They are currently scored on a 1 to 5 scale with a 5 equating to an A in the average college's freshman year introductory course in that subject, a 4 equal to a B, and so forth. A weird aspect of this is that all 5s are not created equal. For example, to get a 5 on the AP Chemistry test, you have to get a little over 60% right. So, people getting 98% right don't get a higher score than people getting 68% right. Test scoring should be kept the same at the lower levels -- a 3 would still be a C -- but the maximum score extended from 5 out to 7, which would be like an A at Caltech. Meanwhile, the GPA boost from taking an AP course would be eliminated, at least before senior year.
The initial winners from this changeover would, of course, tend to be Asians, who currently take a lot of AP tests. But good for them. Whites in heavily Asian areas, who have already started to adapt to the Asian challenge, would do okay. Whites in flyover regions would be challenged to get on the ball with AP. My guess is that it would be good for them and that they would eventually respond well to the challenge.
Overall, my plan looks like it would be better overall for society. There's a huge amount of energy out there looking to get an advantage in the college admissions process, so why not direct it in some positive sum direction?
Yes, sure, obviously it's a win-win, but, does it solve America's most overwhelming problem: Closing The Gap? Will blacks come closer to whites in scores under my system?
I dunno. I haven't thought about it. In fact, not worrying about Closing The Gap has allowed me to put forward a novel reform suggestion that might be better overall, which is not something you see too often these days.
In America today, 98% of the thinking devoted to college admissions goes to figuring out how your own kid can claw his way to the top, and the other 2% goes to airy handwaving theorizing about Closing The Gap. That leaves 0% devoted to thinking about improving the system overall.
Now, if I were truly, fanatically public-spirited, I would devote a lot of energy to dreaming up some bogus but persuasive-sounding theory about how my reform would Close The Gap, which would make it a lot more likely to be adapted. But, I'm not saintly enough to make up an elaborate lie.
The initial winners from this changeover would, of course, tend to be Asians, who currently take a lot of AP tests. But good for them. Whites in heavily Asian areas, who have already started to adapt to the Asian challenge, would do okay. Whites in flyover regions would be challenged to get on the ball with AP. My guess is that it would be good for them and that they would eventually respond well to the challenge.
Overall, my plan looks like it would be better overall for society. There's a huge amount of energy out there looking to get an advantage in the college admissions process, so why not direct it in some positive sum direction?
Yes, sure, obviously it's a win-win, but, does it solve America's most overwhelming problem: Closing The Gap? Will blacks come closer to whites in scores under my system?
I dunno. I haven't thought about it. In fact, not worrying about Closing The Gap has allowed me to put forward a novel reform suggestion that might be better overall, which is not something you see too often these days.
In America today, 98% of the thinking devoted to college admissions goes to figuring out how your own kid can claw his way to the top, and the other 2% goes to airy handwaving theorizing about Closing The Gap. That leaves 0% devoted to thinking about improving the system overall.
Now, if I were truly, fanatically public-spirited, I would devote a lot of energy to dreaming up some bogus but persuasive-sounding theory about how my reform would Close The Gap, which would make it a lot more likely to be adapted. But, I'm not saintly enough to make up an elaborate lie.
September 16, 2011
Mitch explains SAT trends
Respected commenter Mitch explains what is going on with rising Asian SAT scores:
It's a mix of things.
First, the 2005 changes made the math subject material easier, but an actual high score became harder because there were fewer hard problems and the impact of "unforced errors" became higher. Kids who were bright but careless could get a high score because the occasional unforced error was wiped out and more with their performance on the high difficulty questions. That last sentence describes whites more than Asians.
So since 2005, the ability to nail every question and not make unforced errors--something that drill does, indeed, help--has been rewarded, whereas the number of creatively difficult problems is 0 or 1 per test. This hurts white students, on average, more than Asian students just by personality trait, and then the Asian tendency to drill for this test gives even more of an advantage.
The reading test has been made unequivocally easier. I'm not sure what you mean about some reading questions being moved to the writing section. This is not true. The writing test is a near-exact replica of the old English Composition Achievement test, or the English Writing Subject test. There were no changes to it at all from a content perspective--they just changed the type of essay prompt and reduced the number of questions.
Certainly, the easier reading test makes it easier for Asians to get high scores. The writing test rewards attention to detail above all
So the test changes play a part, both in how they reward the traits more likely to be in asians over whites, and then in the Asian prep ritual--which really has to be seen to be believed. I teach in these schools, and the kids are in prep taking tests for 2 years. Even the ones who aren't getting super high scores are getting better scores, and that's bumping the average up.
Then there's the fact that Koreans, Chinese, and Indians are immigrating here in huge numbers, which is presumably offsetting the once larger percentage of Filipinos and Tongans.
I'm assuming you were only looking at US students, right? [I don't know -- it's not obvious from the College Board documents.] Koreans in Korea are taking the test as well, and there they study 40-60 hours a week, instead of school. The prep schools there buy copies of the most recent tests from students and use the tests to prep their students (something that's frowned upon here, although not technically illegal). The kids learn how to write essays by rote, and have whole essays memorized (use this essay for "change", this one for "education", and so on).
There's a general lesson here. Our society is constantly changing and bending procedures in hopes of Closing the Gap between the more feckless and the more feckful racial groups. Yet, perhaps unsurprisingly, the more feckful tend to better exploit these changes for their own benefit.
For example, the changes in federal regulator attitudes toward zero-down and low-doc mortgages that George W. Bush announced in 2002 in the name of fighting racist redlining at his White House Conference on Increasing Minority Homeownership poured hundreds of millions of dollars into Angelo Mozilo's net worth, most of which the SEC allowed him to keep even after fining him. But the net worth of the median Hispanic and black is now lower after mortgage follies.
The changes in the SAT in the last decade were mostly due to a single man, University of California chancellor Richard C. Atkinson. The UC system is the College Board's biggest client, and the UC was told by California voters in 1996 that racial preferences were now a violation of the state constitution.
Atkinson held an inflated notion of his expertise in the field of psychometrics. "When students asked me about IQ testing, I frequently referred them to Stephen Jay Gould’s book The Mismeasure of Man, published in 1981; it is a remarkable piece of scholarship that documented the widespread misuse of IQ tests," he wrote in an essay explaining part of his motivations, although he disingenuously left out all mention of the 800 pound gorilla in the room for UC, Proposition 209. The state legislature's Latino Caucus had threatened to cut the university system's budget unless they could figure out a way to cheat on Prop. 209 and get more Latinos admitted.
Atkinson held an inflated notion of his expertise in the field of psychometrics. "When students asked me about IQ testing, I frequently referred them to Stephen Jay Gould’s book The Mismeasure of Man, published in 1981; it is a remarkable piece of scholarship that documented the widespread misuse of IQ tests," he wrote in an essay explaining part of his motivations, although he disingenuously left out all mention of the 800 pound gorilla in the room for UC, Proposition 209. The state legislature's Latino Caucus had threatened to cut the university system's budget unless they could figure out a way to cheat on Prop. 209 and get more Latinos admitted.
Now, Atkinson wasn't a complete cynic. He was more of a fool. He thought he could kill a whole bunch of birds with one stone. For example, he wanted the SAT to be more of an achievement test than an aptitude test because of his quasi-Gouldian views on IQ testing:
My views are similar to those of Alfred Binet, the French psychologist who, in the early years of the last century, devised the first IQ tests. Binet was very clear that these tests could be useful in a clinical setting, but rejected the idea that they provided a meaningful measure of mental ability that could be used to rank order individuals. Unfortunately, his perspective was soon forgotten as the IQ testing industry burst onto the American scene.
This is like saying:
My views are similar to those of Nicolaus Copernicus, the Polish astronomer who, in the middle of the 16th Century, devised the first heliocentric system. Copernicus was very clear that heliocentrism reduced the number of crystalline spheres and epicycles necessary, but rejected the idea that we could explain the motions of the planets without the Music of the Spheres. Unfortunately, his perspective was soon forgotten as Kepler, Newton, and Einstein burst onto the astronomical scene.
Atkinson's goal was to make the SAT less of an aptitude test and more like the SAT II Subject Tests, which are more achievement tests. (UC long demanded applicants take both the SAT and three SAT Subject Tests, a high degree of overkill.) Since we've all know about how blacks and Hispanics are obviously oppressed by the white man's IQ pseudo-test, the thinking went, this would ameliorate UC's little problem with the Latino Caucus by Closing the Gap.
Except, it didn't work out that way because blacks and Latinos tend to not only have less aptitude, but they tend to be lazier about achieving academically. Also, it made the new three-part SAT 3.75 hours long, which may tend to mentally exhaust blacks and Latinos. The winners from these "reforms" turned out to be Asians, who, on average, work hardest.
Of course, the UC schools already had plenty of Asians. So, a few years ago, UC, having turned the SAT into something like the SAT Subject Tests, announced it wanted to stop requiring the SAT Subject Tests as now being redundant. This sensible reform, however, outraged the Asian-Pacific Islander caucus in the California legislature. The more tests parents have to remember to sign their kids up for and pay for tutoring in how to beat the tests, the better Asians do versus everybody else.
One thing to keep in mind is that elite private colleges don't really seem to want more Asians (at least if they're not going to pay list price. Foreign Asians, who don't qualify for financial assistance, are increasingly fashionable with colleges.). The more Asians score high on the SAT and get inflated GPAs by taking a lot of AP courses, the more the super-prestigious private colleges, which can and do use quotas for admitting blacks and Latinos, appear to discriminate against Asians.
Why they discriminate against Asians has been speculated about, but not, so far as I know, studied in any truly illuminating fashion.
Do they not donate as much money to their alma maters? My vague impression is that Indians, with their ancient tradition of alms-giving, are pretty generous when they have a chance to get their name put on an academic building. (I recently took enthusiastic part in a three minute standing ovation for an Indian gentleman who was the chief donor for the new library at my son's high school. It's a really nice library and it was all paid for and build by the week before my son started school there, so it didn't cost me a penny, so clapping my hands sore was the least I could do.) Chinese benefactors? Maybe not so much ... I don't know. This is the kind of thing that colleges have no doubt studied in intense detail, but their findings are Top Secret.
Or maybe too many Asians is considered uncool by high school students. For example, UC Irvine has long been heavily Asian, but it never seems to climb in coolness with kids.
Do they not donate as much money to their alma maters? My vague impression is that Indians, with their ancient tradition of alms-giving, are pretty generous when they have a chance to get their name put on an academic building. (I recently took enthusiastic part in a three minute standing ovation for an Indian gentleman who was the chief donor for the new library at my son's high school. It's a really nice library and it was all paid for and build by the week before my son started school there, so it didn't cost me a penny, so clapping my hands sore was the least I could do.) Chinese benefactors? Maybe not so much ... I don't know. This is the kind of thing that colleges have no doubt studied in intense detail, but their findings are Top Secret.
Or maybe too many Asians is considered uncool by high school students. For example, UC Irvine has long been heavily Asian, but it never seems to climb in coolness with kids.
Or maybe the elite colleges just don't believe the high SAT scores being recently recorded by Asians. Who knows? Obviously, elite private colleges know what their motivations are for requiring higher paper credential from Asians, but they aren't telling, and nobody seems to be asking.
SAT score trends
The Unsilenced Silence blog has a good graph of Asian v. white SAT scores from 1996 to 2010 in terms of gaps in standard deviations:
So, like most things involving test scores, the gap was stable in the later 1990s. However, Asians improved through much of the 2000s, especially on the Writing test, a new section of the SAT that strikes me as easier to game because it uses subjective (but very quick) grading.
One possibility is that globalization of American colleges has led to more Asian-in-Asia elites having their kids take the SAT, which could be increasing the pool of smart Asians taking the test. I'm also concerned about test security, however, when the test is administered in Asia. The Graduate Record Exam has been plagued by instances of cheating in Asia. Also, the new Writing test is rather like the tests that the Chinese Empire conducted for millennia to pick mandarins.
Another possibility is that the legal immigration system is working in bringing in smart Asians, who are becoming a larger fraction of the Asian-American population over time. But what about regression toward the mean? Amy Chua is very smart, for example, but she's nowhere near as smart as her father, the conceptualizer of the memristor.
The take home lesson for Americans would seem to me to be that Americans shouldn't credulously trust assertions by the College Board / ETS / ACT that their tests can't really be gamed through test prep or other means. The SAT and ACT are, relatively speaking, very good tests, but we live in a highly competitive, globalized world full of people who are smart, hard-working, and less trusting and less believing in Fair Play than naive Americans.
Now, here's his graph of the white-black SAT gap, which has been widening:
The sharp decline on The Gap in 2006 on the Verbal test (renamed Critical Reading) may have something to do with changes in that test, perhaps intended to make it easier on blacks. But, the general trend has been for The Gap to get worse, especially since the early 1990s. Yet, since the early 1990s (the Crack Years) were not a halcyon period for young blacks, the widening of the gap probably reflects concerted efforts since then by the Great and the Good to get more blacks to take the SAT (scraping the bottom of the barrel harder).
The process might work like this: the University of California, prevented from using racial quotas by Prop. 209) demands changes in the SAT Verbal that it hopes will boost black and Hispanic scores. A small amount of initial success in narrowing The Gap, however, leads elites to then over-confidently subsidize more black test-takers. Scraping the bottom of the barrel harder drives The Gap in Verbal back up, and increases The Gap in the other two sections to new heights. (Or I may be over-interpreting this.)
Also, it's likely that as test prep and gaming of the SAT has increased, that whites have benefited more from this than blacks. My mental model of who is doing more and more effectual test prep is Asians > whites > blacks and/or Mexicans > Other Hispanics (many of whom just arrived from Guatemala and are pretty clueless about the kinds of things that lead to higher scores that seem obvious to, say, a Hong Kong Chinese family in San Marino, CA).
Anyway, the White-Black gap graph shows the stability in gaps that is the overwhelming main lesson of generations of test score studies. Despite intense efforts by society since the mid-1960s, we still see about a one standard deviation gap between whites and blacks on a host of tests of cognitive ability (La Griffe du Lion's Fundamental Constant of American Sociology). But, that makes the recent change in Asian v. white scores even more interesting and more deserving of study.
Unsilenced Science also has informative graphs showing trends by year in SAT scores (i.e., not standard deviations).
September 15, 2011
SAT score changes by race since 1996
The oldest SAT score report on the College Board website is from 1996, right after the "recentering" in 1995 that raised scores about 100 points on a 400 to 1600 scale. Over the last 15 years, the average overall score on the original two-part Verbal + Math SAT (i.e., ignoring the new-fangled Writing section of the test introduced in the last decade) fell a grand total of two points, from 1013 to 1011. (See what I mean about baseball statistics being more volatile?)
| 1996 v. 2011 College-Bound Seniors Avg SAT Scores | |||||||||
| Total (V+M) | Verbal | Math | |||||||
| 1996 | 2011 | Chg | 1996 | 2011 | Chg | 1996 | 2011 | Chg | |
| ALL | 1013 | 1011 | (2) | 505 | 497 | (8) | 508 | 514 | 6 |
| Female | 995 | 995 | 0 | 503 | 495 | (8) | 492 | 500 | 8 |
| Male | 1034 | 1031 | (3) | 507 | 500 | (7) | 527 | 531 | 4 |
| Asian | 1054 | 1112 | 58 | 496 | 517 | 21 | 558 | 595 | 37 |
| White | 1049 | 1063 | 14 | 526 | 528 | 2 | 523 | 535 | 12 |
| Black | 856 | 855 | (1) | 434 | 428 | (6) | 422 | 427 | 5 |
| AmerIndian | 960 | 972 | 12 | 483 | 484 | 1 | 477 | 488 | 11 |
| Mexican | 914 | 917 | 3 | 455 | 451 | (4) | 459 | 466 | 7 |
| PR | 897 | 904 | 7 | 452 | 452 | 0 | 445 | 452 | 7 |
| Other Hisp | 931 | 913 | (18) | 465 | 451 | (14) | 466 | 462 | (4) |
You'll note that the average white score went up 14 points form 1049 to 1063. Did white people get smarter over that period? I don't know. The SAT changed a lot over those 15 years, with analogies being dropped and some Verbal multiple choice questions being exiled to Writing. Also, kids appear to have cared more about prepping in 2011, although the College Board doesn't like to talk about this.
Asians went up 58 points, which is pretty striking. Everybody else fell farther behind whites, which wasn't supposed to happen.
Now, it could be that scores actually did pretty well over this 15 year stretch, because the College Board scraped the bottom of the barrel harder. In 1996, 1,085,000 college-bound seniors took the SAT. In 2011, there were 1,647,000 senior SAT-takers. Just between 2006 and 2011, the College Board let an incremental 150,000 students take the SAT free or at reduced cost.
Asians went up 58 points, which is pretty striking. Everybody else fell farther behind whites, which wasn't supposed to happen.
Now, it could be that scores actually did pretty well over this 15 year stretch, because the College Board scraped the bottom of the barrel harder. In 1996, 1,085,000 college-bound seniors took the SAT. In 2011, there were 1,647,000 senior SAT-takers. Just between 2006 and 2011, the College Board let an incremental 150,000 students take the SAT free or at reduced cost.
On the other hand, my impression is that it became a lot more common for students to take both the SAT and ACT over that 15-year stretch, so some of the increase in the number of test-takers comes from people who would only have taken the ACT in 1996. It used to be that East and West Coasters took the SAT and Midwesterners the ACT, but by 2011, lots of students try both to see which one they'll do better on. These kids who take both tests probably tend to be fairly ambitious ones who are looking to game the system by taking both tests, then submitting only the test score they did better upon. So, double-dippers likely scored reasonably well (although, of course, not so 2400 / 36 outstanding that they wouldn't bother taking any test again).
(Has anybody recently done an authoritative study of the trend in overall SAT scores considering all the factors driving scores up or down?)
Between 1996 and 2011, everybody except Other Hispanics got a little better in Math. (I suspect that Other Hispanics used to be mostly Cubans and random fairly well-to-do South Americans, but now it includes a lot of Central Americans.) But Asians got a lot better: 37 points, from 558 to 595.
Verbal scores stagnated or declined slightly, except for Asians, who went up 21 points from 496 to 517.
Now, let's look at scores relative to the white scores in 1996 and 2011. A decade and a half ago, the overall score for everybody was 36 points lower than the white score. Today, it's 52 points lower. Most of that 16 points of relative decline is due to the demographic composition of America's SAT-takers changing for the worse.
| Difference v. whites | |||
| Total (V+M) | |||
| 1996 | 2011 | Chg | |
| ALL | 36 | 52 | (16) |
| Female | 54 | 68 | (14) |
| Male | 15 | 32 | (17) |
| Asian | 5 | 49 | 44 |
| White | 0 | 0 | 0 |
| Black | 193 | 208 | (15) |
| AmerIndian | 89 | 91 | (2) |
| Mexican | 135 | 146 | (11) |
| PR | 152 | 159 | (7) |
| Other Hisp | 118 | 150 | (32) |
The Gap got worse for most of the minority groups that the press gets worked up over. Blacks fell from 193 points behind whites to 208 points (a 15 point relative decline, or a point per year). Mexicans fell from 135 lower to 146 lower. Other Hispanics fell the most, from 118 behind to 150 behind.
These declines are probably mostly due to society (especially the College Board) scraping the bottom of the barrel harder in 2011. What with the recession and all, everybody is convinced that they must go to college, so they try the SAT. The number of people who scored below 400 on Verbal grew from 179,000 to 302,000 and on Math from 172,000 to 251,000.
The number who scored 700 or higher also shot upwards, but that might be due in part to kids taking the SAT more times or taking both the SAT and ACT to see if they can shoot the moon. The number scoring 700 or higher on Verbal went up from 47,000 to 77,000 and on Math from 58,000 to 112,000. High scorers are presumably the most likely to do a lot of test prep and otherwise try to game the system.
The number who scored 700 or higher also shot upwards, but that might be due in part to kids taking the SAT more times or taking both the SAT and ACT to see if they can shoot the moon. The number scoring 700 or higher on Verbal went up from 47,000 to 77,000 and on Math from 58,000 to 112,000. High scorers are presumably the most likely to do a lot of test prep and otherwise try to game the system.
In contrast to all other ethnic groups, who fell farther behind whites over the last 15 years, Asians had a 5 point advantage over whites in 1996, which blossomed to a 49 point lead by 2011, a relative change of 44 points.That's a big change, relative to the near-stasis on everything else.
Subscribe to:
Posts (Atom)
Posts
