June 7, 2012

Don't worry, studies show test prep doesn't work!

From the New York Times, an article on the annual Chinese college admissions test madness, the gao kao, and how Chinese test culture is spilling over to the U.S.:
With more and more Chinese students applying to foreign universities, the emphasis on the rote memorization required for the gao kao has come under criticism from some U.S. educators. Another cause for concern, they say, are the methods being used to study for the Test of English as a Foreign Language, or Toefl, which most U.S. schools require for admission. 
In a story done jointly by The Times and The Chronicle of Higher Education, Patricia J. Parker, assistant director of admissions at Iowa State University, which enrolls more than 1,200 Chinese undergraduates, said “students have proudly told her about memorizing thousands of vocabulary words, studying scripted responses to verbal questions and learning shortcuts that help them guess correct answers.” 
The story’s reporters, Tom Bartlett and Karin Fischer, wrote that Ms. Parker “has seen conditionally admitted students increase their Toefl scores by 30 or 40 points, out of a possible 120, after a summer break, despite no significant improvement in their ability to speak English. Her students, she says, don’t see this intense test-prepping as problematic: ‘They think the goal is to pass the test. They’re studying for the test, not studying English.’ ” 
Zinch China, a consulting company that advises American colleges and universities about China, published a report last year that found cheating on college applications to be “pervasive in China, driven by hyper-competitive parents and aggressive agents.’’ 
An excerpt from the Zinch report: 
The result? Fake achievements, often concocted by agents. Based on our interviews, this happens on about 10 percent of applications. Sometimes a student’s silver medal is turned to gold, and sometimes a student lists an award for an activity he or she never completed. At a top Beijing high school this year, ten students claimed to be Class President!
Most Chinese parents now understand that American schools are looking for “well-rounded” students who combine strong test scores, transcripts, and extra-curricular achievement. The problem is that most Chinese students don’t have time to participate in many extra-curricular activities — they are too busy studying for and taking tests. In fact, many Chinese parents see extra-curricular activities as a dangerous distraction from studying.

44 comments:

Simon in London said...

“students have proudly told her about memorizing thousands of vocabulary words, studying scripted responses to verbal questions and learning shortcuts that help them guess correct answers.”

Hmm - this may explain why my European students often seem so much better at English than some of my overseas students, despite identical IELTS/TOEFL scores. It would seem to argue against just raising the English language requirement for my course.

V said...

I think you've commented on this, Steve, but this is why I like AP tests- if people are going to study like mad, we might as well make them study in ways that impact useful knowledge rather than just signal intelligence.

Duke of Qin said...

“students have proudly told her about memorizing thousands of vocabulary words, studying scripted responses to verbal questions and learning shortcuts that help them guess correct answers.”

The story’s reporters, Tom Bartlett and Karin Fischer, wrote that Ms. Parker “has seen conditionally admitted students increase their Toefl scores by 30 or 40 points, out of a possible 120, after a summer break, despite no significant improvement in their ability to speak English.

This is generally how you learn a new language. Their spoken English might not have shown "significant" improvement but their ability to read and write English has demonstrably. For most individuals, learning to speak a foreign language will require immersion in a foreign language environment, something that these children do not have.

Anonymous said...

When I was in graduate school in physics, the professors on the admissions staff said the Chinese students had amazing scores on the both the TOEFL and verbal GRE. Many of them didn't speak English well when they graduated let alone when they showed up.

Those tests are not effective at what they are trying to do.

Anonymous said...

How about sleep-over tutoring?

How about 'calculus soccer' for xtra-curi? players have to act out the problems.

Anonymous said...

how about After-school-studying-as-performance-art?

Anonymous said...

“students have proudly told her about memorizing thousands of vocabulary words, studying scripted responses to verbal questions and learning shortcuts that help them guess correct answers.”

sound like pc.

vocab: racist, sexist, homophobe, odious,etc

scripted responses: mlk makes me weep, obama is the one, marriage ewuality, race is social contruct, i am soooooo sorry,
etc

short-cuts: uh... diversity is our... strength?

Anonymous said...

‘They think the goal is to pass the test. They’re studying for the test, not studying English.’

I did the same thing in h school and college but not in the chinese way.

I'd ignore studies all semester and then study for the test the night before.

Cliff notes and mt. dew.

Passed the test but learned nothing.
Everything I know I got from movies.

Anonymous said...

Chinese econ is same way. Chinese dump everything into things that create growth but miss bigger picture of sustainability.

Evil Sandmich said...

Being a veteran of many a technical certification test, it is without a doubt the person versus the test, not the person versus the material. The more complicated the material being tested, the more worthless the standardized test.

Anyway, colleges wouldn't want to interrupt their foreign student gravy train.

proudly told her about memorizing thousands of vocabulary words

Boy, what a waste of time ;-)

Anonymous said...

The willingness among some students to bend to any demand is kinda creepy. Kids disdain this phony "book learning" for good reason. You have to actually think. I understand plenty of students are bona fide good students. Still there is this weird segment that do this stuff.

Mitch said...

the professors on the admissions staff said the Chinese students had amazing scores on the both the TOEFL and verbal GRE

The verbal GRE is achieved by cheating. The Chinese pay people to take the test simply to memorize the questions and answer choices. They are then posted on a message board. It is for this reason that the test was banned in China, but students just saved up to fly to Vietnam--where they cheated.

THe GRE was changed solely to fix the problem of Asian, primarily Chinese, cheating.

Anonymous said...

oooooooooooooooooooooooo........ threatening!!!

TD said...

FYI: Looks like it's going to be a month of fireworks in Poland and the Ukraine...

http://www.guardian.co.uk/football/2012/jun/07/euro-2012-holland-racist-abuse

anony-mouse said...

So let me see:

1/ Chinese do many things to game tests.

2/ We know absolutely that Chinese are inherently very intelligent because of the results of the IQ and math tests they have taken.

Makes sense to me.

Ed said...

This is probably not the place to post a sarcastic comment about the historical importance of tests determining ones social status historically in American culture vs. Chinese culture.

TontoBubbaGoldstein said...

They memorize vocabulary words to improve their performance on an English test!

Ye Cats! What is our world coming to?

Anonymous said...

When I was in graduate school in physics, the professors on the admissions staff said the Chinese students had amazing scores on the both the TOEFL and verbal GRE. Many of them didn't speak English well when they graduated let alone when they showed up.

Those tests are not effective at what they are trying to do.


Actually they are. They assess reading competency. The students understand what they read. Administrators who use scores on written assessments to assess speech communication abilities are the ones who are using the test scores inappropriately. In this day and age, if they want to know how well the person communicates, they should require some underling do a skype interview and talk to them about general crap in their field that any underling would be able to evaluate how well they communicate. Then they could use some holistic 1-5 scale to rate them from unintelligible to articulate.

DougRisk said...

Aren't many of the best Scrabble players from places like Viet Nam?

I mean, I remember one Vietnamese guy winning some English Scrabble tournament and he could not speak English at all.

bjdubbs said...

The Chinese are going to test-prep their way to world domination. Good plan.

Btw, this is . . . something. I'm sure Whizkey will tell us what it means.

http://www.dailymail.co.uk/news/article-2154255/Devastated-Ben-Goldsmith-blasts-wife-Twitter-affair-rapper-ended-marriage-Rothschild-heiress.html?ITO=1490

Andrew said...

Why not just stop admitting Chinese students?

The Universities could shrink to a more manageable size and refocus resources on Americans,, and with the reduction in grad students, more professors could focus on American undergraduates.

Eric said...

I mean, I remember one Vietnamese guy winning some English Scrabble tournament and he could not speak English at all.

Why would you have to know the language to play Scrabble? Not only does it not require grammar, you don't even have to know what the words mean. Of course, if you can write in English you have a head start because you already know many of the words. But you can't be a competitive Scrabble player without a lot of rote memorization.

Anonymous said...

South Korea takes itself out of bio-engineering competition.

Matthew said...

"Anyway, colleges wouldn't want to interrupt their foreign student gravy train."

1200 Chinese students just at Iowa State. Mitt Romney has pledged to turn our immigration policy over to the control of the patriotic, conservative, Scots-Irish-Harvard/WASP mafia running our colleges and universities, by automatically giving a green card to any STEM student earning a graduate degree at a US university. Watch as STEM graduate programs expand 300%, and go from 60% to 97% foreign.

Anonymous said...

This is generally how you learn a new language. Their spoken English might not have shown "significant" improvement but their ability to read and write English has demonstrably.

I think you're missing what I think is probably the quoted speaker's (not too clear) point. It's precisely that they really aren't learning or memorizing vocabulary. They are learning to cram for a few thousand words using cues and keys, then forgetting the words, if they ever knew the whole words.

I taught around 1990 in a state school around the Boston area that had near 100% Chinese TAs in the CS Deptartment. (There was also constant pious proclamations about serving the Irish community of families with no previous experience of higher education... but not that many Irish. Go figure.) Anyhow, there was a constant problem with Chinese grad students cheating, in particular, a problem with plagiarism. When caught, the students invariably (and often, it seemed, successfully), resorted to "it is our culture, we are taught copying is a sign of respect, that it is what education is about, that it is what we are supposed to be doing, we really don't understand."

Anonymous said...

I know a half-Chinese student who improved from 1400 to 1530 on the old 1600 SAT scale with some test prep.

I also know plenty of Chinese in Silicon Valley. Their English varies from poor to excellent, but they do good work, and fewer Engish speakers speak Chinese at any level.

Norville Rogers said...

Survival of the preppest

Anonymous said...

Bitter? Because the wasps/Jews made a pyramid scheme college admission system that was designed to promote their own against the Gentiles and it backfired when asians gamed the system better?

I say good riddance.

The Wobbly Guy said...

It's a perennial Chinese problem, not just in China. It took Singapore years of effort before non-academics became a more-or-less accepted part of school life, only because they made these activties count towards the next tier of education. And still, a significant portion of students do not participate in non-academic activities at the high school (Junior College over here) level.

It's often a problem for many ethnic chinese minorities elsewhere too. I remember my girlfriend (ethnic chinese from the Philippines) telling me that her mother discouraged her from joining any non-academic activities when she was on scholarship here in Singapore.

Personally, I am a strong proponent of non-academic activities. You learn how to handle projects, handle people, and acquire life skills you can't get from subject specific education. Sure, you can fake it on your resume, but you will get found out, one way or the other.

The Wobbly Guy said...

South Korea is going to stop teaching evolution! That's great news for China and the rest of us! :D

Anonymous said...

http://www.americanthinker.com/2012/05/mitt_flunks_education_101.html

Anonymous said...

http://www.gq-magazine.co.uk/comment/articles/2012-06/06/johann-hari-twitter-and-the-importance-of-critics

Simon in London said...

Non-academics still don't really count for much in Britain, by the way. When I'm reviewing applications to my graduate course, I really don't care what you've done if it doesn't relate to the subject I'll be teaching you. And undergraduate applications are entirely by academic results.

Anonymous said...


Personally, I am a strong proponent of non-academic activities. You learn how to handle projects, handle people, and acquire life skills you can't get from subject specific education. Sure, you can fake it on your resume, but you will get found out, one way or the other.


Does that include Second Amendment activities?

I think it is important for anyone who wants to to gain experience with lathes and mills and to know how stuff works, like the AR15 and the AK47.

Anonymous said...

Part of me loves the fact that this article means the NY Times is terrified the kids of their readers will face Chinese competition and might lose.

Anonymous said...

Speaking of Chinese cheating, I heard about a trend from someone who used to be in admissions - Chinese students would claim to be authors on papers by looking up someone with the same name - say H. Wu. Very annoying to track down. Another trick, when it came time for interviews, when the interviewer called the prospective chinese student, he would ask "Can I call back?". In awhile, a very articulate English speaker called back that was clearly not the same person. The admissions office got in the habit of saying no when students asked to call back.

Sheila said...

Sometimes tests do correctly cover knowledge of core material. My older son (IQ tested between 145-157)failed a number of public high school courses (neither completed nor turned in the "reports" or "fun projects") but, with absolutely no studying, aced the final exams. We finally let him graduate (he had plenty of credits) and join the army at 17.

pat said...

Asians are different. I know this because of my careful study of "Ninja Warrior".

There is a scene in "True Grit" where Rooster is playing cards with his Chinese landlord and he points at him as says "That's how the China-man bests you". Those orientals are so damn inscrutable.

Another Western that illustrates this is "The Magnificent Seven". This film of course is derived from Kurosawa's "Seven Samurai". One group of seven contract killers are grim and businesslike while the other seven are laughing and jolly. Check it out, the samurai are always smiling and happy. Almost everyone remembers this backwards.

Similarly Clint Eastwood never joked and never smiled when he played Toshiro Mifune. There are a lot of visual jokes in the original Japanese films.

This is doubly odd when you realize that the Tokugawa Samurai were a hated set of brutal and privileged oppressors while the American cowboys were just working class schlubs.

Now we have the American version of the Japanese game show (or is it a sport?) "Ninja Warrior". The Japanese original is charming and fun. The new American version is grim and oppressive.

In the Japanese original almost no one can win. Only three guys have succeeded among the three thousand who have entered. There are no payments and there is no winner's prize. Olympic gold metal winners and professional athletes have tried but the most successful have been just Japanese guys off the street. Two of the champions have been fishermen and one is a shoe salesman.

Only about one third of the Japanese entrants are serious - literally. There are a lot of comedians who enter. They all lose but make everyone laugh doing it. There are a lot of old men also like The Octopus Man - he wears an octopus on his head. He's over sixty and splashes out usually on the first obstacle but he is well loved. There are those who dress up as Superman or Bruce Lee. There are lots of costumed characters. There usually aren't but one or two real athletes in the first fifty entrants.

The American version is almost impossible to watch because it is so humorless. Every competitor it seems has devoted his effort to some relative who has some disease. The Americans have no comedians (where's Jay Leno?). They have no silly fat old men dressed up in a costume. Instead we have earnest young men wearing pink to honor Breast Cancer (or was that Itchy Hemorrhoids?).

The American version promises a half million dollars in prize money and hints that the winners will become sports professionals. The Americans have eliminated all the delightfully fake competitors and brought us more fit young men with washboard abs. Boring.

It is all for naught. In the last American version the announcers described one competitor as being brave enough to compete even though he was only five foot six.

In fact five six is probably too big. The first Japanese champ was five three. Makoto Nagano - by far the best competitor for the last decade - is five two. The new champ Urushihara is only five one and weight 110 lbs. Urushihara is a shoe salesman in real life. A very small shoe salesman.

They asked a Japanese all star about the new third stage at one point. He said,"It's easy - if you're a monkey". More like a Siamang or a Gibbon I would say. The Japanese are selecting for Gibbon like bodies. The Americans are foolishly trying to get gorillas - humorless gorillas.

Albertosaurus

peterike said...

Why not just stop admitting Chinese students?

This.

But that would mean abandoning the raison d'etre of Liberalism, namely, Western cultural suicide.

As Malcolm Muggeridge had it, we are all "sleep-walking to the end of the night."

Hacienda said...

"Asians are different. I know this because of my careful study of "Ninja Warrior"."

Same thing can be said for "Wipeout", an American knockoff of Japanese obstacle shows.

Japanese fall funnier than whites. More spin and rapid momentum changes via collisions. Automatic, physical humour. This is just priceless and can't be programmed. No amount of self-conscious wit or banter can overcome this problem, because the talk and self-consciousness gets in the way. Which, btw, is just a general cultural problem with being white in itself. Whiteness- The "Anti-Negritude".

Matt said...

Another Western that illustrates this is "The Magnificent Seven". This film of course is derived from Kurosawa's "Seven Samurai". One group of seven contract killers are grim and businesslike while the other seven are laughing and jolly. Check it out, the samurai are always smiling and happy. Almost everyone remembers this backwards.

Similarly Clint Eastwood never joked and never smiled when he played Toshiro Mifune. There are a lot of visual jokes in the original Japanese films.


I don't think this is true amongst people who have actually watched "Seven Samurai".

Amongst the people who have not, yes - because the stern, stoic, implacable warrior is the historical samurai ideal. It still shows up in their anime and pop culture - Western ideals of cool tend to embrace the sociable while Asian ideals embrace more socially distant figures.

Kurosawa wasn't particularly a fan of this ideal, or the stoic militarism it supported - so he deconstructs it in his Sanjuro and Seven Samurai films.

But it is a real ideal of the Japanese culture, and assumptions will be made by people who basically are not familar with the director in question. Because it is more the Japanese norm.

The American version is almost impossible to watch because it is so humorless. Every competitor it seems has devoted his effort to some relative who has some disease. The Americans have no comedians (where's Jay Leno?). They have no silly fat old men dressed up in a costume. Instead we have earnest young men wearing pink to honor Breast Cancer (or was that Itchy Hemorrhoids?).

The American version promises a half million dollars in prize money and hints that the winners will become sports professionals. The Americans have eliminated all the delightfully fake competitors and brought us more fit young men with washboard abs. Boring.


Same thing can be said for "Wipeout", an American knockoff of Japanese obstacle shows.

Which, btw, is just a general cultural problem with being white in itself. Whiteness- The "Anti-Negritude".


The British "Total Wipeout" sticks pretty close to the original - http://en.wikipedia.org/wiki/Total_Wipeout

Cross cultural patterns of seriousness and playfulness are interesting to me in that they aren't predictable in a simple and one way fashion.

Japanese (and Asians in general) can be very "ganbatte kudasai" and po faced about things we dismiss (all that "weeping at the company ceremony" business, Asians being oddly and ridiculously concerned with "face" and shaming their family and parents) while not really being invested in a lot of stuff that Westerners tend to care about.

Americans are interesting on a global scale, even compared to similar nations like the Brits and Aussies and North West Europeans, in that (hat tip to blogger HBD chick) they're relatively very invested in voluntary and charitable organisations, that they choose as individuals to care about. East Asians (in East Asia) are at the polar opposite, not really seeming to give much of a flying fuck about anything other than their family, company and nation. Thus earnest young men honoring Breast Cancer in the US?

Anonymous said...

http://youtu.be/bCW95R3ObtY

Ford's last film.

pat said...

I don't know quite how to react to Matt's remarks. Is he claiming that I haven't in fact watched Seven Samurai?

Trust me - or watch it on Netflix. The Seven Samurai are jolly. The Magnificent Seven are grim and they feel sorry for themselves.

My main point is that almost everything everyone belives about the Japanese is wrong. You only need to look around to realize that most popular stuff about the Japanese is pure hokum.

In all Hollywood movies today someone will be using oriental martial arts and/or a Japanese katana. They may prat on about Bushido, honor and duty too. But the simple facts are that the Japanese Samurai were terrible people hated by the peasantry, they were lazy welfare bums and not very good with weapons. They were equipted with bad swords and they didn't keep in practice. They were also slow.

The Samurai only adopted the sword when it was already obsolete. Previously they had been mounted archers - and they were not very good archers either.

It's easy to judge the military effectiveness of oriental soldiers - look at the stats. The Mongols fought everybody, everywhere and the more or less always won. The Samurai never fought anyone and they never beat anyone. However the Samurai bathed more often and wore much more spiffy outfits.

That's not to say they weren't brave just that they weren't skilled. Japanese sword making was nearly a millenium behind that of the West. Modern Japanese manufacturing is admired because of their quality control. The traditional Japanese sword had no quality control. Katana's were not made to specs. They were a hand made objects made with a lot of mystic mumbo-jumbo. When the Mongols invaded the Samurais found that their swords shattered, broke, or delaminated on the swords of the Mongols.

In any case the sword was retained by the Shugunate not because it was effective, rather the opposite. Nobunaga had introduced new firearm tactics but Tokugawa began a policy to eliminate all firearms in Japan. By the time Perry arrived what few cannon they still had were only ceremonial. They were a whole culture intent on "taking a knife to a gun fight".

We now know that Karate isn't a particularly effective fighting style. It was originally thought that Karate would dominate all open fighting shows - what we now call Mixed Martial Arts. Promoters used to mount matches between a Western boxer and a karate fighter. These matches proved disappointing. The boxer always won. Today MMA is dominated by Brazillian jujitsu and a couple other hybrid styles. Real karate is nowhere to be seen.

Samurai sword fighting is similarly inferior. It requires a two handed grip and it is principally a slicing sword. It is also a long sword. All these traits are primitive traits that arose in the West and were later abandonned.

The Romans fought the Celtic barbarians who had long two handed slicing swords. The Romans themselves use a short one handed stabbing sword.

The other type of Western sword was the dueling sword which was also a one handed stabbing sword but with a light thin blade. Moreover in the West men kept in practice. Clemenceau fought a duel with swords.

The Samurai did not resolve disputes with their swords - or at least they weren't supposed to. The next higher up in the authority hierarchy made a determination and ordered one or the other disputants to use his short sword on himself. A common plot point in Samurai movies is when someone decides not to suicide but to fight. In any case such a system will mean that your warriors seldom fight for real.

Another Samurai trait that is seldom shown on the big screen is pederasty. The Samurai like the Spartans had homoerotic relationships with young boys.

Remember also that Mushashi - the greatest of the Samurai - advised his students to always swing the sword slowly. One wonders how long he would have lasted in real duel with any of the Musketeers?

Albertosaurus

Anonymous said...

I don't know quite how to react to Matt's remarks. Is he claiming that I haven't in fact watched Seven Samurai?

I'm claiming that I don't think all the people who have actually watched it remember it as you say, not that you have no watched it.

You: Almost everyone remembers this backwards.

Me: I don't think this is true amongst people who have actually watched "Seven Samurai". (as opposed to having sort of read a brief description about it and the Magnificant Seven). And I also stated that the people who did not watch the film would probably be misled by the fact that Kurosawa was not actually a fan of the stoic, suicidal samurai traditionally idealized by Japanese culture (which should be fairly obvious to you).

The Mongols fought everybody, everywhere and the more or less always won. The Samurai never fought anyone and they never beat anyone.

Notably the Samurai fought the Mongols and did quite well (given that they were on a little island far from the most competitive military arenas on the planet (those parts of Eurasia which border the steppe)).