January 16, 2010

The Stereotype Threat scientific scandal

From my VDARE.com column:
Over the last 15 years, the most popular theory about racial differences in IQ has been “Stereotype Threat.” The New York Times summarized it in its 2009 Year in Ideas featurette on the purported “Obama Effect”—the widespread assumption that the politician’s success might raise black test scores:
“In 1995, two Stanford psychologists, Claude Steele and Joshua Aronson, demonstrated that African-American college students did worse on tests of academic ability when they were exposed beforehand to suggestions that they were being judged according to their race. Steele and Aronson hypothesized that this effect, which they labeled stereotype threat, might explain part of the persistent achievement gap between white and black students. In the years since, this idea has spread throughout the social sciences.”

Stereotype Threat is a beautiful theory for explaining the racial IQ gap. Indeed, I myself have often felt there might even be a little bit of truth to the idea that expectations matter—even though common sense suggests that incentives matter more.

But now, it turns out that the vaunted evidence for this wildly popular concept rests heavily upon another Effect, the File Drawer Effect—defined as “the practice of researchers filing away studies with negative outcomes”. We seem to have another Climate Research Unit scandal on our hands.

A researcher, who doesn’t want his name or any potentially identifying information mentioned, for unfortunately obvious career reasons, recently attended a presentation at a scientific conference. Here is his summary of what he heard:
“One talk presented a meta-analysis of stereotype threat. The presenter was able to find a ton of unpublished studies. The overall conclusion is that stereotype threat does not exist. The unpublished and published studies were compared on many indices of quality, including sample size, and the only variable predicting publication was whether a significant effect of stereotype threat was found. …

“This is quite embarrassing for psychology as a science.”

Here’s the abstract of the presentation he heard (see p. 68 of the PDF)
“Numerous laboratory experiments have been conducted to show that African Americans’ cognitive test performance suffers under stereotype threat, i.e., the fear of confirming negative stereotypes concerning one’s group. A meta-analysis of 55 published and unpublished studies of this effect shows clear signs of publication bias.”

[Stereotype threat and the cognitive test performance of African Americans, by Jelte M. Wicherts & Cor de Haan, University of Amsterdam]

In other words, if a study doesn’t find the existence of stereotype threat, it’s less likely to see the light of day. Positive results are more appealing to journal editors, and politically correct positive results are loveliest of all. In contrast, how much of a market is there for punching holes in society’s fondest hopes?

The Dutch researchers continue:
“The effect varies widely across studies, and is generally small. Although elite university undergraduates may underperform on cognitive tests due to stereotype threat, this effect does not generalize to non-adapted standardized tests, high-stakes settings, and less academically gifted test-takers.”

Note that “Stereotype Threat” mostly seems to exist in settings where test-takers, such as “elite university undergraduates,” are smart enough to pick up on researchers’ hints about what results they hope to publish. In the marketing research industry, it’s well known that survey respondents tend to respond with the answers that they surmise the pollster wants. Human beings like to be cooperative if it doesn’t cost them anything. Similarly, it’s likely not hard for black students at top research universities to gather that they can benefit their professor with a publishable paper just by working less diligently on the meaningless test he's given them.

Moreover, “Stereotype Threat” doesn’t seem to exist where the test is important enough to matter to the students.

My published articles are archived at iSteve.com -- Steve Sailer

41 comments:

Anonymous said...

ANON POSTED

One anonymous here claims that is normal for the rich and poor within a race not to give a damn about one another and that rich elites intermarry between races.
A study of history debunks this.For example most of the nations of Europe have been in bitter conflict with each other for centuries, during these interminable conflicts (ie Ireland, the Balkans)self-idenified ethnicities fought to the death always on the basis of shared nationality, blood and language and never on the basis of social class.
For example the leaders of Irish Republicanism were ofetn the intellectual and social elite.In the war of 1921 they died alongside their foot-soldiers the Irish peasantry.

I REPLY

Yes, you are correct that it is self destructive for the very high IQ whites of Christian European ancestry in California to ignore the interests of their fellow white Christians of European ancestry.

But they do.

Many people choose to do things that destroy the possibility of their genes being passed to a new generation.

Need I remind you that some exceptionally high IQ people choose to be gay. Being gay certainly limits the spread of your genes.

Not all super high IQ people act in their own genetic self interest.

High IQ people that make the decision to move to the "white homeland" will be guaranteeing that they have many great grandchildren that carry their genes. High IQ people that choose not to move to the white homeland are not.

Udolpho.com said...

I am completely shocked.

silk said...

Stereotype Threat must be true. After all, we have this stereotype of liberals as liars. So, it could be the fear of this negative stereotpye that prevents liberals from speaking truthfully. Blame not liberal liars but our stereotypes about liberal mendacity.

Joseph said...

Has the integrity of academia diminished over the last couple decades, or was the higher educational establishment always this corrupt -- willing to sacrifice research and honesty for the sake of advancing ideology?

Don't people pursue the university life because they love knowledge and truth? When did this concern get sidetracked for so many people?

black comb said...

Now, we know why Asians suck at American Idol. They all have booming voices but get all choked up because of the stereotype that they cannot wail out a tune like Aretha Franklin or Teddy Pendergrass. Our fault.

Seems to me what the SWPL crowd is really afraid of is the REALITY THREAT. They are like Ptolemaic astronomers who think the entire world revolves around them rationally, morally, spiritually, cooliciously, and scientifically. If some data comes along to prove that reality doesn't revolve around their assumptions, they panic like the Catholic Church of old and devise yet wonkier theories of how everything does indeed revolve around their TRUTH.

Their most sacred and dogmatic TRUTH involves the non-existence of races--thus non-existence of racial differences. This is their CORE TRUTH whose laws all of social reality must obey.

Problem is available data refuses to revolve around their 'truth'. Instead of ever admitting that their paradigm is wrong, they blame those who provide the data.

Social ptolemaism is the scourge of our age.

Anonymous said...

Blacks should worry more about the Stereophonic Threat. Turn off the damn music and hit the books. Do some homework.

Anonymous said...

Science in service to lies and money.

Its happening more and more.

Remember the Swine Flu (H1N1) Epidemic we were going to have this winter? It didn't happen, despite the CDC, NBC, ABC, CBS, and CNN all assuring it that it would, and dark whispers about FEMA camps for the uninnoculated. I wonder who made the most money off the hysteria? Aventis-Pasteur (vaccine "provider", contracted with the guv'ment) or media/Wall Street insiders who bought the stock early and are shorting it right now?

Anonymous said...

You imply that there was dishonesty and so do some of the individuals commenting. The file drawer problem is a problem for all of science and doesn't mean that people have lied.



There is also a big step from conference presentation to published article. So let's see how this plays out.

Someguy

profile said...

Will this really turn out to be a scandal? MSM is controlled by liberals, right? Just like scientists selectively report data, liberal MSM selectively reports the news. It can't be a full-blown scandal unless it becomes major news.

I suppose there's Fox News and conservative outlets like Newsmax, but climate and race are two different things. Mainstream conservatives can be courageously contrarian/skeptical on environmental issues but rarely on racial issues.

If this does turn out to be scandal, it will be limited within a small academic community, which will bury it since it sides with liberals anyways.

TH said...

Few young scientists have the courage to do HBD-informed research, because they know that they won't get published, or that if they do, they will be denounced and pushed off the tenure track. It seems that just about all of the few who do conduct racially realistic research are full professors, who were silent on racial issues until getting tenure.

I wonder if there's an article coming up on this finding? There's no mention of it on Jelte Wicherts's website, although there's some interesting stuff there.

ricpic said...

Why should Stereotype Threat be more of an obstacle to test performance than stage fright is an obstacle to performance on a stage? The fact is that whatever mental hobgoblins may be dancing in your head before the moment of truth, be it taking a test or performing in a play or running a race, those hobgoblins evaporate the minute the instructor says begin or the curtain rises or the buzzer sounds. This is a universal experience which should have been sufficient to discredit the theory of Stereotype Threat at the time it was put forth.

jody said...

i think for black american students, as a group they could definitely do better academically, but they are smart enough to understand, that while they could do better in school, it does not make much sense to put in all that time and effort simply to improve from "terrible" all the way up to "bad". the end result would be the same, still near the bottom in school performance. but anything that would motivate more black americans to do what it takes to graduate from high school would be a huge positive step. graduating is something that is so easily within reach of millions of black americans who instead opt to drop out after 1 or 2 years. they need the encouragement to graduate, to get a C in algebra, to put in more effort than zero effort.

east asians approach it in completely the opposite way. on average they are maxed out for time and effort put into school, and they ignore almost everything else. some of them have the potential to excel in fields outside of academia, but never even dare. it is so common to hear adult east asians say that they wanted to do so and so, but their parents forced them to become an engineer or a doctor.

everybody is smart enough to play to their strengths. it can be nearly impossible to break these entrenched expectations.

i'm not sold on stereotype threat. there is crushing, simply unbelievable "stereotype threat" pressure applied to white athletes doing the "wrong" things in american sports, but it does not seem to affect them much when they are allowed to show what they can do. despite how badly so many people want them to fail, when given an opportunity to do things "that they should leave to the real athletes", capable white athletes still generally perform at a level that their measured ability and previous performance resume suggest they can.

Polistra said...

@Joseph: The educationists have always been corrupt. I got a degree in education in the late '70s, and this bogus theory was taught in every course.

And here's the latest excrescence of the "let's fool the kids by changing the labels" movement:

http://www.spokesman.com/stories/2010/jan/12/wash-lawmaker-wants-banish-negative-language/

Anonymous said...

Whats about the fear some blacks have of "acting white"? We hear about that from time to time.

Anonymous said...

>If a black guy is caught shop-lifting, what will he say? "I tried to be law-abiding, but it was too difficult because of the stereotype threat of blacks as criminals."<

Of course he won't say that. He will say "I tried to be law-abiding, but it was too difficult because of this racist society." Completely different.

Joseph asked

> was the higher educational establishment always this corrupt<

That's the way I'm betting. In sociology what big ideas did it bring us in the 20th Century that were not the equalivalent of Lysenkoism?

Freudianism, Marxist-Leninism, socialism, Boasianism, progressivism, behaviorism, Gouldianism....all bullshit.

Time to start over. Almost any of us could start a university in his basement and get further, if love of knowledge has anything to do with it.

Le Mur said...

The discoverer of the so-called "Flynn Effect" (an "S. Smith") studied and dismissed this issue in 1942; the science of psychology marches on! (in circles?)

sugral said...

"Few young scientists have the courage to do HBD-informed research, because they know that they won't get published, or that if they do, they will be denounced and pushed off the tenure track. It seems that just about all of the few who do conduct racially realistic research are full professors, who were silent on racial issues until getting tenure."

True, this is why they should all do a dipsy-doo. Pretend to be PC--while accumulating covert data for a later date--and work towards getting tenure. Once tenured, write and say whatever you like. I think Kevin MacDonald is untouchable because he has tenure. Hehe, he got it as a leftist.

Anonymous said...

willing to sacrifice research and honesty for the sake of advancing ideology?


This post is about AGW?

Toadal said...

I feel Stereotype Threat was conceived to deceive the broader public, but not to be taken too seriously by decision makers. Otherwise, it would naturally have led to a return to 1950s style black and white segregation.

Senior VP of Finance to CEO: "JP? Studies have proven that African Americans’ productivity suffers from stereotype threat while working with whites. I've formulated a productivity plan to bolster our ailing accounts receivable, sales and janitorial departments by making all their personnel African Americans."
- or -
Major to battalion commander: "Colonel Stark? Studies have shown that African Americans’ soldiering suffers from stereotype threat while serving with whites. I've formulated a tactical plan to bolster A, B and C rifle companies by making them African American only."

TCO said...

Wait. You wrote a whole article based on what someone told you they heard at a conference? You didn't hear the talk yourself. And you don't have a paper to read or refer to.

Why don't you go that that Dutch fellow's site and read his thesis and his published papers and comment on the those? (The stereotype threat is written about a fair amount). And did you bother contacting the dude and just asking him for a copy of his Powerpoint or have a phone call with him? Prior to writing out this set piece article?

Anonymous said...

Another marketing triumph; take something really old hat and recycle it with a catchy new name: Stereotype Threat! When does the movie about the brave, well meaning white teacher who tackles Stereotype Threat in the classroom, thusly being a deliverance to the minority students, come out? Oh wait, that's been done already? No matter, make it in 3-D, the public will turn out for this message of hope and change.

P Schwartz said...

" TCO said...

Wait. You wrote a whole article based on what someone told you they heard at a conference? You didn't hear the talk yourself. And you don't have a paper to read or refer to."

I know someone who attended the conference and heard the presentation. Steve's account is consistent with what my colleague told me.

Anonymous said...

Haven't Inductivist and the Audacious Epigone looked at GSS data that show blacks think of themselves as actually MORE intelligent than their non-black peers?

Captain Jack Aubrey said...

And just the other day I was arguing with a guy who claimed that conservatives/religious folks were anti-intellectual.

Gee, can't imagine why that (perception) might exist.

I think there may be some truth to the idea that stereotypes affect our views about ourselves and thus our performance in certain ways. I'd wager that to whatever degree it does, though, it affects public behavior more than private. People taught to doubt themselves, for whatever reason, will tend to be less assertive than those told they are smart. Taking a test in a classroom environment does not count as public behavior, however. One does not have to assert oneself to sit and take a test.

Any such stereotype effect may explain individuals; however it is not big enough to explain the broad discrepancies in economic and academic performance in our society. It also doesn't explain the differences between Europe and Sub-Saharan Africa. Remember, the weakness with the "discrimination did it" excuse is that it doesn't just have to explain the economic and educational differences between blacks and whites in the United States - it MUST also explain the consistent disparities between most every white country, on the one hand, and most every black country, on the other.

To do any less is not science.

Svigor said...

Oops. Pantsed again.

Felix said...

“... two Stanford psychologists ... demonstrated that African-American college students did worse on tests of academic ability when they were exposed beforehand to suggestions that they were being judged according to their race"

sounds credible to me. Putting someone under stress tends to reduce their performance

that being said, the issue is whether you've got strategies/relisience to deal with such stresses

as Anon said, "Blacks should worry more about the Stereophonic Threat. Turn off the damn music and hit the books. Do some homework".

Henry Canaday said...

Back up a second from the pseudo-theory of stereotype bias, and think about people. Most young people figure out by high school that they are naturally good at something, test-taking, attracting the opposite sex, bullying the same sex or sports. They tend to try harder at this natural skill, because effort produces pleasurably successful results. They are naturally less successful at other things, so they may put less effort into them, because this excuses or cushions the failure or rejection they have come to expect.

If this is true, then results of a widely given test, like the SAT, will be dispersed more widely than the underlying mental abilities it seeks to measure. The smart kids will try harder, boosting their naturally high results, and slower kids will try less, diminishing naturally lower scores.

If one race possesses, on average, less of the skills measured by a test, they would, on average, be more affected by this ‘diffidence bias.’ The effect would not stem from group stereotypes, inflicted by outside influence. It would simply sum individual experiences and reactions to these experiences. It could account for some overestimate of the gap in innate abilities between races.

I don’t know how to test for this kind of error, but if the possibility occurs to me, it must long have been obvious to SAT preparers. I wonder if they have designed the test, or its scoring, to spot it?

B322 said...

Haven't Inductivist and the Audacious Epigone looked at GSS data that show blacks think of themselves as actually MORE intelligent than their non-black peers?

Indeed. Most people, especially blacks, dramatically overestimate average black intelligence. Blacks also benefit/suffer from unusually high self-esteem.

Jews, on the other hand, have a lackluster reputation for brains, while Jews in particular think Jews are dim. Whatever the effect of Stereotype Threat, it should be hitting Jews hardest.

However, it may not be a Threat at all. Maybe it should be called Stereotype Incentive - the notion that others' negative views of your group gives you a challenge you need to beat. So the blacks rest on their laurels (i.e. TV doctors and computer geniuses being black) while Jews have to beat the opposite (produced by trash-talking Jewish comedians poking fun at themselves?)

It's just a notion - I don't know if it's really true. In any case it is no goofier than Stereotype Threat (which I, incidentally, used to subscribe to, mea culpa).

TCO said...

P Schwartz: That's great and I certainly did not rule that possibility out. And I am happy that a misrepresentation was not spread. My criticism of writing the article on such a basis remains. (Like if you make your overbid in bridge, it does not change that it was the incorrect bid.)

Anonymous said...

I wonder how much 'Settled Science' is simply publication bias? Perhaps there are a number of subjects that have been over looked because of this problem.

Lost Pilgrim

josh said...

Re "san san said':"blacks will get most of the slots" At first read I thought you said,"blacks will get most of the sluts.." ST is related to the previous much beloved idea that blacks(esp males) have low self esteem,and thats why they dont do better in skool.Until science admitted what most observers knew:black males,esp if they can dance,rap,get the "honeys" and fight,have off the charts "sef-esteem'! The poor shmuck who has to keep,literally,two sets of books,so the tough guys wont spot him bringing books home to study,and beat him senseless,THAT guy has some self esteem issue,I bet!!

Mike Courtman said...

"People taught to doubt themselves, for whatever reason, will tend to be less assertive than those told they are smart. Taking a test in a classroom environment does not count as public behavior, however. One does not have to assert oneself to sit and take a test."

True, confidence issues are more likely to be an issue in a field like business than academia, hence the much weaker correlation between IQ and financial success, and IQ and academic achievement.

In Australia and New Zealand there example, there seems to be a big gap between East Asian academic achievement and job success, with many East Asians too introverted for a lot of service jobs.

nooffensebut said...

Great work! I just disagree with the Climategate analogy because the email theft did not uncover any secret conspiracies to hide evidence. This is much more similar to antidepressant-gate.

Mr. Anon said...

The concept of stereotype threat always reminded me of that character from "Mystery Men" - Invisible Boy. He's invisible, but only when no one is looking.

rica said...

Why would stereotype threat matter in written exams where everyone quietly works on his/her own exam and when the exam is often scored by a neutral machine or computer?

It's not like a white guy is looking over the shoulder of a black test-taker to see how he's doing. A black guy is simply one of the many in the classroom, and he quietly work on his own exam. Also, there are many cases where most or majority of the test takers(and test givers)are black. Blacks should feel no stereotype threat in those places but their test scores are low regardless of where or how they take them.

B322 said...

It's not like a white guy is looking over the shoulder of a black test-taker to see how he's doing.... Blacks should feel no stereotype threat in those places but their test scores are low regardless of where or how they take them.

It's important to keep in mind how ad hoc Stereotype Threat is. It was a theory custom-designed to explain low average black test scores in a way that
(a) kept in mind, and explained, why they are so persistent,
(b) made no references to actual cognitive ability, which according to St. Stephen of Cambridge cannot properly be measured and is known to be equal among all races,
(c) made whites look bad, and
(d) would push people to more "anti-racist" action.

The idea is that just hearing the notion that their group does worse on a test either
(a) rattles them or
(b) causes them to conserve energy by not really trying.
(Under no circumstances does it (c) inspire them to beat the stereotype in the sort of way that would make Frederick Douglass proud.)

No actual white scowls or sneers are necessary, or white folks at all. In fact, if all the white folks on earth fell into a well and left the earth to everyone else, Stereotype Threat still COULD hurt blacks scores, those ever-so-hurtful (though not in a way that hurts self-esteem) assertions that "black folks can't do XYZ" would cause black folks to shiver in their boots, or yawn and daydream, so much that their XYZ scores would be artificially lowered.

Briefly: Stereotype Threat is an irrefutable hypothesis.

Truth said...

"but they are smart enough to understand, that while they could do better in school, it does not make much sense to put in all that time and effort simply to improve from "terrible" all the way up to "bad"."

Funny, those are my thoughts about white running backs.

"there is crushing, simply unbelievable "stereotype threat" pressure applied to white athletes doing the "wrong" things in american sports, but it does not seem to affect them much when they are allowed to show what they can do. despite how badly so many people want them to fail, when given an opportunity to do things "that they should leave to the real athletes", capable white athletes still generally perform at a level that their measured ability and previous performance resume suggest they can."

Funny, those were my exact thoughts when I saw Adam Morrison sitting on the Lakers bench last week.

"The poor shmuck who has to keep,literally,two sets of books,so the tough guys wont spot him bringing books home to study,and beat him senseless,THAT guy has some self esteem issue,I bet!!"

Bro, I know we all have "a friend" but you know, the first step to recovery is admission.

Anonymous said...

Truth only acts that way cause he heard it's real hard for blacks to not be snarky.

Statsquatch said...

Well it has been a year and they still have not published the paper.

gwern said...

The PDF link is currently broken. (?)

You can find the PDF at http://web.archive.org/web/20100120032412/http://www.isironline.org/meeting/pdfs/program2009.pdf

Anonymous said...

I remember taking a psychology research methods class back in undergrad. One of my competitors, the most annoyingly loud liberal traitor in the class, had pissed me off by winning the professor's support with his political attitude. For our final project, he conducted research on stereotype threat. He found positive results and won the school contest (one of those elite schools, mind you).

I found a gender difference in my results and received a bad grade, with more than one judge commenting on the offending difference.

After reading this article, I have become convinced that my nemesis invented his entire persona merely for the effect of getting an A. The scary part is that his A took him one step closer than me to becoming a professional scientist. I almost made it.

Maybe that guy made it all the way and is currently publishing his false findings with sociopathic impunity. There are a great many professional scientists with unscientific philosophies, and he would have become yet another one.