October 21, 2013

Is "Stereotype Threat" mostly publication bias?

Although the social sciences are considered a bastion of progressivism, it's remarkable how few data-driven ideas they generate in support of their ideology. We can get a feel for this by noting how rare are the "exceptions to the rule" studies that become immensely popular due to bolstering the dominant worldview, such as Hart & Risley's finding that black people don't talk enough and Claude Steele's little study of Stereotype Threat in which he induces black students at Stanford to score lower on a low stakes test of his devising than their high stakes SAT scores would predict. (I wrote about Stereotype Threat in VDARE.com in 2004, suggesting it's not hard to get across the message to black or female students that the professor wants them to not exert themselves fully on this meaningless test. That you can "prime" groups of people to work less hard on an unimportant test does not prove that you know how to make them score higher on an important test.)

Lately, the evidence has been mounting that the existence of Stereotype Threat is quite dependent upon the file drawer function: studies finding its existence are quickly published while studies not finding its existence are in much less demand.

Developmental Psychology
An Examination of Stereotype Threat Effects on Girls' Mathematics Performance 
Colleen M. Ganley, Leigh A. Mingle, Allison M. Ryan, Katherine Ryan, Marina Vasilyeva, and Michelle Perry 
Online First Publication, January 28, 2013. doi: 10.1037/a0031412
... Conclusion 
Taken together, the findings from published research, unpublished articles, and the present studies reveal inconsistency in the effects of stereotype threat on girls’ mathematics performance. The discrepancy in results from published and unpublished studies suggests publication bias, which may create an inaccurate picture of the phenomenon. A recent review suggests that this publication bias may also be an issue in the literature on stereotype threat in adult women (Stoet & Geary, 2012). Overall, these results raise the possibility that stereotype threat may not be the cause of gender differences in mathematics performance prior to college. Although we feel that more nuanced research needs to be done to truly understand whether stereotype threat impacts girls’ mathematics performance, we also believe that too much focus on this one explanation may deter researchers from investigating other key factors that may be involved in gender differences in mathematics performance. For example, there are a number of factors (e.g., mathematics anxiety, mathematics interest, spatial skills; see Ceci & Williams, 2010) that have been shown to be consistently related to mathematics performance and mathematics-and science-related career choices and may warrant more research attention than does stereotype threat.

And here's another study reaching a similar conclusion.

9 comments:

Anonymous said...

Claude Steele. Not Shelby Steele.

XXXXXXXXXXXXX said...

I've been wondering what happened to Jelte M. Wicherts & Cor de Haan's paper on stereotype threat that they were publicizing at the 2009 ISIR conference and which you pointed to in a January 6, 2010 VDARE article[1]. I'd occasionally look at Wicherts' CV page[2] to see if it had been published yet, but it was always undergoing peer-review, until a couple of months ago when it seems to have been removed completely from the CV. What's going on here? Maybe the paper had some serious flaws and never passed peer-review.

[1] http://www.vdare.com/articles/climategate-and-stereotype-threatgate
[2] http://wicherts.socsci.uva.nl/CVJMW.pdf

B.B.

gwern said...

http://andrewgelman.com/2013/07/07/stereotype-threat/

candid_observer said...

If the failure of Stereotype threat to appear in many experiments is, in a major proportion, due to publication bias, then the ability of some of the best known researchers in the field to replicate their findings over and over, against all statistical explanation as publication bias, is evidence, in their cases, of something approaching fraud.

candid_observer said...

Reading my previous comment, I should have said instead something like:

If, in published experiments, the apparent success of Stereotype threat is in major proportion due to publication bias, then the ability of some of the best known researchers in the field to replicate their findings over and over, against all statistical explanation as publication bias, is evidence, in their cases, of something approaching fraud.

Anonymous said...

"then the ability of some of the best known researchers in the field to replicate their findings over and over, against all statistical explanation as publication bias, is evidence, in their cases, of something approaching fraud"

naah, tis but noble lies as Judith Kleinfeld says.

Do these studies control for the effort put in? If girls were spending less time studying maths than boys, you'd be sure to see that statistic instead of such amusing intellectual theories.

peterike said...

This is like the Global Warming Effect. Two imaginary studies.

* Squirrel populations decreasing in Someplace, USA, because of anything but global warming = not published.

* Squirrel populations decreasing in Someplace, USA, because of global warming = published + grant money to continue studying.

It would be interesting to do a wide ranging study of publications and grants in relation to progressive political issues.

tim said...

Labor Market Discrimination and Racial Differences in Premarket Factors Pedro Carneiro James J. Heckman Dimitriy V. Masterov

Finds no evidence of "stereotype threat"

Anonymous said...

There's a strong visual-spatial component to mental arithmetic, at least the way it is taught in schools - the way western mathematical formats lay it out (e.g. below) and the way it is encouraged using indo-arabic numberals. I know this from personal experience and working with my wife - who has no trouble with mathematical abstractions, but does have difficulties with mental arithmetic (until we figured out a way that works for her).

e.g.

_24
+19
___
_43

The carrying operations are strictly spatial, both when laid out on paper and when done in ones head. Analyze your own mental process in solving the above to see this.