November 14, 2009

The Igony and the Ecstasy

In the NYT, Steven Pinker reviews Malcolm Gladwell's greatest hits book of New Yorker article reprints, What the Dog Saw:
An eclectic essayist is necessarily a dilettante, which is not in itself a bad thing. But Gladwell frequently holds forth about statistics and psychology, and his lack of technical grounding in these subjects can be jarring. He provides misleading definitions of “homology,” “saggital plane” and “power law” and quotes an expert speaking about an “igon value” (that’s eigenvalue, a basic concept in linear algebra). In the spirit of Gladwell, who likes to give portentous names to his aperçus, I will call this the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.

The banalities come from a gimmick that can be called the Straw We. First Gladwell disarmingly includes himself and the reader in a dubious consensus — for example, that “we” believe that jailing an executive will end corporate malfeasance, or that geniuses are invariably self-made prodigies or that eliminating a risk can make a system 100 percent safe. He then knocks it down with an ambiguous observation, such as that “risks are not easily manageable, accidents are not easily preventable.” As a generic statement, this is true but trite: of course many things can go wrong in a complex system, and of course people sometimes trade off safety for cost and convenience (we don’t drive to work wearing crash helmets in Mack trucks at 10 miles per hour). But as a more substantive claim that accident investigations are meaningless “rituals of reassurance” with no effect on safety, or that people have a “fundamental tendency to compensate for lower risks in one area by taking greater risks in another,” it is demonstrably false.

The problem with Gladwell’s generalizations about prediction is that he never zeroes in on the essence of a statistical problem and instead overinterprets some of its trappings. For example, in many cases of uncertainty, a decision maker has to act on an observation that may be either a signal from a target or noise from a distractor (a blip on a screen may be a missile or static; a blob on an X-ray may be a tumor or a harmless thickening). Improving the ability of your detection technology to discriminate signals from noise is always a good thing, because it lowers the chance you’ll mistake a target for a distractor or vice versa. But given the technology you have, there is an optimal threshold for a decision, which depends on the relative costs of missing a target and issuing a false alarm. By failing to identify this trade-off, Gladwell bamboozles his readers with pseudoparadoxes about the limitations of pictures and the downside of precise information.

Another example of an inherent trade-off in decision-making is the one that pits the accuracy of predictive information against the cost and complexity of acquiring it. Gladwell notes that I.Q. scores, teaching certificates and performance in college athletics are imperfect predictors of professional success. This sets up a “we” who is “used to dealing with prediction problems by going back and looking for better predictors.” Instead, Gladwell argues, “teaching should be open to anyone with a pulse and a college degree — and teachers should be judged after they have started their jobs, not before.”

But this “solution” misses the whole point of assessment, which is not clairvoyance but cost-effectiveness. To hire teachers indiscriminately and judge them on the job is an example of “going back and looking for better predictors”: the first year of a career is being used to predict the remainder. It’s simply the predictor that’s most expensive (in dollars and poorly taught students) along the accuracy-­cost trade-off. Nor does the absurdity of this solution for professional athletics (should every college quarterback play in the N.F.L.?) give Gladwell doubts about his misleading analogy between hiring teachers (where the goal is to weed out the bottom 15 percent) and drafting quarterbacks (where the goal is to discover the sliver of a percentage point at the top).

The common thread in Gladwell’s writing is a kind of populism, which seeks to undermine the ideals of talent, intelligence and analytical prowess in favor of luck, opportunity, experience and intuition. For an apolitical writer like Gladwell, this has the advantage of appealing both to the Horatio Alger right and to the egalitarian left. Unfortunately he wildly overstates his empirical case. It is simply not true that a quarter­back’s rank in the draft is uncorrelated with his success in the pros [see here], that cognitive skills don’t predict a teacher’s effectiveness [see here], that intelligence scores are poorly related to job performance or (the major claim in “Outliers”) that above a minimum I.Q. of 120, higher intelligence does not bring greater intellectual achievements.

The reasoning in “Outliers,” which consists of cherry-picked anecdotes, post-hoc sophistry and false dichotomies, had me gnawing on my Kindle. Fortunately for “What the Dog Saw,” the essay format is a better showcase for Gladwell’s talents, because the constraints of length and editors yield a higher ratio of fact to fancy. Readers have much to learn from Gladwell the journalist and essayist. But when it comes to Gladwell the social scientist, they should watch out for those igon values.

I suspect that on the Big Five personality traits, Gladwell, at least in his writing persona (which isn't necessarily the same as his day to day persona), would score very high on:

- Openness ("Wow, what an amazing idea Professor Frink has! I would never have thought of that in a million years! That's so cool!")

- and Agreeableness ("Now that I've met him, I realize that Professor Frink is a wonderful genius and I must help his insights reach the largest possible audience!")

- but very low on Neuroticism ("Could it be that I'll be making a fool of myself? Will that horrible Sailor person point out some obvious crucial flaw in my exposition of Frinkism and make me a laughing stock again? Should I pause before I publish and apply reality tests to Professor Frink's theory ... Nah! Professor Frink is a wonderful genius! This time I'm clearly not overlooking any problems with the basic idea of my article.")

My published articles are archived at iSteve.com -- Steve Sailer

39 comments:

tommy said...

“igon value” (that’s eigenvalue, a basic concept in linear algebra).

LMAO! Keep a careful eye out for Malcolm's internet acolytes to see if they start discussing the uses of "igon values" in the near future.

Theo Meier said...

Yet another German phrase Mr Gladwell should watch out for: igon willig.

Stopped Clock said...

Are you seriously saying Gladwell didn't know how to spell eigenvalue and neither did his proofreaders?

(If this is a duplicate post, don't post it; there was an error after I hit Submit a minute ago.)

Anonymous said...

Nice citations in the NYT Steve! Of course, it's good that the estimable Prof. Pinker has tenure already if he's going to link to you. I read "The Blank Slate" only after reading about here a few years ago, and I can't recommend it enough.

David
Irvine, CA

fbj said...

Gladwell is to sociology and statistics what Deepak Chopra is to medicine...

Anonymous said...

You know you spelled your last name wrong at the end of your post, right?

John Seiler said...

What happened to the fabled New Yorker fact checkers?

And doesn't the book publisher, Little, Brown and Company, do any editing?

I edit books, and immediately would have Googled "Igon Value Problem." I just did, and quickly found an entry from Sept. 3, 2003, posted by "DennisS" on Brad DeLong's blog:

"One of my favorite journalistic gaffes is 'igon value' from one of Malcolm Gladwell's New Yorker pieces. I guess he and his editors never took a single linear algebra course. (I actually really dig Gladwell's pieces in general)."

That was 6 years ago. Maybe Steve's next piece could graph the decline in editing in journalism with the decline in the U.S. economy.

Anonymous said...

" or (the major claim in “Outliers”) that above a minimum I.Q. of 120, higher intelligence does not bring greater intellectual achievements."

Can anyone elaborate on Pinker's claim here?

I haven't heard this before. Charles Murray also claims that 120 is the bare minimum for notable achievement in society. The thinking is that once you are above 120, especially a Verbal IQ, it is a much more even playing field for success in things like business, arts, etc.

Another writer, Dean Simonton, also says this. In his book "Greatness" Simonton chalks up "Greatness" to above average IQ (120 +) coupled with the utmost in will and determination. Simonton thinks that the "X" factor is not super IQ, or super talent, but rather 120 IQ + Talent + irrational driving determination, the last being the true X factor.

Anonymous said...

steve, what movie are you watching this weekend to review?

Collapse?

Scrutineer said...

the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.

A stupid man's report of what a clever man says is never accurate because he unconsciously translates what he hears into something he can understand.

- Bertrand Russell, The History of Western Philosophy

SF said...

Aside from the military, the Census bureau may be the only federal agency that uses an intelligence test for hiring. You can take it by googling on Census Practice Test. The practice test is almost the same as the real one. They can probably get away with it because they are almost an ad-hoc organization. They set up, get the data, and are ready to disband in less than the time it takes a class action suit to get a court verdict.

CP said...

Hilarious.

Steve,

Do a post on: Who is a bigger joker, Gladwell or Tom Friedman?

It's a close call.

CP said...

By the way, I read in this month's Fortune that DuPont had Tom Friedman there to speak.

(For a five figure sum, no doubt.)

Having Tom Friedman speak at your company should be like hearing an iceberg gouge into the side of your ocean-liner. Get to a lifeboat.

Anonymous said...

when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.

And yet Tom Friedman has made a fine career out of doing exactly that!

Anonymous said...

The igon value thing is stunning. I think I first learned about eigenvalues when I was 15 or 16. And it's the freaking New Yorker, not Parade Magazine! And he's sold millions of books.

Which reminds me:

Steve, there are always other people who are in the same boat with you on this one. Imagine what good prose stylists feel about J. K. Rowling and Dan Brown or what good musicians feel about Britney Spears. The mass market is where it's at and the average IQ in this country is a little below 100.

Anonymous said...

the idea that an iq of 120 is sufficient for elite achievement is laughable.

you could simply measure the IQ of a representative sample of the intellectually accomplished, but I suppose it could be argued that such an analysis begs the question because elite schools and graduate schools screen for IQ above 120, thus artificially inflating the IQ "necessary" for achievement. That may sound absurd, but it's the only response if empirical data shows that physics and philosophy professors at Harvard have IQs well above 120.

Such a response does raise the question of why schools would screen out people with IQ around 120 if that group includes many capable of accomplishment at the highest level. But we shouldn't simply make the glib free market argument that if schools wrongly prioritize 120+ IQ they will lose out to schools that do not. It's certainly possible that colleges and graduate programs use poor criteria, or that they use the best criteria possible where other relevant information is unavailable.

So we're being as charitable as possible to the 120 thesis. I don't think it's difficult to devise tests for the hypothesis that IQ stops mattering at 120. All we need to do is measure the IQ of people in graduate programs and see if it correlates with PhD completion, production of important research, measures of impact like citations, eminence, etc.

You could do the same thing for undergraduates--see if IQ correlates with grades within majors/institutions.

Law is also fertile ground for testing the difference between 120 and 120+.

Law school admissions are heavily based on a pure reasoning test, the LSAT. These days you need above a 170 to be competitive for the best schools.


I'd be astounded to discover that there is no difference between an IQ of 120 and 135 in terms of getting a 175 on the LSAT.


You could argue this is begging the question, as was discussed earlier (ie, that law schools irrationally screen for IQs above 120) but that's an empirical question we can answer: LSAT scores do correlate with grades, even though the LSAT range at almost any school is very tightly clustered (probably within the margin of error). And the results of such an analysis at any school with a wide LSAT range would be crushingly dispositive.

This provides interesting evidence: http://infoproc.blogspot.com/2009/01/horsepower-matters-psychometrics-works.html

"SAT-M quartile within top 1 percent predicts future scientific success, even when the testing is done at age 13. The top quartile clearly outperforms the lower quartiles. These results strongly refute the "IQ above 120 doesn't matter" claim, at least in fields like science and engineering; everyone in this sample is above 120 and the top quartile are at the 1 in 10,000 level. The data comes from the Study of Mathematically Precocious Youth (SMPY), a planned 50-year longitudinal study of intellectual talent."

Executive summary: of course 135 is much better than 120 in the upper echelon of intellectually demanding fields (great programmers, physicists, philosophers, supreme court clerks, etc.)

Anonymous said...

SF, your rosy views of the world enchant me :-)

I took the census test in Vegas about 8 months ago. The room was packed and apparently it was similarly packed every other day for a couple of months. The nice fat lady who introduced the test to us told us that:

"People will be hired for census jobs based on how well you do on this test, with the highest scorers being called first and then the second highest scorers and so on. But don't worry too much because to qualify, you only need to get ten questions right and it's not as though many other people are doing all that much better. in the years that I've been proctoring these tests, of the thousands of people who have taken them, only 12 - twelve - ever got all of the questions right. So don't worry if you don't know something and just enjoy the experience and do your best. And remember, if you don't do well enough to qualify of if you're just not satisfied with how well you did, you can always take the test again, up to four times. And I see that some of you are familiar faces who have already been here before. Okay, good luck and open your test pamphlets."

I graded a perfect score but was disqualified for writing in the test pamphlet rather than on the scrap paper.

I'm sure they would have disqualified a perfect-scoring black woman for the same offense. Rules are rules.

mnuez

Harry Baldwin said...

"A stupid man's report of what a clever man says is never accurate because he unconsciously translates what he hears into something he can understand."

That's a great line. It relates to my experience in (delicately) raising HBD issues with supposedly intelligent people. They immediately translate it to "so you're saying black people are inferior," or something equally simplistic.

Black Sea said...

"Gladwell argues, 'teaching should be open to anyone with a pulse and a college degree . . ."

If you have an M.Ed, you don't need the pulse.

Middletown Girl said...

"The thinking is that once you are above 120, especially a Verbal IQ, it is a much more even playing field for success in things like business, arts, etc."

But certainly not in nuclear physics or genetics or serious literature or satire of the highest order or supergreat music. I heard Mozart's IQ was around 220.

Simon said...

There are certain 'elite' applied professions like the practice of Law, where I wouldn't be surprised if IQ correlates only weakly with success above a certain value, probably 125 or so.

For the actual hard stuff though like the real sciences, it's clearly codswallop. The great achievers seem to be IQ 150-160+.

SF said...

". . . disqualified for writing in the test pamphlet rather than on the scrap paper."

Hey, it's a test on knowing how to do things by the standard protocol, which is probably reasonable for that job.

Anonymous said...

SF, all kidding aside, I wonder what the optimal amount of independant thinking is considered positive for various beaurocratic jobs. At airports I always joke with people abou the stone cold rote stupidity of the TSA agents but I also recognize that we're all that much safer because the TSA people are NOT thinkers instead of followers. The fact that they're generally such epsilons that they CAN'T understand nuance may be less important than the fact they know that they're epsilons and thus believe in the rules as in graven images and therefore follow them whether they're rational or not.

In terms of payoff then, I wonder whether there's some mathematical formula that could give you a rough estimation of how many parts "independant intelligent analyzer" are optimal in the mix of majority "dumb block rule-followers".

Meredith said...

Nice citations in the NYT Steve! Of course, it's good that the estimable Prof. Pinker has tenure already if he's going to link to you.

Those links to iSteve were added by Steve not Pinker. As indicated by the use of [...].

In polite company, Pinker prefers that Steve remains below stairs.

meep said...

Yeah, reminds me of explaining to a room full of middle school students [and their math teachers] the proper pronunciation of "Euler". Then I told them they were in the "math insider club". [joking, of course]

I remember Gladwell writing a piece on GM pensions/Social Security that was so full of innumeracy, I wrote to the editors. Doubt my letter [or the letters from other actuaries] was ever run.

As to my own personality traits, I'm extremely low on neuroticism, high on conscientiousness and openness, low on agreeableness, mid for extroversion [I think... I'm all over the place on that, depending on the test]. I would think a lot of those in the "Steve-o-sphere" are low on agreeableness.

Anonymous said...

I would think a lot of those in the "Steve-o-sphere" are low on agreeableness.

WTF is agreeableness?

Deogolwulf said...

Still, at least Mr Gladwell is not beyond parody.

William B Swift said...

>hiring teachers (where the goal is to weed out the bottom 15 percent)

Didn't you mean the top 15%?
I've always liked Sowell's observation from "Inside American Education" that getting a degree in education is a negative ability test, that drives competent people out of the "profession".

Anonymous said...

There are a couple ways in which the concept of a threshold is used in psychometrics.

One way is to translate a continuous quantitative variable into a a qualitative classification. Every teacher understands this process. You have a test on which students can get a score from zero to one hundred. But the school system wants letter grades. You therefore arbitrarily define getting a 'B' for example as a score over the "threshold " of 80. The threshold for an 'A' might be 90.

The more profound use of the term threshold in ststistics is when it describes a non-linearity. Often this comes from using an easy to measure dependent variable instead of a harder to measure but correlated dependant variable. For example consider home run hitting. "Big Mac" (Mark McQuire) is big but not gigantic. When I go to the ball park I routinely see plenty of guys in the stands who are bigger. Why are they in the stands and not in the lineup? Obviously because they can't hit home runs, like Big Mac.

So does that mean that size and strength are irrelevant to home run hitting? Hardly. There are lots of small shortstops and utility infielders but no small home run hitters. Among those guys who can actually hit a major league fastball or curve there is a size threshold. Let's say it's 180 pounds. That means there will be no 130 pound infielders hitting home runs baut it also means that a guy who is 280 or 380 won't hit more as his weight increases. Size does't really matter in home run hitting - as long as you are big enough (above the threshold).

Home run ability is really two abilities: hitting the ball and hitting hard enough to fly 400 feet.

IQ is popular as an explanatory variable not only because it appears everywhere but also because it's so easy to measure. It should of course be called 'g' (for general) not IQ. Binet thought of intelleigence as a quotient because he was trying to assess youthful intelligence versus mature intelligence. Spearman's 'g' came from discovering a statistical factor that was present in so many different ability tests.

Although almost everyting seems to correlate with 'g' there are some exceptions. The most famous is probably musical ability which correlates poorly with intelligence. That's why Mozrt did not have or need an IQ of 220. It is also why so many professional muscians are so stupid. The more fundamantal reason is that musical ability seems to be a function of the posterior superior gyrus and 'g' seems to be a frontal lobe function.

Speaking of Dan Quayle, spelling ability is not particularly correlated with IQ but vocabulary is. So if you know what a word means - you're smart. If you can spell it - you aren't. That seems crazy. But it's true.

The most famous IQ threshold effect has long been in creativity. It was widely recognized that among scientists the most successful were the the ones who were smartest. But among painters for example the best were not noticeably smarter than the worse. However there are no really stupid creative artists. So it was said that creativity was independent of IQ but only above some critical threshold - about 120. That's the theory and it seems to be correspond well enough with everyday experience.

Gladwell seems to have just appropriated the threshold idea into his world view. I don't think he actually made any measurements.

dearieme said...

"I heard Mozart's IQ was around 220". I heard that Mozart died long before IQ tests were invented.

meep said...

=snort=

Anonymous said...

"It relates to my experience in (delicately) raising HBD issues with supposedly intelligent people. They immediately translate it to "so you're saying black people are inferior," or something equally simplistic."

I once stated a basic hbd point to a supposedly intelligent friend and he said, "So how do you explain Barack Obama?"

Most people don't seen to be able to handle generalities. Can this be taught?

Anonymous said...

Yes, but Gladwell is The Love Guru!

The Love Guru

Svigor said...

I once stated a basic hbd point to a supposedly intelligent friend and he said, "So how do you explain Barack Obama?"

A good reply might be "the same way I explain an intelligent person like yourself occasionally asking stupid questions like that."

Anonymous said...

I once stated a basic hbd point to a supposedly intelligent friend and he said, "So how do you explain Barack Obama?"

William Ayers, Jon Favreau, and David Axelrod.

M Stein said...

"So how do you explain Barack Obama?"

What an idiot - it's like saying how do you explain Yao Ming.

Steven Goldberg explained the distribution point well here:

"Those who deny the reality of race will often invoke the fact that, whatever the characteristic in question, the range is greater within race than between races. This is true of nearly any variable for which two groups are compared. But to deny a statistical group difference on this basis would force one to claim that it is meaningless to speak of "men" and "women," or statistical differences between them, because the height difference between the shortest man and the tallest man, or between the shortest woman and tallest woman, is far greater than the few-percent difference between the mean heights of men and women.

This example makes clear the key fact that a small difference in means often complements a huge difference at the extremes; how many seven-foot tall women does one see? The difference in running speed between the average white and average black male is only a few percent, but virtually all of the two hundred fastest men in the world are black. And it is on the upper tail of the curve—the extreme—that public perceptions—stereotypes—are based. That this "within-group" argument is so often made is a measure of the desperation of those who wish to deny that which is undeniable."

http://www.vdare.com/misc/060221_goldberg.htm

Hsu also explains it:

"On the other hand, for most phenotypes (examples: height or IQ, which are both fairly heritable, except in cases of extreme environmental deprivation), there is significant overlap between different population distributions. That is, Swedes might be taller than Vietnamese on average, but the range of heights within each group is larger than the difference in the averages. Nevertheless, at the tails of the distribution one would find very large discrepancies: for example the percentage of the Swedish population that is over 2 meters tall (6"7) might be 5 or 10 times as large as the percentage of the Vietnamese population. If two groups differed by, say, 10 points in average IQ (2/3 of a standard deviation), the respective distributions would overlap quite a bit (more in-group than between-group variation), but the fraction of people with IQ above some threshold (e.g., >140) would be radically different."

http://infoproc.blogspot.com/2008/01/no-scientific-basis-for-race.html

Howard Hughes said...

"Speaking of Dan Quayle, spelling ability is not particularly correlated with IQ but vocabulary is."
Would really like to see some proof of that claim. And what exactly do you mean by "not particularly"? Surely, it has some correlation with IQ.

Anonymous said...

> spelling ability is not particularly correlated with IQ but vocabulary is. So if you know what a word means - you're smart. If you can spell it - you aren't. <

Nonsence.

Truth said...

"I heard Mozart's IQ was around 220"

That's highly impressive, I mean, considering the Flynn Effect, the average white guy's IQ must have been like 11 back then.