The most celebrated nonfiction book of the year is "Freakonomics" by U. of Chicago superstar economist Steven D. Levitt and journalist Stephen J. Dubner. The most admired aspect of the book has been Levitt's theory that legalizing abortion cut the crime rate. Now, it turns out, according to two economists at the Boston Fed who have checked Levitt's calculations in detail, that Levitt's theory is based on two mistakes Levitt made. So far, Levitt admits to making one error, saying it "is personally quite embarrassing."
This fiasco reveals much about what's wrong with public policy discourse in modern America. Fifteen minutes of Googling would have shown that the abortion-cut-crime theory hadn't come close to meeting the burden of proof, but, instead, much of America's intellectual elite fell head over heels for it.
Ever since my 1999 debate with Levitt in Slate.com, Levitt's fans have been telling me that my simpleminded little graphs and ratios of national-level crime trends showing, for example, that the teen homicide rate tripled in the first cohort born after Roe v. Wade couldn't possibly be right because Levitt's state-level analysis was so much more gloriously, glamorously, incomprehensibly complicated than mine, and Occam's Butterknife says that the guy with the most convoluted argument wins. Now, two economists have finally redone Levitt's work and found two fatal mistakes in it. The WSJ reports:
'Freakonomics' Abortion Research Is Faulted by a Pair of Economists
By JON E. HILSENRATH
Staff Reporter of THE WALL STREET JOURNAL
November 28, 2005; Page A2
Prepare to be second-guessed.
That would have been useful advice for Steven Levitt, the University of Chicago economist and author of the smash-hit book "Freakonomics," which uses statistics to explore the hidden truths of everything from corruption in sumo wrestling to the dangers of owning a swimming pool.
The book's neon-orange cover title advises readers to "prepare to be dazzled," and its sales have lived up to the hype. A million copies of the book are in print. The book, which was written with New York Times writer Stephen Dubner, has been on the New York Times best-seller list for 31 weeks and is atop The Wall Street Journal's list of bestsellers in the business category.
But now economists at the Federal Reserve Bank of Boston are taking aim at the statistics behind one of Mr. Levitt's most controversial chapters. Mr. Levitt asserts there is a link between the legalization of abortion in the early 1970s and the drop in crime rates in the 1990s. Christopher Foote, a senior economist at the Boston Fed, and Christopher Goetz, a research assistant, say the research behind that conclusion is faulty.
Long before he became a best-selling author, Mr. Levitt, 38 years old, had established a reputation among economists as a careful researcher who produced first-rate statistical studies on surprising subjects. In 2003, the American Economic Association named him the nation's best economist under 40, one of the most prestigious distinctions in the field. His abortion research was published in 2001 in the Quarterly Journal of Economics, an academic journal. (He was the subject of a page-one Wall Street Journal story1 in the same year.)
The "Freakonomics" chapter on abortion grew out of statistical studies Mr. Levitt and a co-author, Yale Law School Prof. John Donohue, conducted on the subject. The theory: Unwanted children are more likely to become troubled adolescents, prone to crime and drug use, than are wanted children. When abortion was legalized in the 1970s, a whole generation of unwanted births were averted, leading to a drop in crime nearly two decades later when this phantom generation would have come of age.
The Boston Fed's Mr. Foote says he spotted a missing formula in the programming of Mr. Levitt's original research. He argues the programming oversight made it difficult to pick up other factors that might have influenced crime rates during the 1980s and 1990s, like the crack wave that waxed and waned during that period. He also argues that in producing the research, Mr. Levitt should have counted arrests on a per-capita basis. Instead, he counted overall arrests. After he adjusted for both factors, Mr. Foote says, the abortion effect disappeared. [Emphasis mine.]
"There are no statistical grounds for believing that the hypothetical youths who were aborted as fetuses would have been more likely to commit crimes had they reached maturity than the actual youths who developed from fetuses and carried to term," the authors assert in the report. Their work doesn't represent an official view of the Fed.
Mr. Foote, 40, taught in Harvard's economics department between 1996 and 2002; served stints as an economist on the Council of Economic Advisers in 1994, 1995, 2002 and 2003; and served as an economic adviser to the Coalition Provisional Authority in Baghdad, Iraq, in 2003 and 2004.
Mr. Levitt counters that Mr. Foote is looking only at a narrow subset of his overall work on abortion and crime, so his results are of limited value, and not grounds for dismissing the whole theory. He acknowledges the programming error, but says taken by itself, that error doesn't put much of a dent in his work. (Mr. Foote's result depends on changing that formula and on the adjustment for per-capita arrests.) Moreover, Mr. Levitt says the abortion theory has held up when examined in other countries, like Canada and Australia, and when applied to other subjects, like drug use.
"Does this change my mind on the issue? Absolutely not," Mr. Levitt says. [More]
Levitt and Donohue's abortion-cut-crime theory was put together and tested in a quick and dirty fashion in late 1998, and when their draft paper leaked to the Chicago Tribune in August 1999, they hadn't yet done the needed reality checks on their idea.
In our August 1999 debate in Slate, I pointed out to Levitt that the national-level data easily available on federal government websites showed that his theory had radically failed the test of history: the first cohort born after the legalization of abortion had a homicide rate as 14-17 year olds triple that of the last cohort born before legalization.
Rather than damage his nascent career by expressing doubts about his signature theory, Levitt decided to dig his heels in and rely on his extremely complicated state-level analyses to buffalo people into ignoring my easy-to-understand national-level analyses that raised serious doubts about whether he'd come anywhere near meeting the burden of proof.
Hey, it worked. He's now rich and famous.
I told Levitt last month during the Bill Bennett Brouhaha, in which the former Education Secretary was widely denounced for making a reductio ad absurdum argument based on the racial aspect of Levitt's theory, that he should just walk away now from his most famous theory -- just admit that it's too hard to tell what actually happened. He's now a celebrity so he hardly needs this theory anymore to go on being a celebrity. Otherwise, someday, some little-known economist was going to make his reputation by taking the Freakonomist down. Well, Levitt's nemesis has arrived.
A reader writes:
Will this really matter? I guess I have my doubts. We have moved into an era when facts matter less than myths.
Indeed. Virtually nobody will admit they were wrong about this. Way too many important people have too much invested in Levitt's celebrity. This is a fiasco for the economics profession -- the most famous young economist's most famous theory has been exposed after six years of adulation as being based on something approaching malpractice -- but the likelihood that the economics profession will stage an inquiry must range between zero and negative infinity. I wonder how many economics professors have book proposals in right now for that next bestseller "Berserkonomics"? (By the way, Levitt and Dubner are working on a sequel with a title that reflects their characteristic elegant taste: Superfreakonomics.)
In the general media as well, too many influential people publicly endorsed the theory when a small amount of due diligence with Google would have shown them it was deeply dubious. And too many people want his abortion-cut-crime theory to be true for personal or political reasons. I've noticed, for example, that in online discussions, pro-lifers tend to want Levitt's theory to be true. They appear to want to be able to boast, "Even though legal abortion reduces the likelihood of me being a victim of crime, I'm still against it. That's how idealistic I am."
Here's the abstract of Foote and Goetz's paper:
Testing Economic Hypotheses with State-Level Data: A Comment on Donohue and Levitt (2001) [PDF - full paper]
Working Paper 05-15
by Christopher L. Foote and Christopher F. Goetz
State-level data are often used in the empirical research of both macroeconomists and microeconomists. Using data that follows states over time allows economists to hold constant a host of potentially confounding factors that might contaminate an assignment of cause and effect. A good example is a fascinating paper by Donohue and Levitt (2001, henceforth DL), which purports to show that hypothetical individuals resulting from aborted fetuses, had they been born and developed into youths, would have been more likely to commit crimes than youths resulting from fetuses carried to term. We revisit that paper, showing that the actual implementation of DL’s statistical test in their paper differed from what was described. (Specifically, controls for state-year effects were left out of their regression model.) We show that when DL’s key test is run as described and augmented with state-level population data, evidence for higher per capita criminal propensities among the youths who would have developed, had they not been aborted as fetuses, vanishes. Two lessons for empirical researchers are, first, that controls may impact results in ways that are hard to predict, and second, that these controls are probably not powerful enough to compensate for the omission of a key variable in the regression model. (Data and programs to support this comment are available on the web site of the Federal Reserve Bank of Boston.)
Levitt's reply on his Freakonomics blog is here.
All my posting on this issue are at http://www.iSteve.com/abortion.htm
***Permalink***
My published articles are archived at iSteve.com -- Steve Sailer
Posts
No comments:
Post a Comment