The Financial Times has pointed out several data-manipulation errors made by French economist Thomas Piketty in his new bestseller about how the rich get richer. I'm not going to attempt to adjudicate a technical issue.
Some of the problems appear to stem from Piketty building giant models in Excel spreadsheets, which tend to turn into giant hairballs over time. (One unkind theory about Nate Silver's frequent career changes is that his elaborate Excel spreadsheets get too convoluted to maintain, so he's off to something new.) Here's a white paper on "Code and Data for the Social Sciences" with advice on how analysts can learn from professional programmers to make their work more maintainable.
I’m always fascinated by the dynamics of reputation. For example, when Christopher Foote and Christopher Goetz demonstrated in late 2005, a half year after the publication of Freakonomics, that Steven D. Levitt’s most famous theory — that legalizing abortion cut crime — was the result of Levitt’s sloppiness in writing Stata code (not Excel, by the way), the impact on Levitt’s career was negligible. His reputation only started to decline a few years later with the publication of SuperFreakonomics in which Levitt didn’t demonstrate complete fidelity to the climate change orthodoxy.
Basically, there was no money to be made in showing that the facts didn't support Levitt's conclusion. In contrast, there is a lot of money arrayed against Piketty, so we'll hear more about it.
Malcolm Gladwell’s reputation took a major hit from insisting upon tangling with Steven Pinker over a bad NYT review, when Gladwell persisted in defending an overstated article he had written about how it was impossible to predict a college quarterback’s performance in the NFL. But what really hurt Gladwell was misspelling a statistical term (see Dan Quayle's career).
What really makes you unpopular is being right. When Arthur Jensen died a couple of years ago, we had to organize an email campaign to finally get an obituary for him in the New York Times. Jensen's crime was publishing a meta-analysis in 1969 suggesting that the vast social spending to help blacks Close the Gap with whites wasn't going to be very effective. The subsequent 43 years of Jensen's life showed him a prophet, which made him hated.
Conversely, the high repute of Jensen's semi-innumerate critic Stephen Jay Gould is only very slowly being worn down by an inundation of evidence that Gould was, at best, wrongheaded about much.
I just wonder if Gould actually believed the stuff he was spouting or if he saw himself as serving society by kicking the football of truth down field several decades.
ReplyDeleteThe entire case against Piketty should be that he thinks that more immigration raises wages and salaries. That's as dumb as saying that the sun rising on a cloudless morning with no winds will cause the temperature to go down.
ReplyDeleteAll Piketty's screed is is just elect a new people slash hate anyone with two nickels to rub together more than yourself agitprop for the French Socialist Party.
"All Piketty's screed is is just elect a new people slash hate anyone with two nickels to rub together more than yourself agitprop for the French Socialist Party."
ReplyDeleteDunno. I haven't read his book because it's too long and too complicated for me. I would need to have a graduate's level of econ knowledge, but i don't have time to learn that and my undergrad material at the same time.
I highly doubt that the hundreds of thousands of people who bought his book have read it all, or understood the little that they did read.
It's mostly just a status symbol among leftists. I heard one ditsy Australian woman extolling the book on the radio. She sounded like a 13 year old explaining the plot of harry potter.
There's a lot of money arrayed against Piketty's book? Well I presume there's a lot of money pushing the thing, too. Why else would I be hearing so much about it? Maybe Steve knows something about this money war that I don't.
ReplyDeletePiketty's case that mass immigration increases wealth equality boils down to the contention that more people equals more hands in which to spread wealth and income.
ReplyDeleteJust about everyone reading these words are laughing their tails off because the know better.
But even if Piketty was right, so what? Income and wealth inequality is not such big deals to me that I would want to live in a country with the demographics of an international airport just to make things more equal.
most of piketty's research is accurate though, i suspect. the problem is his recommendation for what to do about it.
ReplyDeletethere's zero intelligence involved here. he's not smart, or insightful, or bold, or thinking out of the box. it's raise taxes, raise taxes, raise taxes. every time, every issue, every situation. the answer is always, raise taxes on the wealthy. that was the answer yesterday, it is the answer today, and it will be the answer tomorrow.
tax, tax, tax, then tax some more. it's all liberals know. personally i don't think they really need to spend much money when they campaign, because we already know their platform on every issue. maybe they only need to campaign in the primary, to differentiate themselves from each other, but the pennsylvania primary this month was a hilarious study in how little difference there is between any democrat in 2014. all 4 candidates for governor have the same platform on every issue.
perhaps the only new thing is that they all just state up front, I WILL RAISE TAXES. i promise, i guarantee, i will raise taxes. how they think they are saying anything we didn't already know, or differentiating themselves, is not clear to me, since they don't usually say how high they will raise taxes. like, is one candidate better because she will raise taxes 50% higher than the other guy?
speaking of this, the last good state in the north has now fallen. pennsylvanians held out as long as they could. homosexual marriage enforced, voter ID struck down, state investigations into corrupt african politicans shut down, a cultural marxist democrat is certain to take the governor's office in november, taxes will be raised on energy companies, and the push to bring in mexicans will begin. heck, tom wolf may even get around to raising the state income tax. pennsylvania has the lowest state income tax in the united states, among states which have one. maybe not anymore, after he gets done.
i used to think pennsylvania would still have until november until the homosexuals would win, and wondered whether tom wolf would go for homosexuals first, or raising taxes first. but now the republican governor tom corbett refuses to even battle the homosexuals. wow, we really need to keep voting republican. they are doing so much for us. working so hard for us. republicans are delivering for the historical american nation every day. NOT.
Jensen's crime was publishing a meta-analysis in 1969
ReplyDeleteJensen's 1969 article was a review, not a meta-analysis. 'Meta-analysis' refers to very specific methods that quantitatively summarize a research literature in a particular area.
Piketty's case that mass immigration increases wealth equality boils down to the contention that more people equals more hands in which to spread wealth and income.
ReplyDeleteI've been ignoring every mention of "Piketty" I see online til now, but if that's his case it is a dumb one. Mass immigration does not increase wealth equality, it increases wealth inequality.
NYC and LA actually have the "demographics of an international airport", and they're much more unequal than other parts of the country.
Those massive, un-QA-able, black-box Excel spreadsheets and the "gurus" who build them are a disaster for corporate decision-making. We should have raised the level of professionalism a decade ago, but management doesn't seem to notice.
ReplyDelete"Conversely, the high repute of Jensen's semi-innumerate critic Stephen Jay Gould is only very slowly being worn down by an inundation of evidence that Gould was, at best, wrongheaded about much."
ReplyDeleteAt best? You are being too generous. Gould was a devious marxist creep.
Piketty's book is nothing but a new effort to provide theoretical justification for policies that the left has long been trying to put into practice. The same was true of Keynes's "General Theory," published in 1936 - seven years after the crash of 1929, and after New Deal tax-and-spend policies (or their parallels in other countries) had already been enacted. It should be understood as propaganda rather than analysis.
ReplyDeleteNote that even as the Frenchman Piketty advocates an 80% marginal income tax rate, and a "global wealth tax," France's socialist President Hollande has had to back away from the 75% marginal rate his party enacted upon coming into control of government, because all it has done is to drive successful people from the country. Even as Piketty advocates greater immigration as tending to increase wealth equality, Marine LePen's Front National, with its strong appeal to the français de souche, has had its strongest showing to date in recent local elections. They may be lionized in the American press, but Piketty's ideas are practical failures at home.
Piketty has a lot of money and power backing him, because his prescriptions: tax the "wealthy" (i.e. middle class, Bill Gates and Warren Buffett will never pay one cent in taxes guaranteed, you and I will) and take the money from the "wealthy" middle class and give it to the poor of the world, with a big cut for the new class middlemen, is guaranteed to find enthusiastic takers among Sam Francis's managerial class.
ReplyDeleteWHO will be distributing the money? To pals, cronies, favored people? Why, the Managerial Class of course! The ENArchs, the Oxbridge set, the HYPS folks, etc.
Meanwhile average people will be crushed in poverty. The better to keep power among the elite.
Kudos to the FT for analyzing the thing, which is an admission against their New Class / Managerial Elite interests. The FT is a much, MUCH better paper than the Murdoch-led WSJ Open Borders. Which could not do this sort of analysis.
Second, Excel is the worst tool for big analysis a person could use.
Its easy. Its cheap to learn. And its simple. But for a model that is more complex than a single screen, its worthless.
Far better are REAL statistical tools that actually require competence to use: the free, open source R, with a database like MySQL or MariaDB or Postresql. R will hook into any of those, and you can run vector analysis on a far bigger scale with far less errors than using Excel.
Using Excel for any big analytical job is like using a paring knife for deboning a chicken, chopping vegetables, or carving a turkey. You could do it but you'll hurt yourself.
The reason Excel proliferates in places where it shouldn't is that most of the users have little ability or drive to learn a new tool. R is not magic. Its just a tool. But its more robust for big data sets than Excel that's sure.
Similarly, I have a fascination with belief systems. I mentioned on MR that I've bumped into quite a few people who believe Levitt debunked his critics (you). Even when I send the evidence, they remain convinced Levitt was vindicated. If you are on the right side of popular belief, your reputation can withstand a lot of errors.
ReplyDeletePiketty did his analysis in Excel and people think he's a genius?
ReplyDeletePiketty got tenure at MIT at age 22. He *IS* a genius at a particular kind of theoretical economics.
ReplyDeletePicketty's analysis is partially correct but his remedy is wrong. We need to tax rent-seeking and usury, not general income.
ReplyDeleteIf capital is so valuable, why are the current returns so pathetic? Real interest rates on "risk free" investments are close to zero or less.
ReplyDeleteSince 2000, the return on the S&P 500 was 2.2%. If you include inflation (to get real return), taxes, other frictional costs and then add in dividends -- it is roughly the same or lower.
Large pension funds aren't doing better than average, despite their scale and access to the most expensive advisors.
The 20th century was very good for American capital. At least the part of it that wasn't wiped out in the Depression. In Europe -- not so much. Losing a world war (or even winning one) wiped out capitalists. Same with communism. And the regular financial collapses that are common everywhere else but the US and Switzerland (and its 8 million people).
Capitalism was globally unpopular prior to WW2. It was considered axiomatic that totalitarian government which included centralized planning and state ownership of major industries was inherently much more efficient. And this belief was very common in the US among people of all political beliefs.
Capital markets are brutal on capital. I got the T Shirt.
"giant models in Excel spreadsheets"
ReplyDeleteAfter the passage of Sarbanes Oxley or SARBOX, there was a substantial industry in developing financial control systems. And systems to control those systems.
Excel was always a ripe target. If it was part of a financial control or financial reporting process, it was a problem.
In a sense, it was ironic because as awful as physical, paper spreadsheets were -- they were stable, simple, and understandable. They were huge, messy things but they were dated, initialed, bound in a book or carefully filed work papers, &c.
As soon as PC's got powerful to run humongous Excel spreadsheets -- around 1990 -- there were obvious problems with it. In business, at least.
The assumption persists that Picketty's "more wealth equality" somehow works out to "more wealth." But Picketty may intend the logical result of his prescriptions; he may intend less (overall) wealth.
ReplyDeleteThat poverty and humility are unevenly distributed is the actual beef of the left.
Dane Quaylee spelte thee worde juste likee thee Newe Jerseye Boarde ofe Educatione wantede soe wee shoulde laughe ate Noooooe Joiseyiteses rathere thane Quaylee.
ReplyDeleteR is said to be faster than Python and other languages because it is designed by statisticians for statisticains.
ReplyDeleteExcel is a horrible tool for building large models, or dealing with large datasets. Anyone who is serious about statistics NEEDS to learn at least the basics of R.
ReplyDelete