Tetlock & Forecasting: Foxes v. Hedgehogs, Three Dog Night v. Ramones

This is the time of year when many pundits make predictions (and a few pundits even assess how their predictions of last year worked out), but not me. Granted, I'm always willing to make some predictions -- Compton will be outperformed by San Marino in school test scores! Switzerland will be a nicer place to live than the Congo! -- but nobody seems to find those very interesting.

Political scientist Philip Tetlock published a well regarded book in 2005 Expert Political Judgment recounting the results of having 284 experts make 83,000 forecasts. 

Something I've never seen discussed in all the writing about Tetlock's study is whether his experts were allowed to pick and choose which questions to answer. Or did they, being famous experts, simply volunteer their opinions on everything he asked them?

Let me quote from Robin Hanson's write-up of it:

Tetlock is Professor of Leadership at UC Berkeley, with a background in psychology and political science.  He ran an experiment for many years from the 80s through the early 2000s in which he invited experts on foreign affairs and political science to make predictions about political and economic trends in countries around the world.  Tetlock tracked and evaluated those predictions, developing a large database of both accurate and inaccurate predictions.  ... 

One of the psychological measures or metrics which Tetlock found was well correlated with expert accuracy goes back to a distinction introduced by Isaiah Berlin in his book, The Hedgehog and the Fox. I haven’t read that book, but based on Tetlock’s presentation, Berlin distinguished between two cognitive styles to which he gave these colorful names.  The hedgehog is said to know one thing and know it well.  He sees events and trends in terms of his big idea, and aggressively extends it into new realms.  Hedgehogs tend to be confident in the applicability of their fundamental concepts and impatient with those who "do not get it". 

Foxes in contrast know many small things which they bring to bear in their analyses in a dynamical and flexible way.  They tend to be uncertain and flexible, "on the other hand" types who are skeptical about their own predictive ability and in fact about the whole enterprise of making predictions in such an intractable realm.

Not surprisingly, the foxes did better on average at predicting foreign policy events on Tetlock's surveys. (Hanson presents Tetlock's results here.) According to Tetlock:

“The less successful forecasters tended to have one big, beautiful idea that they loved to stretch, sometimes to the breaking point. They tended to be articulate and very persuasive as to why their idea explained everything.”

Assume there are two polar routes to becoming a prominent forecaster. One is the fox route where you never have any big new ideas, but you strike people as boringly sensible about a lot of stuff (I associate this path with the name David Gergen, although for all I know he may have been routinely wrong about everything -- he was too boringly sensible sounding for me to pay enough attention). Or you can make your name by being right about one new thing once, then pound it into the ground over time as returns diminish. (Maybe Jack Kemp and cutting taxes).

The foxes may be better about predicting a few dozen events. Yet, from the perspective of history, it's the hedgehogs who matter. 

Consider popular music as an analogy. 

The foxiest band of the rock era may have been Three Dog Night, an agglomeration who had a number of major hits in the late 1960s and early 1970s without trying to develop a distinctive sound. They had three different lead singers, they hired experts to play the instruments, and they seldom wrote their own songs, instead preferring tunes by professional songwriters such as Randy Newman, Laura Nyro, Harry Nillson, and Hoyt Axton.

The rock era was longer on hedgehogs than multifaceted foxes, and perhaps the hedgehoggiest band of them all was The Ramones. Occasionally, they would try a little bit different style, but over a 20 year period, their best stuff sounded like their first single Blitzkrieg Bop, a 1976 song that in the 1990s started to get licensed for commercials, and still makes a fortune in licensing (e.g., in 2012 it became the theme song of NFL Thursday Night Football).

The Ramones didn't have the musical talent of Three Dog Night, but for a little while (July 4, 1976 in London) they really were the future of rock and roll. For example, The Clash saw the Ramones on this tour, and their first single White Riot is pure Ramones. But, by nature, The Clash were foxes, so foxish that the doors flew off in a half decade.

P.S. I see Mr. Rockrobster23 came up with the hedgehog v. fox distinction for the Ramones v. Clash back in 2009.