Much of the recent psychological research also suggests that overconfidence is our main cognitive problem, not the reverse. Daniel Kahneman’s book “Thinking, Fast and Slow” describes an exhaustive collection of experiments demonstrating how often people come to conclusions confidently and wrongly. When asked to estimate if more murders happen in Detroit or in Michigan, most people give higher estimates for Detroit even though every murder in Detroit also happens in Michigan.
It didn’t take me long to figure out that, in a not so roundabout way, Kahneman and Tversky had made my baseball story [Moneyball] possible. In a collaboration that lasted 15 years and involved an extraordinary number of strange and inventive experiments, they had demonstrated how essentially irrational human beings can be. In 1983—to take just one of dozens of examples—they had created a brief description of an imaginary character they named “Linda.” “Linda is thirty-one years old, single, outspoken, and very bright,” they wrote. “She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.”
Then they went around asking people the same question:
Which alternative is more probable?
(1) Linda is a bank teller.
(2) Linda is a bank teller and is active in the feminist movement.
The vast majority—roughly 85 percent—of the people they asked opted for No. 2, even though No. 2 is logically impossible. (If No. 2 is true, so is No. 1.) The human mind is so wedded to stereotypes and so distracted by vivid descriptions that it will seize upon them, even when they defy logic, rather than upon truly relevant facts. Kahneman and Tversky called this logical error the “conjunction fallacy.”
This is not to say that Kahneman possesses the self-awareness of a Boy Scout patrol leader hazing the Tenderfeet. Judging from his book, he doesn't really seem to be aware that his questions come out of a long tradition of smart-aleckry and Aspergerness.