"Needless to say, Taleb is right than making an accurate forecast is difficult, especially about the future."
Are you trying to be cute? It's not obvious enough that you didn't have a Bushism there.
Also, having the "Look Inside" arrow over the books looks really tacky. VDare probably makes money if we buy the books from Amazon but can't you scan the covers or something rather copying and pasting?
Steve, you missed the whole point of Taleb's book and the title.
It was not an attack on bell curve models, but rather built-in mental models of how the world works or data is distributed.
The reason he used Black Swans was because they were not rare. Prior to Australia being discovered, the only swans known, AT ALL, were White. There were no known Blacks Swans to Europeans.
That had really no impact on European assumptions, but that was because the data sampling itself being skewed did not distort anything but a meaningless picture of what Swans look like (i.e. Europeans thought they were all White all the time).
If you are making predictions about the likelihood of a terrorist nuclear attack, or earthquakes, or plagues, or anything else where data is hard to come by, there is a large amount of uncertainty about the data, and other questions, you can't really say much.
Your experience about marketing data biases you because the problem about being wrong with P&G brands perception by US consumers is pretty marginal. You won't lose a city of seven million people. You can and will if you bet wrong on say, bio-terror or nuclear weapons by terrorists because you don't know everything, and even more telling YOU DON'T KNOW WHAT THAT YOU ARE MISSING DATA.
That's why the Black Swans. It's about data. If you don't know you've not sample all the data, you can't make predictive models. There was almost no chance you'd miss important data about well known US populations with P&G.
There's a heck of a lot of chance you could bet wrong on any number of things where built-in assumptions about how data "looks" in the universe are skewed by local assumptions.
For example, if you are Ahmadinejad, your risk profile, or that of various factional leaders in Pakistan, or Saudi Arabia, vary quite a great deal from European leaders, who in turn vary from guys like Putin, who in turn have different risk profiles than Hu Jintao.
The difference between the lists which follow the normal distribution and those that have a skewed distribution is that the skewed distributions are based on human actions and human perceptions. The normal distribution are more measurements of the physical world.
- Are we ever going to stop hearing about Hitler being dredged up in the latest fad, book or mathematical theory?
- My impression from your blog was that the housing bubble was not even mathematically planned; it was just a giant swindle where eventual failure was expected. The game in town was that the smartest shakers would sell off their risks to local and foreign suckers and get out quick enough. Did I miss something?
At the same time (without having read the book) Taleb is confronting confirmation bias, where people don’t challenge their mental models enough, and confuse non-rejection of hypothesis with confirmation of hypothesis.
People tend to have representativeness bias, and think too much in terms of examples that come easily to mind. Taleb and Gladwell might be encouraging this, although I suspect any book that uses examples to illustrate general points risks idiots only remembering the examples.
Ps.
Both income and consumption tend to be log-normally distributed, fairly predictable, and allow for a few super-rich.
Taleb's criticisms are frequently inconsistent, but the basic point is as Steve notes: outliers are more important than most everyone realizes. But people fixate on extremes too much, not too little. In his own domain, finance, out-of-the money options are the worst investments (to own), because the implied volatilities are 'too high'. See here for another critical review.
However, the repeated use of No Child Left Behind as a stock example of self-deluding utopianism is too simplistic and demands a response.
It may very well be that IQ realities place an upper bound on how much inner-city schools can improve.
However, this bound is so far above the present performance of average inner-city schools that blaming IQ for low performance and the inevitable failure of NCLB really misses the point.
Culture and institutions can have a huge effect too, as this very website has observed in the case of Barbados.
In the case of public education of poor NAM students, there are any number of charter schools that get excellent results, using the same low-IQ "raw material" as failing public schools. A few are listed below.
These examples need to be reckoned with, before one can use NCLB as an all-purpose negative example.
YES College Prep (Houston) http://www.yesprep.org/about/results.htm
Match charter school (Boston) http://www.matchschool.org/about/results.htm
The point Taleb makes that was new to me was not that outliers exist - most people know that. It was that outliers can be and often are more important than the general tendency. His example is that casinos pay close attention to the odds of someone winning big at their games but less attention to less obvious risks, like the white tiger attack that cost them millions when Siegfried and Roy could not perform for months. So we may carefully put away money every month in mutual funds for years but then lose all those earnings when a 9-11 or subprime mortgage crisis crashes the market.
I think these black swans are more important than even Taleb says. The very Puritan notions of the nobility of thrift and hard work would seem to be attacked at the foundations. What counts is being able to deal with rare and momentous developments.
My problem is the unpredictable nature of such losses and Taleb's inability to provide a usable strategy outside the financial markets to guard against such risks. I think that's why he always looks so nervous.
testing99, the guy across the street from your home looks suspicious. This is a city of many people. We're missing data about that man - and we can't afford to wager mistakenly.
BEAT THAT MAN DOWN AND PROBE HIM - he may have a loose nuke you-know-where. At the very least, let's collect all his data: his whereabouts since birth, his political opinions (search his home and hard drives regularly), his statements, his associates past and present, his lovers, his employment, his blood type, his DNA, his retinal pattern, his fingerprints and dental impression.
We can't afford to bet wrong about an undetected terrorist. After all, this is a city of seven million people and we live in a world of loose nukes and even looser antisemites.
Yes, he looks very suspicious... But you'll excuse me if I don't help you to learn all about him, because I see a strange man outside my own home...he is wearing a long white coat and is hurrying up the driveway, carrying a big net... More later. Stand by on this frequency. Jack Bauer out!
PS: Don't look now, but under your bed there is a...?
There's another big difference between Mediocristan and Extremistan: Networks. The bell curve assumption works most often because of the central limit theorem which suggests that coupling many "independent" non-normal distributions will eventually give us a normal distribution. (Entropy's inevitable increase). The key here is independence, and that is violated in the stock market and in the economy, and in human networks in general. Even in the brain, networks give rise to neurons that fire a lot more often because of a winner-take-all effect that results in virtuous cycle. Globalization and the fast flow of information make us more networked and less independent agents. What is to say that the neighbors of a few guys who just got laid off won't cut spending because of fear and risk aversion?
Most people want to believe in their uniqueness -- not that they are just one more dot on a bell curve. Most people want to believe that they can raise children a certain way and ensure their success or goodness -- not that their children will have determined strengths and weaknesses. As science proceeds apace, the averted gaze will grow more insistent.
There is a dreary inevitability about t99's pronouncements.
"Ah hell, the terrorists are probably going to nuke New York harbor in response to our sticking our nose everywhere and inviting the whole world in...but, what are ya gonna do--not stick your nose in everywhere and not invite the whole effing world in?"
Or women:
"White beta males are doomed. It's a world of single moms married to the government!" [Promote an orthodox Christian ethos instead of secular bureaucratism? Shrink government instead of pandering to everybody? Stop shovelling money to cities that should just go broke?]
I'm reminded of his panicked response to the McCain-Palin nose dive. As if he just now found out that expensive, open-ended wars are unpopular and the Republicans have undercut their own demographic majority and geriatric cancer patients don't make very good candidates.
The guy has trenchant observations, but they're always made from the second phase of the cycle.
For all the flogging testing99 gets about nuclear attacks, he is right, we are going to get one. Of course, nobody is allowed to actually do anything about this fact that would actually preventing an attack from happening, like admitting islam is a religion of terror and that practitioners of it live here and want to do us harm and will help out jihadis from abroad(like they did on 9-11 and the first WTC attacks). We have a 5th column in the nation, but nothing is really done about it...
Anon sed: "For all the flogging testing99 gets about nuclear attacks, he is right, we are going to get one."
Steve was first in line about profiling Arabs. And he never suggested selling off ports to Arab nations. Withdrawing form non-crucial states also helps since very interaction with others is another open door. Everything Steve writes would lead to more security and protection against this type of attack. Running around the world with guns blazing just opens unnecessary doors. It also means lots of innocent people get killed and thus results in moral guilt, something otherwise reserved for losers like Germans and Afrikaners.
I also never read Steve complaining about building anti-ballistic systems, which in addition to tightened Coast Guard and Port/Airport security is the best long-term solution. Basically everything Steve says leads to more security.
Lots of little hints across the culture recently about an eminent nuclear attack (next 6-10 months). Rather suspicious. (The hints are usually made by um certain types under the breath and as an aside.)
Of course, nobody is allowed to actually do anything
Like what? Suspend the Constitution and strip search political dissenters?
We have a 5th column in the nation, but nothing is really done about it...
I was right. Scarf-wearing Rachel Ray should lose her job. And people who don't like Israel should be shot.
Totalitarianism...it's not just for the old Soviet Union anymore.
Good article Steve. Allow me to nitpick a bit.
ReplyDelete"Needless to say, Taleb is right than making an accurate forecast is difficult, especially about the future."
Are you trying to be cute? It's not obvious enough that you didn't have a Bushism there.
Also, having the "Look Inside" arrow over the books looks really tacky. VDare probably makes money if we buy the books from Amazon but can't you scan the covers or something rather copying and pasting?
Steve, you missed the whole point of Taleb's book and the title.
ReplyDeleteIt was not an attack on bell curve models, but rather built-in mental models of how the world works or data is distributed.
The reason he used Black Swans was because they were not rare. Prior to Australia being discovered, the only swans known, AT ALL, were White. There were no known Blacks Swans to Europeans.
That had really no impact on European assumptions, but that was because the data sampling itself being skewed did not distort anything but a meaningless picture of what Swans look like (i.e. Europeans thought they were all White all the time).
If you are making predictions about the likelihood of a terrorist nuclear attack, or earthquakes, or plagues, or anything else where data is hard to come by, there is a large amount of uncertainty about the data, and other questions, you can't really say much.
Your experience about marketing data biases you because the problem about being wrong with P&G brands perception by US consumers is pretty marginal. You won't lose a city of seven million people. You can and will if you bet wrong on say, bio-terror or nuclear weapons by terrorists because you don't know everything, and even more telling YOU DON'T KNOW WHAT THAT YOU ARE MISSING DATA.
That's why the Black Swans. It's about data. If you don't know you've not sample all the data, you can't make predictive models. There was almost no chance you'd miss important data about well known US populations with P&G.
There's a heck of a lot of chance you could bet wrong on any number of things where built-in assumptions about how data "looks" in the universe are skewed by local assumptions.
For example, if you are Ahmadinejad, your risk profile, or that of various factional leaders in Pakistan, or Saudi Arabia, vary quite a great deal from European leaders, who in turn vary from guys like Putin, who in turn have different risk profiles than Hu Jintao.
The difference between the lists which follow the normal distribution and those that have a skewed distribution is that the skewed distributions are based on human actions and human perceptions. The normal distribution are more measurements of the physical world.
ReplyDeleteNice read. 2 things:
ReplyDelete- Are we ever going to stop hearing about Hitler being dredged up in the latest fad, book or mathematical theory?
- My impression from your blog was that the housing bubble was not even mathematically planned; it was just a giant swindle where eventual failure was expected. The game in town was that the smartest shakers would sell off their risks to local and foreign suckers and get out quick enough. Did I miss something?
You know it's t99 when he's ranting about those nuclear attacks again. Meanwhile Hamas is testing the delivery system, Qassam I believe.
ReplyDeleteAt the same time (without having read the book) Taleb is confronting confirmation bias, where people don’t challenge their mental models enough, and confuse non-rejection of hypothesis with confirmation of hypothesis.
ReplyDeletePeople tend to have representativeness bias, and think too much in terms of examples that come easily to mind. Taleb and Gladwell might be encouraging this, although I suspect any book that uses examples to illustrate general points risks idiots only remembering the examples.
Ps.
Both income and consumption tend to be log-normally distributed, fairly predictable, and allow for a few super-rich.
Taleb's criticisms are frequently inconsistent, but the basic point is as Steve notes: outliers are more important than most everyone realizes. But people fixate on extremes too much, not too little. In his own domain, finance, out-of-the money options are the worst investments (to own), because the implied volatilities are 'too high'. See here for another critical review.
ReplyDeleteAn interesting article as always.
ReplyDeleteHowever, the repeated use of No Child Left Behind as a stock example of self-deluding utopianism is too simplistic and demands a response.
It may very well be that IQ realities place an upper bound on how much inner-city schools can improve.
However, this bound is so far above the present performance of average inner-city schools that blaming IQ for low performance and the inevitable failure of NCLB really misses the point.
Culture and institutions can have a huge effect too, as this very website has observed in the case of Barbados.
In the case of public education of poor NAM students, there are any number of charter schools that get excellent results, using the same low-IQ "raw material" as failing public schools. A few are listed below.
These examples need to be reckoned with, before one can use NCLB as an all-purpose negative example.
YES College Prep (Houston)
http://www.yesprep.org/about/results.htm
Match charter school (Boston)
http://www.matchschool.org/about/results.htm
KIPP schools
http://www.kipp.org/01/resultsofkippsch.cfm
Amistad Academy (Achievement First)
http://www.achievementfirst.org/af/index.php?option=com_content&task=view&id=118&Itemid=131
The point Taleb makes that was new to me was not that outliers exist - most people know that. It was that outliers can be and often are more important than the general tendency. His example is that casinos pay close attention to the odds of someone winning big at their games but less attention to less obvious risks, like the white tiger attack that cost them millions when Siegfried and Roy could not perform for months. So we may carefully put away money every month in mutual funds for years but then lose all those earnings when a 9-11 or subprime mortgage crisis crashes the market.
ReplyDeleteI think these black swans are more important than even Taleb says. The very Puritan notions of the nobility of thrift and hard work would seem to be attacked at the foundations. What counts is being able to deal with rare and momentous developments.
My problem is the unpredictable nature of such losses and Taleb's inability to provide a usable strategy outside the financial markets to guard against such risks. I think that's why he always looks so nervous.
Thanks testin99 for the explanation
ReplyDeleteTaleb has a nice video on Charlie Rose who has a pretty decent video archive. All you have to do is tune out Rose
Niederhoffer was a winning hardball squash player who mentored Taleb. His biography is good
testing99, the guy across the street from your home looks suspicious. This is a city of many people. We're missing data about that man - and we can't afford to wager mistakenly.
ReplyDeleteBEAT THAT MAN DOWN AND PROBE HIM - he may have a loose nuke you-know-where. At the very least, let's collect all his data: his whereabouts since birth, his political opinions (search his home and hard drives regularly), his statements, his associates past and present, his lovers, his employment, his blood type, his DNA, his retinal pattern, his fingerprints and dental impression.
We can't afford to bet wrong about an undetected terrorist. After all, this is a city of seven million people and we live in a world of loose nukes and even looser antisemites.
Yes, he looks very suspicious... But you'll excuse me if I don't help you to learn all about him, because I see a strange man outside my own home...he is wearing a long white coat and is hurrying up the driveway, carrying a big net... More later. Stand by on this frequency. Jack Bauer out!
PS: Don't look now, but under your bed there is a...?
There's another big difference between Mediocristan and Extremistan: Networks. The bell curve assumption works most often because of the central limit theorem which suggests that coupling many "independent" non-normal distributions will eventually give us a normal distribution. (Entropy's inevitable increase). The key here is independence, and that is violated in the stock market and in the economy, and in human networks in general. Even in the brain, networks give rise to neurons that fire a lot more often because of a winner-take-all effect that results in virtuous cycle.
ReplyDeleteGlobalization and the fast flow of information make us more networked and less independent agents. What is to say that the neighbors of a few guys who just got laid off won't cut spending because of fear and risk aversion?
Most people want to believe in their uniqueness -- not that they are just one more dot on a bell curve. Most people want to believe that they can raise children a certain way and ensure their success or goodness -- not that their children will have determined strengths and weaknesses. As science proceeds apace, the averted gaze will grow more insistent.
ReplyDeleteThere is a dreary inevitability about t99's pronouncements.
ReplyDelete"Ah hell, the terrorists are probably going to nuke New York harbor in response to our sticking our nose everywhere and inviting the whole world in...but, what are ya gonna do--not stick your nose in everywhere and not invite the whole effing world in?"
Or women:
"White beta males are doomed. It's a world of single moms married to the government!" [Promote an orthodox Christian ethos instead of secular bureaucratism? Shrink government instead of pandering to everybody? Stop shovelling money to cities that should just go broke?]
I'm reminded of his panicked response to the McCain-Palin nose dive. As if he just now found out that expensive, open-ended wars are unpopular and the Republicans have undercut their own demographic majority and geriatric cancer patients don't make very good candidates.
The guy has trenchant observations, but they're always made from the second phase of the cycle.
--Senor Doug
For all the flogging testing99 gets about nuclear attacks, he is right, we are going to get one. Of course, nobody is allowed to actually do anything about this fact that would actually preventing an attack from happening, like admitting islam is a religion of terror and that practitioners of it live here and want to do us harm and will help out jihadis from abroad(like they did on 9-11 and the first WTC attacks). We have a 5th column in the nation, but nothing is really done about it...
ReplyDelete"testing99 said...
ReplyDeleteSteve, you missed the whole point of Taleb's book and the title."
Whenever I read the opening sentence of one of testing99's posts, I am reminded of Dan Ackroyd doing the Point/Counterpoint spoof on SNL:
"Jane, you ignorant slut....."
If you cross a black swan and a white swan do you get Barack Swabama?
ReplyDeleteAnon sed:
ReplyDelete"For all the flogging testing99 gets about nuclear attacks, he is right, we are going to get one."
Steve was first in line about profiling Arabs. And he never suggested selling off ports to Arab nations. Withdrawing form non-crucial states also helps since very interaction with others is another open door. Everything Steve writes would lead to more security and protection against this type of attack. Running around the world with guns blazing just opens unnecessary doors. It also means lots of innocent people get killed and thus results in moral guilt, something otherwise reserved for losers like Germans and Afrikaners.
I also never read Steve complaining about building anti-ballistic systems, which in addition to tightened Coast Guard and Port/Airport security is the best long-term solution. Basically everything Steve says leads to more security.
nuclear attacks[...]we are going to get one.
ReplyDeleteDo Biden and you know something we don't?
Lots of little hints across the culture recently about an eminent nuclear attack (next 6-10 months). Rather suspicious. (The hints are usually made by um certain types under the breath and as an aside.)
Of course, nobody is allowed to actually do anything
Like what? Suspend the Constitution and strip search political dissenters?
We have a 5th column in the nation, but nothing is really done about it...
I was right. Scarf-wearing Rachel Ray should lose her job. And people who don't like Israel should be shot.
Totalitarianism...it's not just for the old Soviet Union anymore.