November 5, 2012

Who is going to win? Does Nate Silver know?

In Maclean's, Colby Cosh of Edmonton (who has always seemed liked the smartest man in North American journalism) gives a balanced assessment of the Silver Age:
The whole world is suddenly talking about election pundit Nate Silver, and as a longtime heckler of Silver I find myself at a bit of a loss. These days, Silver is saying all the right things about statistical methodology and epistemological humility; he has written what looks like a very solid popular book about statistical forecasting; he has copped to being somewhat uncomfortable with his status as an all-seeing political guru, which tends to defuse efforts to make a nickname like “Mr. Overrated” stick; and he has, by challenging a blowhard to a cash bet, also damaged one of my major criticisms of his probabilistic presidential-election forecasts. That last move even earned Silver some prissy, ill-founded criticism from the public editor of the New York Times, which could hardly be better calculated to make me appreciate the man more.
The situation is that many of Nate Silver’s attackers don’t really know what the hell they are talking about. Unfortunately, this gives them something in common with many of Nate Silver’s defenders, who greet any objection to his standing or methods with cries of “Are you against SCIENCE? Are you against MAAATH?” If science and math are things you do appreciate and favour, I would ask you to resist the temptation to embody them in some particular person. ...
Silver is a terrific advocate for statistical literacy. But it is curious how often he seems to have failed upward almost inadvertently. Even this magazine’s coverage of Silver mentions the means by which he first gained public notice: his ostensibly successful background as a forecaster for the Baseball Prospectus website and publishing house. 
Silver built a system for projecting future player performance called PECOTA—a glutinous mass of Excel formulas that claimed to offer the best possible guess as to how, say, Adam Dunn will hit next year. PECOTA, whose contents were proprietary and secret and which was a major selling point for BPro, quickly became an industry standard for bettors and fantasy-baseball players because of its claimed empirical basis. Unlike other projection systems, it would specifically compare Adam Dunn (and every other player) to similar players in the past who had been at the same age and had roughly the same statistical profile.

Colby's on to an interesting distinction here between modern philosophy of science's idealization of forecasting as the acid test of SCIENCE -- which we also assume to be related to the quest for reductionism, Occam's Razor, transparency, peer review, all that kind of elegant stuff -- versus the messy reality of forecasting as a profit-making business. In my corporate career, I got drafted into building sales forecasting models for both companies in the industry, so I've had first hand experience with these issues.
For most players in most years, Silver’s PECOTA worked pretty well. But the world of baseball research, like the world of political psephology, does have its cranky internet termites. They pointed out that PECOTA seemed to blunder when presented with unique players who lack historical comparators, particularly singles-hitting Japanese weirdo Ichiro Suzuki. 

Ichiro is one of baseball history's more consistent players -- across two decades across two continents -- so his future stats have been pretty easy to predict by the most superficial fan:  just project the trendline to account for aging. But Silver's proprietary system couldn't believe Ichiro could continue to get all those obviously fluky infield hits before regression to the mean crashed in, so PECOTA routinely underpredicted his performance, although by less and less as Silver added some kind of specialized secret counter-Japanese-weirdo gizmo or gizmos to his model to make his Ichiro forecasts less wrong.

Keep in mind that PECOTA's prejudice against Ichiro was grounded in a recent advance in general understanding. Sabermetricians had figured out in the late 20th Century that a lot of exceptional seasons really were just luck. When you look at hits as a percentage of balls in play, it turns out that some famous seasons were just a case of a guy getting lucky and hitting 'em where they ain't, which usually turns out not to be replicable across more than one season.

But, Ichiro kept it up for thousands of plate appearances.

Notice, however, that by making his black box forecast more accurate, Silver was making it scientifically less useful. Back when Silver was saying: Based on everything we know about slap hitters, Ichiro is due for a comeuppance this year. Nobody can continue to accumulate seeing-eye singles and slow rollers and all the lucky crud that Ichiro got last year.

And, year after year, Silver was wrong, which suggested that Ichiro wasn't like the other slap hitters, that American baseball minds needed to reverse engineer this Japanese import and figure out exactly how he does what he does. Maybe we could train a few guys over here to do it too, or maybe we could select more young players with some of Ichiro's obscure gifts.

But, over time, as Silver added secret adjustments to his hidden model, that Ichiro Anomaly became less glaring.

Science progresses by the accumulation of wrong predictions. For example, if Isaac Newton had set up a proprietary firm called Astronomy Prospectus that owned a black box model for predicting planetary orbits based on Newton's secret Law of Gravity, his successors could have made minor adjustments in their forecasts to account for the Mercury Anomaly. So, who needs Einstein's General Theory of Relativity when Astronomy Prospectus can just nudge their forecasts?

In general, however, we need to keep in mind the fundamental distinction between the physical sciences and the human sciences: humans can, to some extent, respond to predictions and thus mess up prediction models. The planet Mercury won't respond to predictions whether or not they are public.

I suspect that in the human sciences, prediction models go through a pattern where they become more and more baroque in their complexity, until they fail spectacularly, often because somebody figures out how to exploit the complexity. We saw that with mortgages, where a giant superstructure of math for modeling the actuarial risk of defaults was undermined by simple lowbrow methods like adding a zero to the mortgage applicant's income.

But, it's not clear if electoral prediction models have similar risks.
More importantly, PECOTA produced reasonable predictions, but they were only marginally better than those generated by extremely simple models anyone could build. The baseball analyst known as “Tom Tango” (a mystery man I once profiled for Maclean’s, if you can call it a profile) created a baseline for projection systems that he named the “Marcels” after the monkey on the TV show Friends—the idea being that you must beat the Marcels, year-in and year-out, to prove you actually know more than a monkey. PECOTA didn’t offer much of an upgrade on the Marcels—sometimes none at all.

Einstein famously said that science should be as simple as possible, but no simpler. But, that leaves a lot of latitude.
PECOTA came under added scrutiny in 2009, when it offered an outrageously high forecast—one that was derided immediately, even as people waited in fear and curiosity to see if it would pan out—for Baltimore Orioles rookie catcher Matt Wieters. Wieters did have a decent first year, but he has not, as PECOTA implied he would, rolled over the American League like the Kwantung Army sweeping Manchuria. By the time of the Wieters Affair, Silver had departed Baseball Prospectus for psephological godhood, ultimately leaving his proprietary model behind in the hands of a friendly skeptic, Colin Wyers, who was hired by BPro. In a series of 2010 posts by Wyers and others called “Reintroducing PECOTA”—though it could reasonably have been entitled “Why We Have To Bulldoze This Pigsty And Rebuild It From Scratch”—one can read between the lines. Or, hell, just read the lines.
Behind the scenes, the PECOTA process has always been like Von Hayes: large, complex, and full of creaky interactions and pinch points… The numbers crunching for PECOTA ended up taking weeks upon weeks every year, making for a frustrating delay for both authors of the Baseball Prospectus annual and fantasy baseball players nationwide. Bottlenecks where an individual was working furiously on one part of the process while everyone else was stuck waiting for them were not uncommon. To make matters worse, we were dealing with multiple sets of numbers. 
…Like a Bizarro-world subway system where texting while drunk is mandatory for on-duty drivers, there were many possible points of derailment, and diagnosing problems across a set of busy people in different time zones often took longer than it should have. But we plowed along with the system with few changes despite its obvious drawbacks; Nate knew the ins and outs of it, in the end it produced results, and rebuilding the thing sensibly would be a huge undertaking. We knew that we weren’t adequately prepared in the event that Nate got hit by a bus, but such is the plight of the small partnership. 
…As the season progressed, we had some of our top men—not in the Raiders of the Lost Ark meaning of the term—look at the spreadsheet to see how we could wring the intellectual property out of it and chuck what was left. But in addition to the copious lack of documentation, the measurables from the latest version of the spreadsheet I’ve got include nice round numbers like 26 worksheets, 532 variables, and a 103 MB file size. The file takes two and a half minutes to open on this computer, a fairly modern laptop. The file takes 30 seconds to close on this computer. …We’ve continued to push out PECOTA updates throughout the 2010 season, but we haven’t been happy with their presentation or documentation, and it’s become clear to everyone that it’s time to fix the problem once and for all. 

The stuff in italics is not Colby talking, it's Nate Silver's successors at Baseball Prospectus talking.

I can sympathizes with Silver, because his Excel-based PECOTA model / ball of twine reminds me of the Excel-based sales forecast model I built for the other firm in the CPG marketing data industry after building an almost unbreakable Lotus 1-2-3 3.0 sales forecasting model for the first firm.

The third release of the once-famous Lotus spreadsheet at the end of the 1980s was designed to make decentralized forecasting and  budgeting simple before the Internet by elegantly implementing a 3D model in which identical spreadsheets could be easily stacked and summarized.

And it was simple. In Lotus, I sent each regional sales manager a spreadsheet on which he or she would list every sales proposal, its size, some other details, and its chance of closing this quarter. Multiply the dollars by the probabilities, sum, and there's your regional sales forecast. Fed Ex the diskette (this is c. 1990) back to me at the Chicago HQ, and I have Lotus 3.0 aggregate the numbers across all the regional sheets and give the national forecast for the quarter to the CEO.

Awhile later, after I'd followed the firm's Chairman into his second (and not quite as successful) high-tech startup, the CEO got hired away by the archrival firm, and he hired me to build him the same sales forecasting system.

The only problem was that the other corporation had standardized on Microsoft Office, including Excel, so I had to build the system in Excel, not Lotus 3.0. Even though I knew how to do it now, building the same system in Excel took about three times as long, including about 100 hours on the phone with Excel tech support in Redmond. Each phone call I'd begin by explaining to the MS Excel wizard what the goal of my project was, and each one would reply that that sounded fascinating, but he'd never heard of anybody building a national forecasting system using Excel, and that, as far as he knew, nobody at Microsoft had ever contemplated that use when designing Excel. In theory, Excel supported drilling down through multiple worksheets, but, in practice, it was an ordeal to build, and, worse, practically impossible to explain to anybody else how it worked.

At my first company, when I left for the start-up, my assistants carried on running the Lotus-based sales forecasting system effortlessly. When I left the second company, I printed up a 45 page guide to how keep the sales forecasting system running, which was about an order of magnitude longer than would have been required with Lotus.

Of course, Excel went on to become the global standard in spreadsheets and Lotus 1-2-3 vanished. The idea of non-programmers building large systems in spreadsheets largely vanished with it, as well. Who knows how much productivity has been lost due to Excel's dominion?

It wasn't particularly Silver's fault that the system he pioneered in Excel became unwieldy. That's what Excel does. It's a hard lesson that people have to learn that you can't stay with Excel past a certain point. If you have a real business, at some point you have to bring a programmer in and rebuild the thing from scratch in a regular programming language.

Let's come back to the Philosophy of Science questions. In my sales forecasting systems, I did not attempt to build in ad hoc Ichiro-style adjustments. I kept them super-reductionist. I could have made my forecasts more accurate by putting in stuff like Smith is always overoptimistic by 15% until the week before the quarter closes, while Jones is notorious for sandbagging 10% of her likely revenue. But, I went instead for total transparency for my bosses at HQ and total reporting for their regional sales managers. If the national forecast was wrong, it was because specific regional sales managers were wrong, and it's up to those individuals to correct their biases and delusions, or face the consequences. Being honest and realistic with HQ was part of their job, and part of my job was to make clear when they weren't.

The corporate officers' jobs, however, included reporting profit forecasts to stock market analysts, and they could massage the numbers I reported to them for known biases (or hunches, hopes, or whatever, with only worries about their reputations with analysts and fears of shareholder lawsuits to rein them in).

In contrast to what I did, Silver is running a proprietary non-transparent black box business. He has incentives to be accurate, but he has other incentives as well, such as providing comforting fare to biased readers of the NYT and keeping his system secret.

Colby continues:
If the history of Silver’s PECOTA is new to you, and you’re shocked by brutal phrases like “wring the intellectual property out of it and chuck what was left”, you should now have the sense to look slightly askance at the New PECOTA, i.e., Silver’s presidential-election model. When it comes to prestige, it stands about where PECOTA was in 2006. Like PECOTA, it has a plethora of vulnerable moving parts. Like PECOTA, it is proprietary and irreproducible. That last feature makes it unwise to use Silver’s model as a straw stand-in for “science”, as if the model had been fully specified in a peer-reviewed journal. 
Silver has said a lot about the model’s theoretical underpinnings, and what he has said is all ostensibly convincing. The polling numbers he uses as inputs are available for scrutiny, if (but only if) you’re on his list of pollsters. The weights he assigns to various polling firms, and the generating model for those weights, are public. But that still leaves most of the model somewhat obscure, and without a long series of tests—i.e., U.S. elections—we don’t really know that Nate is not pulling the numbers out of the mathematical equivalent of a goat’s bum. 
Unfortunately, the most useful practical tests must necessarily come by means of structurally unusual presidential elections. The one scheduled for Tuesday won’t tell us much, since Silver gives both major-party candidates a reasonable chance of victory and there is no Ross Perot-type third-party gunslinger or other foreseeable anomaly to put desirable stress on his model.

I'm reminded of the 1996 election, when the consensus was Clinton over Dole by double digits, but it turned out considerably closer. The consensus of the polls was off by enough to have reversed the results of the 2000 and, perhaps, the 2004 election. It didn't matter in 1996, so it's largely forgotten by now.

Among pollsters, only Zogby got the margin right in 1996. This did wonders for Zogby's career, but the whole incident remains shrouded. What did Zogby know that nobody else knew? Anything? Or was he just lucky? Who knows? His system was proprietary.

By the way, the failure of polls in 1996 helps explain why so many pundits didn't question the catastrophic failure of the exit polls in 2004 to pick the winner of the election. The afternoon of Election Day 2004, the word swept the country that the exit polls showed Kerry winning easily. This was widely accepted as true, in part because the reputation of telephone polls before the election had been badly dented by 1996. But, it turned out, the exit polls were biased, not the pre-election polls.

The funniest recent forecasting fiasco was Election Night in 2000 when the networks first called Florida for Gore even though Bush had a big lead in the partially counted vote, making Gore the Presumptive President. Then they switched and called Florida for Bush, declaring the country for Bush, even though Bush's lead in the actual votes counted was shrinking relentlessly. To me, watching at home, a simple trendline suggested that when they got to 99.9% of the votes counted, Florida would be virtually tied. Eventually, the networks figured that out too and switched Florida back to uncalled.

In summary, I like Nate Silver. He works extremely hard and has sensible approaches to presenting consensus forecasts. Eventually, the consensus won't prove right for reasons that might seem obvious in hindsight, but there will be a lot of people with egg on the face in that situation.

His most apparent problem is that he's young and thus can't remember a lot of confidence-sapping events. I've noticed that he naively accepts The Narrative about most late 20th Century political events. He wasn't paying attention so how would he know that the conventional wisdom about Prop. 187 or Willie Horton or whatever is a construct of wishful thinking?

56 comments:

Anonymous said...

Obama FTW!!!

Anonymous said...

OBAMA! wins by 57 states ...direct from Daily Kos/Obama internals.

Norville Rogers said...

I loved this line from the 2010 post he links: "Silver, with his revolutionary disregard for everything but the polling numbers..."

Nobel Memorial Prize in Econ said...

His PEYOTE black box thingamajig failed to predict Ichiro, until he'd added a "gizmo" to fix that... Which still failed to predict it correctly... How'd that one Robert Frost poem go again

Anonymous said...

Ooh, a huge, unwieldy spreadsheet, and no one knows how it works. I can sympathize. There is no reason any model should be like that, Excel or no Excel. This tells us something about Silver.

BTW, the ability to sum (or whatever) across the worksheets had been in Excel since 1993. Did you work with an older version?

Steve Sailer said...

"BTW, the ability to sum (or whatever) across the worksheets had been in Excel since 1993. Did you work with an older version?"

I used the state of the art 1993 version. You could sum across spreadsheets, but the process was unbelievably clunkier than in Lotus 1-2-3 3.0 from a couple of years earlier. Worse was graphing across spreadsheets in Excel.

chucho said...

The joke is that no matter who wins the election, Silver himself is the ultimate winner due to the incredible publicity he's been getting.

If Obama wins, he'll probably win a Nobel or something. If Romney wins, well, he's still a SCIENCE guy and does MATH stuff, so we should still pay attention to him.

sunbeam said...

I'm not a baseball fan, so I had never heard of him before this election.

Why is he such a minor big deal at the moment? People have been touting elections forever, why am I hearing a lot about this guy in particular now?

I kind of lean to thinking that the media has to find a way to spin boring shit like poll numbers into a story, so they latch onto anything they can possibly build a story marketing campaign around that doesn't put people to sleep.

That said, I'm curious as to how many here pay attention to, or put any credence in what guys like this, Dick Morris, who ever else, have to say about these things. They are in the talking head business to get air time, so they can market their consulting services or hawk some other thing like a book they have. Or maybe they get paid directly to appear on these shows, I dunno.

I do pay attention to the polls, as I'm sure most here do. It's pretty easy to label states as sure one candidate or another, then check to see how they are doing in the other states.

Just to let you know, I have correctly picked the winner of every election since 1980, except for 1992 and 2004 using my method. Honestly I was so fed up by Bush in '04 I probably didn't think about it hard enough. I just figured there was no way the guy could get re-elected. Of course, I have found some people actually believed that crap Colin Powell spewed at the UN, so I shouldn't be surprised.

Anonymous said...

"I used the state of the art 1993 version. You could sum across spreadsheets, but the process was unbelievably clunkier than in Lotus 1-2-3 3.0 from a couple of years earlier."

Hmm. Excel also had pivot tables since 1993. Those allowed to easily slice and dice data in any number of dimensions. Pivot tables are the reason no one sums across worksheets today.

Anonymous said...

The whole world is suddenly talking about election pundit Nate Silver


Who?

The "whole world" for some people is encompassed by the New York Times.

Anonymous said...

Senator Sharron Angle could not be reached for comment.

I guess I have to explain the joke - Silver said there was a 75% chance that Angle would win in 2010.

You want to know who'll win tomorrow? Flip a coin. It has at least the same probability of being right as Silver has, and probably more so.

Hunsdon said...

Any author who can reference both the Kwantung Army and a goat's bum in the same piece deserves some closer attention.

stari_momak said...

"Fedex the Diskette"

Wow, a high priced sneakernet. But in Fall 1991 I was sending Word and Excel docs over a university email network which included Macs and PCs. I'm pretty sure it would have been possible even using what was then Internet's unix-based emails, using some flavor of uuencode. And there was for sure ftp file transfer, in the mac world for Stuffed and Binhexed files. (Speaking of which, Stuffit was written by an Asian wonderkind, Raymund Lau , I am nothing if not fair)

58.00207% chance of me going on Charlie Rose tomorrow said...

"Silver himself is the ultimate winner due to the incredible publicity"
My scientific senses tell me a parody Twitter account must be on the way

DaveinHackensack said...

"My scientific senses tell me a parody Twitter account must be on the way"

Already is one: Nate Silver 2.0 (fivethirtynate).

Seems similar in tone to the fake Tilda Swinton one: Tilda Swinton (NotTildaSwinton).

Seneca said...

I guess the keys to this race are voter turnout, the undecided voters, and the swing states.

It looks like voter turnout will be considerably lower for Obama's base than in 2008 from polls I have seen hence a smaller percentage of young people, NAMS, and single women turning up at the polls.

His base just isn't that excited this second time around (the bloom is off the rose).

Regarding the undecided, historically nine time out of ten (or so I read) they break against the incumbent and for the challenger.

This makes sense because if they are still undecided after looking at a candidate's job performance for 4 years it means they are fundamentally unsatisfied.

Because the race is virtually tied amongst those who have already decided the 5% or so who are undecided will determine the election tomorrow.

I have read conflicting reports on how the undecided are breaking from the most recent polls. In places like Tennesee and North Carolina they seem be breaking strongly for Romney. In other states not so much. In the swing states I haven't read much at all about how this group is breaking.

I think it is going to be close but I think Mitt has a chance despite the changing demographic landscape of America.

Romney is hampered by the message he has been saddled with, e.g., cutting programs and a more agressive foreign policy.

In these perilous economic times these messages are not an easy sell even if they are required to get the Republican nomination which is now effectively controlled by Neo-Cons and Wall Street (the Neo-Cons having effectively purged the Paleo-Con isolationist and anit- free trade oriented American firsters like Buchanan, etc....)*

It is so close, it's tough to call.

One wonders of the Bradley effect might come into play You know, people claiming that they will be voting for Obama, because they don't want to seem racist. I think during the last election there was some worry by Democrats that this was taking place which turned out to be unfounded worries.

If O does win it will be the first time a president with such a bad economy has been reelected in a long, long time according to the polling experts at the University of Colorado (think Rosevelt during the depression probably).

*N.B. the Democrats are of course also controlled by Wall Street just a slightly different team in the same industry. Not surprising since finance has replaced manufacturing as the major component of the economy in the last thrity years or so.

Truckee Man said...

I am the last Lotus user, Excel was too hard to learn so I never switched. Anything by microsoft is a huge pain. Now my minions do most of the spreadsheet jockeying.

JI said...

Steve, you really needed to learn how to use some real software. SAS, SPSS, Stata, maybe even Splus, were available in the early-1990's. So were databases in which to store the information and, if you didn't want to export the data from a database to the stats software packages, you could use SQL to do the same sort of thing.

That said, Excel was annoying back then, it was still annoying in the early-2000's, and it is annoying today. Some things never change.

Whiskey said...

Seneca, Romney is anything but a free-trader, his message to the Rust Belt is he'll label China a currency manipulator and impose tariffs to prevent their dumping. No he's not a systematic protectionist, but he's as good as you'll get for a long while and oceans better than Bush or McCain on that subject.

As far the military goes, the UAW blasted Obama for gutting the M1-Abrams and sending lots of auto-workers to the unemployment line. So spending = employment of blue collar White guys, mostly. And having say, 25 instead of now, 10 (active) aircraft carrier groups deters a lot of challenges and mischief that leads to grief.

Part of Steve's problem is his height. Being a tall guy, he doesn't get the idea of pumping up and learning an art to deter attack. Those of us on the shorter side got challenged all the time; until we wised up and lifted. Being 6 foot four inches means a lot of the stuff a guy seven inches shorter experiences just doesn't register.

Russia has a nuclear missile sub off the East Coast, according to Drudge. That wouldn't happen with 20 aircraft carrier groups, its the equivalent of pumping up and attitude.

Whiskey said...

Let me add, Silver is not very proficient with mathematical modeling on a computer. Using Excel for anything other than a few simple sheets is a nightmare (and even with pivot tables that tool sucks more than Lotus 1-2-3 which I recall fondly).

But you can use R, a free, SAS like tool, with MySQL (also free) and tons of R graphical tools to load data and construct well documented formulas. All run on Windows as well as Linux and Mac OSX. All free, well known and documented tools. Most University systems use R (because its free) now over SAS and other tools, like RATS and STORM. On a modern laptop, I've run some fairly sophisticated MySQL-R combos to look for statistical anomalies in public FEC donation data (this was back in 2007-8). It wasn't that hard.

If your data is more than a few hundred rows or so, and more than about twenty columns, you're better off using MySQL and something like R, which can tie into it.

As for Silver himself, and the whole SABREmetrics thing, what limits them is their inability to see the entire world.

A model is just that, a mathematical model with hefty built in assumptions. They did not want to see their favorites juicing or baseball look the other way, so they didn't. At least Beane and other guys are looking for undervalued players and trying to find advantage (so far Belichick with the Pats is the leader in sports). And as soon as one guy does something the advantage is negated as in Wall Street by everyone else reverse-engineering and copying.

What isn't reproduceable is coaching. But most coaches can't actually coach, that is teach systems and execution, particularly well.

My guess is that sports book info guys who charge for their reports use systems built with MySQL and R, and a more simple but robust system. Telling that Silver never sought to sell his own proprietary knowledge, just himself. Is he the next Faith Popcorn?

Clutch cargo cult said...

"Each phone call I'd begin by explaining to the MS Excel wizard what the goal of my project was, and each one would reply that that sounded fascinating, but he'd never heard of anybody building a national forecasting system using Excel, and that, as far as he knew, nobody at Microsoft had ever contemplated that use when designing Excel." ha ha no wonder the stock is $28!

haddox said...

Good overview of the election aggregators, including his own, found here by Sam Wang of Princeton.

Wang is more bullish on Obama than even Silver. It'll be interesting to see how his cocksure "meta-margin" projections hold up. His quant chops seem legit, though his politics are Daily Kos.

By the way, this is a side gig for Wang. He's a neuroscientist, a Stanford PhD with a Cal Tech physics degree. Ironically, Derb happens to mention him here in the context of discussing academic soft totalitarianism on familar iSteve topics.

NOTA said...

The main useful thing about 538, electoral-vote.com, real clear politics, etc. is the electoral vote maps based on state polls. This makes it enormously easier to see what the campaigns are worried about--who gets to be president.

The way it looks to me, the whole meme about the polls being biased and the later attacks on Silver by various hacks and shills, started when Romney's poll numbers looked particularly dire. I assume this was an attempt to keep the campaign from losing all it's steam as volunteers and voters and donors gave it up as a lost cause. It was probably also an attempt to work the refs a bit--make Silver and the big pollsters try to tweak their models to improve Romney's numbers.

lattice of coincidence said...

I've noticed over the run of this blog how any & all inchoate theories of life's inexplicable phenomena and the plenitude of tragicomic happenstances eventually circle back to Steve's concern about why Lotus 1-2-3 lost market share to MS Excel back in the mid '90s

Steve Sailer said...

I had to put down my delicious plate of shrimp to deny that charge 100%.

Anonymous said...

Funny that you bring up the Zogby tidbit. Not to get into some Moneyball scout vs. quant thing, but the whippersnappers never remember that stuff, chiefly 'cos they weren't even in grade school at the time. Yet I can still hear in my head the (then-novel) cable TV yakkers speaking of John Z in worshipful tones even up to the time of Bush/Gore. It's nice work, except when the 2002 election happens!

Anonymous said...

I've noticed over the run of this blog how any & all inchoate theories of life's inexplicable phenomena and the plenitude of tragicomic happenstances eventually circle back to Steve's concern about why Lotus 1-2-3 lost market share to MS Excel back in the mid '90s

Lol!

I think Steve may have mentioned Lotus 1-2-3 vs Excel once before.

Steve Sailer said...

I have a lot on my plate right now, so I don't have time to answer to all these shrimp-sized complaints about me supposedly obsessing over spreadsheet software, the movie Repo Man, and similar complates (of shrimp).

TGGP said...

Google gives 38 results for "lotus" on isteve.blogspot.com

Steve Sailer said...

Yes, but how many mentions are there of a "lotus of coincidence"? Huh?

Steve Sailer said...

"Not to get into some Moneyball scout vs. quant thing, but the whippersnappers never remember that stuff, chiefly 'cos they weren't even in grade school at the time."

My only real complaint about Nate Silver is that he's young, so he mostly innocently accepts The Narrative about 20th Century political events that I remember first hand and thus know didn't actually happen the way you are supposed to remember them happening.

Anonymous said...

Of TGGP's 38 hits, about a third are about lotus-eaters, a third are about how Jim Manzi isn't from Lotus and a final third are directly about spreadsheets. Only once before did Steve tell the story and once he said that Lotus was awesome but didn't compare it to Excel.

Steve Sailer said...

"Only once before did Steve tell the story and once he said that Lotus was awesome but didn't compare it to Excel."

Well, I should have explained how awful Excel is for building anything large. Who knows? It might have saved Nate Silver and his Baseball Prospectus successors a lot of trouble.

Also, I should have used the phrase "lotus of coincidence" before.

ESPN-ification of human affairs said...

I don't just hate the whole illusion of "poll-based purity" around this--even if you "crunch" some polls you're still interpreting and massaging a bunch of someone else's handiwork. But it's anti-scientific at root to ignore the REALLY basic stuff under your nose, things that may not be so lively to commentate upon in a NYT blog but without which you are actually hamstringing your own punditing. For example: elections are decided by those who vote... Internet Consumer From Mars might be impressed by all the colorful libertarians, supposedly including Nate Silver according to one time he was on Charlie Rose's show (not making this up), and their protracted crying about the corrupt system and democracy deficit, blah blah. But it's a huge venerable cliche among their affiliated tribes that they abstain from voting anyway, as a fashion statement I think (now that's the way to conspire to fight The System)

So ultimately the libertarians live like indignant grad students but vote like Amish. And their numerical clout is misunderstood, just as with surveys of people estimating America is 10% made up of gay men (source: any network TV programming schedule). I'd now refer to the "availability heuristic" but don't want to sound like one of these political analyst jerks

Anonymous said...

I've noticed over the run of this blog how any & all inchoate theories of life's inexplicable phenomena and the plenitude of tragicomic happenstances eventually circle back to Steve's concern about why Lotus 1-2-3 lost market share to MS Excel back in the mid '90s


Bad software drives out good.

Seneca said...

"Seneca, Romney is anything but a free-trader, his message to the Rust Belt is he'll label China a currency manipulator and impose tariffs to prevent their dumping. No he's not a systematic protectionist, but he's as good as you'll get for a long while and oceans better than Bush or McCain on that subject."

Yeah, I know, but most people aren't following the election or Romney's positions that closely so the "outsourcing" tag that the Dems have tried to stick on him still has traction with a lot of voters.

"Part of Steve's problem is his height. Being a tall guy, he doesn't get the idea of pumping up and learning an art to deter attack. Those of us on the shorter side got challenged all the time; until we wised up and lifted. Being 6 foot four inches means a lot of the stuff a guy seven inches shorter experiences just doesn't register"


Okay so you got a little bit of a Napoleon complex and don't want people to take advantage of you. Nothing wrong with that and it at least explains your aggressive foreign policy stance.

But doesn't the U.S. spend more on defense than the next twenty nations combined (I've heard the rest of the world combined from some sources)?

I mean we spend 750 billion (a very conservative estimate) on defense while China, who spends the second most, only spends 130 billion or so.

We could cut our defense budget in half and still be outspending them by over a 3 to 1 margin.

I know there are other policy considerations like jobs for enlisted people and defense R&D that make cutting the defense budget in this economic environment difficult...(i.e. people need jobs right now)

but being percieved as the neighborhood weakling is not one of those considerations.

Anonymous said...

Right now--the night before the election--Silver is saying there's a 92% chance of Obama winning, or 2:23. I'd take those odds in a heartbeat and think it's closer to 50:50. Intrade currently has it at 68%.

If Silver were a betting man I think there would be a lot of arbitrage going on.

Steve Sailer said...

"Part of Steve's problem is his height. Being a tall guy, he doesn't get the idea of pumping up and learning an art to deter attack."

That's not unreasonable. I should think about that a little.

DaveinHackensack said...

"That's not unreasonable. I should think about that a little."

But the analogy doesn't support Whiskey's case for 25 Nimitz-class carriers. The US isn't an average-sized guy who needs to lift and learn karate. The US is Vitaliy Klitschko. The big guys won't box us unless they have to, and if they do, they'll probably lose.

The little guys will never box us. They'll trip us when we walk in their houses, and then have their kids bite us and stick their fingers in our eyes, while they tie us down, Gulliver-style. The way to avoid that isn't by buying more boxing gloves, but by not walking into their houses.

Anonymous said...

"I assume this was an attempt to keep the campaign from losing all it's steam as volunteers and voters and donors gave it up as a lost cause."

See, I think the Obama campaign has a strategy of attempting to project inevitability in order to discourage turnout by the opposition.

I poke around Daily Kos once in a while. They're idiots, but they're obedient idiots, and can be relied upon to parrot whatever some campaign flunky tells them to say. As such they're a useful source of data for Obama campaign Kremlinology. The "Obama is winning so you Republicans shouldn't even bother to show up" story line is pretty prominent over there.

PapayaSF said...

I'm with those who think Silver is way off. Too many of those state polls assume a party ID split similar to 2008, which was D+7, while recent polls of party ID are showing D+0 or even R+1. And yet one recent poll, showing a tie, is based on this election being D+11! I don't think so....

In San Francisco, enthusiasm for Obama is WAY down. In 2008 his signs and bumper stickers were everywhere, but now they're actually rare. Not that Romney will take CA, but I think it's a sign that Democrats are demoralized.

Peter A said...

In Boston enthusiasm for Obama (or simply hatred for Romney) is way up. It's a big country - everyone lives in their own bubble. Judging from my acquaintances - and I'm a white guy in the business world - I would guess that Obama has a 99.9% chance of being reelected. I'm sure someone living in Dallas has a completely opposite view.

We do seem to be at the point where the country is so big and so disparate that polling is just guesswork. Direct elections are probably basically meaningless as well at this point.

MQ said...

Part of Steve's problem is his height. Being a tall guy, he doesn't get the idea of pumping up and learning an art to deter attack. Those of us on the shorter side got challenged all the time; until we wised up and lifted. Being 6 foot four inches means a lot of the stuff a guy seven inches shorter experiences just doesn't register.

Ah -- overcompensation. This explains Whiskey's sexual problems and his inane foreign policy prescriptions all in one swoop. It's unfortunately not uncommon for men to take their history of getting bullied in grade school and project it out onto world affairs. I wish people would get it through their head that there is NO connection, none, between getting the experience of getting picked on in junior high and the nature of modern warfare between nuclear-armed powers. Please take your need to prove your masculinity into some other arena. (Ideally not coming up with elaborate explanations for why women will never want to sleep with anyone like you, that is counterproductive).

Anonymous said...

Judging from my acquaintances - and I'm a white guy in the business world - I would guess that Obama has a 99.9% chance of being reelected.


Yeah? I know a few Jews and a lesbian Asian who are voting for Romney in Hudson County, NJ.

Of course, I'm a Republican. And you're a devoted Democrat - I've seen you here before shilling for Obama. People only know the people in their own circles, and those circles tend to be self-selecting. Nationwide, Jews, Asians and lesbians are not going to go for Romney, and white guys "in the business world" are not going to vote for Obama.

I can safely predict that, once again, Obama will clean up among those making over $200k per year.

toto said...

The nice thing about Silver's predictions is that there are many of them - one per state. So after the final numbers are in, everyone can go and check whether the probabilities he gave agreed with the actual outcome - and because there are many numbers, we can actually put confidence intervals around how much we should trust Silver's model.

Contrast with all the pundits claiming that the main election is "a toss-up" - a conveniently irrefutable statement, since it is compatible with any outcome.

Anonymous said...

The way it looks to me, the whole meme about the polls being biased and the later attacks on Silver by various hacks and shills, started when Romney's poll numbers looked particularly dire.


The way it looks to me, NOTA is 32 years old, tops, and lives in one of the major urban conglomerations. He's spent his entire short life absorbing leftist thought by osmosis and though he likes to think of himself as a critical thinker, he's actually quite shallow. He's a "libertarian" because that's the only permissible alternative to leftism in the world he inhabits, and he can no more think outside the box constructed for him than he can flap his arms and fly.

There has never been any reason to imagine that the electorate is +10 Democrat. Those of us who've been around a little longer, and/or are of a more skeptical frame of mind, have learned that the American MSM is exactly as reliable a source of information on US politics as Pravda was on Soviet politics. If you take anything you read in the New York Times without a mountain of salt, you're getting played.

Anonymous said...

Right now--the night before the election--Silver is saying there's a 92% chance of Obama winning, or 2:23. I'd take those odds in a heartbeat


Yeah, if Silver had to put real money of his own - not a thousand dollars, but his house - on the line here, he'd stop that nonsense in a hurry.

Anonymous said...

I've seen more Mitt signs and bumper stickers in north New Jersey - in a bedroom community of NYC - than I have for Obama.

Does that mean Obama will lose NJ? Of course not. But it does mean that the left's hope that there exists a fired-up Dem base which will give them a nine or ten point edge in votes over Republicans is just whistling past the graveyard.

And they have to have that nine or ten point edge, because Romney is cleaning up among independents.

Anonymous said...

Sorry, I simply don't think any real comparison can be made between forecasting sports and forecasting elections because the number of variables of the first are too limited compared to the latter's.

Sports have highly specific rules and measurements that elections simply don't have.

Fact is despite almost a century of presidential polling, no-one and I mean no-one, has ever called more than three successive elections to the percentage point.

There could be a reason for that...

Anonymous said...

if Silver had to put real money of his own - not a thousand dollars, but his house - on the line here, he'd stop that nonsense in a hurry.

With bets that large the downside risk overshadows the confidence you have in the accuracy of the prediction. For example, someone offers to flip a fair coin and gives you 0% of your bet if tails but 210% of your bet if heads. For a bet of a dollar you'd to that all day long and walk away richer.

If it's a one-time offer for the value of your house, not so much, unless you've got a gambler's soul.

NOTA said...

Anon 11:20:

Elections are just *bigger*--there are way more things going on that might matter in an election than in a football game. Many a promising political career is ended early because a bad economy makes your term as governor a failure, or your cabinet post is associated with an unpopular president.

NOTA said...

Just as an aside:

a. God how I wish I were 30 years old.

b. Re the fraction of Democrats, you are just wrong. About 33% of American adults self-identify as Democrats, and about 28% as Republicans. Both those numbers vary a little from year to year, but it's nothing like 10%. (Among voters, the fraction is about the same for both; more Democrats than Republicans don't bother voting.)

c. By contrast, about 38% of voters self-identify as conservative, and 22% as liberal. That proportion is about the same for voters.

This Pew report is the source for these numbers.

NOTA said...

toto:

I think if Silver's predicted presidential result is seriously off, then his state predictions almost certainly will be, as well. If the state poll totals are more or less accurate, then his presidential prediction (which tracks well with much simpler models of predicting electoral votes) is very likely to also be right.

NOTA said...

Anon 8:03:

Republicans do quite a bit better than Democrats among higher income people. I don't know that this holds for the super-rich, though.

A good source for this is this Andrew Gellman presentation summarizing a huge amount of research. But this isn't some kind of secret--there is a lot of data about this stuff out there. Political reporters don't incorporate that stuff into their chatter because they're mostly innumerate, and it doesn't cost them anything to not know what they're talking about.

Michael said...

Speaking of statistical voodoo and the widespread false deification of anything expressed by numbers, can anybody recommend a good book or two about this? I searched for "scientism" on Amazon, but all that comes back are modern Christian apologetics. Of course there is How to Lie with Statistics, but it seems like this idea is ripe for an update since the last edition of it came out in the early 90s. The powers that be have gotten a helluva lot more reliant on models since then.

NOTA said...

Michael:

I think Nassim Taleb's books are a modern corrective for the tendency to think that a complicated model with lots of intimidating equations imply that you know what you're talking about.