November 5, 2012

Should pundits publicly bet?

Former baseball statistics analyst Nate Silver, now blogging at the New York Times on elections, gives Obama an 86.3% chance of winning the Electoral College as of today. (I love the decimal point.)

Recently, Silver offered an even money $1,000 bet to TV talking head Joe Scarborough that Obama would win, for which he has been widely praised by the increasingly dominant-in-the-media sabermetrics boys. (You know, the baseball analysts who lauded stats-savvy baseball executives like Billy Beane, Bill James, and Sandy Alderson for their brilliant stats-driven insights that they could win baseball games with formerly skinny guys like Jose Canseco, Mark McGwire, the Giambi Brothers, David Justice, David Ortiz, and Manny Ramirez who, remarkably, filled out into homer-hitting behemoths, thus proving Science! is always right.)

Yet, shouldn't Silver have defended the honor of his probability percentage forecast by offering Joe Scarborough odds of, say, 3 to 1? In general, shouldn't Silver present his odds not in scientific-sounding percentages all the way to the first decimal point, but as old-fashioned race track fraction odds like 3 to 1 or 11 to 2?

By the way, I have some sympathy for the NYT Public Editor's discomfort with public bets involving pundits, the best known of which was John Tierney winning $5,000 in a 2005 to 2010 bet with oil price forecaster Matthew Simmons that oil would not rise in five years from $65 per barrel to $200. As I wrote in 2010:
I certainly would have bet against Mr. Simmons, as well, but he didn't offer to bet me.  
My suspicion at the time was that Simmons was, more or less, writing a $5,000 check postdated January 1, 2011 to buy his book publicity right now on the NYT op-ed page. That might explain why he didn't haggle and try to split the difference with Tierney, such as putting the win-lose line halfway between $65 and $200, the way a real betting man trying to make money would. But a public bet that the price of oil would be, say, $132.50 or higher would have been kind of a boring story. In contrast, a man confident enough to put $5,000 on the round number of $200 is a man who is acting like he might know something, and thus you'd better buy his book to find out what it is.

I don't want to imply anything bad here about Tierney, who is a great guy. As far as I know, I'm the only one who pointed this out about Simmons bet. It's just not the kind of thing people notice.

Still, there is a lot to be said for public bets. I offered to bet pundit Michael Barone $1,000 in early 2004 that Hispanic turnout in the fall of 2004 would be closer to my estimate of 6.1% of the total vote than his 9% speculation (according to the Census Bureau, it was 6.0%), but Barone prudently shied away. Yet, Barone appears to have learned from the experience, so it was a net benefit for quality of discourse.

Moreover, I've long offered to publicly bet that various notorious Gaps in real world performance among demographic groups are not going to vanish over any tractable time frame.

In general, however, nobody is interested in betting against me that, say, school test scores in San Marino will continue to be higher than in Compton, or however you'd want to specify a bet over The Gap of gaps. That's too boring and depressing to bet over. People are interested in betting as action, not in betting as a tool for learning about reality. The more random the outcome, the more exciting it is to bet upon. The more scientifically predictable, the more dismal the action.

What if in 1969, Arthur Jensen had made public bets with leaders of the conventional wisdom, with Jensen taking the unpopular side that in 2012, 43 years later, The Gap would be about the same as in 1969? Would that have made him more popular at his death?

77 comments:

Anonymous said...

Steve,

Here's an interesting question for you: what odds would you give that San Marino continues to outperform Compton? 100-1?

e-Silver said...

Are we talking 5 year window? Cuz I'd take that action in 25, after California's elected its 1st illegal immigrant governor

x said...

in aus there's a lot of coverage of the election on the t.v, but it's pointless in my opinion given everybody knows who is going to win.

Anonymous said...

No, betting 3 to 1 for Nate Silver would be result in an expected gain of zero if his model is right. If Joe were to bet at the 3 to 1 it would be an admission that Nate is right, which is exactly the opposite of what he is saying.

Steve Sailer said...

They could compromise at 2 to 1, or Silver could demand a bookie's vigorish of 10%.

Anonymous said...

You seem to have a negative view of sabermetrics. I though you'd be pro-sabermetrics, given your stats background and interests.

medvedev said...

Yeah, that's chickenshit. He probably does not want Scarborough to take the bet and knows that at 3:1 he would take it. I know I would!

Anonymous said...

If Silver had to actually bet real money on his prediction, he'd change his prediction to try to make it more accurate.

So I'm in favor of compelling these prognosticators to put money on the things they predict.

WMarkW said...

I don't believe the Census Bureau tabulates voter turnout. Is the 6% figure from a Census publication (like the Statistical Abstract) that contains material from private sources?

Anonymous said...

Recently, Silver offered an even money $1,000 bet to TV talking head Joe Scarborough that Obama would win, for which he has widely praised by the increasingly dominant-in-the-media sabermetrics boys.


Is "sabermetrics boys" the new code word for "Lefties"? Because they are the people who are "increasingly dominant-in-the-media". Silver himself is more a Democratic-operative-posing-as-a-statistician than he is a "sabermetrics boy".

Steve Sailer said...

"I though you'd be pro-sabermetrics, given your stats background and interests."

Sabermetrics does not lack for uncritical cheerleaders.

I started reading Bill James in 1985. But, the Steroids Era offered a huge moral challenge to James, which he completely flunked by virtually ignoring the biggest story in baseball statistics in his lifetime for the first 20 years after Tom Boswell of the Washington Post called out Jose Canseco as a steroid user in 1988.

Bill James shut his mouth and was rewarded with a nice job with the Red Sox, where he helped them win a couple of steroid-driven World Series.

In a lot of ways, the Bill James Story would make a good 21st Century tragedy about how a nerd sold his soul.

Steve Sailer said...

By the way, don't impute any of my moral criticism of Bill James to the much younger Nate Silver. I haven't looked much at all into his writings about steroids. Unlike James, Silver was a kid during the big change in the 1990s.

Anonymous said...

Omitted from your description: it was for a $1000 Red Cross donation. Please don't try to make Silver look like some tough guy.

JeremiahJohnbalaya said...

If Joe were to bet at the 3 to 1 [for Silver] it would be an admission that Nate is right, which is exactly the opposite of what he is saying.

What? If you give me better than even odds on something I think is in my favor ... I take it b/c I think your odds are WRONG.

Anonymous said...

Silver's writings on steroids in baseball seem pretty sensible here: http://www.baseballprospectus.com/article.php?articleid=8846

Anonymous said...

Steve,

I don't recall wanting to take the politically correct side of a bet with you, but I might be forgetting.

Best,
Jim Manzi

Steve Sailer said...

May 7, 2009
Lies, Damned Lies
The Steroids Game

It wasn't hard to be sensible about steroids in 2009. And in fact, I believe I've read something from Silver from 2004 that was much better about steroids than what Bill James was writing at the time. Before that I don't know, but Silver is very young so I wouldn't hold him responsible for his juvenilia.

But old fashioned sportswriter Tom Boswell was right about steroids in 1988. I briefly mentioned that baseball was corrupted by steroids in National Review in 1997. Where were the sabermetricians? Besides angling for jobs with big league teams by not rocking the boat?

Steve Sailer said...

Did we only talk about that by email? I was under the impression we kicked it around in comment threads, but I'll take it out because I shouldn't mention the contents of old emails. My apologies.

Anonymous said...

school test scores in San Marino

San Marino borders Pasadena, and The Huntington sits at the border [just two blocks below E California Blvd - in fact, a block over and two blocks down from the CalTech Athenaeum].

Arcadia and then Pasadena and now San Marino - lots of talk lately here at iSteve about the San Gabriel Valley.

Huh.

helene edwards said...

I wonder if anyone made a bet in 1974 that the people ushered into government as a result of Watergate would end up being more corrupt than Nixon/Mitchell?

Mr Lomez said...

"Where were the sabermetricians? Besides angling for jobs with big league teams by not rocking the boat?"

Why should sabermetricians be any more or less apologetic for steroids than anyone else in baseball? Couldn't these same questions be asked of the trainers, scouts, agents, coaches, etc.? You seem to be implying that sabermetricians had some special moral duty.

You also seem to be implying that sabermetrics only worked because of steroids. James' (and others) data driven methods of player evaluation were, steroids or not, objectively BETTER than orthodox methods. Of course Boston's World Series were steroids-driven. It wouldn't have been possible to succeed in the MLB otherwise, no matter what criteria a team used to evaluate players.

Sabermetrics is by no means a perfect set of tools, and I also think it's given too much credit in certain communities, but equating James' moral failing to his analytical failing, to me, is a weird conflation.

Anonymous said...

Steve, have you looked at any contrarian views on the steroids and home runs issue?

Arthur De Vany, who's an economist but better known as a "paleo" diet and fitness guru, has a stats based contrarian view. He argues that steroids had no effect on home runs. Here's his paper on it and the abstract:

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1790150

"There has been no change in Major League Baseball home run hitting for 45 yr, in spite of the new records. Players hit with no more power now than before. Records are the result of chance variations in at bats, home runs per hit, and other factors. The clustering of records is implied by the intermittency of the law of home runs. Home runs follow a stable Paretian distribution with infinite variance. The shape and scale of the distribution have not changed over the years. The greatest home run hitters are as rare as great scientists, artists, or composers."

Here's a pdf of the paper:

http://steroids-and-baseball.com/DeVany.old/DeVanyHomeRunMS.pdf

De Vany was interviewed on EconTalk where he talks a lot about steroids and home runs:

http://www.econtalk.org/archives/2010/03/de_vany_on_ster.html

Steve Sailer said...

Yeah, I read De Vany's paper a number of years ago, and, here's the thing: He was wrong.

Carol said...

"Arcadia and then Pasadena and now San Marino - lots of talk lately here at iSteve about the San Gabriel Valley."

My home town(s)! There is an intellectual vibe there, that almost but not quite overcomes the whole Hollywood-surfer-Okie influence. Probably due to CalTech, JPL, and Fuller.

Anonymous said...

"What if in 1969, Arthur Jensen had made public bets with leaders of the conventional wisdom, with Jensen taking the unpopular side that in 2012, 43 years later, The Gap would be about the same as in 1969. Would that have made him more popular at his death?"

As Steve likes to say, the bet would have gone down the memory hole.

Mōsh said...

From what I understand his reason for publicly challenging Scarborough wasn't to prove his overwhelming confidence in an Obama win but because Scarborough kept badmouthing Silver's claim that Obama was the favorite, insisted that the election remained a coin toss. There's no reason for Silver to go out of his way to give Scarborough the best possible odds is there?

For examples, Razib has been haunting the intertubes for a while now begging people who claimed they were confident in a Romney win to put their money where their mouth is. About a month ago I took Romney's odds to be about 1/3 but rather than offer him the fair odds of a 1:2 bet I offered him 1:3 (I get $30 for a Romney win and lose $10 in an Obama win) which he accepted.

If Scarborough really thought that it was a coin toss he probably could have offered Silver a $1000:$1100 bet that would have been accepted. The fact however is that Silver appears to be doing his best to analyze the numbers while Scarborough, and pretty much every other major pundit, appear to do little more than construct interesting narratives of little objective worth but which have the benefit of keeping the suspense required for continued viewership of this "fascinating" election.

www.exoticjewishhistory.com

Anonymous said...

86.3% would give Romney at 13/2 and Obama 2/13.

Say 9/2 Romney and 2/17 Obama for 6% overround or tighten the odds on Chi-candidate.

Paddy Power has already paid out at 2/9 but then the freeish publicity of paying out $650 000 was deemed worthwhile.

rwcg said...

Steve,

I've always been unclear exactly what it is you think a baseball statistician like James was supposed to do about steroids.

Steroids or not, home runs were hit, and they can be counted, regressed, and so forth. Seems to me the statistician's job ends just about there.

Steve Sailer said...

"There's no reason for Silver to go out of his way to give Scarborough the best possible odds is there?"

Yeah, there is: good publicity. If you really think 3 to 1 is what the line ought to be, then offer some guy on TV 2 to 1. It makes you look good.

Alternatively, you can just sit back and have everybody in the press write about how offering even odds makes you look good.

Steve Sailer said...

"I've always been unclear exactly what it is you think a baseball statistician like James was supposed to do about steroids."

What Bill James did about everything else in baseball: write about them. Or, at least, mention them.

Steve Sailer said...

It was like if museums were filling up with Hans van Meergeren's awful forgeries of Vermeer's paintings and the Vermeer experts were never mentioning it even though they could see at a glance what was going on.

ben tillman said...

In a lot of ways, the Bill James Story would make a good 21st Century tragedy about how a nerd sold his soul.

To his credit, Roland Beech of 82games.com got hired by Mark Cuban without selling his soul.

Anonymous said...

Tonight as part of the halftime show on monday night football, the sports talking head is "interviewing" both Obama and Romney.

Obama went first.

First question was something like "what have you learned about America and Americans that you didn't know 4 years ago.

Obama: Well, they were a confirmation of what I already knew...




Yep!

"Gamblers" Anonymous said...

Alternatively, you can just sit back and have everybody in the press write about how offering even odds makes you look good

For a charitable donation? Silver isn't the first of his breed to try this and it only makes them all look like even sissier chatterheads. There was supposed to be a Jonathan Chait vs. Jonah Goldberg wager a few years ago wherein the "loser" donated to the other's chosen charity--OH THE HUMANITY. People who blather about politics every day instead of holding down real jobs shouldn't impersonate the cryptic billionaire type of comic book legend, with so much cash he can't spend it fast enough, who only resurfaces at some trumped-up moment of phony mystery to pronounce his oracular judgment.

Anonymous said...

"I started reading Bill James in 1985. But, the Steroids Era offered a huge moral challenge to James, which he completely flunked by virtually ignoring the biggest story in baseball statistics in his lifetime for the first 20 years after Tom Boswell of the Washington Post called out Jose Canseco as a steroid user in 1988."



uhhhhhh........what?


this is the kind of thing that makes people like nate silver laugh at their critics. this literally doesn't address his work at all. not even 1%. it's completely irrelevant.

Steve Sailer said...

"uhhhhhh........what?"

Good. To understand that you don't understand is the first step toward understanding.

rwcg said...


Thought experiment: it's 1992, you're the famous stat guy, and you write something cryptic like 'seems like some players are juiced'. Next question you get is Who and How do you know, and unless you stonewall those questions, suddenly you're accusing the Cansecos of the world based on no evidence whatsoever, just mumbling you can 'see at a glance what is going on'. It's not like you were invited into the locker room to watch Jose stick the needle in Mark's backside.

Sure, you will have been right...eventually. But at the time, one can see how shooting your mouth off about such a thing would not have been the wisest. I can't see holding it against Bill James. I don't even know what he would have accomplished by doing what you're calling him a sellout for not doing. At worst, of all the people to be mad at re: steroids, seems like Bill James should be zillion and oneth on the list.

do the math said...

"lots of talk lately here at iSteve about the San Gabriel Valley" -- yeah, well, it's an increasingly East Asian subregion of the county. And obviously the Spanish placenames and streets aren't unusual to the local ethnic plurality.

Though I think George Patton was born in that vicinity. So there's that

Hari Seldon's Assistant said...

I can't believe Bryan Caplan made Mish pay up in $100 the bet over unemployment. Considering how many people have gone on disability and dropped out, and how the government is doctoring up the stats.

Anonymous said...

Now I know how/why Matt Simmons "drowned" in his bathtub.

Steve Sailer said...

"Thought experiment: it's 1992, you're the famous stat guy, and you write something cryptic like 'seems like some players are juiced'" Next question you get is Who and How do you know, and unless you stonewall those questions, suddenly you're accusing the Cansecos of the world based on no evidence whatsoever, just mumbling you can 'see at a glance what is going on'"

Maybe Bill James could have crunched some numbers?

Look, other sportswriters like Tom Boswell and Rick Telander were writing about steroids a couple of decades ago. But they didn't get to be a senior executive of the steroid-bloated Boston Red Sox, either.

In contrast, as late as 2007, Bill James was telling the Wall Street Journal:

"I strongly suspect that the influence of steroids on hitting numbers is greatly overstated by the public. ...I've never understood why nobody writes about it, but the bats are very different now than they were 20 years ago. [Barry] Bonds's bats are still different from everybody else's."

That's not just "no comment," that's really, really bad.




TGGP said...

I agree with Tabarrok a bet is a tax on bullshit. I gave Morgan Warstler 10-1 odds, because he's one of the most overconfident people I've come across on the internet.

Hari Seldon, the possibility of the stats being juked and people dropping out of the labor force was known in advance to Mish. The terms they agreed on were based on official measurements.

Anonymous said...

Yeah, I read De Vany's paper a number of years ago, and, here's the thing: He was wrong.

What was wrong about it? Did he crunch the numbers wrong? Or is his view that home runs always follow a power distribution wrong?

Anonymous said...

"Good. To understand that you don't understand is the first step toward understanding."


Look, the issue is how well he's done finding the answers to questions he has investigated. Steroids in baseball is simply a completely separate issue from evaluating performance. Barry Bonds was great when he didn't use drugs and he was awesome when he did use drugs. Sabermetrics isn't about what he was doing off the field. You are correct that stats nerds might have noticed the steroids shift but it's still an ancillary issue for sabermetrics, which is all about using quantitative tools to figure out how good someone's performance is. It has nothing to do with whether Mickey Mantle is injured or if he's an alcoholic or if he's lifting weights or if he's taking amphetamines--it's about what he does in baseball games, period.

William Boot said...

Aside from pleasing the NYT's left-leaning readership, the other incentive Silver has to skew his predictions to the left — not so much that he embarrasses himself professionally but a bit — is that he can actually effect the election.

Unless I'm much mistaken (which certainly is possible) it's been shown that polls can become self-fulfilling. People like to vote for a winner, so if the polls show that some guy is a slam dunk, undecided voters think that the majority must know something and they lean that way.

It's like movies advertising that they won the box office last week. Most people would say that the actions of others don't affect what they want to see but study after study shows otherwise.

Steve Sailer said...

"Steroids in baseball is simply a completely separate issue from evaluating performance."

Even at the purely pragmatic level of baseball decision-making, how is, say, Alex Rodriguez's giant contract working out for the Yankees now that's there's stronger drug testing?

By the way, let's keep straight that I'm not criticizing over steroids Nate Silver, who was a stripling back in 1998.

I am criticizing Bill James, and I'm pointing out that steroids was the Original Sin of the rise of sabermetrics. It was their test, and they largely flunked it.

Steve Sailer said...

"What was wrong about it? Did he crunch the numbers wrong? Or is his view that home runs always follow a power distribution wrong?"

The skulls of famous grown men, like Jose Canseco and Barry Bonds, changed shape right about the time they grew giant muscles and hit more home runs.

And then Canseco wrote a book all about the drugs he took and gave to other sluggers.

This isn't really that complicated -- it was obvious from the stats, it was obvious from the pictures, it was obvious from the rumors. And, finally, it turned out to be obvious from the drug tests.

Anonymous said...

This isn't really that complicated -- it was obvious from the stats, it was obvious from the pictures, it was obvious from the rumors. And, finally, it turned out to be obvious from the drug tests.

I agree, but I was curious about what was specifically wrong with De Vany's paper. His claim is something like that home run numbers follow a certain statistical pattern, and that the spike in home runs fell could be accounted for by it, right? Is his view of home runs following this pattern wrong, or are you just ignoring this altogether and looking at the other evidence?

Anonymous said...

Nobody denies steroids came into baseball at some point and affected the game.


The point is that sabermetrics was busy figuring out OBP/OPS vs. BA/RBI and WHIP vs. ERA.


You're just asserting that steroids was the main issue for these guys even though it wasn't.


Anonymous said...

I'd say it is surprising that there were not regular tests for steroid use, but then I'd sound like a man from mars.

In this vein, I would have expected a lot of the players to have been turned off by the negative side-effects of the drugs.

Still, I've actually heard of cases where players had even better seasons after they stopped doping. But this only happened when the players had only slightly dipped into drugs and never used them regularly. I think the negative side-effects may have been inhibiting -without the 'compensatory' effects of regular use.

Steve Sailer said...

Bill James wrote in the 1980s that he always tried to analyze using statistics what people in the game of baseball talked about. By 1993, people in baseball were talking about steroids. A baseball player's agent told me that year that "Jose Canseco is the Typhoid Mary of steroids," an assertion that Jose went to great lengths to document in his autobiography a dozen years later.

In 2007, Bill James was still talking Barry Bonds' maple wood bat. But, hey, it worked out well for Bill. He's got two World Series rings.

When Bill James goes in the Hall of Fame, Michael Lewis, Nate Silver, and Brad Pitt will make laudatory speeches and nobody will mention the unfortunate steroids thing. James will likely be in the Hall of Fame before Bonds.

beowulf said...

Matt Simmons dodged paying on that Tierney bet in the most spectacular way possible.

"Prominent oil investor Matt Simmons died of a heart attack last night at his home in North Haven, Maine, according to police reports."
http://www.businessinsider.com/matt-simmons-dies-2010-8

As for Silver's charitable bet, remember that betting on an election is illegal in most states. Since donating to charity doesn't put money in a gambler's pocket, this allows Silver to avoid the risk of having a blog posting read into evidence at a gambling trial (I doubt his home state of New York would prosecute a social bet in any event, but better safe than sorry).

Anonymous said...

"I read De Vany's paper a number of years ago, and, here's the thing: He was wrong."

If he's right, and greater player strength doesn't affect hitting statistics, then the players are wasting a lot of time in the weight room.

"Nobody denies steroids came into baseball at some point and affected the game. "

De Vany apparently does.

As for the sabermetrics end of things, it would be highly interesting to team owners if they could know that Barry Bonds would be a monster slugger well into his 30's if he had pharmaceutical assistance. You'd tend to sign good hitters that lacked strength, then put them on "strength training programs."

Whiskey said...

Steve, its even worse than that. James ignored the effects of juicing, and the likelihood that teams without effective juicers would turn the tables and demand more stringent testing. See the Dodger, Manny Ramirez, suspension of.

Bill James fundamentally with-held from his analysis because he did not want to see the whole truth. Only that part of it that was palatable.

It is the modern addiction to lies. James like most others depends on propagating and supporting the lies, ever more, to keep a complex system of Platonic Noble Lies out of sight and mind from the people.

If fans knew that nearly all the stars of MLB from the 1980's onwards were nothing but roiders, with some more disciplined than others in using and avoiding detection, interest would collapse as baseball depends on tradition and heritage.

Look at the Tour de France. [Armstrong in his defense claims he never failed a test and had no performance boosts in his career.] It is not just Lance Armstrong (all by hearsay evidence btw, no physical evidence), its pretty much all the top finisher year after year. Going back to the early 1990s. When btw the same effect could be had by living and training in places like Leadville Colorado.

James approach can be criticized on statistical grounds ALONE, let alone moral ones, in that his model is incomplete: roid users break down frequently, their gains are unsustainable with better testing, they are subject to off the field anger-issues, etc.

Whiskey said...

A more complete analysis by James in the late 1980's would have been:

A. Which players fall into the group of "strange injuries derail them frequently"

B. Which players have big and statistically weird jumps in performance and then statistically weird off-the-field issues that detract from their availability?

C. Is there a group of players who collectively have had uncanny jumps in performance that have not historically been seen by peers?

D. Is it likely that at least SOME players, impossible to determine which, have been using performance enhancing drugs to their advantage.

No, James could not say Canseco was juicing. But he could lay statistical odds of collectively, the jump in performance of the Bash Brothers and others with that of age/ability peers historically. Everyone would have understood, and more importantly would have added a more complete understanding of the game, which was James stated mission.

Steve I think you do James a disservice. For years no one in Baseball wanted him, it was unlikely he was ever going to be hired for a long time.

He was blind because he did not want to face the truth, that the game was not "beautiful" but filled with steroid ugliness. Simple as that.

Research data said...

..nobody is interested in betting against me that, say, school test scores in San Marino will continue to be higher than in Compton

Few would bet because San Marino is ALREADY and has been one of the richest countries in the world via GDP per capita and has not had its citizens enslaved for 2 centuries, or subsequent Jim Crow for almost another century. San Marino is also a very small place- a mere 24 sq miles and a population of 30,000 - barely enough to fill at the most 3 big California high schools. In short it is very poor comparison for a bet.


Moreover, I've long offered to publicly bet that various notorious Gaps in real world performance among demographic groups are not going to vanish over any tractable time frame.

^^Then you have already lost the bet. The data of James Flynn shows that in fact, clear performance gains, both relative and absolute, have been made by groups out of slavery and laboring under a century of Jim Crow discrimination. And what is a "tractable" time frame" 100 years? 50 years? Does "tractable" mean that the goalposts will be moved at whim to fit already preset conclusions? The notion of a "tractable" time frame also cuts both ways. It is by no means clear for example, that whites will catch up with Asians within a "tractable" time frame.


What if in 1969, Arthur Jensen had made public bets with leaders of the conventional wisdom, with Jensen taking the unpopular side that in 2012, 43 years later, The Gap would be about the same as in 1969?

But in fact, the data is not where it was in 1969. Jensen has lost his bet. Flynn et al 2006 demonstrates that the claim of Rushton and Jensen that blacks have made no real gains for 100 years, and that a gap of 1 standard deviation still stands is false.

Furthermore the existence of gaps is not necessarily of earth-shattering significance. Southern Italians have long had similar gaps with northern Italians. But how meaningful are they compared to other factors that impact southern Italians? Gaps in and of themselves may not be very meaningful and may only have limited correlation compared to other factors that impact the life outcomes and chances of peoples(Sowell 199s).

Mr Lomez said...

Isn't it possible that James just didn't see the steroids thing like everyone else? And that he wasn't guilty of some kind of moral failing, of "selling out," as you say?

At the time, there were plenty of reasonable seeming explanations for the crazy homerun stats and players' absurd Herculian bodies -- improvements in weight training, smaller parks, harder balls, and, yes, maple bats.

What if, in this one case, James was simply and honestly blinded? Even smart people can be wrong. That doesn't make them villains.

Steve Sailer said...

In the mid-80s, Bill James wrote about rumors he had heard of cocaine users among baseball players. He said that by this point, every player he'd ever heard rumored was a coke addict had been outed, except maybe one.

In other words, in the mid-1980s, James was boasting that he was in on the gossip about very serious private stuff about drug use among baseball players. And he felt it incumbent upon himself to tell his readers that cocaine wasn't going to screw up that many more players' careers. (Of course, young players like Gooden and Strawberry were just getting started.)

A decade later in the 1990s, when steroids were distorting statistics far more than cocaine had, James was much more famous and successful. Surely, he heard far more rumors than I heard. But he virtually avoided the topic of steroids like a plague until 2009.

Anonymous said...

No mention of Steyn's bet that the EU would lose a member?

Mr Lomez said...

Even if James had looked carefully at the issue of steroids in the 90's (I agree that he must have), he still could have reasonably concluded that the spike in homeruns was not on account of rampant steroid use.

Your argument seems to be that he was too smart to have missed something so obvious, and that he probably knew more than he let on but chose not to say anything for selfish reasons having to do with his employment with the Sox.

It's as likely to me that he was fooled like everyone else.

Anonymous said...

The essence of sports betting is that every punter considers himself 'smarter' than the generality of other punters.
You see dedicated sports fans who are so wrapped up and absorbed in their past-time always kid themselves that they have some special sort of insight, gift, knack or whatever that other punters lack, they like to think that they have some sort of 'intuition' or 'fee' for their sport, in the main using 'systems' that are personal to them and are hardly ever written down on paper, but exists as balances of various factors, externalities etc in their own minds.
Whether there is any truth in this and any punter actually comes up on top, over the long run, is a different story.

jack strocchi said...



Warning: Preemptive Gloat

Just checking in to remind everyone that back in Nov 2008 I stated that Obama would win two terms. mainly due to the voters wanting someone to stick around long enough to clean up the colossal Bush mess, a task I described as "janitorial" rather than "messianic".

I followed up in Apr 2010 when I predicted that the Tea Party would  "burn out" and Obama would win the 2012 election "comfortably". I also argued that Obama needed to focus on winning a bigger share of the "white working class" vote. It looks like the auto bailout has swung working class white Ohio into the DEM column.

Everyone goes on about Ray Fair, Doug Hibbs and Nate Silver in the US and Possum Polytics, Poll Bludger and Mumbles in AUS. They are all great quants but I am now sitting on a 6 on 6 winning streak picking US/AUS federal elections through the naughties. With a good chance of making it 7 on 7 with Obama.

Its still possible for an upset Romney victory to spoil my winning streak. Especially if base turnout favours the REPs. If the poll goes as I predicted then I would be interested to see if any other psephs can top that.

Either I am very lucky or as Jack Nicklaus once remarked, "the more I practice the luckier I get".




(Silvio) Silver said...

The notion of a "tractable" time frame also cuts both ways. It is by no means clear for example, that whites will catch up with Asians within a "tractable" time frame.

I doubt anyone here would claim that whites must at some point catch up with NE Asians so this isn't an example of tractability "cutting both ways."

Furthermore the existence of gaps is not necessarily of earth-shattering significance.

Then why the resistance (tantrums, smearing, etc) to recognizing it as one important factor among others?

Re: bets

I think the risk is too great. On important issues the cost to credibility of losing is too great compared to the benefit of winning. Consider how easy it became to be dismissive of Paul Ehrlich's line of reasoning after he lost his bet with Julian Simon. While Ehrlich, in my opinion, was a bit of a nutcase, and I'm not terribly displeased that he was discredited, was right to think about the world the way he did. Thinking people willing to ponder the sorts of issues he highlighted had no problem recognizing the bet was insignificant with respect to the big picture. But such thinking types are in short supply, and losing the bet made it that much harder to influence the bulk of the populace to think in terms that in the long run, sheesh, it's obviously incredibly important for people and governments to think in. (Ehrlich wasn't helped by having to fend off charges of raaaacism either, making his case at a time when the thinking had become that if you're white, you have a moral duty to treat the needs of people of other races as incomparably more important than the needs of people of your own race.)

Anonymous said...

The fact however is that Silver appears to be doing his best to analyze the numbers while Scarborough, and pretty much every other major pundit, appear to do little more than construct interesting narratives of little objective worth but which have the benefit of keeping the suspense required for continued viewership of this "fascinating" election.

The intransigent rejection of polling results may not be related to either a stirring of the pot for ratings purposes or an acute affliction of psychological denial and wishful thinking (my previous take).

Today on Morning Joe, Mark Halperin mentioned something to the effect of "people want to vote for the winner." I don't know the degree to which this statement is in reference to voter turn-out or 'undecideds' actually changing their votes.

In any case, it seems political operatives may have an incentive to manage, err inflate, expectations.

Anyway, that was news to me. Seems counter intuitive.

Anonymous said...

I'm amazed at how long this thread has gone on without someone mentioning Long Bets:

http://longbets.org/

Steve, you should really offer to publicly bet someone that the black/white test score gap will not close in 5 years, or 10 years, or whatever window they want.

Find every single person involved in any K-12/educational/minority uplift thing and bet them that it won't work. Free money.

rwcg said...

It seems like Steve is assuming the effect of steroids must be easily-detectable via discontinuities in a bunch of statistics. I am skeptical that this assumption is warranted or that the effect could ever have been definitively separated out from other things. So all we're left with is that James was supposed to go around blabbing about players' skulls. Ok.

In the larger picture the overall accusation seems to be that James kept his mouth shut because (after 25+ years on his hobby, statistics) he was mostly angling for a front-office job and couching all his behavior toward that end. This strikes me as unlikely, and seems an odd thing to harbor jealousy and resentment over.

It's not as if, in getting that role with the Red Sox, James was displacing a bunch of worthy people who make intelligent contributions and decisions. How many unworthy idiots and morons are given such jobs in sports like baseball just because they are former players? If you want to be jealous of unworthy people getting fun jobs in baseball, why not resent all of them, rather than Bill James?

Oh well. I have my own hobby horses.

David M. said...

From what I've read, Silver's analysis is based on one primary input - state opinion polls. Sure, he does lots of funny, complicated things to the polls, but at the end of the day, he's basically just saying that the state poll numbers show Obama likely to win, which doesn't really require a complicated spreadsheet.

Everyone in the blogosphere, whether they agree with Silver or not, seems united in their belief that the pundits are all flaming idiots because they aren't approaching matters from a statistical viewpoint like Silver. Surely many of the pundits are indeed idiots, but is it really idiotic to say that there are elements affecting the election that are difficult to throw into an equation? And therefore isn't there some plausible reason to take into account a "gut feeling" derived from other factors observed such as: a) voter enthusiasm, b) candidates' demeanor, c) rumors, d) campaigning strategies, e) fraud, etc, etc. Isn't this in a sense a more complete analysis than one simply based on state opinion polls, whose results have certainly been far from perfect in swing states.

I certainly don't have a strong feeling on who will win even now as voting starts. But I do feel that the pundits who are considering these other factors and coming to the conclusion that it's a toss up, seem to be more level-headed than Silver. But that's my gut feeling, so I guess that makes me a mouth-breathing science hater.

The Legendary Linda said...

Aside from pleasing the NYT's left-leaning readership, the other incentive Silver has to skew his predictions to the left — not so much that he embarrasses himself professionally but a bit — is that he can actually effect the election.

Unless I'm much mistaken (which certainly is possible) it's been shown that polls can become self-fulfilling. People like to vote for a winner, so if the polls show that some guy is a slam dunk, undecided voters think that the majority must know something and they lean that way.



And this is exactly why pundits should be forced to bet big money (6 figure+) on their predictions so that they are publicly humiliated every time they are wrong. That will force them to actually make predictions they believe and be more objective and analytical.

A major show like "meet the press" or "This week" could simply say to pundits, you want to come on our influential platform and give your opinions, you must be willing to back them up to the tune of 100 grand. And then if the pundits refuse, they could humiliate them by implying they are not rich enough to wager that kind of money, and perhaps when they start making some real money, they can return to the show.

I think this would really raise the level of discourse, bring much needed accountability to the chattering class, and would be great for TV ratings to boot.

It would also have been great if pundits had been forced to bet on their predictions for the Iraq war.

Anonymous said...

cocaine users among baseball players

Lefty Driesell, the old Maryland Terrapins basketball coach, studied cocaine use for his Master's degree, in 1957, and got in a boatload of controversy for pointing out that, if used properly, cocaine was definitely a performance-enhancing drug.

I guess the big problem with cocaine was the part about "if used properly" - apparently it was so fun to use that folks just kept shovelling it in their noses until they ended up dead.

ben tillman said...

Obama now at -455 with the comeback on Romney at +355.

ben tillman said...

Even if James had looked carefully at the issue of steroids in the 90's (I agree that he must have), he still could have reasonably concluded that the spike in homeruns was not on account of rampant steroid use.

Your argument seems to be that he was too smart to have missed something so obvious....

It's as likely to me that he was fooled like everyone else.


I don't believe anyone over the age of 12 was fooled.

Anonymous said...

I think you nailed Nate Silver.

I'd also suggest an alternative to the 3-1 odds you recommend. Namely that Silver call the exact percentage by which Obama will win.

Is he willing to give 86.3% on that? Because I'll happily take him on a $1000 bet on that.

Romney v Obama is a 50/50 proposition.

Anonymous said...

It is not just Lance Armstrong (all by hearsay evidence btw, no physical evidence), ...When btw the same effect could be had by living and training in places like Leadville Colorado

It wasn't hearsay evidence, it was eyewitness testimony. Eyewitness testimony is good enough to convict someone of murder and throw them in jail, so it's good enough to ban them from the sport for doping.

And no, living at altitude does not have the same effect on athletic performance as EPO or blood doping. Bjarne Riis was widely rumored to be above 60% hematocrit during his TdF win; you can perhaps drive up the hematocrit level to the high 40's or low 50's after a couple months of living at 10K+ ft. You also won't be able to train as effectively at altitude because you can't drive your body as hard without enough O2. And this leaves aside the other drugs Armstrong was using, such as T and HGH.

Anonymous said...

I followed up in Apr 2010 when I predicted that the Tea Party would "burn out"

They didn't burn out. They migrated from attending demonstrations to infiltrating the Republican party infrastructure. They then proceeded to knock off a number of powerful incumbents in the primaries.

Demonstrations and rallies ultimately don't mean much, which is perhaps why lefties are so devoted to them.

keypusher said...

Look, the issue is how well he's done finding the answers to questions he has investigated. Steroids in baseball is simply a completely separate issue from evaluating performance. Barry Bonds was great when he didn't use drugs and he was awesome when he did use drugs. Sabermetrics isn't about what he was doing off the field. You are correct that stats nerds might have noticed the steroids shift but it's still an ancillary issue for sabermetrics, which is all about using quantitative tools to figure out how good someone's performance is. It has nothing to do with whether Mickey Mantle is injured or if he's an alcoholic or if he's lifting weights or if he's taking amphetamines--it's about what he does in baseball games, period.

Check out James' Baseball Abstract, all 1000+ pages of it. You'll see that he writes about everything under the sun connected with baseball. Except steroids.

The Baseball Crank wrote something long ago about how Barry Bonds' power surge late in his career had no counterpart in any previous slugger. Bill James could have written that article in his sleep. But he didn't.

Anonymous said...

James will likely be in the Hall of Fame before Bonds.

Not if Bonds buys a ticket before him.

tommy said...

Then you have already lost the bet. The data of James Flynn shows that in fact, clear performance gains, both relative and absolute, have been made by groups out of slavery and laboring under a century of Jim Crow discrimination. And what is a "tractable" time frame" 100 years? 50 years? Does "tractable" mean that the goalposts will be moved at whim to fit already preset conclusions? The notion of a "tractable" time frame also cuts both ways. It is by no means clear for example, that whites will catch up with Asians within a "tractable" time frame.

It helps to look at all evidence and not merely evidence that supports your biases. Jensen and Rushton already replied to this one years ago.