May 5, 2006

What's worse than Freakonomics? Pseudo-Freakonomics!

Why, when economists venture out of their normal ruts, do they routinely make obvious errors? And, more puzzlingly, why don't other economists notice their mistakes?

For example, Tyler Cowen of Marginal Revolution is playing up a study by economist David Romer claiming that NFL coaches are too conservative. According to Romer, they punt way too much on fourth down instead of going for a first down, which risks turning the ball over to the other team if they fail to make the needed yardage. (By the way, for the non-fans of American football, you need to advance the ball at least 10 yards within four downs to get another first down. A "down" is a play.)

That may be true. NFL teams have gotten much better over the decades at completing short passes. The legendary Joe Namath, for example, had a career completion percentage of only 50.1%, while Peyton Manning's percentage so far is 63.9%. My vague impression, however, is that coaches haven't gotten proportionally more optimistic about going for it on fourth down, although I could be wrong. After all, the competition to become an NFL coach is fierce, and NFL offensive coordinators are not stupid or innumerate men.

Unfortunately, Romer's study is pretty close to worthless because he makes a catastrophically bad assumption (at least in the version of his paper that Tyler links to). Romer writes on p. 4 of his PDF:

"Decisions to go for it on fourth down (that is, not to kick) are sufficiently rare, however, that they cannot be used to estimate the value of trying for a first down or touchdown. I therefore use the outcomes of third down plays instead."

In other words, he didn't actually look at whether or not teams make it on fourth down. What he looked at was whether they make it on third down.

But any football fan can realize the big difference between, say, third and two and fourth and two. It's easier to pick up, say, at least two yards on third down than on fourth down, just as it's easier to pick up at least two yards on first down than on third down.

Why? Because the lower the down, the more tries you have left to get a first down, so the safer it is to try a risky play now for a long gain. If on third and two you try to throw a long pass into the end zone for a touchdown, but it falls incomplete, well you still have fourth down. But on fourth down, the penalty for failure is severe -- you turn the ball over to the other team. So, you'll probably run a conservative play to grind out the needed yardage and not much more. Knowing that, on lower downs, the defense tends to play back to prevent giving up a touchdown or long gain.

This leads Romer to make absurd recommendations to coaches. He writes:

On the team’s own half of the field, going for it is better on average as long as there are less than about 4 yards to go.

His Figure 5 (the last page in the PDF) says that when you are on your own ten yard line, you are better off going for it on fourth-and-two than you are punting. That's ridiculous. If you don't make it, you are giving your opponent an almost automatic 3-point field goal and a likely 7 point touchdown. And even if you do make it, the chance of going on to score from deep in your own territory is relatively small. Except when losing and utterly desperate at the end of the game, any coach who repeatedly went for it on fourth-and-two from his own ten yard line would be fired the next day.

The reason this economist comes up with this prima facie stupid recommendation is because on third and two on your own ten yard line (which are the actual plays he's looking at, not fourth down) the defense plays loosely because they don't much mind giving up a three yard gain for a first down out to your own 13, but they would hate giving up a 90 yard touchdown play. So, the last thing the defense will do on your 10 yard line is stack eleven men on the line of scrimmage, because if your man with the ball gets through that line, he's gone. Touchdown.

We seem to be entering an era of Imperialist Economists (a.k.a., Freakonomists) who smugly advise the human race on myriad issues only vaguely related to the traditional field of economics ... while often not knowing what the hell they are talking about. Despite all the grief I give Steven D. Levitt, he appears to be above-average compared to other would-be Freakonomists.

My published articles are archived at -- Steve Sailer

No comments: