Peyton Manning v. Tom Brady

Let's continue kibbitzing in the argument between Steven Pinker and Malcolm Gladwell over Gladwell's contention that "In fact, Berri and Simmons found no connection between where a quarterback was taken in the draft—that is, how highly he was rated on the basis of his college performance—and how well he played in the pros."

The mass of evidence suggests that, yes, there is a correlation between where a quarterback is selected in the draft and how well he'll do. Let's note, however, that the correlation glass is half full. For example, Peyton Manning, winner of tonight's 35-34 come-from-behind win over Tom Brady's New England Patriots, was chosen first overall in the 1998 NFL draft. On the other hand, Brady, whose 4th and 2 pass on his own 28 with two minutes left, was juggled by the receiver, costing New England the win, was chosen 199th in the 2000 NFL draft.

I've now read the most recent paper by Gladwell's favorites, economists David J. Berri and Rob Simmons, "Catching a Draft:"

Our analysis revealed that there was a relationship between aggregate performance and where a player was chosen. But when we looked at per play performance, the relationship between production and draft position was quite weak. In contrast, a much stronger relationship existed between how many plays a quarterback ran and where he was selected. In sum, draft position can get a quarterback on the field. But quarterbacks taken higher do not appear to perform any better.

But, Berri is using a very, very slippery approach.

First, he likes to compare quarterbacks picked in the top 10 draft picks in a year to those picked 11 to 50 or to 11 to 100. (And, he ignores the many picked below # 100, where the accuracy of the draft becomes even more apparent.) But because the teams pick in inverse order of how well they did the previous season, those top ten draft picks are going to, on average, bad teams: the worst 10 teams in the league (leaving out trades of draft choices). In contrast, picks 11 to 50 or 11 to 100 will go, on average, to better teams. All else being equal, it’s easier to be successful on a good team than a bad team, if they let you play.

And here’s Berri's other major trick: he wants to measure success on a per play basis, rather than some more useful cumulative measure, such as Pro Bowl selections.

There are obvious problems with measuring success on a per play basis, such as if you’re no good, the coaches don’t let you get many plays. Here are all the quarterbacks drafted since 1980 with their career statistics.

They’re arranged per draft order for each year. You’ll notice that a high proportion of high draft choices played a lot. Some of the low draft choices played a lot, but a lot of them barely played at all in the NFL: the team didn’t invest much in them, and when they proved in practice, unsurprisingly, to be less than NFL starting quality, they went to the bench or into insurance sales.

So, there’s a huge selection bias built into Berri’s measure of success. If you turn out in training camp to be better than the NFL draft consensus (e.g., Tom Brady), they let you play. But if you are a low draft pick and you don't prove to be better than the NFL thought you were, they don't let you play.

For example, the year Brady was picked 199th, Tee Martin was picked 163rd. In Tee's career, he completed 6 passes in 16 attempts for 69 yards, 0 touchdowns, and 1 interception. In other words, Tee Martin proved to be exactly as mediocre as you would expect a 5th round draft choice to be.

So, Brady's statistics would weight much more heavily on a per play basis than Tee Martin's.

On the other hand, if they think you are such hot stuff that they'll burn a high draft choice and millions of dollars on you because they really need a new quarterback right now, well, then they make you play a fair amount at a young age, even if you aren't ready for the NFL, and even if you aren't as good as they thought you were.

Moreover, when those lower drafted quarterbacks did play, they played typically under conditions more fruitful for success per play. Typically, they weren’t thrown in as 22-year-old rookie starters on lousy teams. In their younger years, they probably played against second-string defenses in the last minutes of blowouts. Or the starter went down on a good team, and they stepped into the driver’s seat of a high-powered machine (like Matt Cassell taking over for Tom Brady last year, whose having a harder time this year in St. Louis where he can't just throw the ball in the general direction of Randy Moss.)

My published articles are archived at iSteve.com -- Steve Sailer