It is amusing to me, and I bet it is pretty amusing to him, that Nate Silver has become so venerated even as the popular understanding of what he does remains so impoverished. ... Silver went to the trouble of writing a whole book about what he does, but it is in the nature of such books to be bought more than they are read and read more than they are understood. The proof is that Silver’s name has become inextricably linked with one of the oppressive buzzwords of 2013: “big data”.
People know “big data” has something to do with statistics, and, hey, who’s the most famous statistics guy on the planet? I’ll let you in on a bluffer’s secret: what “big data” denotes are massive realtime streams of ever-changing information, such as web traffic or the Twitter “firehose”, that can potentially be bent to commercial purposes using powerful and bleeding-edge computational techniques. Silver has always worked exclusively, at least in public, with what might be called “small data”: sets of a few hundred political polls or ballplayers’ statistical lines. He is living proof that there is money to be made applying 50- or 100-year old statistical nostrums and exploratory techniques to such small data.