Occam's Butterknife applied to PISA

A lot of the ideas that are broached early on iSteve end up being kicked around later in the prestige press, but they tend to get dumbed down in the process of replacing Occam's Razor with Occam's Butterknife. For instance, I've been saying for a decade that looking at test scores internationally can say a lot about the future of a country. Today, Tom Friedman today writes in the NYT about PISA scores. 

A team from the Organization for Economic Cooperation and Development, or O.E.C.D., has just come out with a fascinating little study mapping the correlation between performance on the Program for International Student Assessment, or PISA, exam — which every two years tests math, science and reading comprehension skills of 15-year-olds in 65 countries — and the total earnings on natural resources as a percentage of G.D.P. for each participating country. In short, how well do your high school kids do on math compared with how much oil you pump or how many diamonds you dig? 

 The results indicated that there was a “a significant negative relationship between the money countries extract from national resources and the knowledge and skills of their high school population,” said Andreas Schleicher, who oversees the PISA exams for the O.E.C.D. “This is a global pattern that holds across 65 countries that took part in the latest PISA assessment.” Oil and PISA don’t mix. (See the data map at: http://www.oecd.org/dataoecd/43/9/49881940.pdf.) 

I can't reproduce the scatter plot here, but I'm not blown away by the correlation I see when I look at it. It's better to be smart than endowed with lots of natural resources, but, overall, it's best to be both.

As the Bible notes, added Schleicher, “Moses arduously led the Jews for 40 years through the desert — just to bring them to the only country in the Middle East that had no oil. But Moses may have gotten it right, after all. Today, Israel has one of the most innovative economies, and its population enjoys a standard of living most of the oil-rich countries in the region are not able to offer.” 

Actually, Israel's overall PISA scores are mediocre. Israel does worse on the PISA than Russia, which has a resource-driven economy. Israel has a smart fraction, definitely, but even that doesn't appear to be all that spectacular according to what PISA measured. (This may say more about limitations in PISA than about Israel.)

So hold the oil, and pass the books. According to Schleicher, in the latest PISA results, students in Singapore, Finland, South Korea, Hong Kong and Japan stand out as having high PISA scores and few natural resources, while Qatar and Kazakhstan stand out as having the highest oil rents and the lowest PISA scores. (Saudi Arabia, Kuwait, Oman, Algeria, Bahrain, Iran and Syria stood out the same way in a similar 2007 Trends in International Mathematics and Science Study, or Timss, test, while, interestingly, students from Lebanon, Jordan and Turkey — also Middle East states with few natural resources — scored better.) 

Also lagging in recent PISA scores, though, were students in many of the resource-rich countries of Latin America, like Brazil, Mexico and Argentina. Africa was not tested. Canada, Australia and Norway, also countries with high levels of natural resources, still score well on PISA, in large part, argues Schleicher, because all three countries have established deliberate policies of saving and investing these resource rents, and not just consuming them. 

Norway did mediocre on the latest PISA relative to, say, Finland. Canada and Australia score well because they are the only rich countries whose immigrants, first and second generation, don't drag down the national averages. That's because they carefully select immigrants to, explicitly, boost the welfare of natives. (Finland has very few 15-year-old immigrants, too)

Add it all up and the numbers say that if you really want to know how a country is going to do in the 21st century, don’t count its oil reserves or gold mines, count its highly effective teachers, involved parents and committed students. “Today’s learning outcomes at school,” says Schleicher, “are a powerful predictor for the wealth and social outcomes that countries will reap in the long run.”

Indeed.

But, the reverse correlation between test scores and natural resources asserted here is mostly a statistical illusion due to 2 problems:

1) Which countries participate in PISA and which don't. Countries that are dumb but rich due to oil are more likely to take part in PISA than countries that are dumb but poor. For example, Saudi Arabia has more money than brains, so it took part in PISA. Its Arab neighbor Yemen would probably score near the bottom of PISA if it had enough oil to pay for a bureaucracy to implement the tests. But it doesn't have much of anything, so it doesn't bother. Same with most African countries, few of which participate in PISA or TIMSS.

There can often be a resource curse where mineral wealth causes civil wars, coups, and laziness. But, if you've got smart, hard-working people, more resources is better than fewer resources, on the whole.

2) They are measuring how much resources a country has by what percentage of its economy stems from natural resources. So, dumb countries that don't have anything else going for them beside natural resources are measured as having lots of natural resources, while smart countries with lots of resources, like Australia, Canada, and the United States, also do other things economically, so they don't appear to be as dependent upon natural resources as dumb countries. 

In general, the happiest thing to be is a smart, large, resource rich, relatively-lightly populated English-speaking country with a British-descended system of government and culture. Australians don't call Australia "the lucky country" for no reason.

My old articles are archived at iSteve.com -- Steve Sailer