December 11, 2012

New TIMSS and PIRLS test results

There are new results out from two international school achievement tests:

The U.S. average mathematics score at grade 4 (541)
was higher than the international TIMSS scale average,
which is set at 500.
• At grade 4, the United States was among the top 15
education systems in mathematics (8 education systems
had higher averages and 6 were not measurably
different) and scored higher, on average, than 42
education systems.
•  The 8 education systems with average mathematics
scores above the U.S. score were Singapore, Korea,
Hong Kong-CHN, Chinese Taipei-CHN, Japan,
Northern Ireland-GBR, North Carolina-USA, and
Belgium (Flemish)-BEL. ...
• At grade 8, the United States was among the top 24
education systems in mathematics (11 education
systems had higher averages and 12 were not
measurably different) and scored higher, on average,
than 32 education systems.
• The 11 education systems with average mathematics
scores above the U.S. score were Korea, Singapore,
Chinese Taipei-CHN, Hong Kong-CHN, Japan,
Massachusetts-USA, Minnesota-USA, the Russian
Federation, North Carolina-USA, Quebec-CAN, and
Indiana-USA.
At grade 8, the United States was among the top
23 education systems in science (12 education
systems had higher averages and 10 were
not measurably different) and scored higher,
on average, than 33 education systems.
• The 12 education systems with average science scores
above the U.S. score were Singapore, MassachusettsUSA,
Chinese Taipei-CHN, Korea, Japan, MinnesotaUSA,
Finland, Alberta-CAN, Slovenia, the Russian
Federation, Colorado-USA, and Hong Kong-CHN. 

For example, here is 8th grade math:
Grade 8
Education systemAverage score
TIMSS scale average500
Korea, Rep. of613△
Singapore1611△
Chinese Taipei-CHN609△
Hong Kong-CHN586△
Japan570△
Russian Federation1539△
Israel2516
Finland514
United States1509
England-GBR3507
Hungary505
Australia505
Slovenia505
Lithuania4502
Italy498▽
New Zealand488▽
Kazakhstan487▽
Sweden484▽
Ukraine479▽
Norway475▽
Armenia467▽
Romania458▽
United Arab Emirates456▽
Turkey452▽
Lebanon449▽
Malaysia440▽
Georgia4,5431▽
Thailand427▽
Macedonia, Rep. of6426▽
Tunisia425▽
Chile416▽
Iran, Islamic Rep. of6415▽
Qatar6410▽
Bahrain6409▽
Jordan6406▽
Palestinian Nat'l Auth.6404▽
Saudi Arabia6394▽
Indonesia6386▽
Syrian Arab Republic6380▽
Morocco7371▽
Oman6366▽
Ghana7331▽
Benchmarking education systems
Massachusetts-USA1,4561△
Minnesota-USA4545△
North Carolina-USA2,4537△
Quebec-CAN532△
Indiana-USA1,4522△
Colorado-USA4518
Connecticut-USA1,4518
Florida-USA1,4513
Ontario-CAN1512
Alberta-CAN1505
California-USA1,4493▽
Dubai-UAE478▽
Alabama-USA4466▽
Abu Dhabi-UAE449▽
△ Average score is higher than U.S. average score.
▽ Average score is lower than U.S. average score.
There are a bunch of different tables like this for different subjects in different grades, so don't take this one all that seriously. I just plunked it in because it was handy.

Here's 8th grade science:

Grade 8
Education systemAverage score
TIMSS scale average500
Singapore1590△
Chinese Taipei-CHN564△
Korea, Rep. of560△
Japan558△
Finland552△
Slovenia543△
Russian Federation1542△
Hong Kong-CHN535△
England-GBR2533
United States1525
Hungary522
Australia519
Israel3516
Lithuania4514▽
New Zealand512▽
Sweden509▽
Italy501▽
Ukraine501▽
Norway494▽
Kazakhstan490▽
Turkey483▽
Iran, Islamic Rep. of474▽
Romania465▽
United Arab Emirates465▽
Chile461▽
Bahrain452▽
Thailand451▽
Jordan449▽
Tunisia439▽
Armenia437▽
Saudi Arabia436▽
Malaysia426▽
Syrian Arab Republic426▽
Palestinian Nat'l Auth.420▽
Georgia4,5420▽
Oman420▽
Qatar419▽
Macedonia, Rep. of407▽
Lebanon406▽
Indonesia406▽
Morocco376▽
Ghana6306▽
Benchmarking education systems
Massachusetts-USA1,4567△
Minnesota-USA4553△
Alberta-CAN1546△
Colorado-USA4542△
Indiana-USA1,4533
Connecticut-USA1,4532
North Carolina-USA3,4532
Florida-USA1,4530
Ontario-CAN1521
Quebec-CAN520
California-USA1,4499▽
Alabama-USA4485▽
Dubai-UAE485▽
Abu Dhabi-UAE461▽
△ Average score is higher than U.S. average score.
▽ Average score is lower than U.S. average score.


I have no idea how representative the samples are, or how hard the students felt like trying.

These things are a lot of work to set up. Think of how hard it would be to coordinate all over the world in all these different languages. Then try to think about all the things that could go wrong if you were in charge. It's pretty daunting.

Looking at all 53 education systems that participated in PIRLS at grade 4 (i.e., both countries and other education systems), the United States was among the top 13 education systems in average reading scores. The five education systems that had higher average scores were Hong Kong-CHN, FloridaUSA, the Russian Federation, Finland, and Singapore. Seven education systems, Northern Ireland-GBR, Denmark, Croatia, Chinese Taipei-CHN, Ontario-CAN, Ireland, and England-GBR, had average scores not measurably different from the U.S. average score. The United States had higher average reading scores than 40 education systems.

Education system Overall reading average scale score
   PIRLS scale average 500
Hong Kong-CHN1 571
Russian Federation 568
Finland 568
Singapore2 567
Northern Ireland-GBR3 558
United States2 556
Denmark2 554
Croatia2 553
Chinese Taipei-CHN 553
Ireland 552
England-GBR3 552
Canada2 548
Netherlands3 546
Czech Republic 545
Sweden 542
Italy 541
Germany 541
Israel1 541
Portugal 541
Hungary 539
Slovak Republic 535
Bulgaria 532
New Zealand 531
Slovenia 530
Austria 529
Lithuania2,4 528
Australia 527
Poland 526


France 520
Spain 513
Norway5 507
Belgium (French)-BEL2,3 506
Romania 502
Georgia4,6 488
Malta 477
Trinidad and Tobago 471
Azerbaijan2,6 462
Iran, Islamic Rep. of 457
Colombia 448
United Arab Emirates 439
Saudi Arabia 430
Indonesia 428
Qatar2 425
Oman7 391
Morocco8 310
Benchmarking education systems
Florida-USA1,4 569
Ontario-CAN2 552
Alberta-CAN2 548
Quebec-CAN 538
Andalusia-ESP 515
Dubai-UAE 476
Maltese-MLT 457
Dhabi-UAE 424


on

23 comments:

  1. IHTG12/11/12, 3:30 AM

    Israel went up 17 places. I hope there was no cheating involved.

    ReplyDelete
  2. Anonymous12/11/12, 4:16 AM

    The rankings appear to be quite different from the PISA results. For example, the US and Russia do rather well here, and some countries, e.g., Australia and New Zealand, surprisingly poorly.

    ReplyDelete
  3. Anonymous12/11/12, 4:34 AM

    I don't believe these scores. The US is ranked too high.

    ReplyDelete
  4. Anonymous12/11/12, 5:06 AM

    It's striking how poor Muslim countries are performing, even superrich oil nations like Qatar. You can also see this effect in Western countries in that those with a large Muslim minority, like for instance Sweden is clearly outperformed by Finland in spite of the two countries being so similar in most other ways.

    ReplyDelete
  5. rightsaidfred12/11/12, 5:26 AM

    Interesting results, though it is as we expect: not so much the education system as it is the inherent abilities of the people in the system.

    Florida USA ranked high a couple of times, which tells me the results come from some selected areas. Florida's school results overall are a bit dismal. From my experience, less ambitious people move there because the "living is easy".

    ReplyDelete
  6. TH12/11/12, 6:12 AM

    I calculated some correlations between the PISA 2009 (15-y/o's), TIMSS 2011 (8th grade), and PIRLS 2011 (10-y/o's).

    The correlation between PISA 2009 math and TIMSS 2011 math is 0.87 (n=26). In both studies, East Asians are at the top, white-majority countries at the middle, and others at the bottom. However, if you look only at white-majority countries, the correlation is 0.19 (n=13). Russia and Israel do particularly well in the TIMSS compared to the PISA. The former is supposed to be a more math-heavy test compared to the latter which is a test of "mathematics literacy".

    In the US, the racial breakdown of the TIMSS scores in grade 8 is as follows (SD=100):

    White 530
    Black 465
    Hispanic 485
    Asian 568
    Multiracial 513

    Hispanics slightly outscore Norway and Sweden in the TIMSS, while Norway and Sweden score only slightly (0.1 SD or so) higher than US blacks. In the PISA math test, Norway and Sweden outscored US Hispanics by 0.3-0.4 SD and US blacks by about 0.7 SD.

    The correlation between PISA 2009 reading and PIRLS 2011 is 0.81 (n=36). Among white-majority countries (n=18) the correlation is 0.24.

    In the US, the racial breakdown of the PIRLS scores is as follows (SD=100):

    White 575
    Black 522
    Hispanic 532
    Asian 588
    Multiracial 578

    The black average is higher than that of, for example, France, Spain, Norway, and Belgium. In the PISA reading test, each of those four countries outscored US blacks by more than 0.5 SD.

    ReplyDelete
  7. Anonymous12/11/12, 6:17 AM

    Florida or Texas do bad because of hispanics. Granted, there hispanics do better than Californias but the big urban areas there are like La or Santa Ana do many hispanics. North Carolona does good for 19th percent black.

    ReplyDelete
  8. Anonymous12/11/12, 6:22 AM

    Where the hell is India? Is it possible that they scored worse than Ghana? Maybe they should be booted from the whole BRICs club. Seriously, India got more poor people than all 54 african countries combined!

    And where does india score on the IQ scale, I'm really curious. Maybe this whole asians are smarter thing, only applies to the chinese and japs.

    And how does an Afro-Caribean country like Trinidad beat so many mostly caucasian muslim and south american states? Even in PISA Trinidad ranks ahead of states like Qatar. You would think that theyd be down the list with their cousin Ghana. Maybe its time to redefine the racial categories of the world, coz some of our caucasian 'brothers' are letting us down.

    ReplyDelete
  9. Anonymous12/11/12, 7:42 AM

    I don't believe these scores. The US is ranked too high.

    Russia also may be ranked too high:
    http://alfin2100.blogspot.com/2012/11/putins-russia-crumbling-of-potemkin.html

    http://alfin2100.blogspot.com/2012/11/business-in-russia-etc-worrisome-future.html

    ReplyDelete
  10. Eric Rasmusen12/11/12, 8:22 AM

    It would be nice to have the standard deviations for each country. They are interesting in themselves. Also, one could do a racial-composition adjustment then.

    ReplyDelete
  11. Anonymous12/11/12, 8:24 AM

    ..

    Here's an HBD Dictionary:

    http://www.humanbiologicaldiversity.com/#Dictionary

    ..

    ReplyDelete
  12. corvinus12/11/12, 8:25 AM

    I wonder if the USA scores are more heavily represented by white and Asian students than in other surveys which show the USA not doing very well. I would agree that the stereotypical stupidity of Americans, especially white Americans, is somewhat overblown.

    ReplyDelete
  13. Southern Man12/11/12, 9:00 AM

    I've taught undergraduate and graduate students, both domestic and international, for fifteen years, and I can assure you that the average US domestic undergrad is VASTLY more prepared in math and science than the average incoming international grad student.

    ReplyDelete
  14. Anonymous12/11/12, 9:01 AM

    Yeah. Americans just aren't trying hard. That's why we aren't number 1.

    ReplyDelete
  15. Anonymous12/11/12, 9:52 AM

    Way to go Alberta! No idea why they do so poorly in math compared to science and reading.

    ReplyDelete
  16. Anonymous12/11/12, 10:15 AM

    I'm surprised that California ranked as high as it did (although still pretty dismal IMHO).

    Probably relied on test results from wealthy white/Asian suburbs in Silicon Valley.

    ReplyDelete
  17. Anonymous12/11/12, 10:23 AM

    No China or Shanghai results/participation?

    goatweed

    ReplyDelete
  18. gmr12/11/12, 11:27 AM

    I wonder if it makes more sense to compare the top quarter of students, instead of the averages. The top quarter are the ones that are going to be in charge. For the bottom quarter, who will never really contribute much to society, they just need basic skills.

    Comparing reading across countries is problematic because some languages are much easier to learn than others. For instance, while Finnish may be difficult for native English speakers to learn, if you grow up speaking and reading Finnish, spelling will almost never be a problem: there's pretty standard orthography, without weird spellings. English, on the other hand, is filled with odd spellings, with silent letters, dropped letters, homonyms, etc.

    ReplyDelete
  19. Anonymous12/11/12, 11:37 AM

    http://www.amren.com/news/2012/12/whites-stayed-home-and-re-elected-obama/

    ReplyDelete
  20. FirstComment12/11/12, 12:43 PM

    Here's a nature versus factoid. Israeli Arab 8th graders got 465 on the math test, which is higher than UAE, Turkey, Iran, Syria. Jordan. Morocco, PA, Qatar, Saudi Arabia and Lebanon.

    ReplyDelete
  21. jody12/11/12, 12:47 PM

    seems too affected by motivation and interest levels.

    ReplyDelete
  22. rec1man12/11/12, 5:20 PM

    @Anon, Trinidad has 45% Indian population mainly derived from low caste farm workers

    ReplyDelete
  23. Florida resident12/17/12, 10:37 AM

    I am surprised that Mr. Sailer uses this term:
    “Benchmarking education systems Florida-USA 569”.
    I doubt that it should be “system”.
    See “Bad Students, not Bad Schools”
    by Robert Weissberg,
    http://www.amazon.com/Bad-Students-Not-Schools/dp/141281345X

    Meanwhile I am glad for my state of Florida.
    Two my kids did OK here.

    ReplyDelete

Comments are moderated, at whim.