The Amazing Truth About PISA Scores: USA Beats Western Europe, Ties with Asia

Once we correct (even crudely) for demography in the 2009 PISA scores, (PISA is the Program for International Student Assessment) American students outperform Western Europe by significant margins and tie with Asian students. Jump to the graphs if you don't want to read my boring set-up and methodology.

The main theme in my blog is that we shouldn’t confuse policy with culture, and with demographic factors.

For instance, education scholars have known for decades that the home environment of the kids and the education levels of the parents are very important for student outcomes. We also know that immigrant kids have a more difficult time at school, in part because they don’t know the language.

Take me as an example. The school me and my brother attended was in a basement in Tehran, had no modern resources, and largely focused on religious indoctrination. But we had a good home background. Our father attended a college in the west a few years (our mother didn’t, despite stratospheric scores test scores, because at the time you didn’t send a good Kurdish girl to another city to study). So we did well in school. Conversely, the first few years in Sweden I had bad grades, in part because I didn’t master the language.

The point I am trying to make is that the school in Sweden was objectively superior to the school in Iran. But I scored lower in Sweden, because of factors outside the control of the education system. If you want to compare the effect of the school, you have to isolate those external factors and make an apples-to-apples comparison.

However, this is not at all how the media is presenting the recent PISA scores. For example there is a lot of attention of the score of the kids in Shanghai, the according to the NYT is supposed to “stun” us or something.

It's dumb to compare one of the most elite cities in a country with entire nations, and to draw policy-inference from such a comparison. Shanghai has 3 times the average income of China! It is also naive to trust the Chinese government when they tell us the data is representative of the entire nation. Either you compare Shanghai to New York City, or you compare the entire country of China, including the rural part, with other large nations. Most of the news and policy conclusions we read about PISA-scores in the New York Times is thus pure nonsense.

1. Correcting for the demography:

In almost all European countries, immigrants from third world countries score lower than native born kids.

Why? No knows exactly why. Language, culture, home environment, income of parents, the education level of the parents and social problems in the neighborhood and peer groups norms are among likely explanations. But it is generally not true that the schools themselves are worse for immigrants than natives. In welfare states, immigrants often (thought not always) go to the same or similar schools and have as much or likely more resources per student.

So the fact that immigrant students in mixed schools do worse than Swedish kids used to a few decades ago in homogeneous schools does not it out of itself prove that Swedish public schools have become worse.

Of course, the biggest myth that the media reporting of PISA scores propagates is that the American public school system is horrible.

The liberal left in U.S and in Europe loves this myth, because they get to demand more government spending, and at the same time get to gloat about how much smarter Europeans are than Americans. The right also kind of likes the myth, because they get to blame social problems on the government, and scare the public about Chinese competitiveness.

We all know that Asian students beat Americans students, which "proves" that they must have a better education system. This inference is considered common sense among public intellectuals. Well, expect for the fact that Asian kids in the American school system actually score slightly better than Asian kids in North-East-Asia!

So maybe it’s not that there is something magical about Asian schools, and has more to do with the extraordinary focus on education in Asian culture, with their self-discipline and with their favorable home environment.

There are 3 parts to the PISA test, Reading, Math, and Science. I will just make it simple and use the average score of the 3 tests. This is not strictly correct, but in practice it doesn’t influence the results, while making it much easier for the reader. (the reason it doesn't influence the results is that countries that are good at one part tend to be good at other parts of the test.)

The simplest thing to do in order to get an apples-to-apples comparison is to at least correct for demography and cultural background. For instance, Finland scores the best of any European country. However first and second generation immigrant students in Finland do not outperform native Swedish, and score 50 points below native Finns (more on this later).

On PISA, 50 points is a lot. To give you a comparison, 50 points is larger than the difference between Sweden and Turkey. A crude rule of thumb here is that 50 points is 0.5 standard deviations.

The problem is that different countries have different share of immigrants. Sweden in 2009 PISA data had 17%, and Finland 4%. It’s just not fair to the Swedish public school system to demand that they must produce the same outcome, when Sweden has many more disadvantaged students. Similarly schools with African-American students who are plagued by racism, discrimination, crime, broken homes, poverty and other social problems are not necessarily worse just because their students don’t achieve the same results as affluent suburbs of Chicago. In fact, the most reliable data I have seen suggests that American minority schools on average have slightly more money than white schools. It’s just that the social problems they face are too much to overcome for the schools. It is illogical to blame the public school system for things out of its hands.

So let’s start by removing those with foreign background immigrants from the sample when comparing European countries with each other. I define immigrants here as those with a parent born outside the country, so it includes second generation immigrants. This is fairly easy for Europe.

In the case of America, 99% of the population originates from other countries, be they England, Italy, Sweden, India, Africa, Hong-Kong or Mexico. If we want to isolate the effect of the United States public school system, we should compare the immigrant groups with their home country. For those majority of Americans whose ancestors originate from Europe, we obviously want to compare them with Europe. For some groups, such as Indians, this is inappropriate. The reason is that mainly the most gifted Indians get to migrate to America to work or study.

However, as I have argued previously, there is strong reason to believe that this problem of so called biased selection does not apply to historic European migration to the United States at the aggregate level. The people who left Europe were not better educated than those who stayed. Immigrants were perhaps more motivated, but often poorer than average.

So similar to my comparison of GDP levels, let us compare Americans with European ancestry (about 65% of the U.S population, and not some sort of elite) with Europeans in Europe. We remove Asians, Mexicans, African-Americans and other countries that are best compared to their home nations. In Europe, we remove immigrants.

The results are astonishing at least to me. Rather than being at the bottom of the class, United States students are 7th best out of 28, and far better than the average of Western European nations where they largely originate from.

The mean score of Americans with European ancestry is 524, compared to 506 in Europe, when first and second generation immigrants are excluded. So much for the bigoted notions that Americans are dumb and Europeans are smart. This is also opposed to everything I have been taught about the American public school system.

For Asian-American students (remember this includes Vietnam, Thailand and other less developed countries outside Northeast Asia), the mean PISA score is 534, same as 533 for the average of Japan, South Korea, Singapore and Hong Kong. Here we have two biases going in opposite directions: Asians in the U.S are selected. On the other hand we are comparing the richest and best scoring Asian countries with all Americans with origin in South and East Asia.


2. Policy-Implications

Libertarians in the United States have often claimed that the public school system (which has more than 90% of the students) is a disaster. They blame this on government control and on teachers unions. However, it is completely unfair to demand that a public school in southern California where most of the students are recent immigrants from Mexico whose parents have no experience in higher education (only 4% of all Mexican immigrates have a college degree, compared to over 50% of Indian immigrants) should perform as well as a private school in Silicon Valley.

The libertarians have no answer why European and Asian countries that also have public school systems score higher than the United States (unadjusted for demography). Top scoring Finland has strong teacher unions, just as California.

Similarly, the left claims that the American education system is horrible, because Americans don’t invest enough in education. The left has no answer when you point out that the United States spends insanely more than Europe and East Asia on education. According to the OECD, the United States spends about 50% more per pupil than the average for Western Europe, and 40% more than Japan.



Another policy implication is that Europe can learn from American public schools, which appear to be better than most European countries. I can only compare Sweden with the U.S, but I can tell you that from my experience, the American system is superior. I always thought this was just anecdotal evidence, but I am beginning to realize that American schools are indeed better.

For example, we don’t have any real equivalent to Advanced Placements classes. We have cheaper and worse textbooks. The teachers on average have far less education. I could go on.

Nor is it any longer a mystery to me why Americans spend so much more on education and (falsely appear) to get out less in output.

But of course the biggest implication is that most Europeans and all American liberals have lost the bragging right about their side being smarter than Americans.

3. Immigrant PISA scores compared to natives

This is again the mean difference of the 3 parts of PISA.


Australia is the only country with a negative gap, which means Australian immigrants actually score better than natives. Canada is similar. The Australian-Canadian skill based migration system is at work here, generating less inequality (even short term).

The other pattern appears to be that the gap is almost constant in the remaining Western European countries. This may be important to keep in mind, whenever people claim that uniquely Swedish policies are causing poor immigrant educational outcomes.

Tino Sanandaji is of Kurdish origins, and was born in Iran in 1980. In 1989 he moved together with his mother and brother to Sweden. He has a degree from the Stockholm School of Economics, his M.A in Economics from the University of Chicago and is expected to receive his PhD in Public Policy from the University of Chicago in 2011. His work has appeared in The Wall Street Journal, The New York Times, National Review, and numerous Swedish newspapers. Tino has been a resident of Hyde Park Chicago, since 2004.