The notion that US public schools are mediocre, or even in “crisis,” has been fed in recent decades by comparisons of American students and their international peers, often using scores on the Programme for International Student Assessment (PISA), or other tests. But are such comparisons of the “average student” useful, or even valid? What do they actually indicate? And how could we better assess what is working, what is not, and what is likely to actually improve US public education? This BBA MOOC session tackles these important questions.

Required Reading

  • Mel Riddile (2014). PISA: It’s Still ‘Poverty Not Stupid.’ National Association of Secondary School Principals. In this blog post, former Virginia state principal of the year and National Principal of the Year Mel Riddile compared each country’s poverty rate to its score on the PISA, demonstrating that countries with low poverty rates, like Finland, tend to score higher. Riddile then compared US schools by free and reduced lunch rates to their PISA scores and found a correlation between poverty rate and test scores. When he adjusted scores by taking into account student poverty, the result was a major boost in US scores, prompting him to suggest that in order to raise PISA scores in the United States, we should address poverty.
  • “Equity in the Distribution of Learning Opportunities.” In Viewing Education in the United States through the Prism of PISA (2010), pp. 32-38. Organization for Economic Cooperation and Development. This report compares the country’s share of socio-economically disadvantaged students and its PISA reading scores. The United States is one of only four countries (along with Israel, Slovenia, and Turkey) that favor socio-economically advantaged schools with more teachers, driving an unequal allocation of resources. The OECD reports that US students who scored the lowest on PISA were from lower socio-economic backgrounds. It also explores other factors that influence scores, such as single-parent families, community size, immigrant students, and students’ level of resilience.
  • James Harvey, Charles Fowler, John McKay, Gary Marx (2015). Iceberg Effect. National Superintendents Roundtable and Horace Mann League. The study employs data on inequality, support for schools, social stress/violence, and support for families to compare inputs to education across nine countries as well as assessing student and system outcomes among them to determine how they rank in each category. The United States showed poor results for three of the four inputs — economic equity, social stress, and support for families — when compared to the other countries, prompting the authors to conclude that our education system should address these issues in order to substantially improve outcomes.
  • Too Many Children Left Behind: The US Achievement Gap in Comparative Perspective. 2015. Bruce Bradbury, Miles Corak, Jane Waldfogel, and Elizabeth Washbrook. Russell Sage Foundation. Chapter 1 (introduction) and Chapter 2 (Meaning and Measurement of Equal Opportunity). This rigorous analysis of comparable data from the United States and three politically and economically similar countries – the UK, Canada, and Australia – enables the authors to assess how a variety of factors at specific points in students’ lives influence their academic achievement. Their troubling conclusion is that the combination of higher levels of inequality in the United States and a weak social safety net compared to the other nations puts American students at severe disadvantage (much of it in place before kindergarten) and limits schools’ abilities to substantially improve those students’ prospects.
  • Tienken, Christopher (2014). “PISA Problems.” AASA Journal of Scholarship & Practice, v10 n4 p4-18 Winter. Tienken asserts that PISA test scores are not an important indicator of economic success and should not be used by policymakers as meaningful insight to future economic outcomes. He argues that there are many factors that affect test scores, such as parent educational attainment and poverty, and these factors vary by country. Tienken urges Americans not to focus on PISA scores because other measures indicate that the United States is successful in preparing students to compete internationally, regardless of the scores its students are receiving on PISA.
  • Martin Carnoy, Emma Garcia, and Tatiana Khavenson (2015). Bringing it back home: Why state comparisons are more useful than international comparisons for improving US education policy. Economic Policy Institute. This paper’s authors argue that policymakers should not draw conclusions about the United States’ level of educational success based on international tests, and, that US education reform should not be based on other countries’ education systems. Carnoy, Garcia, and Khavenson use scores from the National Assessment of Educational Progress (NAEP) to illuminate major differences across states that are closely tied to student, school, and state poverty levels and that change over time. Based on these trends, the authors assert that the best way to improve education in the United States is to use information on states’ progress rather than across high-scoring countries, given the much-larger differences between the United States and these countries across the broad range of hard-to-manipulate factors that influence scores.

 

Optional Reading

  • William H Schmidt and Nathan Burroughs (2015). “Puzzling Out PISA: What Can International Comparisons Tell Us About American Education?American Educator. In keeping with the conclusions of Carnoy, Garcia, and Khavenson, Schmidt and Burroughs discuss why it makes more sense to explore variation in student performance within countries than among them. The United States has a high rate of within-country variation in formal mathematics compared to other countries that participate in the PISA. For example, US students from disadvantaged backgrounds are more likely to receive poor quality mathematics instruction. And while the United States is not the only country that systematically disadvantages low-income students in mathematics, the authors suggest that making instruction more equitable across schools would help raise PISA scores.
  • Improving Teacher Quality Around the World: The International Summit on the Teaching Profession (2012). The Asia Society Partnership for Global Learning. This paper suggests that the countries with the highest PISA scores have the most professional outlook on teachers. The authors point to evidence demonstrating the need for teachers to be highly professionalized and to have mentors and ample professional development opportunities. They urge the implementation of deliberate policy strategies that promote high-quality teaching, rather than attempting to change the nation’s cultural perception of teachers. School and curriculum reform must also be prioritized to improve the system as a whole, as teacher improvement is only one piece of the puzzle.
  • Eric Hanushek and Ludger Wöβmann (2003). Does Educational Tracking Affect Performance and Inequality? Differences-in-Differences Evidence Across Countries. National Bureau of Economic Research. In this study, tracking is defined as hierarchically structuring students by performance level. Hanushek finds that countries that track students result in larger achievement gaps, and that the younger tracking begins, the larger the inequalities on the country’s PISA scores.
  • Sahlberg, Pasi (2012). “A Model Lesson: Finland Shows Us What Equal Opportunity Looks Like.” American Educator, v36 n1 p20-27, 40 Spring. Finland has some of the highest PISA scores among tested countries, and Sahlberg, a Finnish teacher and researcher, asserts that his country’s success is the result of enhanced equitable opportunities through the development of a more comprehensive public school system. Since 1945, Finland has abolished student tracking, limited standardized testing, and professionalized teachers to a degree virtually unparalleled across the world. That said, the most reliable predictors of success in Finland’s educational system may be the country’s low poverty rates and a strong welfare system that supports the whole family, as per Linda Darling-Hammond in What we can learn from Finland’s successful school reform. She also emphasizes, however, the major impact of the education policy changes, which she finds reflected in Finland’s high-quality teacher training, equitable resources for students of all income levels with added supports for students with special needs, effective evaluation systems, and healthy balance of national control and decentralization.
  • Kuan Chen Tsai and Osman Ozturgut (2013). “PISA and Beyond: What Can We Learn from Asian Education.” Pacific-Asian Education, v25, no. 2. This paper examines the test-taking culture in Asia, and suggests that students across that region conform to a rote learning style that reflects their collectivist societies in contrast to the Western emphasis on individualistic and engaging learning. So while it is true that Asian countries outperform the United States on the PISA, Tsai encourages policymakers and educators to look at each country’s cultural climate before comparing their scores.
  • Chudgar, Amita; Luschei, Thomas F. (2009). “National Income, Income Inequality, and the Importance of Schools: A Hierarchical Cross-National Comparison.” American Educational Research Journal In this study, Chudgar finds that family background is a better indicator of student performance than schools. However, this only proved to be true in economically advanced and equitable countries. In poorer and less equitable countries, in contrast, schools had more significant impacts on student performance.

 

Slides/Graphics:

This graphic showing PISA scores by socioeconomic status/SES level across 14 countries illustrates that, across these wealthy nations, high-SES students consistently score much higher – roughly twice as high – as their lowest-SES peers. That said, at each percentile level, the United States scores among the lowest, while Canada, Finland, and Korea consistently outperform their counterparts.

average student performance by ESCS percentile, 14 countires

The Children Left Behind: A league table of inequality in child well-being in the world’s rich countries. While it is true that, across wealthy nations, high-income students far outperform their lower-income peers, the United States stands out for having many more poor students. Indeed, this is a major reason for middle-of-the-pack PISA scores; our high share of poor students drags our average down more than is true of other countries. As this table illustrates, US levels of inequality put us at the bottom among our developed counterparts, a parallel drag on scores (and students’ odds of academic and life success).

inequality in child well-being in the world's richest countries

Mel Riddile (2014). PISA: It’s Still ‘Poverty Not Stupid.’ As summarized above, Riddile compared US schools with less than 10% student poverty to other countries with similar poverty rates. In this comparison, the United States ranked first, showing that US students who attend low-poverty schools easily compete with – and indeed, far outperform – their Finnish counterparts.

 US schools with less than 10% poverty compared to similar countries

 

Film: Korean High School While Korean students tend to get high scores on the PISA, that achievement comes with a cost. This documentary preview, made by an American student, follows Korean teens through high school as they strive to reach their standard of perfect – both in school and beauty. Students are under immense pressure to study and get the highest score on the Sunung, their college entrance exam, which results in many committing suicide. Other students admit that they have no hobbies outside of school and will study for as many as 16 hours per day. Meanwhile, young girls are undergoing plastic surgery in attempt to live up to their country’s impossible beauty standards.

 

Book club:  Yong Zhao. (2009). Catching Up or Leading the Way: American Education in the Age of Globalization. ASCD. Given the impacts of globalization and technology on our world and the future of today’s students, how well are US education policies adapting to new realities? In particular, are our schools emphasizing the knowledge and skills that students need to succeed in a global society–or are they actually undermining their strengths by overemphasizing high-stakes testing and standardization? Are education systems in China and other countries really as superior as some people claim? Born and raised in China and now a distinguished professor at education at the University of Kansas, Zhao bases many of his observations on firsthand experience as a student in China and as a parent of children attending school in the United States. He asserts that American education is at a crossroads and that we need to change course to maintain leadership in a rapidly changing world. He explains what he believes is right with American education; why much of the criticism of schools in the United States has been misleading and misinformed; and why China and other nations in Asia are actually reforming their systems to be more like their American counterparts. In particular, he believes that protecting space for individualism and creativity has been key to consistent American economic strength and ingenuity and that those same emphases will serve us even better in today’s context.

 

Discussion Questions:

  • This session discusses several factors that affect PISA scores — including poverty, parent education and job status, and teacher quality — across different countries. Given all of these factors, how can we interpret scores in a valid and reliable manner? Or is it the case, as some critics charge, that there is no value in PISA and the United States should not continue to participate? Alternatively, how might we expand the analysis of scores on our own rigorous, nationally sampled exam, the National Assessment for Educational Progress (NAEP/the “Nation’s Report Card”) to advance better policies?
  • Sahlberg, Darling-Hammond, and Zhao all explain why we should not compare average US PISA scores to those of high-scoring countries, such as Finland and China. Given both unique challenges to improving the US education system and barriers that are true across countries, what would it take, policy-wise, to raise US scores? Does it even matter if the United States scores as high as Finland, and if so, do you think that would drive our education system to favor less creativity and innovation, as Zhao discussed in his book?
  • Implementation of the Every Student Succeeds Act (ESSA), which replaced NCLB as our premiere national education legislation, allows states to assess students using performance assessments and portfolios instead of standardized, traditional testing. Would international scores look different if PISA switched to this method of assessment? How might this affect global education systems?