PISA 2022 Results (Volume I)

The State of Learning and Equity in Education

Report

PISA

5 December 2023

5. Changes in performance and equity in education between 2018 and 2022

Abstract

This chapter discusses short-term changes in student mean performance and the performance of high- and low-achieving students between 2018 and 2022. The chapter also analyses how these changes relate to students’ gender and socio-economic advantage.

For Australia, Canada, Denmark, Hong Kong (China), Ireland, Jamaica, Latvia, the Netherlands, New Zealand, Panama, the United Kingdom and the United States, caution is required when interpreting estimates as one or more PISA sampling standards were not met (see Reader’s Guide, Annexes A2 and A4).

This chapter examines changes in performance between the previous PISA assessment, which took place in 2018, and the latest 2022 assessment. The next chapter discusses long-term trends in student performance stretching over a decade or more.

What the data tell us

Between 2018 and 2022, and on average across 35 OECD countries, mean performance dropped by almost 15 score points in mathematics and 10 score points in reading but did not change significantly in science. In mathematics and reading, most countries and economies that can compare PISA 2022 results to PISA 2018 dropped in mean performance (41 countries/economies in mathematics, 35 in reading). In contrast, science performance remained broadly stable in many countries/economies (33 out of 71) between 2018 and 2022, and even improved in 18.
Only four countries and economies improved their performance between PISA 2018 and 2022 in all three subjects: Brunei Darussalam, Cambodia, the Dominican Republic and Chinese Taipei. Performance improved in reading and science but not in mathematics in Japan, Panama* and Qatar. Performance improved in mathematics but declined in reading and remained stable in science in Saudi Arabia.
The gap between the highest-scoring students (10% with the highest scores) and the weakest students (10% with the lowest scores) increased only modestly in reading and mathematics on average across OECD countries: performance dropped in these subjects to a similar extent for both high- and low-achievers. In contrast, in science the average gap widened by about 10 score points between 2018 and 2022 on average across OECD countries: declines in science performance were only observed among lower-achieving students. The range of performance observed among 15-year-olds widened significantly in all three subjects in Finland and the Netherlands*; it narrowed significantly in the Republic of Moldova, the Republic of North Macedonia, Qatar and Saudi Arabia.
The socio-economic gap in mathematics performance did not change between 2018 and 2022 in 51 out of the 68 countries/economies with available PISA data; it widened on average across OECD countries and in 12 countries/economies; and it narrowed in five countries/economies (Argentina, Chile, the Philippines, Saudi Arabia and the United Arab Emirates).
The gender gap in mathematics performance did not change between 2018 and 2022 in most countries/economies (57 out of the 72 with comparable data). The gender gap in mathematics performance widened on average across OECD countries (by four score points in favour of boys) and in 11 countries/economies, and narrowed in four (Albania, Baku [Azerbaijan], Colombia and Montenegro).

When interpreting these comparisons, it is important to remember that the most recent years have been marked by the COVID-19 pandemic as well as other changes in education and society. First, children born around 2002 and 2006 who took the PISA test in 2018 and 2022 likely had different educational and life experiences from previous cohorts, but not all differences are due to the pandemic. Differences in the educational experiences of 15-year-olds across countries and over recent years are discussed in Volume II. Second, social and demographic trends such as international migration and widening participation in secondary education may have altered the student population that sat the most recent PISA assessments. Such changes are described and analysed in later chapters of this volume. Chapter 6 compares not only children born around 2006 to those born four years before them but explores how education performance and equity have changed over the past decade and more. The final chapter focuses on students with an immigrant background.

Three benchmarks for interpreting performance changes over time

What represents a small or large change in PISA test scores? Test scores, unlike physical units such as metres or grams, do not have meaning that readers can relate to in their own life experience. Describing a difference in test scores in terms that are familiar to most readers is not easy.

In this report, we propose three benchmarks for interpreting test-score differences.

A first benchmark, which defines a “large” change, is 20 score points. This is approximately equivalent to the typical annual learning gain by students around the age of 15 (Box I.5.1). Put differently, 20 points represents the average pace of learning of 15-year-olds in countries that participate in PISA.

A second benchmark, which distinguishes between “small” differences and differences that are “medium” or “large”, is 10 score points. Changes of up to 10 points have been routinely observed in PISA over periods of three years, the typical interval between two consecutive assessments (Box I.5.2).

The third benchmark considers the statistical uncertainty intrinsic to PISA indicators. This uncertainty produces variation in mean scores over time, even in the absence of any real change in how students perform on the test (see Box I.5.3). Differences likely to be observed in the data even in a perfectly controlled setting due to this intrinsic uncertainty are described as “non-significant” differences. Countries/economies whose results do not differ significantly between two consecutive assessments are classified as having “stable” results.

These three benchmarks help interpret score differences. However, significant differences in PISA scores, whether small, medium, or even, large, do not automatically mean that such differences reflect real differences in what PISA intends to measure – namely, what students know and can do. For example, PISA results may also reflect differences in student motivation and effort on testing day or more generally, the conditions that surrounded the administration of the test, which can affect how students engage with the test. Appendix A8 reports on a number of analyses to monitor student engagement in PISA 2022 and how it compares to PISA 2018. Throughout this chapter, these analyses are mentioned whenever they provide meaningful context for interpreting comparisons between 2018 and 2022 results.

Changes in performance between 2018 and 2022

Changes in mean performance between 2018 and 2022

Figure I.5.1 shows the changes in mean performance between 2018 and 2022 in mathematics, reading and science. On average across 35 OECD countries, mean performance dropped by almost 15 score points in mathematics and about 10 score points in reading. However, it did not change significantly in science. Given that change in the OECD average over consecutive PISA assessments up to 2018 had never exceeded four score points in mathematics and five score points in reading, these 2022 results are unprecedented. They point to a shock that pushed down performance in many countries over the 2018-2022 period.

In mathematics and reading, about half of countries/economies that can compare PISA 2022 and 2018 (or 2017) results showed a drop in mean performance (41 out of 72 in mathematics; 35 out of 71 in reading). A drop in performance was also observed in Spain (where the most recent comparison is to 2015 results). In contrast, science performance remained broadly stable in many countries and economies (33 out of 71) between 2018 and 2022.

Figure I.5.1. Change between 2018 and 2022 in mean mathematics, reading and science performance

In many cases, the drop exceeded 20 score points, i.e. the yearly gain in test scores that is typically observed among students around the age of 15. This means that 15-year-olds in these countries in 2022 scored at or below the level expected of 14-year-olds in 2018.

In mathematics, the decline in performance was most pronounced in Albania, Jordan, Iceland, Norway and Malaysia (in descending order), where it exceeded 30 score points. Drops of more than 20 score points in mean mathematics scores were also observed in Baku (Azerbaijan), Denmark*, Finland, France, Germany, Greece, Montenegro, the Netherlands*, Poland, Portugal, the Slovak Republic, Slovenia, Sweden and Thailand.
In reading, the decline in performance exceeded 30 score points in Albania, Iceland and North Macedonia (in descending order). Drops between 20 and 30 score points were observed in Baku (Azerbaijan), Finland, Hong Kong (China)*, Malaysia, Morocco, the Netherlands*, Norway, Poland and Slovenia, as well as (between 2015 and 2022) in Spain.
In science, the decline in performance exceeded 20 score points in Albania, Iceland, Malaysia and North Macedonia.

Many more countries/economies that are not listed in previous paragraphs experienced performance declines between 2018 and 2022. In contrast, four countries and economies improved their performance in all three subjects: Brunei Darussalam, Cambodia, the Dominican Republic and Chinese Taipei. Performance improved in reading and science but not mathematics in Japan, Panama* and Qatar. Performance improved in mathematics but declined in reading and remained stable in science in Saudi Arabia (Table I.5.1).

Box I.5.1. How much do 15-year-olds learn over one year of schooling?

Two recent publications (Avvisati and Givord, 2023[1]; Avvisati and Givord, 2021[2]) estimate the average yearly learning gain of students based on data sets from 2018 and earlier PISA assessments of more than 30 countries and economies. These studies show that around the age of 15, students’ test scores in mathematics, reading and science increase by about one-fifth of a standard deviation over a year of schooling (and age) on average across countries, a gain equivalent to about 20 score points in PISA (Avvisati, 2021[3]). They also show that yearly learning gains can vary significantly across countries: in mathematics, for example, the estimates reported in Avvisati and Givord (2023[1]) imply that the test scores of students in Austria, Scotland* and Singapore increase about twice as fast as those of students in Brazil and Malaysia, which increase by about 12 points over a 12-month period.

In this report, a single, round number (20 score points) is used as a common benchmark for all countries, reflecting approximately the average pace of learning of 15-year-olds in countries that participate in PISA. Readers should avoid using this to convert any difference in terms of years-of-schooling equivalents (or months-of-schooling equivalents): first, because there are significant differences in the pace of learning at a given age across countries. This reflects differences in how schooling is organised, the resources invested in education, and the quality of education itself. And second, because there is no reason to expect the pace of learning to remain constant over time: the average pace of learning measured at age 15 may give only a limited indication of the test-score gains that can be expected in a particular country over two or three years.

Table I.5.1. Change between 2018 and 2022 in mean performance in mathematics, reading and science

Changes between 2018 and 2022 in the context of previous trends in mean performance

For some countries and economies, the changes in PISA performance observed between 2018 and 2022 deviate significantly from the trend observed over earlier assessments; for others, they confirm or reinforce a trend which already began before 2018. Figure I.5.2 shows the average trend up to 2018 across 23 OECD countries that can compare performance across all PISA assessments together with the mean performance observed in 2022 in these same countries.

Figure I.5.2. Trends in mathematics, reading and science performance up to 2018

Figure I.5.3 provides similar information for all participants in PISA 2022, PISA 2018, and at least one assessment prior to 2018. The pre-2018 trends reported in Figure I.5.3 correspond to the average change observed between the earliest available measurement in PISA and PISA 2018, calculated using a linear regression: they represent the slope of the trend line. The average change is reported over a four-year period to facilitate comparisons with the change observed between 2018 and 2022. Countries and economies at the top of each chart improved in mean performance in the corresponding subject between their first participation in PISA and 2018; countries at the bottom of each chart experienced a declining trend in mean performance up to 2018.

Among countries and economies where the pre-2018 trend was positive, several experienced a full or partial reversal of these gains in 2022:

In mathematics, such a reversal was observed in Albania, Bulgaria, Estonia, Italy, Macao (China), Malaysia, Mexico, Montenegro, North Macedonia, Peru, Poland and Portugal;
In reading, in Albania, Estonia, Germany, Macao (China), Moldova, Montenegro, Poland, Portugal and Singapore;
In science, in Albania, Malaysia, Moldova and North Macedonia.

Many other countries/economies which improved performance over earlier cycles, however, were able to maintain their 2018 performance level despite the shock of the COVID-19 epidemic:

In mathematics, mean scores in 2022 remained close to their 2018 level in Argentina, Australia*, Brazil, Chile, Colombia, Croatia, Georgia, Israel, Japan, Kazakhstan, Korea, Lithuania, Malta, Moldova Panama*, Qatar, Romania, Serbia, Singapore, Switzerland, Türkiye and the United Arab Emirates;
In reading, Argentina, Australia*, Austria, Brazil, Chile, Colombia, Croatia, the Czech Republic, Georgia, Hungary, Ireland*, Israel, Italy, Kazakhstan, Korea, Latvia*, Lithuania, Malta, Mexico, New Zealand*, Peru, Romania, Serbia, Switzerland, the United States* and Uruguay;
In science, in Argentina, Australia*, Austria, Brazil, Bulgaria, Canada*, Chile, Colombia, Costa Rica, the Czech Republic, Denmark*, Estonia, France, Georgia, Hong Kong (China)*, Hungary, Israel, Korea, Lithuania, Macao (China), New Zealand*, Peru, Portugal, Romania, Serbia, the Slovak Republic, Sweden, Switzerland, the United Arab Emirates, the United Kingdom*and the United States*.
In reading and/or science, three countries even improved their results in 2022, extending their positive pre-2018 trend. This is the case of Qatar (in both subjects) and in Singapore and Türkiye (in science only).

Figure I.5.3. Changes in performance between 2018 and 2022 in the context of pre-2018 performance trends

Among countries where the pre-2018 trend was neither improving nor declining (in the long term and on average), results turned to the negative in most cases. This pattern, which corresponds to what was observed on average across OECD countries in mathematics (see Figure I.5.2), also held in:

mathematics in Austria, Costa Rica, Denmark*, Germany, Greece, Hong Kong (China)*, Indonesia, Ireland*, Jordan, Kosovo1, Latvia*, Norway, Slovenia, Sweden, Thailand, the United Kingdom*, the United States* and Uruguay;
reading in Belgium, Bulgaria, Canada*, Denmark*, France, Greece, Hong Kong (China)*, Indonesia, Kosovo, Malaysia, North Macedonia, Norway, Slovenia, Türkiye, the United Arab Emirates and the United Kingdom*;
science in Indonesia, Mexico, Montenegro, Norway, Poland and Thailand.

Other countries and economies were experiencing a decline in mean performance already prior to 2018. These negative trends were often confirmed and reinforced over the most recent period in:

mathematics in Belgium, Canada*, the Czech Republic, Finland, France, Hungary, Iceland, the Netherlands*, New Zealand* and the Slovak Republic
reading in Costa Rica, Finland, Iceland, the Netherlands*, the Slovak Republic, Sweden and Thailand;
science in Belgium, Finland, Germany, Greece, Iceland, Kosovo, the Netherlands*, and Slovenia.

A small number of previously declining countries and economies, however, experienced positive changes in 2022, and bounced back: this rebound was observed in mathematics in Chinese Taipei, reading in the Dominican Republic, and science in Croatia and Ireland*.

Box I.5.2. How large are typical test-score changes in PISA between two consecutive assessments?

To get a sense of how unique the changes observed between 2018 and 2022 are, it is useful to compare them to typical changes that were reported in previous years over similarly short periods of time (three years). Of the 81 countries and economies that participated in PISA 2022, data allow 73 to compare their mathematics results to PISA 2018 (or to the results of PISA for Development in 2017); and of these, 60 can compare their PISA 2015 and PISA 2018 results (Tables I.B1.5.4, I.B1.5.5, I.B1.5.6). In mathematics, only about half of these 60 countries/economies had a difference in mean scores larger than (plus or minus) five score points (this corresponds to the median absolute difference). Fewer than one out of four countries reported positive or negative changes in mean scores in mathematics that were larger than nine score points.

When considering the full set of results for countries that participated in PISA 2022 and all three subjects, more than 200 score-point differences across two consecutive PISA assessments can be computed (257 in mathematics, 281 in reading, and 218 in science) over the 2000-2018 period:

In mathematics, the median difference observed across consecutive assessments is 6.2 score points and only one out of four showed a difference larger than 11.1 score points.
In reading, the median difference observed is 7.4 score points and only one out of four showed a difference larger than 14.4 score points.
In science, the median difference observed is 6.4 score points and only one out of four showed a difference larger than 10.9 score points.

In other words, for individual countries/economies, increases or decreases of up to 10 score points have been relatively common in PISA and certainly not unprecedented – even over short time intervals.

In larger aggregates such as the OECD average, the typical changes observed in the past are much smaller than for individual countries/economies. Indeed, score differences for individual countries/economies typically result from improvements or deteriorations unique to each country/economy as well as the uncertainty intrinsic in statistical indicators (see Box I.5.3). However, idiosyncratic changes and statistical uncertainty tend to cancel out in larger aggregates such as the OECD average: changes in one direction, for one country, are compensated by opposite changes for other countries. As a result, the change observed in the OECD average over consecutive assessments up to 2018 has never exceeded four score points in mathematics and five score points in reading. Changes in the OECD average that are more pronounced are unprecedented. They point to a shock affecting many countries simultaneously and in the same direction.

Changes in performance distributions between 2018 and 2022

The decline in mean mathematics and reading performance across OECD countries (on average) and in most PISA-participating education systems was not uniform in terms of the distribution of student performance. One way this can be seen is by examining performance trends for low- and high-achieving students. The 10th percentile is the point on the scale below which 10% of students score. In other words, if all students were ranked from lowest- to highest-scoring, the 10th percentile would be the highest-scoring of the lowest-performing 10% of students. Likewise, the 90th percentile is the point on the scale below which 90% of students score (or, conversely, above which only 10% of students score). The median or 50th percentile divides the performance distribution into two equal halves, one above and one below that position on the scale.

In mathematics, mean performance was about 15 score points lower in 2022 compared to 2018 on average across OECD countries. But the performance decline was slightly less pronounced at the 90th percentile (-11 score points): almost all students performed worse but low-achieving students declined by slightly more than high-achieving students did. A similar pattern is observed in reading. As a result, learning gaps between the highest- and lowest-performing students widened. That said, the difference between the 10th and 90th percentiles increased only by about four score points in reading between 2018 and 2022, and even less in mathematics on average across OECD countries.2

In science, no change was observed in average mean scores across OECD countries. But significant declines were observed among the weakest students (at the 10th and 25th percentile) on average (Table I.B1.5.9). As a result, the gap between the highest- and lowest-performing students widened by more than 10 score points in science (as measured by the inter-decile range, i.e. the difference between the 10th and 90th percentiles) (Figure I.5.4).

The previous paragraphs refer to the average trend across OECD countries; often, the distribution in performance in individual countries and economies evolved differently. For example, in mathematics, the inter-decile range widened significantly in 12 countries and economies (as did the OECD average); narrowed significantly in 26 countries/economies; and did not change significantly in the remaining 35 countries/economies for which comparable data for 2018 (or 2017) and 2022 are available (Table I.5.2). In reading and science, the inter-decile range did not change significantly in most countries/economies (i.e. 55 countries/economies in reading and 44 countries/economies in science).

Table I.5.2 lists countries and economies according to whether their performance distributions in mathematics, reading and science narrowed, widened or did not change significantly (as measured by the inter-decile range). When this can be ascertained with confidence,3 the table also shows whether the change or lack thereof is primarily due to changes among low-achieving students, high-achieving students or both. For example, in Chile, performance differences widened in science because low-achieving students performed worse while high-achieving students performed better (Table I.B1.5.9 and Table I.B1.5.12).

The performance distribution widened between 2018 and 2022 in all three subjects in Finland and the Netherlands* as well as on average across OECD countries. The performance distribution narrowed in all three subjects in North Macedonia, Moldova, Qatar and Saudi Arabia (Table I.5.2).

Figure I.5.4. Average change in science scores for high- and low-achieving students (2018-2022)

Table I.5.2. Change between 2018 and 2022 in the performance distribution in mathematics, reading and science

Box I.5.3. Statistical significance of trend indicators

Statistical sources of uncertainty can be quantified. They refer to aspects of the PISA methodology that may produce variation in the reported results even in the absence of any real change in how students respond to the test. A difference in mean scores (or in other population-level estimates of performance in PISA) is called statistically significant if it is unlikely that such a difference could be observed when, in fact, no true difference exists in the populations from which student samples were drawn.

Statistical uncertainty in trend comparisons has three different sources: the sampling of students and schools; the design of PISA tests (measurement precision); and the use of a common scale to report the results of tests that were scaled independently. When results from multiple assessments are compared, small differences can be observed because a sample of students – rather than the full population of 15-year-old students – took each assessment; because the students sat slightly different tests, each including only a finite number of test items, thus yielding only an approximate measure of performance; and because PISA scores are based on estimates of test item properties (such as their difficulty) which are themselves subject to uncertainty and which, as a result, can vary across calibrations. The latter source of uncertainty – quantified in the link error – is unique to trend comparisons.

The link error represents uncertainty around scale values (“is a score of 432 in PISA 2022 the same as 432 in PISA 2018?”) and is therefore independent of the size of the student sample. As a result, it is the same for estimates based on individual countries, subpopulations or the OECD average.1 For comparisons between mathematics results in PISA 2022 and mathematics results in 2018, the link error corresponds to 2.24 score points. The link error tends to be smaller for comparisons of reading scores (1.47 points) and science scores (1.61 points).2 It tends to be larger for comparisons between 2022 scores and scores from even earlier assessments. Link errors for indicators involving more than two scores (e.g. linear trends) and complex scale transformations such as the percentage of students above/below a threshold of proficiency are discussed in Annex A7.

These three independent sources of uncertainty are combined in the estimates of standard errors for trend indicators. Standard errors are then used to construct “confidence intervals”, a range of values that excludes only 5% of the differences that would be observed in the absence of true change.

It should be kept in mind that the difference between significant and non-significant changes is, itself, often non-significant, and that in some situations it may be impossible to say with confidence that there has been a change, even if such a change actually happened: non-significance does not imply no change.

1. In PISA the link error is assumed to be constant across the scale. For PISA 2022 (as was the case for PISA 2018 and PISA 2015), link errors are estimated based on the variation in country means across distinct scale calibrations (see Annex A7).

2. Link errors between 2022 and previous assessments were computed only based on countries that administered the PISA 2022 test on computer, as was already the case for link errors between 2018 and previous assessments. In the analysis, the same link errors are used for all countries, including those that administered PISA 2022 using paper-based instruments.

Changes in equity between 2018 and 2022

Up to this point, the chapter has examined trends in student performance; the remainder of the chapter looks at trends in measures of equity in education. As defined in the first chapter of this report, equity in education is examined in PISA 2022 in terms of fairness and inclusion. Fairness is examined in the following sections by looking at socio-economic and gender disparities in student performance. Inclusion is examined in chapter 6 by looking at changes in enrolment rates and achievement of basic proficiency among 15-year-old students.

Overall, PISA data show that the unprecedented decline in mathematics performance in PISA 2022 does not translate into significantly greater disparities in performance in terms of socio-economic status or gender in most countries and economies. Nevertheless, widespread declines in disadvantaged students’ performance have meant that a greater proportion of students have failed to achieve baseline levels of proficiency. And at the top end of the spectrum, declines in advantaged students’ performance has meant that a smaller proportion of students achieved the highest proficiency levels of 5 and 6 in many countries.

Changes in socio-economic disparities

Changes in socio-economic disparities between 2018 and 2022 are measured in this chapter by the difference in average performance in mathematics between socio-economically advantaged and disadvantaged students (hereafter, this will be referred to as the “socio-economic gap”). A narrower socio-economic gap means there is less disparity in performance between advantaged and disadvantaged students; by contrast, a wider gap indicates greater disparity.

It is important here to emphasise that a smaller socio-economic gap does not necessarily mean a greater fairness in the system that is desirable. A smaller socio-economic gap can result from the performance of disadvantaged students failing to improve and that of advantaged students declining. This was the case of two countries that participated in PISA 2022 (Chile and the United Arab Emirates), as shown below in Table I.5.3. In cases such as these, the benefits of more fairness in terms of socio-economic status should take into account the detriment of advantaged students performing worse.

Changes in the socio-economic gap

Disadvantaged students’ performance in mathematics declined between 2018 and 2022 on average across OECD countries (by 17 score points) and in 34 countries/economies (Figure I.5.5). Declines that were larger than 20 score points were observed in 20 countries/economies and declines larger than 30 points were observed in seven countries (Albania, Iceland, Jordan, the Netherlands*, Norway, the Slovak Republic and Slovenia). A decline of almost 70 score points occurred in Albania.

Disadvantaged students’ performance in mathematics did not change significantly in another 29 countries/economies and it improved in five countries/economies (Argentina, Brunei Darussalam, the Dominican Republic, the Philippines and Saudi Arabia). The increase in performance ranged from 12 points (in Argentina) to 27 points (in Saudi Arabia). In these five countries, the mean score of disadvantaged students in PISA 2018 was lower than 400 points; in other words, these disadvantaged students improved their scores starting from a performance level that was very low.

Advantaged students’ performance in mathematics declined on average across OECD countries (by 10 score points) and in 30 countries/economies. Advantaged students’ performance in mathematics declined by more than 20 points in 11 countries/economies; by more than 30 points in Iceland, Malaysia and Thailand; by almost 50 points in Jordan; and by almost 60 score points in Albania.

Advantaged students’ performance in mathematics did not change significantly in 34 countries/economies and it improved in four countries (Brunei Darussalam, Japan, Singapore and Chinese Taipei). In these four countries, the increase in performance ranged from 14 points in Brunei Darussalam to 30 points in Chinese Taipei. Except for Brunei Darussalam, these are countries/economies where advantaged students’ performance in PISA 2018 was already among the highest across all PISA-participating countries (ranging between 558 and 611 score points in mathematics); in other words, these advantaged students improved their scores starting from a performance level that was already very high.

Figure I.5.5. Change between 2018 and 2022 in mean performance in mathematics, by national quarter of socio-economic status

Considering that mathematics performance generally declined among disadvantaged and advantaged students, it is not surprising that in most countries the socio-economic gap in mathematics performance did not change between 2018 and 2022. Table I.5.3 shows changes in the difference in mathematics performance between socio-economically advantaged and disadvantaged students (i.e. the “socio-economic gap”) over this period.

The socio-economic gap in mathematics performance did not change between 2018 and 2022 in 51 out of the 68 countries/economies with available PISA data; it widened on average across OECD countries (by seven score points) and in 12 countries/economies; and it narrowed in five countries/economies.

The socio-economic gap in performance narrowed the most (38 score points) in the Philippines where disadvantaged students’ performance improved greatly (20 points) and advantaged students’ performance declined by a similar margin (18 points). In Argentina and Saudi Arabia the socio-economic gap narrowed because disadvantaged students’ performance improved whereas advantaged students’ performance did not change. Conversely, in Chile and the United Arab Emirates the gap narrowed because advantaged students’ performance declined and disadvantaged students’ performance did not change.

The socio-economic gap in performance widened the most in Chinese Taipei (27 score points) where advantaged students’ performance improved (30 score points) and disadvantaged students’ performance did not change. Similarly, the socio-economic gap widened greatly (22 points) in Singapore because advantaged students’ performance improved (16 score points) and disadvantaged students’ performance did not change.

In seven other countries/economies (Australia*, Austria, Estonia, Macao [China], New Zealand*, Sweden and Switzerland), the socio-economic gap widened because disadvantaged students’ performance declined whereas advantaged students’ performance did not change. Within this group of countries, disadvantaged students’ performance declined by more than 20 score points in Estonia, New Zealand* and Sweden.

Table I.5.3. Change between 2018 and 2022 in the socio-economic gap in mathematics performance

Changes in socio-economic disparities at different levels of proficiency

Differences in performance in terms of socio-economic status can also be examined by looking at the change in the proportion of advantaged and disadvantaged students that scored below baseline proficiency Level 2 (“low performers”) and at proficiency Level 5 or 6 (“top performers”) in mathematics.4

As shown in Figure I.5.6, the percentage of disadvantaged students who scored below proficiency Level 2 in mathematics increased on average across OECD countries (by nine percentage points) and in most countries/economies (47 out of 68 with available data) between 2018 and 2022. In 19 of these countries/economies, the share of disadvantaged students scoring below Level 2 in mathematics increased by more than 10 percentage points. In eight countries/economies this share increased by more than 15 percentage points.

In some countries/economies where the share of socio-economically disadvantaged low-performing students increased the most (i.e. more than 10 percentage points), at least three out of four disadvantaged students scored below proficiency Level 2 in mathematics in PISA 2022 (e.g. Albania, Bulgaria, Jordan, Malaysia and Thailand) (Table I.B1.5.25). That said, in Finland and Poland the share of low performers among disadvantaged students continued to be lower than 40% despite the large increase in this share between PISA 2018 and PISA 2022.

In another 19 countries/economies the percentage of disadvantaged students who scored below proficiency Level 2 in mathematics did not change whereas in Brunei Darussalam and in Saudi Arabia the share decreased (by five percentage points).

Among socio-economically advantaged students, the share of low performers typically did not change (this was observed in 39 countries/economies); it increased in 28 countries/economies and decreased in Brunei Darussalam (by eight percentage points).

Figure I.5.6 also shows that trends between 2018 and 2022 in some countries/economies sharply contrast with countries’ trajectories before 2018. Most noticeably, all countries/economies in which the share of disadvantaged low performers increased between 2018 and 2022 had experienced a decrease or stability in their share of disadvantaged low performers between 2012 and 2018 (except Romania). In Montenegro, for example, the share of disadvantaged low performers decreased by 16 percentage points between 2012 and 2018 but has increased by 15 percentage points since 2018. On average across OECD countries, the share of disadvantaged low performers did not change between 2012 and 2018 but increased by nine percentage points between 2018 and 2022.

Similarly, in all countries/economies in which the share of advantaged low performers increased between 2018 and 2022 had seen stability or a decrease of this share in previous PISA assessments (except Canada*). In Peru, for example, the share of advantaged low performers decreased by 13 percentage points between 2012 and 2018 but increased by six percentage points between 2018 and 2022. On average across OECD countries, the share of advantaged low performers did not change between 2012 and 2018 but increased by three percentage points between 2018 and 2022.

Figure I.5.6. Change between 2018 and 2022 in percentage of low performers in mathematics in the context of pre-2018 trends, by national quarter of socio-economic status

Between 2018 and 2022, among advantaged students, the share of top performers decreased on average across OECD countries (by three percentage points) and in 18 countries/economies, and increased in five (Australia*, Japan, Macao [China], Singapore and Chinese Taipei) (Figure I.5.7). Among disadvantaged students, the share of top performers decreased on average across OECD countries (by one percentage point) and in 15 countries/economies, and increased in two countries (Japan and Chinese Taipei).

In Japan and Chinese Taipei, the share of top performers increased by between 10 and 13 percentage points among advantaged students and by about three to four percentage points among disadvantaged students. In Germany and Poland, the share of top performers fell by between 9 and 11 percentage points among advantaged students and by roughly three percentage points among disadvantaged students.

Countries/economies that had increased their share of socio-economically advantaged top performers between 2012 and 2018 did not change or had a decrease between 2018 and 2022. In Sweden, the share of top performers among advantaged students was 16% in PISA 2012 and 24% in 2018, but it did not change significantly between 2018 and 2022. In the United Arab Emirates, a reversal of the previous trend is observed: the share of advantaged top performers increased by four percentage points between PISA 2012 (8%) and PISA 2018 (12%) but decreased by three percentage points between 2018 and 2022 (9%).

Among disadvantaged students, the share of top performers remained stable in most countries between 2012 and 2018. In all countries/economies where the share of disadvantaged top performers decreased between 2018 and 2022, this share had not changed between 2012 and 2018 (except Denmark*, where it increased). By contrast, Japan and Chinese Taipei show promising trend reversals: while the share of disadvantaged top performers decreased in Chinese Taipei (by eight percentage points) and Japan (by four percentage points) between 2012 and 2018, it has increased in both countries/economies since 2018.

Figure I.5.7. Change between 2018 and 2022 in percentage of top performers in mathematics in the context of pre-2018 trends, by national quarter of socio-economic status

Box I.5.4. A context for interpreting trends

This chapter reports changes in performance and equity between 2018 and 2022. In order to attribute changes in performance over successive PISA assessments to changes in student learning or differences in the composition of student populations, the PISA test and how it is conducted must remain equivalent from assessment to assessment. Overall, PISA 2022 and PISA 2018 were conducted in much the same way:

The assessment was primarily conducted on computer (as was also the case in both 2015 and 2018). Seven countries/economies (see below) switched from paper to computer in 2022. Some countries continued to administer a paper-based assessment, as in prior years (Cambodia, Guatemala, Paraguay1 and Viet Nam). Because response patterns in 2022 in all subjects deviated significantly from those observed in Viet Nam in earlier assessments, no reliable trend could be established for Viet Nam, and comparisons of scale scores to those reported in past assessments are not reported in this volume (see Annex A4).
In countries that used computers to assess students in 2022, students answered questions in just two subjects, devoting one hour to each. This was already the case for most students in 2018; however, a minority of students in 2018 was tested in three subjects within the same two-hour testing period. In previous rounds of PISA, the number of subjects varied even more across students.

A small number of countries experienced major changes in test-administration conditions between 2018 and 2022, and care must be taken when interpreting their trends.

Argentina, Jordan, Moldova, North Macedonia, Romania, Saudi Arabia and Ukraine switched from paper to computer assessment in 2022. Although measures were taken to align the reporting scales at the international level in order to report trends, differences in familiarity with the test format or student motivation when taking the test may interfere with performance trends. Furthermore, in the case of Jordan, past reading and science scores were computed on a scale that was only weakly linked to the international scale; for this reason, this volume does not report trends in reading and science for Jordan, and limits trend reporting to mathematics.
Three countries changed their testing period by more than one or two months in 2022, moving it to a different period in the school year. Ireland* and the Netherlands* tested students between October and December 2022 (previously, in March and April 2018). Cambodia tested students in June 2022; their previous results were collected in December 2017 as part of the PISA for Development initiative. While the age-based definition of the target population implies that neither the average age nor the average amount of schooling of students in the PISA sample changes, test-period changes do affect the grade composition of the PISA cohort; furthermore, it is possible that students’ motivation and test performance are subject to seasonal patterns, which may confound differences over time.
Iceland and Norway were the first countries/economies in PISA to rely on a server-based administration (using Chromebooks) in some schools. They reported that students in these schools experienced difficulties moving through the cognitive assessment early in the testing period. Further investigation traced the problem back to overload on the PISA contractor’s server. The problem was rapidly solved for students who were tested later and did not affect other countries that used a server-based administration. In Norway, it affected at most 9% of the final sample (584 students). In Iceland, it affected at most 13% of the final sample (438 students). During data adjudication, these data were thoroughly reviewed, and considered to be fit for reporting. Furthermore, analyses conducted by the PISA National Centre for Iceland (where, due to the census nature of the survey, schools’ results in PISA could be tracked over time), confirmed that the issue affected only students’ ability to complete the test but not the way in which these students responded to the parts that they completed: performance changes were very similar in affected and non-affected school (OECD, Forthcoming[4]).

Finally, 21 countries and economies did not meet PISA technical standards for overall exclusion rates, student response rates, and/or school participation rates in 2022. For 12 of these, it is probable that more than minimal bias resulted from these deviations from standards (see Reader’s Guide). In Portugal and two of the 12 countries mentioned above (the Netherlands* and the United States*), a response-rate issue affected PISA 2018 results. The results of these 13 countries and economies are not included in the OECD average-26. There is more detailed information about the potential bias, its most likely direction, and how it might affect trend comparisons in the Reader’s Guide at the beginning of this volume and in Annex A2 and A4; and, for the Netherlands*, Portugal and the United States*, in the corresponding Annexes for 2018 (OECD, 2019[5]).

1. Cambodia, Guatemala and Paraguay participated in the PISA for Development project in 2017.

Changes in gender disparities

Table I.5.4 shows changes between 2018 and 2022 in the difference between boys and girls in average mathematics performance (hereafter, this difference will be referred to as the “gender gap”).

In this analysis, the gender gap is measured by the score difference between boys and girls (boys – girls). Thus, when describing trends in the gender gap between PISA 2018 and PISA 2022, it will be said below that the gap “narrowed” if the gap became more favourable to girls; similarly, it is said that the gender gap “widened” if it became more favourable to boys.

As shown in Table I.5.4, the gender gap in mathematics performance did not change between 2018 and 2022 in most countries/economies (57 out of the 72 with comparable data). The gender gap widened on average across OECD countries (by four score points) and in 11 countries/economies, and it narrowed in four (Albania, Baku [Azerbaijan], Colombia and Montenegro).

In three out of the four countries/economies where the gender gap in performance narrowed, performance declined among boys and girls but boys’ performance declined more than girls’. The gender gap narrowed the most (roughly 15 score points) in Albania and Baku (Azerbaijan). In Albania boys’ and girls’ performance in mathematics was not significantly different in PISA 2018 but a gap of 19 points in favour of girls was observed in PISA 2022. In Baku (Azerbaijan) a gender gap in favour of boys was observed in PISA 2018 but reversed in PISA 2022 when girls outperformed boys by seven points.

The gender gap in performance widened the most (20 score points) in Israel where girls’ performance declined (by 15 score points) and boys’ performance did not change. Similarly, the gender gap widened in Chile, Hong Kong (China)*, Macao (China) and Malta because girls’ performance declined and boys’ performance did not change.

Most typically, in 26 countries/economies, the gender gap did not change between PISA 2018 and 2022 in the context of a performance decline for both boys and girls. In 10 of these countries/economies (Costa Rica, Estonia, France, Germany, Italy, Latvia*, Mexico, New Zealand*, Portugal and the United Kingdom*) boys outperformed girls in PISA 2018 and 2022. In seven (Bulgaria, Greece, Kosovo, Poland, the Slovak Republic, Slovenia and Sweden), boys’ and girls’ performance in mathematics was not significantly different in either assessment. In three other countries/economies (Finland, Indonesia and Malaysia), girls outperformed boys in both assessments.

Another typical case is that neither girls’ nor boys’ performance changed significantly between PISA 2018 and 2022, resulting in a gender gap that did not change between the assessments. This was observed in 16 countries and economies.

Girls’ performance in mathematics declined between 2018 and 2022 in 47 countries/economies; it did not change significantly in another 20 countries/economies and improved in five countries/economies (by roughly 15 score points in all of them). Girls’ performance in mathematics declined by more than 60 score points in Albania and 43 points in Iceland; in Jordan, the Netherlands* and Norway girls’ performance declined by more than 30 points.

Performance trends among boys were also predominantly negative, though less so than for girls. Boys’ performance in mathematics declined in 33 countries/economies; it did not change significantly in another 31 countries/economies and improved in eight (Brunei Darussalam, the Dominican Republic, Guatemala, Paraguay, Qatar, Saudi Arabia, Singapore and Chinese Taipei). Boys’ performance in mathematics declined by 76 score points in Albania, more than 40 points in Jordan, and more than 30 points in Malaysia.

Table I.5.4. Change between 2018 and 2022 in mean performance in mathematics, by gender

Changes in gender disparities at different levels of proficiency

As shown in Figure I.5.8, the percentage of girls who scored below proficiency Level 2 in mathematics increased on average across OECD countries (by six percentage points) and in most countries/economies (52 out of 72 with available data) between 2018 and 2022. In 12 of these countries/economies, the share of girls scoring below Level 2 in mathematics increased by more than 10 percentage points, and in five countries/economies this share increased by more than 15 percentage points (Albania, Iceland, Jordan, Malaysia and Thailand). Iceland, the Netherlands* and Norway are examples of countries/economies that had relatively small shares of low-performing girls in PISA 2018 but saw large increases in PISA 2022.

In another 18 countries/economies, the percentage of girls who scored below proficiency Level 2 in mathematics did not change whereas in Brunei Darussalam and in Paraguay the share decreased (by roughly seven percentage points).

There was a marked drop in boys’ performance as well, though less pronounced than for girls. As shown in Figure I.5.8, between 2018 and 2022, the share of low-performing boys increased on average across OECD countries (by six percentage points) and in most countries/economies (46); it did not change in 22 countries/economies and decreased in four (Brunei Darussalam, Guatemala, Paraguay and Saudi Arabia).

The share of low performers in mathematics increased among boys and girls between PISA 2018 and 2022 in 43 out of 72 countries with comparable data. In 13 countries/economies, the share of low performers in mathematics did not change among boy or girls.

The percentage of low-performing girls in mathematics had been decreasing or holding steady between PISA 2012 and PISA 2018 in all countries/economies where it increased between PISA 2018 and 2022. In Iceland, for example, the share of low performers among girls was 20% in PISA 2012 and 18% in PISA 2018 but 34% in PISA 2022 (increase of 16 percentage points between 2018 and 2022). Similarly, the share of girls who scored below proficiency Level 2 in mathematics in Costa Rica was 67% in PISA 2012, 65% in PISA 2018 but 76% in PISA 2022 (increase of 12 percentage points between 2018 and 2022).

Among boys, the increases in the share of low performers between 2018 and 2022 also occurred in the context of stability or decreases in previous years for several countries/economies. The percentage of low-performing boys in mathematics had been decreasing or holding steady in most countries/economies where it increased between PISA 2018 and 2022. In Sweden, for example, the share of low performers among boys decreased by nine percentage points between 2012 and 2018 (from 28% to 19%) but increased by the same amount between 2018 and 2022.

Figure I.5.8. Change between 2018 and 2022 in percentage of low performers in mathematics in the context of pre-2018 trends, by gender

When it comes to trends in the share of top performers in mathematics, declines were somewhat more pronounced among girls than boys (Figure I.5.9). Among girls, the share of top performers decreased between PISA 2018 and 2022 on average across OECD countries (by three percentage points) and in 30 countries/economies, and increased in only two (Japan and Chinese Taipei). The decrease was the largest (between five and eight percentage points) in Hong Kong (China)*, Iceland, Norway and Poland.

Among boys, the share of top performers decreased on average across OECD countries (by two percentage points) and in 18 countries/economies; it increased in Australia*, Japan, Macao (China), Singapore and Chinese Taipei.

The share of top performers in mathematics increased between PISA 2018 and 2022 among boys and girls in Japan and Chinese Taipei whereas it decreased among boys and girls in 16 out 72 countries with comparable data. In 33 countries/economies, the share of top performers in mathematics did not change among boy nor girls.

The decrease in the share of top performers in mathematics among girls and boys was generally greater between PISA 2018 and 2022 than in the six-year period that preceded it. The percentage of top-performing girls in mathematics was increasing moderately or holding steady in most countries/economies where this share decreased between PISA 2018 and 2022 (e.g. Albania, Norway, Portugal, Sweden). Among boys, the percentage of top performers in mathematics had also been increasing moderately or holding steady in most countries/economies where this share decreased between PISA 2018 and 2022; in others a decrease in the share of top performers is observed as well between PISA 2012 and 2018 (e.g. Belgium, Finland, Germany).

Figure I.5.9. Change between 2018 and 2022 in percentage of top performers in mathematics in the context of pre-2018 trends, by gender

Figure I.5.1	Change between 2018 and 2022 in mean mathematics, reading and science performance
Table I.5.1	Change between 2018 and 2022 in mean performance in mathematics, reading and science
Figure I.5.2	Trends in mathematics, reading and science performance up to 2018
Figure I.5.3	Changes in performance between 2018 and 2022 in the context of pre-2018 performance trends
Figure I.5.4	Average change in science scores for high- and low-achieving students (2018-2022)
Table I.5.2	Change between 2018 and 2022 in the performance distribution in mathematics, reading and science
Figure I.5.5	Change between 2018 and 2022 in mean performance in mathematics, by national quarter of socio-economic status
Table I.5.3	Change between 2018 and 2022 in the socio-economic gap in mathematics performance
Figure I.5.6	Change between 2018 and 2022 in percentage of low performers in mathematics in the context of pre-2018 trends, by national quarter of socio-economic status
Figure I.5.7	Change between 2018 and 2022 in percentage of top performers in mathematics in the context of pre-2018 trends, by national quarter of socio-economic status
Table I.5.4	Change between 2018 and 2022 in mean performance in mathematics, by gender
Figure I.5.8	Change between 2018 and 2022 in percentage of low performers in mathematics in the context of pre-2018 trends, by gender
Figure I.5.9	Change between 2018 and 2022 in percentage of top performers in mathematics in the context of pre-2018 trends, by gender

Statlink

References

[3] Avvisati, F. (2021), “How much do 15-year-olds learn over one year of schooling?”, PISA in Focus, No. 115, OECD Publishing, Paris, https://doi.org/10.1787/b837fd6a-en.

[1] Avvisati, F. and P. Givord (2023), “The learning gain over one school year among 15-year-olds: An international comparison based on PISA”, Labour Economics, Vol. 84, p. 102365, https://doi.org/10.1016/j.labeco.2023.102365.

[2] Avvisati, F. and P. Givord (2021), “How much do 15-year-olds learn over one year of schooling? An international comparison based on PISA”, OECD Education Working Papers, No. 257, OECD Publishing, Paris, https://doi.org/10.1787/a28ed097-en.

[5] OECD (2019), PISA 2018 Results (Volume I): What Students Know and Can Do, PISA, OECD Publishing, Paris, https://doi.org/10.1787/5f07c754-en.

[4] OECD (Forthcoming), PISA 2022 Technical Report, PISA, OECD Publishing, Paris.

Notes

← 1. This designation is without prejudice to positions on status, and is in line with United Nations Security Council Resolution 1244/99 and the Advisory Opinion of the International Court of Justice on Kosovo’s declaration of independence.

← 2. In mathematics, no increase was observed on average among the smaller set of OECD countries that reached response-rate standards in both years or where biases due to non-response could be excluded. Even among the larger set of OECD countries, the standard deviation – an alternative measure of the dispersion – did not increase significantly.

← 3. This discussion only considers changes that were statistically significant. In most cases, estimates of percentiles are subject to greater uncertainty than estimates of means. Just like changes in mean performance, changes in percentiles over time are also subject to link errors; in contrast, link errors can be ignored in the estimation of changes in the inter-decile range (i.e. when determining whether the distribution narrowed or widened). For this reason, it is sometimes possible to conclude that the performance distribution widened even if neither the 10th nor the 90th percentile exhibit significant changes.

← 4. For trends in the percentage of low performers among all students, see Box I.3.2 (in Chapter 3), and Figure I.6.5 and Table I.6.5 (in Chapter 6). For trends in the percentage of top performers among all students, see Figure I.6.5 and Table I.6.5 in Chapter 6.

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment

Governance

Health

Industry, business and entrepreneurship

Regional, rural and urban development

Science, technology and innovation

Society

Taxation

Trade

Energy

Nuclear energy

Transport

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment

Governance

Health

Industry, business and entrepreneurship

Regional, rural and urban development

Science, technology and innovation

Society

Taxation

Trade

Energy

Nuclear energy

Transport

Countries A - C

Countries D - I

Countries J - M

Countries N - R

Countries S - T

Countries U - Z

Regional and global engagement

Countries

Countries A - C

Countries D - I

Countries J - M

Countries N - R

Countries S - T

Countries U - Z

Regional and global engagement

Publications

Publications

Featured publications

Data

Data

Featured data

News & Events

News & Events

Featured Events

About OECD

About

Engage with us

Work with us

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment