This annex to the PISA 2022 results provides further technical details on how the assessment covered its target population of 15-year-olds, how its national samples represent this population across participating countries and economies, and how the sampling procedure was adapted to accurately represent diverse education systems worldwide.
PISA 2022 Results (Volume II)
Learning During – and From – Disruption
PISA
Annex A2. The PISA target population, the PISA samples, and the definition of schools
Abstract
What is the PISA target population?
PISA 2022 assessed the cumulative outcomes of education and learning at a point at which most young people are still enrolled in formal education: when they are 15 years old.
International surveys of education outcomes must guarantee the comparability of their target population across participating countries and economies. One way to do this is to assess students at the same grade level. However, differences between countries in the nature and extent of early childhood education and care, age at entry into primary education, and the overall institutional structure of education systems do not allow for a definition of internationally comparable grade levels.
Other international assessments have defined their target population by the grade level that provides maximum coverage of a particular age cohort. However, this definition leads to a population particularly sensitive to the distribution of students across age and grade levels, where small changes – of assessment dates, or month of entry into primary education – can lead to the selection of different target grades. There also may be differences across or within countries in whether students who are older or younger than the desired age cohort are represented in the modal grade, further rendering such grade level-based samples difficult to compare.
To overcome these problems, PISA uses an age-based definition of its target population, one that is not tied to the institutional structures of national education systems.1 PISA assesses students who are aged between 15 years and 3 (complete) months and 16 years and 2 (complete) months2 at the beginning of the assessment period, plus or minus an allowed 1-month variation, and who are enrolled in an educational institution3 at grade 7 or higher. All students who met these criteria were eligible to sit the PISA test in 2022, regardless of the type of educational institution in which they were enrolled and whether they were enrolled in full- or part-time education. This also allows PISA to evaluate students shortly before they are faced with major life choices, such as whether to continue with education or enter the workforce.
Hence, PISA makes statements about the knowledge and skills of a group of individuals who were born within a comparable reference period, but who may have been exposed to different educational experiences inside and outside of school. These students may be distributed over different ranges of grades (both in terms of the specific grade levels and the spread in grade levels) in different countries/economies, or over different tracks or streams within their respective education systems. It is important to consider these differences when comparing PISA results across countries/economies. In addition, differences in performance observed when students are 15 may diminish or disappear entirely later in life.
If a country’s mean scores in mathematics, reading or science are significantly higher than those of another, it cannot automatically be inferred that schools or particular parts of the education system in the first country are more effective than those in the second. However, one can legitimately conclude that it is the cumulative impact of learning experiences in the first country, starting in early childhood and up to the age of 15, and including all experiences, whether they be at school, home or elsewhere, that have resulted in the better outcomes of the first country in the subjects that PISA assesses.4
How were students chosen?
The accuracy of the results from any survey depends on the quality of the information drawn from those surveyed as well as on the sampling procedures. Quality standards, procedures, instruments and verification mechanisms were developed for PISA that ensured that national samples yielded comparable data and that results could be compared across countries and economies with confidence. Experts from the PISA Consortium selected the samples for most participating countries/economies and monitored the sample-selection process closely in those countries that opted to select their own samples.
All samples in PISA 2022 were designed as two-stage stratified samples. The first stage sampled schools in which 15-year-old students may be enrolled. Schools were sampled systematically with selection probabilities proportional to the estimated size of their (eligible) 15-year-old population. At least 150 schools5 were selected in each country, although the requirements for national analyses often demanded a larger sample. Replacement schools for each sampled school were simultaneously identified, in case an originally sampled school chose not to participate in PISA.
The second stage of the selection process sampled students within sampled schools. Once schools were selected, a list of each sampled school’s 15-year-old students was prepared. From this list, 42 students were then selected with equal probability (all 15-year-old students were selected when less than 42 eligible students were enrolled). The target number of students in a school who were to be sampled could deviate from 42 when agreed by PISA’s sampling contractor but could not fall below 20 students.
Data-quality standards in PISA require minimum participation rates for schools and for students. These standards were established to minimise potential bias arising from non-response. Indeed, it was likely that any bias resulting from non-response would be negligible – typically smaller than the sampling error – in countries that met these standards.6
At least 85 % of the schools initially selected to take part in the PISA assessment were required to agree to conduct the test when accounting for the number of enrolled 15-year-olds. Where the initial response rate of schools was between 65% and 85%, however, an acceptable school-response rate could still be achieved using replacement schools.
Whenever a school is selected for PISA, two other schools – the most similar according to the statistical criteria used for sampling – are selected as replacement schools in case of non-response or other contingencies. However, statistical similarities notwithstanding, sampling bias is still possible if the replacement schools differ from sampled schools in ways that might not be considered for sampling. Therefore, countries/economies were encouraged to persuade as many of the schools in the original sample as possible to participate.
Schools that were included but where student participation rates of 25-50% were observed were not considered to be participating schools when determining participation rates; but data collected from these schools (from both the cognitive assessment and background questionnaires) were included in the database and contributed to the estimation of the various quantities derived from the assessment. Data from schools with a student participation rate of less than 25% were excluded from the database.
In PISA 2022, 14 countries/economies – the United States (51%), Hong Kong (China) (60%), New Zealand (61%), the Netherlands (66%), the United Kingdom (67%), the Flemish community (Belgium) (72%), Ukrainian regions (18 of 27) (80%), Belgium (80%), Brazil (81%), Canada (81%), Chinese Taipei (83%), Latvia (84%), Panama (84%) and Chile (84%) – did not meet the standard of 85% weighted school participation rate; three of them did not meet the 65% threshold for schools initially selected for PISA. Even after replacement schools were included, seven countries – the United States (63%), New Zealand (72%), Hong Kong (China) (80%), the United Kingdom (82%), Chinese Taipei (84%), Canada (86%) and the Netherlands (90%) still failed to reach target participation rates;7 all other participating countries/economies reached the threshold for an acceptable participation rate after including replacement schools.
PISA 2022 also required that at least 80% of the students chosen in participating schools sat the PISA test. This threshold was calculated at the national level and did not have to be met in each participating school. Follow-up sessions were required in schools where too few students had participated in the planned assessment sessions. Student-participation rates were calculated over all originally selected schools and over all participating schools, including replacement schools. Students who participated in either the planned or follow-up sessions were counted in these rates; those who attended only the questionnaire session were included in the international database and contributed to the statistics presented in this publication if they provided at least a description of either parent’s occupation.
The standard of 80% student participation rate was not met by nine countries/economies: Jamaica (68%), New Zealand (72%), the United Kingdom (75%), Hong Kong (China) (75%), Australia (76%), Ireland (77%), Panama (77%), Canada (77%) and Malta (79%).
Table I.A2.6 shows the response rate for students and schools, before and after including replacement schools.
Column 1 shows the weighted participation rate of schools before replacement; it is equivalent to Column 2 divided by Column 3 (multiplied by 100 to give a percentage).
Column 2 shows the number of responding schools before school replacement, weighted by student enrolment.
Column 3 shows the number of sampled schools before school replacement, weighted by student enrolment. This includes both responding and non-responding schools.
Column 4 shows the unweighted number of responding schools before school replacement.
Column 5 shows the unweighted number of sampled schools before school replacement, including both responding and non-responding schools.
Columns 6 to 10 repeat Columns 1 to 5 for schools after school replacement, i.e. after non-responding schools were substituted by the replacement schools identified during the initial sampling procedure.
Columns 11 to 15 repeat Columns 6 to 10 but for students in schools after school replacement. Note that the weighted and unweighted numbers of students sampled (Columns 13 and 15) include students who were assessed and those who should have been assessed but who were absent on the day of assessment. As mentioned above, any students in schools where the student response rate was less than 50% were not considered to be attending participating schools and were thus excluded from Columns 14 and 15 (and, similarly, from Columns 4, 5, 9 and 10).
What proportion of 15-year-olds does PISA represent?
All countries/economies attempted to maximise the coverage of 15-year-olds enrolled in education in their national samples, including students enrolled in special education institutions. As such, the technical standards used in PISA only allowed countries/economies to exclude up to 5% of the desired target population (i.e. 15-year-old students enrolled in educational institutions at grade 7 or higher) either by excluding schools or students within schools.
Sixteen countries and economies did not meet this standard in PISA 2022: Ukrainian regions (18 of 27) (14.9%), Denmark (11.6%), the Netherlands (8.4%), Latvia (7.9%), Sweden (7.4%), Norway (7.3%), Australia (6.9%), Scotland (United Kingdom) (6.6%), Lithuania (6.5%), the United States (6.1%), Estonia (5.9%), Canada (5.8%), Switzerland (5.8%), New Zealand (5.8%), Türkiye (5.6%) and Croatia (5.4%). In 31 countries/economies, the overall exclusion rate was less than 2% (Table I.A2.1). When language exclusions8 were accounted for (i.e. removed from the overall exclusion rate), Switzerland, Türkiye and the United States no longer had exclusion rates greater than 5%. In Ukraine, almost all excluded students were so considered due to the war. More details can be found in the PISA 2022 Technical Report (OECD, 2023[1]).
Exclusions that should remain within the above limits include:
At the school level:
schools that were geographically inaccessible or where the implementation of the PISA assessment was not considered feasible
schools that provided teaching only for students in the categories defined under “within-school exclusions”, such as schools for students with special education needs.
The percentage of 15-year-olds enrolled in such schools had to be less than 2.5% of the nationally desired target population (0.5% maximum for the former group and 2% maximum for the latter group). The magnitude, nature and justification for school-level exclusions are documented in the PISA 2022 Technical Report (OECD, 2023[1]). In addition, due to differences in when schools re-opened and returned to full, in-person instruction after the COVID-19 pandemic, an additional code for student exclusions (Code 6) was used in PISA 2022 to account for those who were enrolled but received instruction virtually.
At the student level:
students with an intellectual disability, i.e. a mental or emotional disability resulting in the student being so cognitively delayed that he/she could not perform in the PISA testing environment
students with a functional disability, i.e. a moderate to severe permanent physical disability resulting in the student being unable to perform in the PISA testing environment
students with limited assessment-language proficiency (these students were unable to read or speak any of the languages of assessment in the country at a sufficient level and were unable to overcome such a language barrier in the PISA testing environment; they were typically students who had received less than one year of instruction in the language of assessment)
students who were not attending in-person classes or going to school for tests/assessments during the PISA testing period but, rather, were receiving all of their instruction on line
other exclusions, a category defined by the PISA national centres in individual participating countries and approved by the PISA international consortium
students taught in a language of instruction for the major domain for which no materials were available.
Students could not be excluded solely because of low proficiency or common disciplinary problems. The percentage of 15-year-olds excluded within schools had to be less than 2.5% of the national desired target population.
Table I.A2.1 describes the target population of the countries/economies that participated in PISA 2022. Further information on the target population and the implementation of PISA sampling standards can be found in the PISA 2022 Technical Report (OECD, 2023[1]).
Column 1 shows the total number of 15-year-olds according to the most recent available information, which in most countries and economies means from 2021, the year before the assessment.
Column 2 shows the number of 15-year-olds enrolled in school in grade 7 or above, which is referred to as the “eligible population”.
Column 3 shows the national desired target population. Countries/economies were allowed to exclude up to 0.5% of students a priori from the eligible population, essentially for practical reasons if agreed upon with the PISA consortium.
Column 4 shows the number of students enrolled in schools that were excluded from the national desired target population, either from the sampling frame or later in the field during data collection. In other words, these are school-level exclusions.
Column 5 shows the size of the national desired target population after subtracting the students enrolled in excluded schools. This column is obtained by subtracting Column 4 from Column 3.
Column 6 shows the percentage of students enrolled in excluded schools. This is obtained by dividing Column 4 by Column 3 and multiplying by 100.
Column 7 shows the number of students who participated in PISA 2022. Note that in some cases, this number does not account for 15-year-olds assessed as part of additional national options.
Column 8 shows the weighted number of participating students, i.e. the number of students in the nationally defined target population that the PISA sample represents.
Column 9 shows the total number of students excluded within schools. In each sampled school, all eligible students – namely, those 15 years of age, regardless of grade – were listed, and a reason for the exclusion was provided for each student who was to be excluded from the sample. These reasons are further described and classified into specific categories in Table I.A2.4.
Column 10 shows the weighted number of students excluded within schools, i.e. the overall number of students in the national defined target population represented by the number of students from the sample excluded within schools. This weighted number is also described and classified by exclusion categories in Table I.A2.4.
Column 11 shows the percentage of students excluded within schools. This is equivalent to the weighted number of excluded students (Column 10) divided by the weighted number of excluded and participating students (the sum of Columns 8 and 10), multiplied by 100.
Column 12 shows the overall exclusion rate, which represents the weighted percentage of the national desired target population excluded from PISA either through school-level exclusions or through the exclusion of students within schools. It is equivalent to the school-level exclusion rate (Column 6) plus the product of the within-school exclusion rate and 1 minus the school-level exclusion rate expressed as a decimal (Column 6 divided by 100).9
Column 13 shows an index of the extent to which the national desired target population was covered by the PISA sample. As mentioned above, 15 countries/economies fell below the coverage of 95%. This is also known as Coverage Index 1.
Column 14 shows an index of the extent to which 15-year-olds enrolled in school were covered by the PISA sample. The index, also known as Coverage Index 2, measures the overall proportion of the national enrolled population that is covered by the non-excluded portion of the student sample, and takes into account both school- and student-level exclusions. Values close to 100 indicate that the PISA sample represents the entire (grade 7 and higher) education system as defined in PISA 2022. This is calculated in a similar manner to Column 13; however, the total enrolled population of 15-year-olds in grade 7 or above (Column 2) is used as a base instead of the national desired target population (Column 3).
Column 15 shows an index of the coverage of the 15-year-old population. The index is the weighted number of participating students (Column 8) divided by the total population of 15-year-old students (Column 1). This is also known as Coverage Index 3.
A high level of coverage contributes to the comparability of the assessment results. For example, even assuming that the excluded students would have systematically scored worse than those who participated, and that this relationship is moderately strong, an exclusion rate of 5% would likely lead to an overestimation of national mean scores of less than 5 score points on the PISA scale (where the standard deviation is 100 score points).10
Given the significant disruption caused by COVID-19 global pandemic to education systems in general, and to the administration of the PISA 2022 Main Survey in particular, coverage is of particular concern in the 2022 cycle, as it is feasibly affected both by changes in student behaviour (e.g., not returning to school when those were reopened) and by operational factors of administering PISA itself (e.g. less participating students due to interference between PISA dates and a country/economy’s school reopening plan).
Table I.A2.2 provides an across-cycle perspective on:
the estimated size of the 15-year-old cohort in a given country/economy (Column 1 for PISA 2022),
the estimated population size of 15-year-olds enrolled at school in grade 7 or above (Column 2 for PISA 2022),
the number of students that sat PISA 2022 weighted by how much they represent the population (Column 3 for PISA 2022), and
the coverage of the 15-year-old population (Coverage Index 3, Column 4 for PISA 2022).
The same information is provided for previous PISA cycles until 2003. A decrease in the Coverage Index 3 between PISA 2018 and PISA 2022 was observed for 23 countries/economies. However, in only five of them this decrease was larger than 5%: the Dominican Republic, Germany, Hong Kong (China)*, the Netherlands* and Ukrainian regions (18 of 27). Nonetheless, these elevated drops in coverage are to be interpreted with due caution: sampling outcomes for Hong Kong (China) and the Netherlands struggled to meet PISA sampling standards. In Ukrainian, schools in several regions were not accessible in 2022; Coverage Index 3 decreased from 86.7% in PISA 2018 to 63.9% in PISA 2022.
Conversely, all other participating countries/economies either kept or increased their coverage of the population between PISA 2018 and PISA 2022. Small increases, up to 5%, were observed in 31 countries/economies, with others showing quite elevated increase in coverage in the 2022 cycle compared to PISA 2018.
The PISA Adjudication Group, comprising the Technical Advisory Group and the Sampling Referee, reviewed the PISA 2022 data. Overall, the review found that national implementations of PISA generally adhered to PISA’s technical standards despite the challenging circumstances that affected not only PISA operations but schooling more generally during the COVID-19 pandemic. Nevertheless, a number of deviations from the standards were noted and their consequences for data quality were reviewed in depth. The following overall patterns of deviations from sampling standards were identified:
About one in five adjudicated entities had exclusion rates exceeding the limits set by the technical standards (Standard 1.7).
Seven entities failed to meet the required school-response rates, with three of them failing to meet the stricter level of 65% before replacement (Standard 1.11). This is not inconsistent with earlier cycles of PISA, however.
A significantly larger number of entities failed to meet the required student-response rates (Standard 1.12): ten entities did not meet this standard in PISA 2022, while only one entity did not meet the standard in PISA 2018.
Countries/economies that failed to meet the response-rate standards were requested to submit a non-response bias analysis (NRBA) report. These reports, evaluated by the PISA Adjudication Group, contained additional analyses using the national context and data sources to assess potential bias arising from school and student non-participation.
Details on the PISA Adjudication Group’s assessments of the deviations from PISA standards are described in the Reader’s Guide and Annex A4.
Definition of schools
In some countries, subunits within schools were sampled instead of schools, which may affect the estimate of the between-school variance. In Austria, the Czech Republic, El Salvador, Germany, Hungary, Japan and Romania, schools with more than one programme of study were split into the units delivering these programmes. In the Netherlands, locations were listed as sampling units. In the Flemish community (Belgium), each campus of a multi-campus school was sampled independently, whereas the larger administrative unit of a multi-campus school was sampled as a whole in the French community (Belgium).
In Australia and Colombia each campus of a multi-campus school was sampled independently. In Argentina each campus of a multi-campus school was sampled independently and campuses with more than one programme of study were split into the units delivering these programmes. Schools in the Basque Country (Spain) that were divided into sections by language of instruction were split into sections for sampling based on those languages.
Some schools in the United Arab Emirates were sampled as a whole unit, while others were divided by curriculum and sometimes by gender. Due to reorganisation, some schools in Sweden were split into two parts, each part with its own principal. Some schools in Portugal were organised into clusters where all units in a cluster shared the same teachers and principal; each of these clusters constituted a single sampling unit. Some schools in Singapore were sampled as a whole unit while others were split by campus or language of instruction. Some schools in Türkiye were sampled as a whole unit while others were split by programme of study. Schools in Uruguay were sampled as a whole unit, except for schools offering classes at night; night-shift sections were sampled independently from the school.
The distribution of PISA students across grades
Students assessed in PISA 2022 were enrolled in various grade levels. The percentage of students at each grade level is presented, by country, in Tables I.A2.8 and I.A2.9, and by gender within each country/economy in Tables I.A2.12 and I.A2.13.
Table II.A 2.1. PISA target populations and samples, 2022 [1/4]
Table II.A2.1. PISA target populations and samples, 2022 [2/4]
Table II.A2.1. PISA target populations and samples, 2022 [3/4]
Table II.A2.1. PISA target populations and samples, 2022 [4/4]
Table II.A 2.2. Change in the enrolment of 15-year-olds in grade 7 and above (PISA 2003 through PISA 2022) [1/6]
Notes: Costa Rica, Georgia, Malaysia, Malta, Moldova and United Arab Emirates conducted the PISA 2009 assessment in 2010 as part of PISA 2009+.
For Albania, Brazil, Chile, Jordan, Netherlands, Romania and Uruguay, estimates of the Total population of 15-year-olds across years have been updated to align data sources with those used in 2018. Therefore, the estimates reported in this table do not match those that appear in previous PISA reports.
For Mexico, in 2015, the Total population of 15-year-olds enrolled in grade 7 or above is an estimate of the target population size of the sample frame from which the 15-year-old students were selected for the PISA test. At the time Mexico provided the information to PISA, the official figure for this population was 1 573 952.
Table II.A2.2. Change in the enrolment of 15-year-olds in grade 7 and above (PISA 2003 through PISA 2022) [2/6]
Notes: Costa Rica, Georgia, Malaysia, Malta, Moldova and United Arab Emirates conducted the PISA 2009 assessment in 2010 as part of PISA 2009+.
For Albania, Brazil, Chile, Jordan, Netherlands, Romania and Uruguay, estimates of the Total population of 15-year-olds across years have been updated to align data sources with those used in 2018. Therefore, the estimates reported in this table do not match those that appear in previous PISA reports.
For Mexico, in 2015, the Total population of 15-year-olds enrolled in grade 7 or above is an estimate of the target population size of the sample frame from which the 15-year-old students were selected for the PISA test. At the time Mexico provided the information to PISA, the official figure for this population was 1 573 952.
Table II.A2.2. Change in the enrolment of 15-year-olds in grade 7 and above (PISA 2003 through PISA 2022) [3/6]
Notes: Costa Rica, Georgia, Malaysia, Malta, Moldova and United Arab Emirates conducted the PISA 2009 assessment in 2010 as part of PISA 2009+.
For Albania, Brazil, Chile, Jordan, Netherlands, Romania and Uruguay, estimates of the Total population of 15-year-olds across years have been updated to align data sources with those used in 2018. Therefore, the estimates reported in this table do not match those that appear in previous PISA reports.
For Mexico, in 2015, the Total population of 15-year-olds enrolled in grade 7 or above is an estimate of the target population size of the sample frame from which the 15-year-old students were selected for the PISA test. At the time Mexico provided the information to PISA, the official figure for this population was 1 573 952.
Table II.A2.2. Change in the enrolment of 15-year-olds in grade 7 and above (PISA 2003 through PISA 2022) [4/6]
Notes: Costa Rica, Georgia, Malaysia, Malta, Moldova and United Arab Emirates conducted the PISA 2009 assessment in 2010 as part of PISA 2009+. For Albania, Brazil, Chile, Jordan, Netherlands, Romania and Uruguay, estimates of the Total population of 15-year-olds across years have been updated to align data sources with those used in 2018. Therefore, the estimates reported in this table do not match those that appear in previous PISA reports. For Mexico, in 2015, the Total population of 15-year-olds enrolled in grade 7 or above is an estimate of the target population size of the sample frame from which the 15-year-old students were selected for the PISA test. At the time Mexico provided the information to PISA, the official figure for this population was 1 573 952.
Table II.A2.2. Change in the enrolment of 15-year-olds in grade 7 and above (PISA 2003 through PISA 2022) [5/6]
Notes: Costa Rica, Georgia, Malaysia, Malta, Moldova and United Arab Emirates conducted the PISA 2009 assessment in 2010 as part of PISA 2009+.
For Albania, Brazil, Chile, Jordan, Netherlands, Romania and Uruguay, estimates of the Total population of 15-year-olds across years have been updated to align data sources with those used in 2018. Therefore, the estimates reported in this table do not match those that appear in previous PISA reports.
For Mexico, in 2015, the Total population of 15-year-olds enrolled in grade 7 or above is an estimate of the target population size of the sample frame from which the 15-year-old students were selected for the PISA test. At the time Mexico provided the information to PISA, the official figure for this population was 1 573 952.
Table II.A2.2. Change in the enrolment of 15-year-olds in grade 7 and above (PISA 2003 through PISA 2022) [6/6]
Notes: Costa Rica, Georgia, Malaysia, Malta, Moldova and United Arab Emirates conducted the PISA 2009 assessment in 2010 as part of PISA 2009+. For Albania, Brazil, Chile, Jordan, Netherlands, Romania and Uruguay, estimates of the Total population of 15-year-olds across years have been updated to align data sources with those used in 2018. Therefore, the estimates reported in this table do not match those that appear in previous PISA reports. For Mexico, in 2015, the Total population of 15-year-olds enrolled in grade 7 or above is an estimate of the target population size of the sample frame from which the 15-year-old students were selected for the PISA test. At the time Mexico provided the information to PISA, the official figure for this population was 1 573 952.
Table II.A2.4. Exclusions, PISA 2022 [1/4]
* For this entity, the use of code 6 exclusions was expanded beyond the scope of exclusion just for Covid and used for students who met the definition but due to the war in addition to Covid.
Table II.A2.4. Exclusions, PISA 2022 [2/4]
* For this entity, the use of code 6 exclusions was expanded beyond the scope of exclusion just for Covid and used for students who met the definition but due to the war in addition to Covid.
Table II.A2.4. Exclusions, PISA 2022 [3/4]
* For this entity, the use of code 6 exclusions was expanded beyond the scope of exclusion just for Covid and used for students who met the definition but due to the war in addition to Covid.
Table II.A2.4. Exclusions, PISA 2022 [4/4]
* For this entity, the use of code 6 exclusions was expanded beyond the scope of exclusion just for Covid and used for students who met the definition but due to the war in addition to Covid.
Table II.A 2.6. Response rates, PISA 2022 [1/4]
Table II.A2.6. Response rates, PISA 2022 [2/4]
Table II.A2.6. Response rates, PISA 2022 [3/4]
Table II.A2.6. Response rates, PISA 2022 [4/4]
Table II.A2.8. The PISA target population, the PISA samples, and the definition of schools tables
References
[1] OECD (2023), PISA 2022 Technical Report, OECD Publishing.
Notes
← 1. To accommodate countries that requested grade-based results for the purpose of national analyses, PISA 2022 provided a sampling option to supplement the age-based sampling from the target population with an additional grade-based sample.
← 2. More precisely, PISA assessed students who were at least 15 years and 3 complete months old and who were at most 16 years and 3 complete months old (i.e., younger than 16 years, 2 months and roughly 30 days old), with a tolerance of one month on each side of this age window. If the PISA assessment was conducted in April 2022, as was the case in many countries and economies, all students born in 2006 would have been eligible.
← 3. Educational institutions are generally referred to as schools in this publication, although some educational institutions (in particular, some types of vocational education establishments) may not be referred to as schools in certain countries.
← 4. Such a comparison is complicated by first-generation immigrant students, who received part of their education in a country other than the one in which they were assessed. Mean scores in any country or economy should be interpreted in the context of local student demographics. In addition, the PISA target population does not include residents of a country who attend school in another country. It does, however, include foreign nationals who attend school in the country of assessment.
← 5. In education systems inherently too small (due to demographics for instance), all schools and all eligible students were included in the sample. In PISA 2022, all eligible schools were selected in North Macedonia and Qatar. All students in all schools were selected in Brunei Darussalam, Iceland, Macao (China), and Malta.
← 6. Non-response and other standards enforced to achieve consistent, precise, generalisable, and timely data collection in PISA 2022 are available on its Technical Standards (OECD, 2023).
← 7. The threshold for an acceptable participation rate after replacement varies between 85 % and 100 %, depending on the participation rate before replacement.
← 8. These exclusions refer only to those students with limited proficiency in the language of instruction/assessment. Exclusions related to the unavailability of test material in the language of instruction are not considered in this analysis.
← 9. The overall exclusion rate includes those students who were excluded at the school level (Column 6) and those students who were excluded within schools (Column 11); however, only students enrolled in non-excluded schools were affected by within-school exclusions, hence the presence of the term equivalent to 1 minus Column 6 (expressed as a decimal).
← 10. If the correlation between the propensity of exclusions and student performance were 0.3, then resulting mean scores would likely have been overestimated by 1 score point if the exclusion rate were 1 %; by 3 score points if the exclusion rate were 5 %; and by 6 score points if the exclusion rate were 10 %. If the correlation between the propensity of exclusions and student performance were 0.5, then resulting mean scores would likely have been overestimated by 1 score point if the exclusion rate were 1 %; by 5 score points if the exclusion rate were 5 %; and by 10 score points if the exclusion rate were 10 %. For this calculation, a model was used that assumed a bivariate normal distribution for performance and the propensity to participate.