Education in Eastern Europe and Central Asia

1. Eastern Europe and Central Asia participation and outcomes in PISA 2018

Education in Eastern Europe and Central Asia

Countries in Eastern Europe and Central Asia1 (EECA) have undergone tremendous social and political changes in the last 30 years. Most have transitioned from centralised and planned societies to market-based ones and economic development, as measured by gross domestic product (GDP) per-capita, has risen overall (World Bank, 2021[1]). Regional growth has been led by Bulgaria, Croatia and Romania, which have also acceded into the European Union. Other countries, such as Azerbaijan and Kazakhstan, have seen less consistent development from year to year, but still show positive economic progress.

Despite the overall economic growth of the region, EECA countries still face several common challenges. In most countries, the level of development is well below those of most OECD countries. Moreover, the increasing prosperity and wealth of the region has not been equally distributed. Economic inequality, as measured by the Gini coefficient, remains particularly high in Georgia and Romania, and is both higher than the OECD average and rising in Bulgaria and Turkey (World Bank, 2021[2]). Finally, good governance is a critical issue in the region and there is a recognised need to build trustworthy and effective systems of government, particularly in Belarus, Moldova and Ukraine (EU, 2020[3]).

Education is central to achieving regional development goals, as knowledgeable and skilled populations are important in creating dynamic, sustainable economies and inclusive, participatory societies. EECA countries have a strong educational tradition and have produced students who achieve top marks in international competitions. However, the focus on identifying and developing top performers can also divert attention and resources away from helping all students realise their potential. A higher share of EECA students, especially those from disadvantaged backgrounds, drop out before completing secondary school, and many who stay in school do not master the basic competences needed to lead productive lives (UNICEF, 2017[4]; OECD, 2019[5]). Addressing these challenges will be crucial to the region’s future economic development and social cohesion.

Purpose of this report and sources of evidence

This report uses data from the OECD Programme for International Student Assessment (PISA), policy findings from the United Nations Children’s Fund (UNICEF)-OECD country reviews and other international research to identify strengths and challenges that are common to EECA education systems, recognising that there is scope for further analysis on issues relevant to specific countries (Box 1.1). This report also compares the outcomes from EECA countries to global benchmarks, which can reveal the unique features of education in the region. This kind of multi-country analysis can help determine regionally relevant practices that can help improve student outcomes, particularly in secondary school.

Box 1.1. Areas for further analysis

This report focuses on insights from PISA that can help inform the most salient and common educational challenges facing the EECA region. In developing this report, several areas were identified that might benefit from further analysis, but are not addressed here because they do not concern all education systems in the region. For instance, PISA results typically highlight differences between public and private schools. However, across the region only 4% of students on average attend private schools. Similarly, PISA focuses on the differences in outcomes between non-immigrant and immigrant students, who represent 13% of PISA-participating students across the OECD, but only 3% of students in EECA countries. While these issues may be important in some systems (for example, 12% of 15-year-old students in Turkey attend private schools), they are not significant factors to the overall performance of the region.

Other issues might be important in the region, but are not captured by PISA data. Students with disabilities, for example, are excluded from the PISA sample. Identifying different ethnic groups, in particular the Roma, is not possible in PISA. Nevertheless, countries need to understand these issues systemically, such as how to enable schools to support diverse students where they are concentrated. To aid these efforts, many countries analyse PISA data in association with national indicators and publish these results as part of their annual reporting and cyclical strategic planning. UNICEF and the OECD are currently working with Turkey to analyse which school and student characteristics are associated with differences in outcomes with a view to identifying policy interventions that can improve equity.

Participation in PISA

PISA is a triennial survey (due to the COVID-19 epidemic, PISA will be administered next in 2022) of 15‑year-old students around the world. It assesses the extent to which they have acquired the knowledge and skills in reading, mathematics and science that are essential for full participation in social and economic life. PISA does not just assess what students know, but examines how well students can extrapolate from what they have learned and apply their knowledge in real-life settings.

In addition to benchmarking performance, PISA also collects a diverse array of information about students’ families and their socio-economic background, which can be used to better understand the educational equity of countries. Since 2000 when two countries from the region took PISA, EECA countries have continuously increased their engagement and ten participated in 2018 (Table 1.1). Kyrgyzstan also participated in 2006 and 2009, while Mongolia and Uzbekistan are expected to participate in PISA 2022.

In 2018 the PISA assessment was computer-based in most countries (the transition to the computer-based assessment started in 2015), but was still paper-based in 9 out of 79 PISA-participating countries and economies, including Moldova, Romania and Ukraine (Table 1.2). Data between the two modes are comparable, but the paper-based assessment does not include interactive and adaptive items (OECD, 2019[5]).

All countries and economies in PISA 2018 distributed the student and school questionnaires and some participants also administered optional background questionnaires. These included questionnaires for students (about their educational careers, information and communication technology (ICT) familiarity, well-being and financial literacy), parents and teachers. Table 1.2 shows the optional questionnaires taken by EECA countries.

Table 1.1. Participation in PISA cycles

	Baku (Azerbaijan)	Belarus	Bulgaria	Croatia	Georgia	Kazakhstan	Moldova	Romania	Turkey	Ukraine
PISA 2000			X					X
PISA 2003									X
PISA 2006	X		X	X				X	X
PISA 2009	X		X	X	X	X	X	X	X
PISA 2012			X	X		X		X	X
PISA 2015			X	X	X	X	X	X	X
PISA 2018	X	X	X	X	X	X	X	X	X	X

Notes: Azerbaijan as a whole country participated in 2006 and 2009.

Bulgaria conducted the PISA 2000 assessment in 2001 and Romania in 2002, as part of PISA 2000+.

Georgia and Moldova conducted the PISA 2009 assessment in 2010 as part of PISA 2009+.

Kazakhstan participated also in 2015 but coverage was too small to ensure comparability, so the data were not published.

Table 1.2. Aspects of PISA 2018 participation

		Baku (Azerbaijan)	Belarus	Bulgaria	Croatia	Georgia	Kazakhstan	Moldova	Romania	Turkey	Ukraine
Computer format of the assessment		X	X	X	X	X	X			X
Global competence assessment					X		X
Financial literacy assessment/questionnaire				X		X
Optional questionnaires	Educational Career			X	X		X
	ICT			X	X	X	X			X
	Parent				X	X
	Teacher	X
	Well-being			X		X

Notes: The PISA assessment had a computer format in 70 countries/economies. The global competence assessment was conducted in 27 countries, the financial literacy assessment in 21, the educational career questionnaire in 31, the ICT questionnaire in 50, the parent questionnaire in 17, the teacher questionnaire in 19 and the well-being questionnaire in 9 countries.

Regional analyses

UNICEF and the OECD have regularly studied education in the EECA region. Since 2006, the UNICEF Europe and Central Asia Regional Office has conducted analysis of PISA results for several countries in the region. UNICEF and the OECD have recently completed education policy reviews on schooling for Romania (2017), Turkey (2019) and Georgia (2019). The OECD has also conducted reviews in Kazakhstan (2020, 2015 and 2014) and Ukraine (2017). These studies focused on policies related to evaluation and assessment, school resources, skills development, vocational education and integrity. In 2020, UNICEF and the OECD also developed a report based on PISA data for countries in the Western Balkans region. The knowledge base and analytical frameworks built by these activities greatly inform and shape this report.

Key features of Eastern Europe and Central Asian countries and their implications for student learning, as measured by PISA

In each participating country, PISA 2018 assessed a representative sample of children between the ages of 15 years and 3 months and 16 years and 2 months who were enrolled in an educational institution at Grade 7 or above. A two-stage sampling procedure selected a sample of at least 150 schools and roughly 42 students within each of those schools. The majority of countries assessed between 5 000 and 7 000 students. Kazakhstan tested roughly 20 000 students in order to produce representative results for each region. The national context of each country that participates in PISA affects greatly the students who are sampled to participate in the survey. This section discusses some of the key contextual features of EECA countries, and how these contexts are represented in their PISA 2018 student samples.

Socio-economic context

EECA countries have more socio-economically disadvantaged students compared to OECD countries

An important concern for all countries is how students from disadvantaged backgrounds perform compared to their advantaged peers, which helps indicate the extent to which the school system helps students overcome socio-economic inequalities. While there is variation between countries, EECA countries are, on average, lower income than those in the OECD. EECA countries had an average GDP per-capita of USD 20 839 (United States dollars) purchasing power parity (PPP) in 2018, compared to the OECD average of USD 44 994 (Table 1.3).

While wealth is an important measure of socio-economic status, other factors also influence a student’s level of advantage. In PISA, a student’s background is represented through the index of economic, social and cultural status (ESCS), which is created based upon information about a student’s home environment, parents’ level of education and parents’ employment. This index is calculated such that the OECD average is 0.0 and one standard deviation is 1.0. The average ESCS across EECA countries is -0.4. However, there are disparities within the region. Belarus has an ESCS of 0.1, while Turkey has an average ESCS of -1.1. Since socio-economic context and student performance are closely related, it is important to consider these data when interpreting and comparing the educational outcomes of EECA countries.

Table 1.3. Socio-economic indicators

	Per-capita GDP in 2018 (PPP, USD)	PISA 2018 ESCS
Azerbaijan	14 556	-0.6
Belarus	19 345	0.1
Bulgaria	22 611	-0.3
Croatia	28 215	-0.2
Georgia	14 604	-0.4
Kazakhstan	26 167	-0.4
Moldova	12 674	-0.6
Romania	29 193	-0.5
Turkey	28 395	-1.1
Ukraine	12 629	-0.2
EECA average	20 839	-0.4
OECD average	44 994	0.0

Note: The data for this table were collected before Costa Rica became an OECD member.

Sources: The World Bank (n.d.[6]), GDP per-capita (current international ), https://data.worldbank.org/indicator/NY.GDP.PCAP.PP.CD (accessed 19 February 2021); (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

A relatively higher share of students in EECA countries attend schools in rural areas

The EECA region is vast and includes a variety of communities from small, rural villages to large, urban cities. On average, the share of students who attend school in rural communities (defined as having populations of 3 000 people or fewer) is relatively larger across the EECA region (15% compared to 9% across the OECD), but some countries have considerably higher shares. In fact, Moldova (47%), Georgia and Kazakhstan (both 30%) are three of the four most rural countries that participate in PISA. Research has shown that rural schools can face several challenges, ranging from infrastructure to human resources (Echazarra and Radinger, 2019[7]). Where relevant (and focusing on countries with large shares of students who attend schools in rural areas), this report will explore how school location can shape student learning outcomes.

Educational landscape

PISA coverage rates vary in EECA countries

As PISA only assesses students attending an education institution, the learning outcomes of 15-year-olds who are out of school are not captured in PISA data. The share of the total population of 15-year-olds in a country that is eligible to participate in PISA is known as the coverage index. In some EECA countries, the general age at which compulsory education ends is 15 or earlier (Table 1.5). In these countries, some students might already have left school when PISA is administered, which can lower the countries’ coverage indices. Other factors, such as a high rate of dropout or grade repetition, can also affect a country’s coverage index.

Across EECA countries, the coverage index is slightly lower than the OECD average (80% compared to 88%) (Table 1.5). Disparities at the country-level are quite wide. While Kazakhstan and Moldova have coverage indices above 90%, Baku (Azerbaijan) has a coverage index of 46%, which is the lowest among all PISA-participating countries and reflects the relatively low leaving age. Readers of this report should interpret PISA results in light of these differences in coverage.

Table 1.4. Duration of compulsory education/training and student age groups, 2018-19

	Official entrance age to primary education (years), 2019	General leaving age
Azerbaijan	6	14
Belarus	6/7	14/15
Bulgaria	7	16
Croatia	6/7	15
Georgia	6	14
Kazakhstan	6	15
Moldova	7	16
Romania	6	17
Turkey	6	18
Ukraine	6	17

Notes: Grade that corresponds to end of compulsory education is from UNICEF-OECD country reviews.

Starting age refers to the official age at which students start compulsory education/training.

The possibility of early entry to primary education is not taken into account nor are the specific admission conditions of pupils officially recognised with special educational needs.

Leaving age refers to the statutory age at which students are expected to complete compulsory education/training.

Source: (UNESCO-UIS, 2021[8]), UIS dataset, http://data.uis.unesco.org/, (accessed 29 June 2021).

Students in EECA countries take PISA in both lower and upper secondary education

In some countries, 15-year-old students are transitioning from lower secondary to upper secondary education, which means that PISA participants in those countries are often from both these levels of education. In EECA countries, more students are in upper secondary education when they take PISA compared to the OECD average (76% vs 52%). Nevertheless, less than 62% of students in Baku (Azerbaijan), Belarus and Kazakhstan were in upper secondary education, and less than 10% of students in Moldova were. Which level students are in when they take PISA could affect their results. As mentioned previously, in many EECA countries compulsory education ends before upper secondary education, and thus upper secondary students may be a more self-selective group.

EECA countries generally track upper secondary students into general education and vocational pathways and more specific programmes

Many countries divide students into different types of educational pathways, or tracks. Among these pathways, the two most common are general education, which typically prepares students for academic tertiary studies, and vocational education, which equips students with practical skills to enter the workforce (in most countries vocational students can also enter tertiary education). Internationally, countries vary in terms of when students are selected into different tracks. While some systems, such as Austria, start sorting students after primary education, the majority start offering distinct tracks to students at the beginning of upper secondary school.

In the EECA region, 28% of upper secondary students are enrolled in a vocational pathway (compared to 21% across the OECD) but the size and nature of vocational sectors varies greatly across countries. Although in Baku (Azerbaijan), Georgia and Moldova have almost no students in vocational pathways, 49% of students in Bulgaria and 68% of students in Croatia are enrolled in vocational pathways at the upper secondary level. In Kazakhstan, a sizeable vocational sector operates, but is considered largely separate from the upper secondary education system and is often classified at ISCED 4 and 5 levels. A distinguishing feature of EECA education systems is that many select students into specific programmes within pathways (e.g., general education schools that specialise in mathematics). Chapter 2 of this report explores issues around student grouping and segregation in greater depth.

Table 1.5. Characteristics of the students in the PISA 2018 sample

		Baku (Azerbaijan)	Belarus	Bulgaria	Croatia	Georgia	Kazakhstan	Moldova	Romania	Turkey	Ukraine	EECA average	OECD average
Number of students		6 827	5 803	5 294	6 609	5 572	19 507	5 367	5 075	6 890	5 998	-	-
Percentage of the 15-year-old population covered by the PISA sample (Coverage Index 3)		46	88	72	89	83	92	95	73	73	87	80	88
Modal grade (grade most represented by 15-year-olds)		Grade 10	Grade 10	Grade 9	Grade 9	Grade 10	Grade 10	Grade 9	Grade 9	Grade 10	Grade 10	-	-
Share in upper secondary education		62	56	100	100	85	54	10	93	99	100	76	52
Students' PISA index of economic, social and cultural status		-0.6	-0.1	-0.3	-0.2	-0.4	-0.4	-0.6	-0.5	-1.1	-0.2	-0.4	0.0
Share of girls		47	48	47	50	48	49	49	48	50	47	48	50
Share of students with an immigrant background		5	4	1	9	1	8	1	1	1	2	3	13
Share of students who speak the test language at home		89	96	87	97	94	90	91	97	93	64	90	88
Share of students enrolled in vocational programmes	All students	0	14	49	67	0	20	3	12	33	28	23	12
	Students in upper secondary education	0	25	49	68	0	36*	-	13	33	28	28	21
Share of students enrolled in schools located in:	A village or rural areas (fewer than 3 000 people)	1	17	3	1	30	30	47	7	1	19	15	9
	Towns (from 3 000 to about 100 000 people)	52	33	57	59	23	19	32	53	32	34	40	53
	Cities (over 100 000 people)	47	50	40	40	47	51	21	40	67	47	45	38
Share of students in private schools		0	0	1	2	11	8	1	2	12	1	4	18

* Classified as ISCED 5

Note: The data for this table were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Learning outcomes in Eastern Europe and Central Asia

Overall performance

PISA results show that student outcomes in some EECA countries have improved over time. In Moldova and Turkey, student outcomes in reading have improved between the first year the countries participated and 2018. These countries have also increased their coverage indices, showing that gains in educational access and learning outcomes are not mutually exclusive (Table 1.6) (also see Box 1.3 for a discussion on how rising coverage indices might be reflected in different countries).

In other countries, student outcomes in reading have not changed between the first year they participated in PISA and 2018. From cycle to cycle, however, some differences can be observed. Georgia, for instance, improved in reading from an average of 374 score points in 2009 to 401 in 2015, before declining to 380 in 2018. On the other hand, outcomes in Bulgaria decreased from 430 on average in 2000 to 402 in 2006, before increasing in subsequent years2.

Table 1.6. PISA performance in reading over time

	Score points in earliest year of availability	Score points in 2018	Coverage index in earliest year of availability	Coverage index in 2018
Bulgaria	430 (2000)	420	83% (2006)	72%
Croatia	477 (2006)	479	85% (2006)	89%
Georgia	374 (2009)	380	76% (2009)	83%
Kazakhstan	390 (2009)	387	89% (2009)	92%
Moldova	388 (2009)	424	90% (2009)	95%
Romania	428 (2000)	428	66% (2006)	73%
Turkey	441 (2003)	466	36% (2003)	73%

Notes: Bulgaria and Romania conducted the PISA 2000 assessment in 2001 as part of PISA 2000+. Georgia and Moldova conducted the PISA 2009 assessment in 2010 as part of PISA 2009+.

Statistically significant performance differences are represented in bold.

Coverage index refers to the percentage of the 15-year-old population represented in a country’s PISA sample.

Data for the coverage index were not available before 2003.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Though results in the region are generally improving, overall outcomes in the EECA region are still lower than international benchmarks (Figure 1.1). All countries in the region performed below the OECD average in reading, mathematics and science, though there is considerable variation. Students in Belarus and Croatia perform similarly to OECD countries such as Italy and Latvia. Meanwhile, Georgia and Kazakhstan perform similarly to lower-middle income countries like Panama and Thailand.

Figure 1.1. Average performance in Reading, Mathematics and Science

Note: The data for this figure were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

As mentioned previously, one should interpret PISA results in light of participants’ economic development, as 44% of performance differences in mean reading scores between countries in PISA 2018 can be accounted for by national income (OECD, 2019[5]). Figure 1.2 shows the performance of education systems relative to their per-capita GDP. In general, education systems in the EECA region perform around what would be predicted by their levels of economic development. However, some countries perform higher relative to others with similar income levels. Ukraine for example, performs better than several wealthier countries, which indicates the potential for policy to help overcome resource limitations.

Figure 1.2. GDP per-capita and average reading performance

Notes: EECA economies are marked and labelled in red. Green lines indicate the OECD average. The data for this figure were collected before Costa Rica became an OECD member.

Sources: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020), World Bank (n.d.[6]). GDP per-capita (current international $), https://data.worldbank.org/indicator/NY.GDP.PCAP.PP.CD (accessed 10 February 2021).

Statlink

To help understand differences in student knowledge and skills, PISA categorises student performance into different proficiency levels. These levels in reading, which was the main assessment domain in PISA 2018, range from the highest (Level 6) to the lowest (Level 1c) proficiency (Table 1.7). Level 2 is considered the minimum level of proficiency students need to acquire to advance in their education and participate in modern societies.

Table 1.7. Summary description of the eight levels of reading proficiency in PISA 2018

Level	Lower score limit	Percentage of students able to perform tasks at each level or above (OECD average)	Characteristics of tasks
6	698	1.3%	Readers at Level 6 can comprehend lengthy and abstract texts in which the information of interest is deeply embedded and only indirectly related to the task. They can compare, contrast and integrate information representing multiple and potentially conflicting perspectives, using multiple criteria and generating inferences across distant pieces of information to determine how the information may be used.
5	626	8.7%	Readers at Level 5 can comprehend lengthy texts, inferring which information in the text is relevant even though the information of interest may be easily overlooked. They can perform causal or other forms of reasoning based on a deep understanding of extended pieces of text. They can also answer indirect questions by inferring the relationship between the question and one or several pieces of information distributed within or across multiple texts and sources, and can establish distinctions between content and purpose, and between fact and opinion.
4	553	27.6%	At Level 4, readers can comprehend extended passages in single or multiple-text settings. They interpret the meaning of nuances of language in a section of text by taking into account the text as a whole. In other interpretative tasks, students demonstrate understanding and application of ad hoc categories. They can compare perspectives and draw inferences based on multiple sources.
3	480	53.6%	Readers at Level 3 can represent the literal meaning of single or multiple texts in the absence of explicit content or organisational clues. Readers can integrate content and generate both basic and more advanced inferences. They can also integrate several parts of a piece of text in order to identify the main idea, understand a relationship or construe the meaning of a word or phrase when the required information is featured on a single page.
2	407	77.4%	Readers at Level 2 can identify the main idea in a piece of text of moderate length. They can understand relationships or construe meaning within a limited part of the text when the information is not prominent by producing basic inferences, and/or when the text(s) include some distracting information.
1a	335	92.3%	Readers at Level 1a can understand the literal meaning of sentences or short passages. Readers at this level can also recognise the main theme or the author’s purpose in a piece of text about a familiar topic, and make a simple connection between several adjacent pieces of information, or between the given information and their own prior knowledge.
1b	262	98.6%	Readers at Level 1b can evaluate the literal meaning of simple sentences. They can also interpret the literal meaning of texts by making simple connections between adjacent pieces of information in the question and/or the text.
1c	189	99.9%	Readers at Level 1c can understand and affirm the meaning of short, syntactically simple sentences on a literal level, and read for a clear and simple purpose within a limited amount of time.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Figure 1.3 shows that on average in the EECA countries, 42% of 15-year-old students did not attain the baseline proficiency level in reading (vs. 23% in the OECD). These students cannot identify the main idea of a text of moderate length, find information based on explicit, but sometimes complex, criteria, and reflect on the purpose and form of texts when explicitly directed to do so. However, there are large differences between countries in the region: Belarus, Croatia, Turkey and Ukraine were close to the OECD average, with about one student in four not reaching this baseline level. On the other hand, in Baku (Azerbaijan), Georgia and Kazakhstan, more than 60% of students do not reach this level.

Figure 1.3. Proficiency levels in reading

Notes: Countries are sorted by the percentage of students below Level 2 in reading.

The data for this figure were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Box 1.2. Meta-cognitive skills

In addition to measuring students’ reading literacy in general, PISA 2018 measured a specific set of reading skills, called meta-cognitive skills. PISA 2018 defines meta-cognitive skills as knowing how to guide one’s own understanding and learn in different contexts. Having meta‑cognitive skills is crucial in modern societies because they help individuals navigate, interpret and solve unanticipated problems. To measure meta-cognitive skills, PISA asked students about the usefulness of various strategies (understanding and remembering; summarising; assessing credibility) for accomplishing different types of reading tasks and compared their responses to those given by a group of experts. All EECA countries except Ukraine are below the OECD average in terms of students’ meta-cognitive skills, and some by considerable margins (Figure 1.4)

Students in the region generally struggle more when asked to choose the best strategies for assessing the credibility of a source (especially in Baku (Azerbaijan) and Kazakhstan) and when summarising information. For example, PISA asked students what is an appropriate response to receiving an email from a mobile phone operator informing them that they have won a smartphone. EECA students were more likely to say that clicking on the associated link and filling out an online form was appropriate. Students from OECD countries were more likely to be sceptical of such an offer, saying that they would check the website of the mobile phone operator to see if the offer is mentioned or delete the email without clicking on the link. This finding has economic and social implications, as it suggests that students from the region might be less discerning and critical of the information that they access.

Figure 1.4. Meta-cognitive skills

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Performance and equity

In addition to overall performance, PISA measures the outcomes of different student groups within an education system. This type of disaggregation helps policy makers understand if all students are achieving similar outcomes, or if some students are performing very well while others are falling behind. This report concentrates primarily on equity according to students’ socio-economic status, gender and, where relevant, school location (in a rural or urban area), which are important issues in the EECA region.

Figure 1.5 shows that, when looking across all PISA-participating economies, there is a strong, positive relationship between overall performance and variation in performance, likely owing to the wider range of possible student outcomes in higher performing countries. As EECA countries typically have lower performance compared to the OECD average, disparities between student groups in EECA countries might be smaller in absolute terms, but that does not mean these gaps are less meaningful. Readers should keep this information in mind as they interpret the PISA results. Where appropriate, this report will also report results in terms of country-level standard deviations to help contextualise comparison.

Figure 1.5. Average performance and within-country variation in reading

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Socio-economic status

Socio-economically advantaged students3 perform better on PISA than disadvantaged students in all PISA-participating countries and economies. On average across EECA countries, socio-economically advantaged students score 80 points more than socio-economically disadvantaged students (the gap across OECD countries is 89). Such gaps are highest in Romania (109) and Bulgaria (106), and lowest in Baku (Azerbaijan) (41) and Kazakhstan (40).

Figure 1.6. Socio-economic status and reading performance

Notes: Countries are sorted by the difference in reading score. All differences are statistically significant.

The coverage index for Baku (Azerbaijan) was only 46%, so many 15-year-olds with a disadvantaged background will not even have been at school and do not appear here: the relationship between socio-economic status and performance may have been greater if it could have been observed on the entire 15-year-old population (OECD, 2019, p. 56[9]).

The data for this figure were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Box 1.3. Performance and participation of vulnerable students over time

As indicated in Table 1.6, in many EECA countries the share of students who are eligible to participate in PISA (coverage index) has increased over time. Increased coverage generally means that an education system is enrolling more students from disadvantaged backgrounds who may face more obstacles to learning (for example, less support from less educated parents), which can influence a country’s overall outcomes (OECD, 2019[9]). However, specific changes in the composition of the student sample can differ according national contexts, and these differences can also shape how changes in overall performance are interpreted.

Figure 1.7 shows the change in the population of sampled students whose parents do not hold a higher education qualification, and the change in their achievement. These students were selected for further analysis because they are more likely not to be in school, and thus less likely to be covered in the PISA sample. Furthermore, the definition of higher education qualification is consistent over time and thus the numbers of students from this group are comparable across years.

Figure 1.7. Reading proficiency among students whose parents do not hold a higher education qualification

Sample is restricted to students whose parents do not hold a higher education qualification

Notes: The width of the columns represents the number of students whose parents do not hold a higher education qualification and are scaled to be proportionate within each country.

The area of each column represents the number of students whose parents do not hold a higher education qualification who performed below Level 2 proficiency in reading.

Data from Bulgaria and Romania are from 2006 because coding for parental education was different in 2000, when they first participated.

The four countries are selected because their coverage indices in 2018 were below that of the OECD average. Baku (Azerbaijan) is excluded because it did not previously participate as a municipality.

Sources: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020); (World Bank, 2021[1]), Data Bank, https://data.worldbank.org/indicator/SE.TER.CUAT.BA.ZS?end=2017&locations=GE-RO&start=1975&view=chart (accessed 26 June 2021).

Statlink

Changes in the achievement of this student group vary across the analysed countries. In Bulgaria and Georgia, a greater share of sampled students achieved below Level 2 proficiency in reading in 2018 than in 2006 and 2009, respectively. These results suggest that these countries might not be effectively supporting vulnerable students, despite there being no significant increase in the number of such students, or even a decrease as in the case of Georgia. In Romania and Turkey, a smaller share of students whose parents do not hold a higher education qualification now achieve below Level 2 proficiency in reading, which indicates that these countries might be targeting more support at potentially vulnerable students. The situation is particularly noteworthy in Turkey given that the country also increased its population of such students.

The factors explaining the changes over time in the share of students whose parents do not hold a higher education qualification vary considerably across countries. In Georgia and Romania, the population of students whose parents do not hold a higher education qualification have decreased considerably, even though coverage indices increased, which is likely related to the increasing share of adults with a higher education qualification in these countries. In Bulgaria, the number of students from this group remained constant even though the coverage index decreased, which might be explained by high rates of brain drain (upcoming review). In Turkey, where compulsory education was extended to 12 years in 2011 (see Chapter 2), the coverage index doubled and the number of students in the PISA sample whose parents do not hold a higher education qualification increased

Gender

PISA results consistently show that girls tend to outperform boys by about 30 points in reading. In mathematics, boys outperform girls by roughly 5 points, and differences in science are not significant on average. In EECA countries, girls outperform boys by 32 points on average in reading in PISA 2018, which is similar to the difference across the OECD (30 points on average). Like OECD countries, there is considerable variation across countries. Six EECA countries have gender gaps greater than the OECD average, with the highest in Moldova and Bulgaria (40 score points). However, in terms of standard deviations, eight out of ten EECA countries have a larger gap than the OECD average.

Performance differences according to gender have decreased over time. Six out of eight countries in the region have reduced their gender gaps between their first years of participation and 2018 (Figure 1.8). These decreases were often because boys increased in performance while girls decreased, which was the case in Croatia, Georgia and Kazakhstan. In Bulgaria, both boys and girls decreased in performance, but girls decreased more than boys.

Figure 1.8. Differences in reading performance by gender over time

Notes: In 2006, Azerbaijan participated in PISA as a country. The data for this figure were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

School location

In most PISA-participating countries and economies, students enrolled in urban areas have higher performance than students in rural schools (OECD, 2019[9]).Among EECA countries where more than 3% of 15-year-old students were enrolled in rural schools, the urban-rural gaps in Moldova (89 points) and Romania (110 points) are considerably larger than the same gap across the OECD (35 points) (Figure 1.9). In terms of standard deviations, Kazakhstan’s gap (0.55 standard deviations) is also larger than that of the OECD (0.51 standard deviations). After accounting for student and school socio-economic status, the relationship between geography and performance weakens but remains statistically significant in Georgia, Kazakhstan and Moldova.

Figure 1.9. Average reading performance by school location

* Includes only the 22 OECD countries with more than 3% of students in rural schools.

Notes: From principals’ reports on community in which their school is located.

Baku (Azerbaijan), Bulgaria, Croatia and Turkey have few 15-year-olds in rural schools (3% or less) so are not included in the figure.

The data for this figure were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Educational tracks

Like in OECD countries, reading performance in EECA also varies according to education tracks, and gaps in three EECA countries are as large or larger than the OECD average (Figure 1.10). In terms of standard deviations, however, five EECA countries have gaps as large or larger than the OECD average, with only Kazakhstan and Ukraine having smaller differences. The observed gap in learning achievement between general and vocational pathways reflects not only a difference in curriculum but also a difference in student intake. Boys and socio-economically disadvantaged students are more likely to be enrolled in vocational programmes in all EECA countries where such tracks are offered (see Figure 1.11). These data suggest that student grouping and tracking in EECA countries reflect educational inequities at lower levels of education, and, without careful interventions, could risk exacerbating them.

Figure 1.10. Reading performance at the upper secondary level by educational tracks

* Includes only the 23 OECD countries with at least 3% of students in vocational/pre-vocational schools.

Notes: In Moldova most 15-year-old students are at the lower secondary level and in Baku (Azerbaijan) and Georgia there are few vocational students at the upper secondary level, so they are not included in the figure.

The data for this figure were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Figure 1.11. Profile of general and vocational students at the upper secondary level

Difference in the share of students in vocational pathways by:

 

* Includes only the 23 OECD countries with at least 3% of students in vocational/pre-vocational schools.

Notes: In Moldova most 15-year-old students are at the lower secondary level and in Baku (Azerbaijan) and Georgia there are no separate general and vocational programmes even at the upper secondary level, so they are not included in the figure.

The data for this figure were collected before Costa Rica became an OECD member.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

Box 1.4. Language of instruction

In Baku (Azerbaijan), Kazakhstan, Moldova and Ukraine, school instruction occurs in Azerbaijani, Kazakh, Romanian and Ukrainian, respectively, and also Russian in each country. PISA 2018 data were analysed to better understand if there are differences in learning outcomes according to students’ languages of instruction (and, by proxy, the schools that instruct in those languages).

For each country, the population subject to analysis was limited to students who speak the more common national language at home (Azerbaijani, Kazakh, Romanian and Ukrainian). Students’ socio-economic background, and the socio-economic status of the schools they attend, was also accounted for. Results show that students in Kazakhstan and Moldova who speak Kazakh and Romanian, respectively, who attend Russian-speaking schools perform better than those who attend Kazakh- and Romanian-speaking schools.

Figure 1.12. Language of instruction and reading performance

Difference in reading between students who attend school in Russian and students who attend school in Azerbaijani, Kazakh, Romanian or Ukrainian (Russian minus other language)

Note: Results that are statistically significant are shaded.

Source: (OECD, 2019[5]), PISA 2018 Database, https://www.oecd.org/pisa/data/2018database/ (accessed 17 November 2020).

Statlink

These findings suggest that school-level factors related to the language of instruction could affect student performance in Kazakhstan and Moldova (e.g. textbooks). Nevertheless, the findings should be interpreted carefully. While language spoken at home and socio-economic background are important factors, they do not represent completely all the differences between Kazakh/Romanian- and Russian-speaking populations in Kazakhstan and Moldova. It is possible that unaccounted for contextual variables (e.g. parental background) can help explain these differences.

References

[7] Echazarra, A. and T. Radinger (2019), “Learning in rural schools: Insights from PISA, TALIS and the literature”, OECD Education Working Papers, No. 196, OECD Publishing, Paris, https://dx.doi.org/10.1787/8b1a5cb9-en.

[3] EU (2020), Partnership for Good Governance, https://pjp-eu.coe.int/en/web/pgg2/home (accessed on 16 March 2021).

[5] OECD (2019), PISA 2018 Results (Volume I): What Students Know and Can Do, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/5f07c754-en.

[9] OECD (2019), PISA 2018 Results (Volume II): Where All Students Can Succeed, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/b5fd1b8f-en.

[6] The World Bank (n.d.), GDP per capita (current US$) | Data, https://data.worldbank.org/indicator/NY.GDP.PCAP.CD?locations=XK-AL-ME-MK-RS&view=chart (accessed on 1 February 2019).

[8] UNESCO-UIS (2021), UIS dataset, http://data.uis.unesco.org/ (accessed on 4 July 2018).

[4] UNICEF (2017), Improving Education Participation: Policy and Practice Pointers for Enrolling All Children and Adolescents in School and Preventing Dropout, https://www.unicef.org/eca/media/2971/file/Improving_education_participation_report.pdf (accessed on 17 December 2020).

[1] World Bank (2021), GDP Per Capita, PPP, https://data.worldbank.org/indicator/NY.GDP.PCAP.PP.CD (accessed on 11 January 2020).

[2] World Bank (2021), Gini index (World Bank estimate), https://data.worldbank.org/indicator/SI.POV.GINI? (accessed on 16 March 2021).

Notes

← 1.  This report focuses on PISA-participating countries in Eastern Europe and Central Asia that are supported by the UNICEF ECARO office. The ten countries from this region that participated in PISA 2018 are Azerbaijan (only the city of Baku participated in PISA 2018), Belarus, Bulgaria, Croatia, Georgia, Kazakhstan, Moldova, Romania, Turkey and Ukraine.

← 2.  PISA scores do not have a substantive meaning but are set in relation to the variation in results observed across all test participants. The results are scaled to fit approximately normal distributions, with means around 500 score points and standard deviations around 100 score points. The metric for each scale was set when it was first developed as a major domain. The mean reading score for the 28 OECD member countries at the time was set at 500 score points, with a standard deviation of 100 points, in PISA 2000; the OECD mean mathematics score was set at 500 in PISA 2003; and the OECD mean science score was set at 500 in PISA 2006.

← 3.  PISA measures a student’s socio-economic status through responses on the student questionnaire in three areas—parents’ level of education, parents’ employment and household possessions.