OECD Reviews of Evaluation and Assessment in Education: Bulgaria

Assessment and recommendations

Education in Bulgaria

Educational improvement aims to support inclusive and resilient growth in Bulgaria

Over the past three decades, Bulgaria has carried out important structural reforms which, alongside accession to the European Union (EU) in 2007, have contributed to robust economic growth and improved living standards (OECD, 2021[1]). Despite these efforts, convergence towards EU and OECD income levels has been slower than in other Central and East European countries. Economic growth is concentrated in regions with large cities, exports are dominated by primary goods and poverty remains high, particularly among ethnic minorities (OECD, 2021[2]). Demographic decline is producing additional pressures, with many rural regions suffering from both depopulation and rapid ageing. Positively, Bulgaria has seen strong performance in high value-added sectors, such as information and communication technology (ICT), where labour productivity appears more than twice the average rate recorded for the non-financial business economy (OECD, 2021[2]). Continued growth in these sectors could enable Bulgaria to boost its economic competitiveness and create more highly skilled jobs but this will be contingent on reforms in a range of areas, including education and skills.

The national development strategy, Bulgaria 2030, sets out an ambitious reform plan that identifies the importance of raising educational attainment and addressing inequities in order to sustain socio-economic growth. In December 2021, the government also announced plans for a new education programme to revise and update laws regulating the sector with the goal of making education more inclusive and improving co-ordination among stakeholders at the school level, state and local authorities, as well as across broader society (Fileva, 2021[3]). While these are positive developments, Bulgaria will require better data (particularly on the needs and outcomes of vulnerable groups), a closer monitoring of progress and stronger policy alignment to implement national education goals.

Bulgaria’s education system has evolved over recent years but challenges remain

Participation in early childhood education and care is a central education priority

Bulgaria introduced legislation to make pre-school education compulsory for all children from age four starting in 2020. This move was designed to increase participation in pre-school education, which had been low and actually decreasing since 2015 (Eurostat, n.d.[4]; EC, 2020[5]). In 2018, only 82.4% of children aged 4 to 7 were enrolled in Bulgarian pre-schools, compared to the EU average of 94.8% (EC, 2020[5]). Programmes are also in place to support the most disadvantaged children by financing care-related fees, providing parental education and pedagogical, psychological and social support for children (EC, 2020[5]). Bulgaria’s emphasis on raising pre-school participation reflects trends in many EU and OECD countries, which have also made investments in policies to increase pre-school enrolments as a means to support children’s long-term development and improve overall equity in their education systems (OECD, 2020[6]).

Despite previous gains in secondary enrolments, participation in basic education appears to be falling

Over the past ten years, Bulgaria has made impressive gains in raising participation at the secondary education level but progress appears to have stalled and may be moving backwards (see Figure 1). For example, net enrolment in upper secondary education climbed steadily from 81% in 2010 to 90% by 2017 but has since been declining (UIS, 2018[7]). Enrolment in other levels of education have also been declining and, as of 2018, Bulgaria now has one of the lowest rates of net enrolment at the lower secondary level among regional peers (85%) – with Hungary (97%), Poland (98%) and Serbia (98%) all maintaining “full” participation at this level of education (ibid). While a mass-tracking campaign carried out in 2017 and 2018 suggested that over 80% of unenrolled school-age children were actually living abroad (EC, 2018[8]), there is also evidence that a significant number of students are leaving school for other reasons. Data compiled by the National Statistical Institute (hereafter the NSI) for the 2018/19 academic year suggests that 41% of primary students and 39% of lower secondary students who dropped out of schooling did so for “family reasons” (NSI, 2020[9]). There is a risk that Bulgaria’s falling enrolment and completion rates will be compounded by the ongoing COVID-19 pandemic, as disruptions in learning may lead students, especially the most vulnerable, to not return to school.

Figure 1. Net enrolment rate by level of education in Bulgaria, 2003-18

Source: UIS (2018[7]), UIS.Stat, http://data.uis.unesco.org/ (accessed on 4 February 2021).

Statlink

Many young people have not mastered foundational competencies

Data from OECD Programme for International Student Assessment (PISA) suggest that a sizeable share of young people in Bulgaria have not acquired the foundational cognitive skills they need for lifelong learning and productive employment. In 2018, 32% of 15-year-old students scored below the baseline of Level 2 in all PISA subject domains, compared to an OECD average of 13% and an EU average of 14% (OECD, 2019[10]). PISA data also suggest that, like participation rates, progress in learning outcomes may have stalled or begun to move backwards over recent years (see Figure 2). There has been a significant statistical decline in reading outcomes between the 2012 and 2018 rounds of PISA (436 to 420), as well as in science (446 in 2015 compared to 424 in 2018). However, the lack of a national assessment that produces reliable and timely trend data makes it difficult to understand students’ learning progression across grades and over time.

Figure 2. Trend in average reading, mathematics and science scores by PISA cycle

Source: OECD (2019[10]), PISA 2018 Results (Volume I): What Students Know and Can Do, https://doi.org/10.1787/5f07c754-en.

Statlink

Teaching practices and the school environment are not conducive to student learning and engagement

Similar to many European countries, Bulgaria has an ageing teaching profession, which creates concerns about teacher shortages and outdated pedagogical skills. Data from TALIS 2018 indicate that over half of Bulgaria’s teachers at the lower secondary level are aged 50 or above, compared to an average of 34% in OECD countries (see Figure 3) (OECD, 2019[11]). Until recently, the country also faced difficulties in attracting new entrants to the teaching profession, especially for posts in rural areas and high-demand subjects. While recent increases to teacher salaries have helped facilitate the recruitment of new teachers, many practising teachers report a need for training in modern teaching methods, such as using ICT in the classroom, managing student behaviour, teaching in a multicultural or multilingual environment and supporting children with special needs (OECD, 2019[11]). Where training does exist, it is often of poor quality and prohibitive cost (OECD, 2019[11]).

There are other concerns with the teaching and learning environment in Bulgaria. Demographic trends have provoked school closures in rural areas and the overcrowding of schools in urban areas. This context, combined with the squeeze in the teacher workforce, has contributed to shorter instructional time in Bulgarian schools for key subjects, compared with other EU countries (IEA, 2017[12]; 2020[13]). Findings from PISA 2018 also suggest issues around student engagement and the school environment. Some 44% of students report that they had skipped a whole day of school at least once in 2018, compared to an OECD average of 21%, and 34% of students report that they are bullied at least a few times a month, compared to an OECD average of 23% (OECD, 2019[14]). Moreover, during the COVID-19 pandemic, keeping students engaged in an online environment where classes often took place, added extra pressure on teachers, parents and students themselves. This is particularly the case for students from disadvantaged backgrounds, as they are more likely to lack the parental support, resilience, learning strategies or engagement to learn on their own (OECD, 2020[15]).

Figure 3. Teachers’ age by level of education that they teach in, 2019

Note: ECEC – Early childhood education and care.

Source: Eurostat (2021[16]), Classroom Teachers and Academic Staff by Education Level, Programme Orientation, Sex and Age Groups [educ_uoe_perp01], https://ec.europa.eu/eurostat/data/database (accessed on 20 July 2021).

Statlink

Learning outcomes are notably lower for ethnic minorities and those living in rural areas

International assessments suggest significant variation in learning outcomes among different demographic groups. PISA 2018 found a 106 score point difference in reading between students from disadvantaged versus advantaged socio-economic backgrounds in Bulgaria, compared to an 89 score point difference on average across the OECD and smaller gaps in neighbouring North Macedonia (80 score points) and Serbia (73 score points) (OECD, 2019[10]). Participation in schooling is also much lower among minority groups, with Roma children significantly under-represented in non-compulsory pre-school education and an estimated 45% of Roma leaving school before they complete secondary education (Republic of Bulgaria, 2019[17]). Variations in access to education also persist between rural and urban regions within Bulgaria, which risk compounding inequalities. The closure of schools and fewer educational facilities in rural and remote areas may be contributing to a higher rate of students dropping out before completing lower secondary school. Data reveal that the share of students who do not complete lower secondary school reaches 30% in villages and over 15% in small towns (EC, 2018[8]). The COVID-19 crisis has created new risks of learning loss and deepened inequalities. Early studies suggest that at least 50 000 school-age children in Bulgaria experienced significant learning disruption, a fifth of surveyed students reported performing worse and around half reported feeling lonely, insecure and angry (UNESCO, 2020[18]).

Bulgaria is working to improve teaching and learning

Over the past five years, Bulgaria has embarked on an ambitious path to modernise its education system. The national development strategy, Bulgaria 2030, sets out the country’s aim to become a knowledge- and innovation-intensive economy by 2030, with a high-technology industrial base (Ministry of Finance, 2020[19]). This calls for more students to develop higher-order competencies, such as critical thinking, creative problem solving and entrepreneurial mindsets, as well as stronger digital skills to facilitate the widespread adoption of ICTs. The country’s Pre-school and School Education Act in 2016 has also paved the way for a wave of structural reforms – including the introduction of a new competency-based curriculum, a dual vocational education and training (VET) system, a compulsory pre-primary year, a modern school inspectorate, an increase in teachers’ salaries and a more formative role for the regional departments of education (REDs). These reforms set out an ambitious vision and actionable steps to ensure that all students have access to quality schooling and gain relevant skills. At the same time, Bulgaria continues to face pressures that may challenge its ability to deliver on these reforms, not least relatively low per-student funding and notable funding gaps between municipalities. Institutional and professional capacities are also relatively weak, as is the technological infrastructure. The recently announced national education programme and pandemic recovery efforts represent a critical opportunity to strengthen the foundations for implementing Bulgaria’s education reform agenda.

Evaluation and assessment in Bulgaria

Applying the OECD framework for evaluation and assessment policies to Bulgaria’s education system

Over the past decade, the OECD has reviewed evaluation and assessment frameworks in over 30 education systems to help identify the factors associated with improving educational quality in different contexts. This research revealed three hallmarks of a strong evaluation and assessment framework that promotes the quality and equity of student learning. First, such a framework sets clear standards for what is expected nationally of students, teachers, schools and the system overall. Second, it directs the collection of data on performance, helping to ensure that stakeholders receive the information and feedback they need to reflect critically on their own progress and identify steps that will help them advance. Third, it promotes coherence and alignment, so the whole education system can work in the same direction and use resources effectively. This report recommends ways in which Bulgaria can strengthen its evaluation and assessment framework in the school education sector. Such policies are particularly important in the wake of COVID-19, as they play a crucial role in helping teachers, schools and policy makers to identify students who have fallen behind or dropped out, adapt instruction and redirect resources to where they are needed most.

While Bulgaria has created new learning standards, as well as broader evaluation and assessment policies that are aligned with the Pre-school and School Education Act (2016), the country’s high-stakes sorting and examinations culture continues to reinforce the perception of student assessment as a primarily summative exercise. Going forward, this review recommends that Bulgaria develop a new student assessment framework focused primarily on improving student learning. The country will need to remove practices that stand in the way of this goal, such as the National External Assessment (NEA), which currently helps select students into different pathways and does not produce trend data to fully support its system monitoring function. Redesigning the NEA as part of a new national assessment framework would help ensure it serves as a formative tool to support system monitoring and student learning. This review also sets out recommendations for how Bulgaria can close implementation gaps between its stated education policies and the school practices. For example, despite having a set of school quality standards, stakeholders often make narrow comparisons of schools using results from national examinations and academic competitions. Bulgaria could communicate a more comprehensive understanding of school quality and help facilitate a culture of improvement by providing more contextualised information on school performance and sharing examples of school practices that exemplify the national quality standards.

This report aims to support the Bulgarian government not only in strengthening its education policies but also in reviewing how it spends educational resources. Such processes are especially important as the COVID-19 pandemic continues to put additional pressure on public budgets. In a positive way, Bulgaria’s new school funding model has started directing more public education resources towards vulnerable groups and regions. This review proposes ways in which Bulgaria could further target available resources to where they can have the greatest impact. For example, investing in reporting templates and resources to improve the quality of start-of-year diagnostic assessments could help identify and address learning gaps before they become more problematic. Another example relates to the incentives and rewards for teachers and schools. While financial incentives, such as increases in salaries or extra funding for good results on external school evaluations can help drive improvements, these policies are associated with high costs. Bulgaria should instead consider targeting financial incentives and exploring non-financial rewards that would help free up resources to raise standards and learning outcomes in low-performing schools. Evaluating, reporting and adapting education policies in these ways can help Bulgaria strengthen trust in its reforms and create an enabling environment that not only improves teaching and learning but also advances the country’s wider socio-economic development goals.

Student assessment supports learning by helping teachers, students and parents determine what learners know and what they are capable of doing. This information can help identify specific learning needs before they develop into serious obstacles and enable students to make informed decisions about their educational pathways.

Bulgaria’s new competency-based curriculum has introduced important changes to student assessment policy, such as the use of start-of-year diagnostic tests, qualitative marking and a set of expected learning outcomes for each subject and grade level. While these policies have the potential to enhance the quality of education, practical changes in school and classroom practices have been slow to take effect. As a result, classroom-based assessments continue to focus on traditional summative tests, with a narrow emphasis on performance in a limited range of tasks as opposed to broader, deeper learning. The ability of teachers to adopt new assessment practices is also constrained by a lack of training and support, as well as political and public expectations of how to assess students and demonstrate achievement. These factors have important implications for Bulgarian students, as they encourage an educational approach that can undermine student agency, engagement and progress, and is not coherent with the goals of the country’s new competency-based curriculum.

Improving student assessment: Three areas for policy action

 

Policy issue 2.1. Building a shared understanding of student assessment as a means to support teaching and learning

Bulgaria has demonstrated a clear political will to modernise pedagogical approaches within its school system. This can help address some of the country’s major education challenges like tackling drop-out and raising student achievement since such practices can create a supportive learning environment that encourages all students to succeed. However, extensive changes to policy documentation have not translated into substantial pedagogical innovation or practical changes in student assessment at the classroom level. This is symptomatic of both a traditional culture of simple, summative assessment and a lack of attention to the resources and capacity needed to implement education reforms. Bulgaria needs to communicate the need and rationale for adopting new approaches to assessment, especially in the classroom. Enhancing the link between assessment and learning in a clear and coherent policy framework can help in this regard. At the same time, school leaders and teachers need support and practical resources to implement pedagogical changes successfully.

• Recommendation 2.1.1. Establish a clear and coherent national vision of student assessment. Bulgaria needs to establish student assessment as a central part of the learning process. Developing a common vision that provides an overview of the various components and instruments included in Bulgaria’s national assessment framework, as well as their different purposes, value and how they work together should be formalised in both legislation and accompanying explanatory materials. This can help shift the existing emphasis on summative assessments and high-stakes testing, towards a more balanced and comprehensive approach to assessing students. Engaging key stakeholders in the elaboration of the national vision of student assessment and using the exercise to make the country’s learning standards more coherent, accessible and practical can help ensure that it serves as a clear reference point for actors across the education system in years to come.

• Recommendation 2.1.2. Adapt the reporting of student learning information to promote a broader understanding of assessment. As in other countries with a strong history of summative assessment, Bulgaria faces the challenge of balancing tensions between stated commitments to using a wider range of assessments on the one hand and public, parental and political pressure for accountability in the form of scores and rankings on the other. To implement new assessment techniques in the classroom, Bulgaria should change student marking and reporting procedures so that they are more conducive to learning. Specifically, the government should reduce the frequency of required continuous assessments and instead consider requiring teachers to provide more granular, descriptive feedback at key moments in the school year. The Ministry of Education and Science (hereafter the Ministry) could also reframe qualitative descriptors to better promote progress (e.g. “exemplary” or “undeveloped” rather than “excellent” or “poor”) and should develop a range of substantive guidance materials to support students, teachers and parents in using reports and feedback more constructively. Such adaptations can create a more inclusive and individualised learning environments in Bulgarian classrooms, helping to address educational disparities among student demographic groups and raise overall learning outcomes.

Policy issue 2.2. Developing the capacity of teachers to use assessment formatively

Many teachers and principals in Bulgaria are committed to making assessments more informative for their practice and more meaningful and motivational for students. However, formative assessment is commonly misunderstood as “summative assessment done more often”. Without a deeper understanding of formative assessment and the confidence to use these practices in their classrooms, teachers in Bulgaria will likely struggle to address the diverse learning needs of their students, which in turn, this risks leading some students to disengage in the learning process. Bulgaria therefore has considerable scope to clarify teachers’ understanding of formative assessment and develop their skills in this area.

• Recommendation 2.2.1. Promote the use of diagnostic assessments to help teachers better understand and adapt to the learning needs of students. In Bulgaria, where large shares of students do not master basic skills and where learning gaps and disengagement start early, embedding formative assessment practices in the classroom has the potential to have a considerable positive impact on the learning of all students. To strengthen teachers’ formative assessment practices, Bulgaria should optimise the existing start-of-year diagnostic tests by enhancing their design quality and use. For example, introducing requirements for reporting diagnostic assessment results would help teachers in using this instrument for its intended purpose. At first, Bulgaria could focus on improving the administration and use of diagnostic assessments in priority subjects (e.g. mathematics) and priority years (e.g. the early years of primary education), before scaling to include other areas. This approach can help target investments in the diagnostic assessments to where they are likely to have the greatest impact on improving student performance.

• Recommendation 2.2.2. Foster real change at the classroom level by making training on formative assessment a priority for all teachers. Another way that Bulgaria should build capacity for formative assessment is by making this topic a prominent feature of initial teacher education and the teaching practicum. For in-service teachers, Bulgaria should provide methodological support on formative assessment (an initiative that could be facilitated through REDs), as well as create incentives for more experienced or engaged teachers to support colleagues in developing formative assessment practices within their schools. Together, these types of formal and peer learning structures can help teachers strengthen their assessment literacy while also aligning intended, implemented and assessed curricula.

• Recommendation 2.2.3. Equip teachers with a range of practical support to facilitate formative assessment in the classroom. Supporting teachers to integrate formative assessment in their classrooms will require ongoing support and resources that are easy to use in daily teaching practices. To do this, Bulgaria should gradually build an online library of guidance materials on formative assessment tools. This platform could include exemplar student report cards, rubrics for assessing students’ learning against expected learning outcomes and video tutorials on key aspects of formative assessment featuring good practices modelled in real classroom environments. The Ministry could also identify guidelines for REDs to support teachers with formative assessment while still allowing them to develop their own training programmes. Drawing on the expertise of REDs and collecting feedback from teachers and school leaders about the kinds of support they receive can help improve the quality and relevance of training and support offered to teachers already working in Bulgarian schools.

Policy issue 2.3. Enhancing the validity and fairness of examination and selection processes into and out of upper secondary education

Bulgaria provides multiple pathways into upper secondary education, which in principle encourages students to select study programmes that match their ambitions and aptitudes. In practice, however, selection processes appear to distort both student learning and progression. For example, the Grade 7 NEA, implemented initially as a monitoring tool, plays an outsized role in determining students’ educational destinies without safeguards to mitigate the adverse effects of high-stakes testing and a negative backwash on the curriculum. Moreover, student selection occurs at age 13 in Bulgaria, markedly earlier than in most countries across Europe and the OECD (see Figure 4), exacerbating challenges to system quality and equity. At the end of upper secondary education, the vast majority of students now take the State Matriculation examination, which is considered a valuable tool in facilitating student transitions. The integrity and reputation of the State Matriculation exam have increased in recent years thanks to its secure development, administration and marking procedures. However, there is scope to align this examination more closely with the subject areas covered in Bulgaria’s national curriculum and with broader goals, given that few students choose to take the examination in high-demand science, technology, engineering and mathematics (STEM) subjects.

Figure 4. Age at first selection and equity in reading performance

Source: OECD (2021[20]), "PISA: Programme for International Student Assessment Database", https://doi.org/10.1787/data-00365-en.

Statlink

• Recommendation 2.3.1. Reform the selection process into upper secondary to increase equity and facilitate quality learning in Grade 7 of lower secondary school. While in the long term, Bulgaria should reconsider the use of an examination at age 13, in the immediate term, there is a need for reliable, external input at the transition point between lower and upper secondary education. This is particularly true for those students applying to the most in-demand schools in the country. Bulgaria should therefore decouple the selection process from the Grade 7 NEA and introduce a new, optional selection examination better suited to generating useful information about a student’s suitability for a certain school type or educational programme. Bulgaria should also consider ways to enhance the reliability of the assessment data used for selection, by reducing the influence of teacher-assigned marks from the process, at least until – through reinforced training and support for teachers – classroom assessment has become more reliable and valid.

• Recommendation 2.3.2. Enhance the validity of the State Matriculation examination to ensure it more fully fulfils its dual purpose of certifying achievement against national learning standards and signalling suitability for transition to higher education. While Bulgaria’s State Matriculation examination demonstrates a high degree of reliability (it consistently measures what it sets out to measure), there is scope to improve its validity (its alignment with stated objectives). Bulgaria should continue to align the State Matriculation with the national competency-based curriculum and other curricular priorities. Since the distribution of marks on the State Matriculation varies considerably between subjects (i.e. an “excellent” in one subject may indicate a level of proficiency not matched by an “excellent” in another subject), there is speculation about the perceived difficulty of certain subject tests. This may be influencing students’ choices about which subjects to take more than their own ambitions or aptitudes. Bulgaria should therefore increase the examination’s power of discrimination by removing the predetermined pass/fail score and investigating current imbalances in the distribution of scores across different subject tests. Such efforts would help to ensure that the State Matriculation is a useful indicator of student proficiency and help accurately signal a student’s level of competency to future education providers or employers.

Teacher appraisal supports teaching and learning by providing teachers with feedback on their performance and competencies. Well-designed appraisals support teachers’ professional development and hold them to account for their practice, in turn helping to raise student achievement.

Bulgaria has introduced reforms to attract new teachers and develop teachers’ competencies in line with a broader shift towards more student-centred instruction. These reforms include a new differentiated teacher career structure, a significant increase in teachers’ salaries and mandatory continuous professional learning requirements. In addition, a new teacher appraisal process aims to inform promotion decisions. Bulgaria has also updated the core content for initial teacher education, eliminated tuition fees for many initial teacher education programmes and updated the teacher professional profile. The overall number of reforms introduced in recent years is impressive and shows a clear commitment to investing in the teaching profession. However, many of these reforms have significant financial implications for the Bulgarian government and there is a need to ensure coherence across initiatives. For instance, linking appraisal to the new professional profile and differentiating the profile to align with stages of the teacher career structure could better support teachers’ professional development. Importantly, if Bulgaria does not link recent investments in teachers to structural policies that help recruit the best and most motivated candidates, as well as encourage practising teachers to develop their competencies, it is unlikely these reforms will contribute to overall improvements in teaching and learning.

Improving teacher appraisal: Three areas for policy action

 

Policy issue 3.1. Ensuring that appraisals support teachers’ ongoing development

It is positive that Bulgaria already has a professional profile for teachers that provides shared language around expectations for what teachers should know and be able to do, as well as a range of appraisal processes that serve a variety of purposes, such as certifying new teachers, rewarding them with financial bonuses and informing career progression. These policies can help inform teachers’ self-evaluations and continuous professional development. However, the professional profile does not relate to the teacher career path, nor does it serve as the main criteria to appraise teachers’ performance for career progression. Bulgaria should differentiate the professional profile and make appraisals more consistent and reliable. This will help reward teachers for developing their competencies, therefore leveraging public funds to improve teaching quality.

• Recommendation 3.1.1. Revise the professional profile for teachers to support appraisal and motivate development throughout a teacher’s career. Bulgaria’s professional profile does not define the specific competencies teachers are expected to develop for each stage of their careers and the career path itself does not meaningfully distinguish between the functions of senior and chief teachers or offer substantial salary increases over time. Moreover, the competencies for trainee teachers do not relate to the professional profile. This context makes it difficult to encourage teachers to develop their competences in areas of national importance for the education system, such as using ICTs, managing classroom behaviour and supporting diverse cohorts of students, especially those with special education needs. Bulgaria should therefore revise the professional profile to align with the entire teacher career structure, from new entrant to chief teacher, which would help create a more unified and consistent system of teacher development. Importantly, this new profile should serve as the basis for decisions about performance-based career progression and associated salary increases. Such changes stand to help motivate teachers in Bulgaria to develop their teaching practice throughout their career.

• Recommendation 3.1.2. Modify the attestation appraisal to objectively and consistently assess real teaching practice and support teacher development. Bulgaria’s new attestation appraisal is not based on a common set of standards and will be carried out by appraisers who have a working relationship with the teacher, rather than by external evaluators. This might raise doubts about the fairness and reliability of promotion decisions. To improve the integrity of appraisal for promotion, Bulgaria should introduce more objectivity to the process by requiring that actors external to the school lead the commission for teacher attestation appraisals or validate the decisions made by local actors. The attestation appraisal should also require teachers to demonstrate how they are supporting the learning of all students. Adding classroom observations and removing appraisal elements that promote a narrow focus on the top-performing students (e.g. winning Olympiads) as well as developing training and guidance on how to conduct the attestation appraisal can help in this regard.

• Recommendation 3.1.3. Provide feedback on teachers’ performance and support their ongoing development between attestation appraisals. While school principals in Bulgaria periodically monitor teachers’ work, there is no regular appraisal process to support teachers’ professional development. The feedback teachers receive from these types of appraisals can help encourage their self-efficacy, for example in using more student-centred approaches, and help them better understand and direct their own learning. To strengthen the use of formative appraisals, Bulgaria should introduce an annual school-based appraisal process that is led by school-based actors who are familiar with the teacher and can encourage open and honest sharing of needs and feedback. This type of low-stakes appraisal can be an effective way to strengthen teaching and learning in schools.

• Recommendation 3.1.4. Use a more objective process to reward teachers for their performance. Bulgaria has an annual assessment of teachers that result in “additional labour remuneration”. This performance-based reward is a longstanding supplement to teacher salaries. However, since Bulgaria has significantly increased teacher pay and may need to continue doing so over time, now is an opportune moment to evaluate how the funds allocated to additional labour remuneration could be used more effectively and efficiently. In the short term, for example, Bulgaria might redirect funds to incentivise teachers to work in hard-to-staff schools or high-demand subject areas. In the medium to longer term, Bulgaria’s professional career structure for teachers should reward performance through promotion to higher career levels. Importantly, these higher career levels will need to be associated with substantial raises that extend well into a teacher’s career so that they do not reach a maximum salary within too short a period. There may be resistance to these changes and the government should work with the teachers’ unions to carefully plan for the transition and make better use of available public resources.

Policy issue 3.2. Meeting the demand for new teachers and supporting their development

Bulgaria is recruiting a large number of new teachers to replace the ageing teaching population and has introduced reforms to improve the initial preparation of teachers. However, teacher trainees are still not sufficiently prepared in the student-centred approaches needed for Bulgaria’s competence-based school curriculum. As a priority, Bulgaria should introduce additional measures to improve the quality and relevance of initial teacher education programmes and establish a minimum threshold for admission. Without proactively managing the supply and demand of teachers, as well as providing incentives for teachers to work in harder-to-staff regions of the country and teach subject areas facing shortages, Bulgaria will likely continue to struggle to address educational inequities and improve teacher quality.

• Recommendation 3.2.1. Make sure that initial teacher education programmes help teachers develop the competencies they will need at the start of their careers. While it is positive that Bulgaria recently amended legislation to establish minimum standards and content for initial teacher education courses, many providers do not yet meet these requirements. Bulgaria will need to take action to ensure that initial teacher preparation actually changes to better prepare teachers for their work in the classroom. To help implement the new requirements, Bulgaria should establish specific accreditation criteria that require initial teacher education providers to demonstrate how their programme addresses and evaluates the competencies expected of new teachers. Bulgaria could also establish a working group to support providers in redesigning their programme curricula, practicum requirements and practical-applied examinations.

• Recommendation 3.2.2. Make sure that the best candidates become teachers and fill shortage areas. Bulgaria needs to ensure the quality of new graduates from initial teacher education programmes and address remaining teacher shortages in specific locations and subject areas. These actions are crucial to make the most of Bulgaria’s recent investments in the teaching profession. Ways to do this include establishing entry requirements for initial teacher education that identify candidates who are well-suited to teaching, as well as incentivising experienced teachers to work in harder-to-staff areas. The Ministry should base any decisions about entry requirements and incentives on systematic forward planning to manage the labour market, which would help improve the efficiency of public resources.

• Recommendation 3.2.3. Formalise the appraisal of new teachers and provide them with effective induction support. Bulgaria currently lacks a consistent process to appraise new teachers’ performance against common standards. Moreover, despite the fact that schools are required to provide new teachers with mentors, these arrangements are generally insufficient, partly because mentors themselves do not receive training and support for their role. Bulgaria should therefore consider introducing an appraisal process based on “new teacher” competencies outlined in a revised professional profile and regulating specific induction support for all novice teachers. Guaranteeing a more supportive induction period will help to retain new teachers, which has been a challenge for Bulgaria.

Policy issue 3.3. Ensuring that continuous professional development addresses the learning needs of teachers and students

Bulgaria has reformed its teacher professional development system over the past decade, which has encouraged more teachers to participate in professional learning. However, teachers have little information on the quality and relevance of different providers, which is critical to navigating the professional development system and ensuring that training actually helps to improve their teaching practice. Addressing these concerns and aligning the system more closely to national education goals (e.g. inclusive education and using formative assessment to improve learning outcomes) can help Bulgaria further leverage the significant public investment it is making in the teaching workforce.

• Recommendation 3.3.1. Enhance the relevance and quality of professional learning. As of 2021, 247 training organisations offering 4 431 programmes were listed on the Ministry’s online information register (Ministry of Education and Science, 2021[21]). Having such a large continuous professional development market without rigorous quality assurance and monitoring procedures makes it difficult to ensure that programmes align with the professional profile for teachers and meet other requirements. While Bulgaria has already taken steps to collect feedback on training programmes, there is still a need for more formal quality assurance mechanisms. For example, the Ministry should devote sufficient staff to review the accreditation of providers, establish a process to investigate complaints lodged by participants and make better use of data to annually identify areas of teaching and learning that require the most improvement. These actions can help guarantee that professional learning programmes cover essential competencies and support Bulgaria’s broader education goals.

• Recommendation 3.3.2. Support teachers’ collaborative learning in schools and online. While enhancing the relevance and quality of formal training providers should be the Ministry’s immediate priority, developing new measures to further support teachers’ school-based and virtual learning is a cost-effective way to complement these efforts. Positively, Bulgaria already supports teachers’ learning in the school and established on line platforms and networks to support peer learning among teachers. The Ministry’s efforts to support peer learning on line should be commended, especially since so many teachers have been required to teach remotely because of the COVID-19 pandemic. However, the Ministry could do more to raise awareness about its online learning platforms, and provide guidance and training on the range of collaborative in-school and on line professional learning activities, such as peer classroom observations and providing feedback to other teachers to improve student outcomes.

School evaluation, if well-designed, supports teaching and learning by helping schools to improve their practice and holding them accountable for the quality of the education that they provide to students.

Bulgaria has a longstanding culture of elite schools that often serve the country’s highest achieving students, recruit the most qualified staff and have access to additional resources. This has contributed to one of the highest shares of school social and academic segregation among PISA participants (see Figure 5) (OECD, 2019[22]). Bulgaria has made significant progress to address these issues by developing a modern school evaluation framework that includes many features commonly found in OECD countries. For example, there is a new national school inspectorate, a differentiated inspection cycle that targets low-performing schools as well as new school quality standards. These efforts have the potential to reduce inequities in the education system because they focus attention and resources on schools that need the most support. However, the concept of school quality is not fully understood in Bulgaria. There is also a lack of clarity around the roles and responsibilities of the newly formed National Inspectorate of Education (hereafter the Inspectorate) and the REDs, which are now responsible for providing methodological support to schools. While clarifying how the new external school evaluation system should work in practice will be important to improving teaching practices and learning outcomes, such efforts will likely take time to implement. It is therefore imperative that Bulgaria simultaneously proceed with plans to develop instruments for school self-evaluation so that schools can immediately start driving their own improvement.

Figure 5. Isolation of disadvantaged students from high-achieving students in reading

Note: All analyses are restricted to schools with the modal International Standard Classification of Education (ISCED) level for 15-year-old students (see Annex A3 of PISA 2018). The isolation index of disadvantaged students from high-achieving students measures whether socio-economically disadvantaged students are concentrated in schools distinct from those that enrol high-achieving students. The index is related to the likelihood that a representative disadvantaged student attends a school that enrols high-achieving students. It ranges from 0 to 1, with 0 corresponding to no segregation and 1 to full segregation (see Annex A3 for a more complete description). A socio-economically disadvantaged student is a student in the bottom quarter of the PISA index of economic, social and cultural status (ESCS) in his or her own country/economy.

Source: OECD (2020[23]), PISA 2018 Results (Volume V): Effective Policies, Successful Schools, https://doi.org/10.1787/ca768d40-en.

Statlink

Improving school evaluation: Three areas for policy action

 

Policy issue 4.1. Building a common understanding of school quality

Bulgaria’s move to establish a school evaluation framework linked to national standards is a positive development that can help direct schools and the education system as a whole towards a common set of goals (OECD, 2013[24]). However, much of the public and education actors maintain a narrow view of school quality focused on students’ performance on State Matriculation examinations rather than effective practices to improve the outcomes of all students. Changing this perspective is key if Bulgarian schools are to reduce the share of students who do not achieve baseline levels of proficiency in core domains. Bulgaria should promote a more modern and comprehensive understanding of school quality to ensure that key stakeholders understand and engage with the work of the new Inspectorate and embrace broader national education reforms.

• Recommendation 4.4.1. Clearly communicate what school quality means. At present, the Inspectorate does not publish school inspection reports because it is not required to do so and wants to avoid the negative consequences associated with rankings and decontextualised comparisons between schools. However, in the absence of more transparent and contextualised information about a school’s performance, many stakeholders in Bulgaria still compare schools, relying mainly on State Matriculation examination results. This prevents school actors and the public from developing a better, more comprehensive understanding of school quality. Bulgaria should revise the school evaluation framework to confirm that quality means supporting the progress of all students and reduce the emphasis on the achievements of top-performers in academic competitions. With careful management, the Inspectorate should also publish inspection reports to provide a more holistic view of performance in relation to school quality standards. These efforts can help communicate to a better understanding of school quality that aligns with national education goals.

• Recommendation 4.4.2. Help schools develop a better understanding of school quality and lead their own development. Once school self-evaluation becomes a requirement (see Policy Issue 4.3), the Ministry will need to ensure that schools understand how this process relates to school improvement. Developing an online platform to make self-evaluation and school improvement resources more easily accessible would be a good way to help schools understand the relationship between these processes and their role in determining and enacting improvements in teaching and learning. In addition, Bulgaria could require all schools to develop action plans in response to external school evaluations that sets out specific actions to improve teaching and learning practices. For example, the Inspectorate could provide a template for school development plans and, over time, publish examples of good plans as a resource tool.

Policy issue 4.2. Ensuring that external school evaluations support school improvement, especially in at-risk schools

Prior to the creation of Bulgaria’s new Inspectorate, the Ministry’s REDs were responsible for monitoring, controlling and supporting schools. These responsibilities are now divided between the new national Inspectorate, which conducts external school evaluations, and REDs, which are expected to provide hands-on support to schools following inspections. This change has the potential to strengthen and level-up education quality, especially in a country like Bulgaria where there are major regional disparities in the provision of education and significant gaps in student outcomes according to socio-economic status and ethnic background. However, REDs currently lack clear direction on the specific support activities that fall under their new remit and face significant capacity issues. They also do not appear to use the school quality standards to inform the type of support they provide to schools. Addressing these factors can help facilitate the successful implementation of Bulgaria’s new external school evaluation system and improve the quality of learning opportunities for students across the country.

• Recommendation 4.2.1. Clarify and formalise REDs’ new mandate for monitoring and supporting schools. To ensure that their new role is clear, REDs need clarification to help distinguish the tasks required by their new school support role in relation to those of the new Inspectorate. Without such provisions, it is unclear what exactly REDs should be doing. For example, representatives of the Inspectorate told the OECD review team that they do not know how or whether REDs are making use of school inspection results. Ensuring complementarity between REDs and the Inspectorate will require adjusting regulations and creating formal opportunities for the two bodies to work together (e.g. through partnership agreements). These efforts can give legal weight to the government’s new arrangements for supporting school quality.

• Recommendation 4.2.2. Build REDs’ capacity to support school quality. At present, most expert staff in the REDs are qualified to provide schools with methodological support in specific subject areas. While there is often one RED expert with a specific mandate to support principals, this does not include helping them lead improvement efforts following an external school evaluation. The support function of REDs is also hindered by staff shortages and workload challenges, and some RED experts have not recently worked in schools. Despite this, experts are not required to participate in training. Bulgaria will need to address these capacity issues to ensure that REDs can better provide schools with relevant and meaningful support.

• Recommendation 4.2.3. Ensure the Inspectorate can fulfil its mandate. Alongside building the capacity of REDs, Bulgaria’s new Inspectorate will also need support. At present, the Inspectorate is unable to conduct external evaluations of all schools, hindering the effective implementation of its school evaluation framework. Moreover, the minimum selection criteria for Inspectorate staff are less stringent than in many other EU countries, jeopardising their credibility. For instance, internal inspectors are not required to have a background in education and there are no processes to reduce political interference in the appointment of the Inspectorate’s director. Providing sufficient resources, raising the requirements for internal inspectors and bolstering the agency’s independence will be crucial to ensuring this technical body operates as intended and can fulfil its mandate of improving school quality in Bulgaria.

• Recommendation 4.2.4. Use external school evaluations and the Innovative Schools initiative to support equity and inclusion. A key priority for Bulgaria is to address significant gaps in the participation and learning outcomes of students in different districts and from different ethnic groups. The new external school evaluation framework can support this objective by providing follow-up support after an external evaluation, such as school improvement funding and networking opportunities. These measures should primarily target low-performing schools to help them improve. Bulgaria should also consider how the existing Innovative Schools initiative, which aims to foster creative teaching, learning and school management strategies, can be leveraged to support more equitable school education. This may not be the case since schools currently need to find their own sources of funding to support their innovative projects. The Ministry could conduct a review to identify if certain schools are under-represented in the programme because they lack funding. Such efforts will help ensure that attention and resources stay focused on students and schools at risk of falling behind.

Policy issue 4.3. Making regular school self-evaluation mandatory and building schools’ capacity for development

Bulgaria’s efforts to strengthen external school evaluation are important but will likely take time to yield the desired results. The Ministry should therefore proceed with plans to develop instruments for school self-evaluation, so that schools can start driving their own improvement immediately. While it is positive that the Ministry will soon introduce a new ordinance on school quality management that makes regular self-evaluation mandatory, schools will need additional support to make this exercise meaningful. For example, schools will require flexibility to adapt the self-evaluation process to fit their needs, they will need data to easily benchmark their outcomes against comparable schools and school leaders will need the capacity to develop and implement improvement plans. Such efforts can ensure that self-evaluation and external evaluation are complementary and mutually reinforcing processes so that all schools are encouraged to focus on areas that are most important to quality provision (OECD, 2013[25]).

• Recommendation 4.3.1. Ensure that new school self-evaluation requirements support school development. Bulgaria will need to make sure that the new ordinance on school quality management covers key aspects of self-evaluation, such as core quality indicators and a clearly defined purpose of school self-evaluation. This will help ensure that schools have a clear understanding of how to conduct meaningful self-evaluations. To give the new self-evaluation requirements legal weight and help schools make use of this process, the government should also include self-evaluation in the revised school quality standards, use external evaluations to assess whether schools are conducting self-evaluations and provide feedback on this process as well as the school’s follow-up efforts to improve teaching and learning.

• Recommendation 4.3.2. Build schools’ capacity to conduct self-evaluations and act on results. Schools may find it challenging to gather and analyse evidence, engage with the school community and devise recommendations for improvement. Therefore, Bulgaria should consider producing guidelines, online resources and training on self-evaluation. In particular, a self-evaluation manual that provides an overview of steps in the self-evaluation process and a small number of core quality indicators could help schools make meaningful judgements about their practices. Importantly, the Ministry should also provide schools with access to quality data to support their self-evaluation efforts; the new Education Management Information System (EMIS) is well positioned to do this (see Policy Issue 5.1).

• Recommendation 4.3.3. Strengthen principals’ instructional leadership. Bulgaria has taken positive steps recently to hold principals accountable for their performance and ensure their continuous professional development. However, training and mentoring for instructional leadership, which is critical for school improvement, is lacking. Bulgaria should require principals to participate in mandatory initial training on all major domains of their role. In addition, Bulgaria should also encourage and support the development of existing school leaders by making career advancement more rewarding and incentivising talented school leaders to work in struggling schools. These efforts can strengthen the capacity of principals to address system wide challenges in their school, such as improving learning outcomes and addressing inequalities.

System evaluation supports teaching and learning by generating information on how an education system is performing and uses this information to improve policy and hold policy makers to account for progress against established goals.

Bulgaria has some of the basic building blocks needed to monitor and evaluate education policy and guide system improvement. Recently, the country has improved this framework further by establishing a modern EMIS. However, there remain major issues with available evidence to review performance at different levels of the system. In particular, the country’s NEA cannot support trend analysis, meaning that Bulgaria does not have a national instrument to monitor learning outcomes over time. In addition, the Ministry provides limited public reporting on system performance and on how evidence has been used to inform policy. Investing in better quality education data, particularly on learning outcomes, and improving reporting on system performance will be essential to help different actors track progress towards achieving national education goals, inform policy and build public trust in reform efforts.

Improving system evaluation: Three areas for policy action

 

Policy Issue 5.1. Ensuring Bulgaria’s new EMIS becomes a source of quality data for a variety of users

Historically, Bulgaria has had issues with the availability and collection of education data but the Ministry is currently upgrading its EMIS, which will introduce important developments. For example, there will be unique identification numbers, as well as new data on school participation and education outcomes. To optimise the investment in this new information system, the Ministry should continue to review its practices and standards to compile and share education data, in partnership with critical users like the NSI. This will help ensure that new data is secure, accurate and can serve a variety of purposes, making the new EMIS a valuable tool for a variety of users.

• Recommendation 5.1.1. Prepare to establish the new EMIS as Bulgaria’s central source of education data. Bulgaria’s new EMIS represents an important opportunity to modernise and integrate the collection, management and use of education data. Nevertheless, planning gaps remain in terms of the protocols for defining and collecting data and verifying its quality. The Ministry should work with relevant agencies to align data definitions and protocols with national and international reporting standards. This effort will help ensure the new EMIS becomes the official go-to source of information for all education stakeholders. To ensure the quality of data, the Ministry should create new quality assurance procedures for data entry and gradually transition all school reporting to a digital format. This will make it easier for actors to collect and use data in the EMIS to improve the performance of the education system.

• Recommendation 5.1.2. Develop the functionalities of the new EMIS and improve the availability of quality data to support accountability and policy making. Considering the investment made in the new EMIS, it is important this tool is easily accessible and can support accountability and policy making. Bulgaria should therefore build in functionalities that allow the new system to support evaluation and monitor progress towards national goals. For example, the Ministry could map existing education indicators against available sources of information to help identify data gaps. This type of analysis could help develop important new indicators and guide the future development of the EMIS. Creating a public data interface would also support broader system evaluation, by allowing different users to easily view and download a variety of education data based on their needs.

Policy Issue 5.2. Establishing a national assessment system that supports system monitoring and helps improve learning outcomes

Bulgaria established its NEA in 2007 and it has gradually expanded coverage to collect system and student-level data for three grades of schooling. The NEA has stated objectives that are broadly positive and reflect the main purposes of standardised assessments found in OECD countries. However, features of the NEA’s design and implementation mean that, while it can rank students in a particular cohort by their achievement levels, it cannot meaningfully support learning or inform system evaluation through reliable trend data. Currently, the NEA serves primarily as a summative test with high stakes for students. This is a concern – in particular in Grade 7 where it is used as an examination to select students into elite secondary schools. Bulgaria should consider decoupling the assessment from its selection function in all grades. While the government is generally aware of these issues, reforming the NEA will require political will, as well as financial resources and technical capacity.

• Recommendation 5.2.1. Reinforce the monitoring and formative potential of the NEA. The conflation of purposes currently attributed to Bulgaria’s NEA makes it difficult for policy makers and the Center for Assessment of Pre-school and School Education to navigate which design options would best ensure the assessment system fulfils its stated goals. For example, the Grade 7 NEA has undergone several changes in the last decade with limited consultation, leading to confusion among stakeholders about the main role of the assessment (i.e. if it is for system monitoring or selecting students). Bulgaria should refocus the primary purposes of the NEA on monitoring system performance and providing formative information to support teaching and learning. Specifically, the Bulgarian government should remove the selective function of the NEA in all grades. This change in purpose would better support Bulgaria’s national goal of improving educational equity. Now is an opportune moment to consider such a major change, since it would also give Bulgaria a chance to align NEA instruments with the new competency-based curriculum. At present, the tests do not assess the types of complex, high-order abilities that Bulgaria wants students to master.

• Recommendation 5.2.2. Ensure the design of the NEA system aligns with its monitoring and formative purposes and supports national education goals. As Bulgaria moves to reform the NEA, the government will need to reflect on key features of the assessment’s design. This review puts forward several recommendations for Bulgaria to consider. For example, Bulgaria should move to a criterion-referenced scoring system and change test administration to avoid critical transition points in a student’s schooling (e.g. moving the census-based primary school NEA in Grade 4 to Grade 2). Bulgaria may also consider adjusting the coverage of subjects at different levels (i.e. focusing on numeracy and literacy in Grades 2 and 6 while expanding to include other subjects in Grade 10). All changes should aim to reinforce the assessment’s system monitoring function and maximise its formative potential as a tool for driving system improvement.

• Recommendation 5.2.3. Disseminate results from NEAs to inform education policy and support learning. While it is positive that the Ministry commissions ad hoc analysis of NEA data, there is no regular report that summarises results and provides relevant insights for policy making. Moreover, schools do not receive detailed information about how their students performed and stakeholders cannot make comparisons based on similar characteristics like socio-economic background. To ensure that NEA results can inform policy and support school improvement efforts, Bulgaria should consider new ways to disseminate results. For instance, Bulgaria could provide reports for teachers that show how students perform on particular test items, disaggregated by different comparison groups. This could help to identify common errors and emphasise areas in need of improvement.

Policy Issue 5.3. Strengthening regular performance monitoring to guide system improvement

Bulgaria has initiated many major education reforms over recent years, most of which are costly and require sustained implementation. The COVID-19 pandemic has created an unexpected challenge and risks throwing a number of these reforms off track. However, there are also new opportunities to fund education reforms through COVID recovery initiatives and the recently announced national education programme, which gives new impetus to key goals such as strengthening inclusive education. These fast-changing and complex circumstances create a growing need for Bulgaria to improve its planning processes and more clearly communicate its reform agenda to different education stakeholders and the public. This should help the Ministry build support behind reform efforts, avoid roadblocks, and crowdsource new solutions, in light, for instance, of the fact that the country’s education reforms are likely to affect certain school environments differently.

• Recommendation 5.3.1. Establish an independent body to produce regular analytical reports on system performance. Data and capacity constraints hinder the Bulgarian government’s ability to produce regular reports on education system performance. This means that different education actors – not least, the Ministry and its REDs – do not have timely and comprehensive analysis to flag issues, track progress and make evidence-informed policy decisions. To improve system evaluation and reporting, many OECD countries have established independent bodies to ensure regular, objective monitoring and commission research on major policies and issues. The autonomy of these bodies strengthens trust in their findings and the likelihood that their research will inform constructive debates. Bulgaria should consider establishing this type of body, which could be responsible for compiling an annual “state of education” report, showing how the education system is performing against key indicators. Rather than carrying out the research itself, the body could commission research tasks. For instance, the Inspectorate could be tasked with producing an annual report, which may help it capture a qualitative picture of performance by showing how reforms play out in different school environments.

• Recommendation 5.3.2. Ensure that education authorities can track how the system is performing against national goals. The Ministry must design education policies for implementation in very different regional contexts, which brings particular challenges and opportunities. To ensure that centrally planned policies meet their goals, Bulgaria should consider developing customised tools for regional and municipal authorities, to help guide improvement in school sub-systems. Specific tools could include a regional “state of education” profile, as well as additional information that would not be included in the profile, such as national assessment results disaggregated by sub-groups within the region (for instance, municipalities).

• Recommendation 5.3.3. Make better use of system evaluation results for policy making and planning. To optimise investments in education data and the national assessments system, Bulgaria should carry out regular, robust implementation planning – for instance, through establishing annual or biannual action plans linked to its mid-term strategy. This would enable policy makers to sequence and adjust policy interventions, keep implementation on schedule and facilitate co‑ordination. To build trust and facilitate a better understanding of education reforms, Bulgaria should improve public reporting on how evidence has helped guide policy. For instance, the government could dedicate a session of the Parliamentary Committee on Education and Science to discuss the findings of a state of education report.

References

[5] EC (2020), Education and Training Monitor 2020: Bulgaria, European Commission, https://op.europa.eu/en/publication-detail/-/publication/0b2b116f-2499-11eb-9d7e-01aa75ed71a1/language-en/format-PDF/source-171315948 (accessed on 24 February 2021).

[8] EC (2018), “Tracking school dropouts in Bulgaria”, ESPN Flash Report, Vol. 2018/30, https://ec.europa.eu/social/BlobServlet?docId=19725&langId=en (accessed on 31 August 2021).

[16] Eurostat (2021), Classroom Teachers and Academic Staff by Education Level, Programme Orientation, Sex and Age Groups [educ_uoe_perp01], https://ec.europa.eu/eurostat/data/database (accessed on 20 July 2021).

[4] Eurostat (n.d.), Eurostat Database, https://ec.europa.eu/eurostat/data/database (accessed on 23 February 2021).

[3] Fileva, L. (2021), “Education policies: New funding for schools and fewer textbooks per subject [ Политиките за образованието: ново финансиране за училищата и по-малко учебници по предмет]”, Dnevnik [Дневник], https://www.dnevnik.bg/bulgaria/2021/12/15/4293494_politikite_za_obrazovanieto_novo_finansirane_za/.

[13] IEA (2020), TIMSS 2019 International Results in Mathematics and Science, TIMSS & PIRLS International Study Center, Lynch School of Education and Human Development, Boston College and International Association for the Evaluation of Educational Achievement.

[12] IEA (2017), PIRLS 2016: International Results in Reading, TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College and International Association for the Evaluation of Educational Achievement, http://pirls2016.org/wp-content/uploads/structure/CompletePDF/P16-PIRLS-International-Results-in-Reading.pdf (accessed on 24 February 2021).

[27] Maghnouj, S. et al. (2020), OECD Reviews of Evaluation and Assessment in Education: Serbia, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/225350d9-en.

[21] Ministry of Education and Science (2021), “Replies to additional questions after mission two”, Ministry of Education and Science of Bulgaria, Sofia.

[19] Ministry of Finance (2020), Vision, Goals and Priorities of the National Development Programme Bulgaria 2030, https://www.minfin.bg/en/1394.

[9] NSI (2020), Students and Drop-outs: Reasons and Level of Education, National Statistical Institute, Bulgarian, https://nsi.bg/en/content/3435/students-and-drop-outs-reasons-and-level-education.

[26] NSI (n.d.), Students and Drop-outs by Reasons and Level of Education, National Statistical Institute, https://www.nsi.bg/en/content/3435/students-and-drop-outs-reasons-and-level-education (accessed on 4 November 2021).

[1] OECD (2021), Decentralisation and Regionalisation in Bulgaria: Towards Balanced Regional Development, OECD Multi-level Governance Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/b5ab8109-en.

[2] OECD (2021), OECD Economic Surveys: Bulgaria 2021: Economic Assessment, OECD Publishing, Paris, https://dx.doi.org/10.1787/1fe2940d-en.

[20] OECD (2021), “PISA: Programme for International Student Assessment”, OECD Education Statistics (database), https://doi.org/10.1787/data-00365-en (accessed on 2 December 2021).

[6] OECD (2020), Early Learning Matters, OECD, Paris, https://www.oecd.org/education/school/Early-Learning-Matters-Project-Brochure.pdf.

[15] OECD (2020), “Learning remotely when schools close: How well are students and schools prepared? Insights from PISA”, OECD Policy Responses to Coronavirus (COVID-19), OECD Publishing, Paris, https://dx.doi.org/10.1787/3bfda1f7-en.

[23] OECD (2020), PISA 2018 Results (Volume V): Effective Policies, Successful Schools, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/ca768d40-en.

[10] OECD (2019), PISA 2018 Results (Volume I): What Students Know and Can Do, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/5f07c754-en.

[22] OECD (2019), “Isolation of disadvantaged students from high-achieving students in reading”, in PISA 2018 Results (Volume II): Where All Students Can Succeed, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/b5fd1b8f-en.

[14] OECD (2019), PISA 2018 Results (Volume III): What School Life Means for Students’ Lives, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/acd78851-en.

[11] OECD (2019), TALIS 2018 Results (Volume I): Teachers and School Leaders as Lifelong Learners, TALIS, OECD Publishing, Paris, https://dx.doi.org/10.1787/1d0bc92a-en.

[24] OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/9789264190658-en.

[25] OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/9789264190658-en.

[17] Republic of Bulgaria (2019), Bulgaria 2030: Part 1 Analysis of the Socio-economic Development of the Country After its Accession to the EU, https://www.minfin.bg/upload/41549/Bulgaria%202030%20analiz.pdf.

[7] UIS (2018), UIS.Stat, UNESCO Institute for Statistics, http://data.uis.unesco.org/ (accessed on 4 February 2021).

[18] UNESCO (2020), “Rapid assessment of COVID-19 impact on education in Bulgaria”, https://www.unicef.org/eca/rapid-assessment-covid-19-impact-education-bulgaria.