This chapter looks at how Serbia can align school evaluation with its core purposes of accountability and improvement. Serbia has one of the most advanced external school evaluation systems in the Western Balkans and school self-evaluation is required on an annual basis. However, major gaps in these processes prevent the country from making the most of school evaluation to improve teaching and learning. In particular, schools receive a limited amount of technical follow-up and evaluation reports are commonly perceived as summative rather than formative. These gaps are exacerbated because school leaders often lack the capacity to use evaluation exercises to develop and implement improvement efforts on their own. Serbia needs to strengthen external and self-evaluation processes and embed these in a larger framework for school improvement.
OECD Reviews of Evaluation and Assessment in Education: Serbia
Chapter 4. Developing schools’ capacity for improvement
Abstract
Introduction
School evaluation serves the dual purpose of helping schools improve their practices and keeping them accountable for the quality of their work. Serbia has made a strong push in the past two decades to develop both an external school evaluation system and school self-evaluation. The former, modelled on long-standing inspectorates in Europe, has some of the elements of a good evaluation system including development-oriented indicators, use of diverse sources of information and data, and a focus on helping schools improve their practices. School self-evaluation is also a requirement and virtually all schools in Serbia reflect yearly on their practices to inform planning.
However, some major gaps remain and prevent Serbia from making the most of its school evaluation system to help schools improve their teaching and learning practices. While the external system compares positively on paper to evaluation systems in OECD countries, it has not been appropriated by schools. This is partly due to a limited technical follow-up, helping schools make the most of school evaluation and the summative nature of school evaluation reports and feedback. Additionally, schools lack the capacity to make the most of external and self-evaluations to inform improvement. This is due to a lack of focus on training and hiring competent instructional leaders as school principals but also to the lack of external financial and technical support provided to schools.
For Serbia to make the most of improvements in its school evaluation system, it needs to make sure that it is fully embedded in a larger framework of school improvement focused on building in-school agency for change and adapting external support to the needs of schools. Such effort is necessary if Serbia is to meet the ambitious goals it sets for its education system.
Key features of an effective school evaluation system
In most OECD countries, school evaluations ensure compliance with rules and procedures, and focus increasingly on school quality and improvement (see Figure 4.1). Another recent trend has been the development of school self-evaluation, which has become a central mechanism for encouraging school-led improvement and objective setting. Internationally, strengthened systems for external and school-level monitoring and evaluation are seen as essential complements to the increasing decentralisation of education systems to ensure local and school accountability for education quality.
Frameworks for school evaluation ensure transparency, consistency and focus on key aspects of the school environment
Frameworks for school evaluation should align with the broader aims of an education system. They should ensure that schools create an environment where all students can thrive and achieve national learning standards. As well as ensuring compliance with rules and procedures, effective frameworks focus on the aspects of the school environment that are most important for students’ learning and development. These include the quality of teaching and learning, teacher development support, and the quality of instructional leadership (OECD, 2013[1]). Most frameworks also use a measure of students’ educational outcomes and progress according to national learning standards, such as assessments results or teachers’ reports.
A number of OECD countries have developed a national vision of a good school (OECD, 2013[1]). The vision guides evaluation, helping to focus on the ultimate purpose of ensuring that every school is good. Visions are often framed around learners, setting out how a good school supports their intellectual, emotional and social development.
Figure 4.1. School evaluation
Countries’ external evaluations balance accountability and improvement
The vast majority of OECD countries have external school evaluation (see Figure 4.2). Schools tend to be evaluated on a cyclical basis, most commonly every three to five years (OECD, 2015[2]). Within the broad purpose of evaluating school performance, some countries emphasise accountability for teaching quality and learning outcomes. In these countries, national assessment data, school ratings and the publication of evaluation reports play an important role. In contrast, in countries that place greater emphasis on improvement, evaluations tend to focus more on support and feedback to schools. They also place a strong emphasis on helping schools develop their own internal evaluation and improvement processes.
Evaluations aim to establish a school-wide perspective on teaching and learning
Administrative information for compliance reporting is a standard source of information for evaluations, although it is now collected digitally in most countries (OECD, 2015[2]). This frees up time during school visits to collect evidence of school quality. Most evaluations are based on a school visit over multiple days. Visits frequently include classroom observations. Unlike for teacher appraisal, these observations do not evaluate individual teachers but rather aim to cover a sample of classes across different subjects and grades to establish a view of teaching and learning across the school. Inspectors also undertake interviews with school staff, students and sometimes collect the views of parents. Since much of this information is qualitative and subjective, making it difficult to evaluate reliably, countries develop significant guidance such as rubrics for classroom observations to ensure fairness and consistency.
Many countries have created school inspectorates in central government
External evaluations are led by national education authorities, frequently from central government (OECD, 2013[1]). Across Europe, most countries have created an inspectorate that is affiliated to but frequently independent of government. This arrangement ensures integrity and enables the inspectorate to develop the significant professional expertise necessary for effective evaluation. School inspectors may be permanent staff or accredited experts contracted to undertake evaluations. The latter provides flexibility for countries, enabling them to meet the schedule of school evaluations and draw on a range of experience, without the costs of maintaining a large permanent staff. Inspectors across OECD countries are generally expected to have significant experience in the teaching profession.
The consequences of evaluations vary according to their purpose
To serve improvement purposes, evaluations must provide schools with clear, specific feedback in the school evaluation report, which helps them understand what is good in the school and what they can do to improve. To follow up and ensure that recommendations are implemented, countries often require schools to use evaluation results in their development plans. In some countries, local authorities also support evaluation follow-up and school improvement. Around half of OECD countries use evaluation results to target low-performing schools for more frequent evaluations (OECD, 2015[2]).
In most countries, evaluations also result in a rating that highlights excellent, satisfactory or underperforming schools. To support accountability, most OECD countries publish evaluation reports (OECD, 2015[2]). Public evaluation reports can generate healthy competition between schools and are an important source of information for students and parents in systems with school choice. However, publishing reports also risks distorting school-level practices such as encouraging an excessive focus on assessment results or preparation for evaluations. Evaluation frameworks must therefore emphasise the quality of school-level processes and an inclusive vision of learning, where all students, regardless of ability or background, are supported to do their best. Evaluation systems that emphasise decontextualised outcome data such as assessment results are likely to unfairly penalise schools where students come from less advantaged backgrounds since socio-economic background is the most influential factor associated with educational outcomes (OECD, 2016[3]).
Figure 4.2. School evaluation in OECD countries
Source: OECD (2015[2]), Education at a Glance 2015: OECD Indicators, https://dx.doi.org/10.1787/eag-2015-en.
Self-evaluation is an internal tool for improvement
Most OECD countries require schools to undertake self-evaluations annually or every two years (see Figure 4.2). Self-evaluations encourage reflection, goal-setting and inform school development plans (OECD, 2013[1]). To be an effective source of school-led improvement, many countries encourage schools to appropriate self-evaluation as an internal tool for improvement rather than an externally imposed requirement. In some countries, schools develop their own frameworks for self-evaluation. In others, they use a common framework with external evaluation but have the discretion to add or adapt indicators to reflect their context and priorities.
The relationship between external and internal evaluations varies across countries. In general, as systems mature, greater emphasis is placed on self-evaluation while external evaluation is scaled back. Most OECD countries now use the results from self-evaluations to feed external evaluations with, for example, inspectors reviewing self-evaluation results as part of external evaluations. However, the relationship is also shaped by the degree of school autonomy – in centralised systems, external evaluations continue to have a more dominant role, while the reverse is true for systems that emphasise greater school autonomy.
Effective self-evaluation requires strong school-level capacity
Effective self-evaluation requires strong leadership and strong processes for monitoring, evaluating and setting objectives (SICI, 2003[4]). Many OECD countries highlight that developing this capacity in schools is a challenge. This makes specific training for principals and teachers in self-evaluation – using evaluation results, classroom and peer observations, analysis of data and developing improvement plans – important (OECD, 2013[1]). Other supports include guidelines on undertaking self-evaluations and suggested indicators for self-evaluations.
While a principal’s leadership plays a critical role in self-evaluation, creating teams to share self-evaluation roles is also important. The most effective self-evaluation teams involve a range of staff members who are respected by their colleagues and have a clear vision of how self-evaluation can support school improvement. To support collective learning, the self-evaluation team should engage the whole school community in developing a plan for school improvement. This process should include students, who have a unique perspective on how their schools and classrooms can be improved (Rudduck, 2007[5]). The views of students and their parents also help to understand how the school environment impacts student well-being and their overall development. This is important for evaluating achievement of a national vision focused on learners.
Data systems provide important input for evaluation
Administrative school data – like the number of students, their background and teacher information – provides important contextual information for internal and external evaluators. Increasingly, countries use information systems that collect information from schools for multiple purposes including evaluation and policymaking.
Most countries also collect information about school outcomes. Standardised assessments and national examinations provide comparative information about learning to national standards. Some countries also use this information to identify schools at risk of low performance and target external evaluations (European Commission/EACEA/Eurydice, 2015[6]). However, since assessment results do not provide a full picture of a school, they are often complemented by other information such as student retention and progression rates, student background, school financial information and previous evaluation results. A number of countries use this data to develop composite indicators of school performance that frequently inform evaluation and support school accountability.
Principals must be able to lead school improvement
Strong school leadership is essential for effective school self-evaluation and school improvement more generally. Principals support evaluation and improvement through a number of leadership roles: defining the school’s goals, observing instruction, supporting teachers’ professional development and collaborating with teachers to improve instruction (Schleicher, 2015[7]). This diversity points to a major shift in their role in recent years, with principals increasingly leading instructional improvement.
Principals need a deep understanding of teaching and learning, and strong leadership skills to become instructional leaders
Most principals bring significant experience in the teaching profession – among the countries participating in the OECD Teaching and Learning International Survey (TALIS), the average principal has 21 years of teaching experience. Teaching experience alone, however, is not sufficient and the ability to demonstrate strong leadership of the school community is particularly important. Nearly 80% of principals in TALIS participating countries reported that they received training in instructional leadership either before or after taking up their position, or both (OECD, 2014[8]).
Principals’ initial training must be complemented by opportunities for continued professional development once in post. One of the most effective types is collaborative professional learning activities, where principals work together to examine practices and acquire new knowledge (DuFour, 2004[9]). In countries where international assessment results suggest that learning levels are high, such as Australia, the Netherlands and Singapore, more than 80% of principals reported participating in these kinds of activities in the last 12 months (OECD, 2014[8]).
Professionalising school leadership – standards, selection and appraisal
Given the important role that principals occupy, OECD countries are taking steps to professionalise the role. A number of countries have developed professional principal standards that set out what a school leader is expected to know and be able to do. Principal standards should include how principals are expected to contribute to self-evaluation and improvement. Similar to teachers, principal standards guide the recruitment of principals, their training and appraisal.
Around half of OECD countries have legislated appraisal of school leaders (see Figure 4.3) (OECD, 2015[2]). These kinds of appraisals hold principals accountable for their leadership of the school, but also provide them with valuable professional feedback and support in their demanding role. Responsibility for principal appraisal varies. In some countries, it is led by central authorities, such as the school inspectorate or the same body that undertakes external teacher appraisals. In others, it is the responsibility of a school-level body, such as the school board. While the latter provides the opportunity to ensure that appraisal closely reflects the school context, boards need significant support to appraise principals competently and fairly.
Figure 4.3. Existence of school leader appraisal in OECD countries, 2015
Note: Data for Lithuania are drawn from European Commission/EACEA/Eurydice (2015[6]).
Sources: OECD (2015[2]), Education at a Glance 2015: OECD Indicators, http://dx.doi.org/10.1787/eag-2015-en; European Commission/EACEA/Eurydice (2015[6]), Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe, https://publications.europa.eu/en/publication-detail/-/publication/4a2443a7-7bac-11e5-9fae-01aa75ed71a1/language-en.
School governance in Serbia
Schools in Serbia have some autonomy in how they allocate their budget and manage instruction compared to OECD countries. On paper, the school board plays an important role in overseeing the quality of the school and the school principal is responsible for both managerial and instructional leadership. However, while there has been progress in making appointments more merit-based, the capacity of school leaders remains limited. School principals and school boards receive very little training and technical guidance on how to steer school improvement or provide oversight. Schools also receive very little public funding to implement improvement plans. As a result, most schools rely on external impetus and support if they are to change the quality of their practices in a meaningful way.
School principals in Serbia receive very limited training in their core areas of responsibility
School principals in Serbia are required to have a master’s degree and some teaching experience (at least eight years) (European Commission/EACEA/Eurydice, 2018[10]; OECD, 2014[8]). However, in contrast to practices in OECD countries, there are no mandatory initial education requirements for school principals in Serbia. The majority (50.7%) of school principals in Serbia who participated in TALIS1 reported having never received any formal training in school administration or participated in a principal training course, compared to only 15% across OECD countries. Until 2018, there were no mandatory requirements for in-service professional development either and principals had to find and pay for their own training. The Ministry of Education, Science and Technological Development (hereafter the ministry) introduced in 2018 some training preparation for the certification exam as described below but this training remains of limited scope and length. A large majority (70%) of school principals participating in TALIS 2013 reported that the cost of training was the main barrier to their participation, compared to less than 30% across OECD countries on average. As a consequence, more than 60% of school principals in Serbia have never benefitted from in-service training in instructional leadership (e.g. school self-evaluation, goal-setting, monitoring of teaching and learning and planning) (OECD, 2014[8]).
In recent years, the ministry has taken some important steps to try to improve access to leadership training for school principals. Most significantly, in 2016, Serbia launched a master’s programme in school leadership as part of an EU-funded project. The programme is currently offered by two universities (University of Kragujevac and University of Novi Sad) and targets teachers wanting to move into a school principal role as well as principals already in service. While the programme was tuition-free during the time of the EU project in the school year 2016/17, students now have to pay, which has limited take-up – the number of students dropped from 55 in the school year of 2016/17 to 10 in 2018/19. Moreover, this master’s programme is not part of the eligibility requirements for becoming a school principal, which also contributes to the low take-up rate.
Serbia has taken action to reduce the politicisation of school principal appointment and dismissal
School principals in Serbia are appointed for four years. Prior to 2017, they were selected by the school board based on a recommendation from the teacher council. While school boards should theoretically base their decision on the competency standards for school principals introduced in 2013, it is unclear whether these standards are systematically used. School boards received no training or guidance on how to ensure the integrity of the process and are susceptible to political interference from local authorities. This resulted in many principals being appointed based on political affiliation and personal relationship with school staff or local government rather than on merit, creating concerns that some school principals were more focused on serving the interests of individuals rather than the broader interests of students and the school as a whole. In 2017, the ministry changed the selection process in an attempt to increase transparency and independence. According to the new process, the school board appoints a selection commission comprising teachers and administrative staff. The commission reviews all candidates’ applications and prepares a selection report with its opinion on each candidate together with their documentation. The commission is required to take into account the latest regular appraisal results for candidates who have already worked as school principals and reference the competency standards. The school board then creates a list of pre-selected candidates based on the commissions’ report and submits it to the minister, who makes the final selection decision.
The ministry introduced a certification programme for school principals
In 2018, as part of efforts to professionalise the school leadership role and reduce political influence, the ministry introduced for the first time a certification process for all in-service school principals and a mandatory training programme to prepare them to take the certification assessment. A national commission interviews each candidate for two hours. The interview includes an oral presentation by the candidate on the key findings from a research project on educational practice that they undertook and a discussion of the candidate’s leadership competencies based on the school principal standards. A portfolio of work is also submitted to the commission in advance. School principals are required to pass the certification assessment to continue serving as school principals in Serbian public schools.
School advisors are responsible for regularly appraising school principals but this is a rare occurrence
School advisors from the Regional School Authorities (RSAs) are responsible for regularly appraising school principals to help them improve their practices. However, this process of “regular supervision” is rarely carried out. School advisors have little time to dedicate to the regular supervision of school staff, as they are also responsible for carrying out the external school evaluation. The rulebook on regular supervision sets the broad areas the school advisor should look at during regular supervision. However, these guidelines do not differentiate between teacher and school principal supervision and thus do not take into account the specific competencies needed for each of the two functions.
The external school evaluation team evaluates some aspects of school principals’ leadership capacity as part of the overall external evaluation process. The external evaluation looks at the school principals’ capacity to encourage innovation in the school, appraise teachers and provide them with feedback, and organise professional development for school staff. According to the school quality standards, school principals are expected to plan their own professional development activities based on the result of the external evaluation. The external school evaluation report does not provide the school principal with recommendations for improvement.
The school boards lack the capacity to play a steering role in school governance
School boards in Serbia are responsible for monitoring the quality of school management. Each school board includes nine members appointed by their respective professional body or council for four years, comprising three school employee representatives, three parents and three members chosen by the local self-government. In vocational education and training (VET) schools, the three members delegated by local self-government must be a so-called social partner (e.g. companies, associations of employers, unions, etc.). The school board adopts the school programme, development plan and annual work plan. It is also responsible for validating the school budget proposal that will be submitted to and validated by the RSA. Finally, since the reform of school principal appointment, the board makes a pre-selection of candidates to lead the school and submits this proposal to the minister for a final decision. Despite these extensive responsibilities, the members of school boards receive no training on how to carry out their role. There are also no guidelines or manuals for school boards to follow.
The number of school support staff in Serbian schools is limited
School principals in Serbia can rely on some administrative and pedagogical leadership support from school support staff and deputy principals depending on the size and needs of the school. There are different profiles of experts and associate staff who support Serbian schools through a variety of tasks. These include, among other things, supporting child development and well-being, providing professional support to educators, teachers and school principals, and promoting inclusive policies within the school. In particular, experts and associates are often: school psychologists, who provide students with counselling and help school staff address behavioural issues; special educational needs (SEN) specialists – commonly referred to in Serbia as “defectologists” – who work in special schools and are responsible for diagnosing students with SEN, teaching and identifying adequate learning supports; and, school pedagogues, who are primarily responsible for supporting teaching and learning. However, these profiles are rarely present in a school at the same time. For example, the majority of basic education schools with eight to 23 classes can only hire one school support staff. Most schools have either a pedagogue or a school psychologist who helps the school principal in leading the school self-evaluation, setting the classroom observation plans for regular appraisal and defining the school development plan. In some large schools, principals might also have a deputy responsible for overseeing administrative tasks.
School evaluation in Serbia
Serbia has put in place a strong framework for school evaluation. The country has a comprehensive set of school quality standards that are development-oriented and draw upon the experience of long-established European inspection systems. All schools have now undergone a first cycle of external evaluations and a new cycle will start in the school year 2019/20, drawing on a framework that has been refined based on experience. School self-evaluation is also relatively well established, with schools required to evaluate their practices yearly and use the information in their planning. However, some significant obstacles stand in the way of this evaluation framework becoming a vehicle for school improvement. The governance arrangements for school evaluation are fragmented, which weakens accountability. The developmental impact of evaluation is hindered by several factors, in particular, weak national capacity to provide constructive feedback and support to schools and the limited understanding within schools on how to undertake a meaningful self-evaluation for improvement purposes.
Table 4.1. Types of school evaluation in Serbia
Source: MoESTD (2018[11]), OECD Review of Evaluation and Assessment: Country Background Report for Serbia, Ministry of Education, Science and Technological Development.
Serbia’s school quality standards are a strength of the school evaluation system
Serbia introduced school quality standards for the first time in 2012. The standards were inspired by the Dutch School Quality Standards and compare favourably to those used in OECD countries. They focus on seven domains of quality (programming, planning and reporting; teaching and learning; student learning outcomes; student support; ethos; school organisation and management; school resources) and had a strong focus on teaching and learning. These school quality standards were the key reference document for external school evaluation in Serbia and were also used by schools in their self-evaluation processes.
In 2018, the Institute for Education Quality and Evaluation (IEQE) revised and further improved the standards, taking into account lessons from the first cycle of external school evaluations (2012-17). In particular, the number of quality domains was reduced from seven to six with the aim to focus more centrally on the quality of learning and teaching in the classroom. For example, the list of standards under the “teaching and learning” quality area was condensed and descriptive indicators were added to place teaching at the centre of this goal. The language of the standards was also improved by specifying which actors’ behaviours or outcomes would be looked at for each indicator (e.g. teachers, school principal, students, etc.).
The external school evaluation has a limited impact on improving quality in schools
Responsibility for external school evaluation is split between the IEQE and the ministry of education
Contrary to practices in most OECD countries, where one institution usually has clear responsibility for school evaluation (OECD, 2013[1]), responsibility in Serbia is split between the IEQE’s Centre for Quality Assurance of Educational Institutions and the ministry through its RSAs. The Centre for Quality Assurance is responsible for setting school quality standards, designing the evaluation process, developing guidelines and training school evaluators. However, evaluators do not report to the IEQE. At present, most are taken from the body of advisors based in the RSAs accountable only to the ministry through the Department for Co‑ordination of Regional School Authorities. Advisors do not report to the IEQE and the latter has a limited mandate for monitoring the quality of the evaluation process.
The external evaluation process is well-designed and focuses on practices which are linked to student learning
Until 2019, the external school evaluation focused on ten core school quality standards. These core standards had corresponding indicators and were well aligned with factors associated with improving student learning in schools, such as classroom teaching practices and school instructional leadership (e.g. school planning and monitoring of learning progress at the school level). The evaluation team could also include five additional indicators from other standards to focus on during the school visit. These were selected based on the school’s profile and development plan. Schools were evaluated every 5 years by an evaluation team of advisors from the 17 Regional School Authorities.
The external evaluation process was revised in 2019. Schools are now evaluated against all quality standards, each of which is associated with several indicators (indicators in total). This change goes in the opposite direction of trends observed in OECD countries, which try to limit school evaluation to a core set of indicators. The evaluation cycle was also lengthened to every six years instead of five. However, external evaluations may occur more frequently at the request of the school administration or ministry. In addition, schools that receive the lowest level of quality (Level 1) are evaluated again after 3 years.
124 24 advisors who conduct the evaluations come mostly from the school’s own RSA but advisors from other RSAs or evaluators from the IEQE may join the evaluation team if capacity in the school’s RSA is limited. Before visiting the schools, the evaluating team collects and reviews school records such as the school self-evaluation report, the school programme, the school annual work plan, the school development plans and previous reports from advisors. During the school visit, which lasts at least two days, the evaluation team meets with the school principal and interviews the school support staff, teachers, students and parents. The team also observes teaching and learning practice in the classroom (at least 40% of teachers need to be observed for at least 20 minutes each).
Feedback to schools is descriptive and provides little guidance on what schools can do to improve
At the end of the visit, the evaluation team briefs the school principal orally about the results. They also produce a written report summarising the key findings on each standard, as well as providing an overall score. The report is shared with the RSA and the ministry of education. School principals are required to brief school staff and parents on the results and data from the report is publicly available in a registry on the ministry’s website.
Schools are given a numerical score from one to four for each indicator, whereby four signals that an indicator has been fully achieved. A judgement of how a school performs against a particular standard is then determined by averaging the indicator scores for that standard. This contributes to the school’s overall score (grade). To receive an overall grade of 4 (very good), schools need to have scored a Level 4 for more than 50% of the standards and the rest must have a score of Level 3. In 2017, under the previous scoring system, 60% of the evaluated basic education schools received a score of 3 (good school), about a fifth (22%) received a score of 4 (very good) while 2% received the lowest score of 1 (very weak) (IEQE, 2017[12]).
The written report is mostly descriptive and does not include recommendations for improvement. The evaluation team provides a summary of key findings for each of the evaluated standards but does not indicate how the school can address specific issues in each domain. Rather, schools are expected to use the evaluation results to develop their own goals and improvement plans but not all schools have the capacity or support to do this. Moreover, schools also tend to focus on the overall evaluation grade rather than the domain-specific description of issues. For example, some of the schools visited by the review team could cite their overall grade but were not able to list the key strengths and issues identified by the external evaluation. Reasons for this could be related to the school’s lack of capacity to digest and use evaluation results or that the report itself did not clearly present information about the school’s strengths and weaknesses.
The advisors evaluating schools are often the same persons in charge of follow-up advice and support
Following the external evaluation, schools are required to set up a school improvement plan to address the key issues raised. This plan is sent to the RSA advisors who performed the external school evaluation for review and validation. The same advisors are likewise given the responsibility of helping schools implement their action plans. This dual mandate puts at risk the transparency and independence of the evaluation process. Providing technical support and advice to the school requires building strong and close relationships with the school staff. This may interfere with the neutrality of the external evaluation.
The ministry is introducing new follow-up processes to improve the evaluation’s developmental and accountability goals
Currently, external school evaluation does not trigger additional financial and technical support for schools that performed badly (“weak” or “very weak” schools). To help address this gap, the United Nations Children's Fund (UNICEF) and the IEQE piloted in 2015-17 a peer-learning scheme called SHARE. The SHARE project pairs high performing schools (grade of four) with low-performing schools (grades of one and two) as a means to help improve practices in the lowest-performing schools (see Box 4.7). The ministry intends to scale up the project with the new round of school evaluation. In the new cycle, the ministry is also planning to make school principals more accountable for evaluation results. Notably, principals in schools that receive the lowest grade of one in two consecutive school evaluations can be dismissed from their position.
The number of advisors is insufficient to carry out all of the responsibilities of this function
The number of school advisors in Serbia is insufficient to undertake all the tasks for which they are responsible. There are 100 advisors across the 17 RSAs covering over 1 700 basic education and upper secondary schools. The freeze in hiring in the Serbian public sector is such that many RSAs do not have enough advisors to carry out external school evaluations and thus rely on advisors from other RSAs. For instance, the RSA of Novi Sad has 6 advisors covering 19 municipalities. This has implications for the quality and depth of school evaluations as well as for the advisors’ many other tasks. In addition to the external school evaluation, advisors are responsible for following up with schools on the implementation of the school development plan, teachers’ regular appraisal and appraisal for promotion as well as school principals’ and pedagogues’ appraisal. Due to limited human resources, advisors have had difficulties in carrying out their appraisal and school development follow-up roles since the introduction of the external school evaluation process in 2013, which takes up most of their time. In some cases, staff from the IEQE joins the evaluation teams when the number of advisors is not sufficient.
To address this shortage, the ministry recently contracted 200 teachers to take part in the new cycle of external school evaluations starting in 2018 to help assess teaching and learning. These teachers will receive training from the IEQE and will first join school visits as observers before contributing as evaluators under the supervision of advisors. This new policy aims to free up some of the advisors’ time to carry out their other responsibilities.
The advisors receive some training to carry out external school evaluations
As is the case in most European countries, advisors in Serbia are former teachers, school principals or pedagogues with a minimum of eight years of experience in schools. Advisors received two weeks of training organised by the IEQE in collaboration with the Standing International Conference of Inspectorates (SICI), association of national and regional inspectorates of education in Europe, and the Dutch Inspectorate in 2013 when the new school evaluation process was first introduced. The length of this training is well below that of inspectors in most European countries, which varies from several months to one year (European Commission/EACEA/Eurydice, 2015[6]). This training focused on familiarising advisors with the school evaluation process, including how to conduct the school visit and observe teaching and learning in the classroom, and was part of their licensing process as external evaluators. The IEQE organises co‑ordination meetings with all advisors twice a year to discuss ongoing evaluation and ensure harmonisation of practices. Advisors also receive training on new reforms such as the ongoing curriculum reform.
The IEQE produces a detailed analysis of school quality in Serbia based on evaluation results
The IEQE publishes an annual report summarising the key findings from external school evaluations. These reports provide valuable information about how schools in Serbia perform against school quality standards. They identify the key challenges that schools face in providing quality education and information is disaggregated by level of education, school administration and quality domains. Education actors met by the review team both in the ministry and in the regions (RSAs) and schools were aware of the main challenges identified by the reports. However, it is unclear to what extent recommendations from this annual report are used to inform policy development by the ministry (see Chapter 5).
School self-evaluation is mandated by law but it is unclear if it is meaningfully implemented in Serbian schools
Schools are required to carry a self-evaluation yearly
All schools in Serbia must conduct yearly self-evaluations. According to PISA 2012, 96% of students in Serbia are in schools whose principals reported that self-evaluations are being carried out compared to 87% on average across OECD countries (OECD, 2013[13]). Schools are mandated to carry out school self-evaluations at least once a year and over a five-year period to have evaluated all seven (now six) areas of quality in the school quality standards. Schools are expected to form a school self-evaluation team of at least five members, including teachers, school administration staff (e.g. the school principal, pedagogue or deputy school principal) and representatives of the parents and students’ councils. The team has been put in charge of reviewing school practices and performance in relation to the identified quality areas and drafting the school evaluation report. Schools are required to share the report with the teachers’ council, parents’ council and the school board, as well as with any other interested party.
Schools receive very little guidance and training on how to carry out self-evaluations
School principals and other staff receive no mandatory training in school evaluation as part of their initial training or while in service. In contrast, in most OECD countries, principals’ initial training includes modules on school self-evaluation and planning (OECD, 2013[1]). The IEQE’s Centre for Quality Assurance of Educational Institutions is responsible for developing and providing training on school self-evaluation upon request from schools. However, because of limited financial and human resources, such training is rarely carried out and must be paid for by the school.
Schools receive very limited external guidance on how to conduct self-evaluations and what indicators to use to assess and compare their performance. While the IEQE’s Centre for Quality Assurance of Educational Institutions has a mandate to develop guidelines and tools for self-evaluation, it has not done so for almost a decade. According to representatives from the IEQE, this is because the centre’s limited resources are dedicated almost entirely to supporting external school evaluation. Therefore, the school self-evaluation manual developed by the ministry in 2005 in collaboration with the British Council has never been updated to reflect the new school quality standards. It is also unclear to what extent this manual is used by schools to inform their evaluation practices. For example, none of the schools met by the review team referred to the manual or even seemed aware of its existence. Moreover, while the external school evaluation team checks whether the school carries school self-evaluation, it does not systematically monitor the quality of these among the school quality standard indicators.
The use of external school evaluation to inform in-school planning and external monitoring and accountability is limited
Results of school self-evaluation are not systematically used to inform in-school practices. While national regulations require schools to draft and update their three- to five-year school development plan based on the results of school self-evaluation, this does not appear to be happening in many schools. However, a school’s self-evaluation is one of the sources of evidence used to inform the external school evaluation results (MoESTD, 2019[14]).
Serbia has a separate process for checking schools’ compliance with legal requirements carried out by audit inspectors at the municipal level
In Serbia, municipal inspectors (hereafter referred to as audit inspectors) are responsible for checking schools’ compliance with laws and regulations related to school safety, inclusion and access for all children and labour laws. Inspectors audit schools at least once a year and check a list of documents requested from the school. These include documents from the school councils and professional bodies such as the teacher council. The audit team provides the school principal with the audit report, including the conclusions and measures that need to be implemented by the school within 15 days after the school visit. Inspectors can also carry out exceptional audits when problems are reported in a given school by teachers, parents, students or the school principal.
The law on inspections (audits) was changed in 2018 to simplify the process and make the most of available human resources. Audits will be carried out in some schools only based on an initial risk assessment, the criteria of which are yet to be determined. Municipalities will also be able to ask inspectors from other municipalities to join the audit team. There are very limited links between the audit carried out by the inspectors and the external school evaluation carried out by the RSA. Inspectors reported to the OECD review team that in some cases advisors are invited to join the audit teams but this is not consistently done.
Standardised data on school outcomes is very limited
Schools in Serbia have very limited tools to compare their practices and results to those of other schools in similar contexts. The main tool used by schools to understand how their learning outcomes compare to that of other schools is the end-of-primary exam report from the IEQE, which compares each school’s results to that of other schools with a similar profile. However, upper secondary schools do not have any standard measure of their students’ learning outcomes. Both basic education schools and upper secondary schools do not receive comparative information about enrolment, completion, drop-out and inclusion levels. Moreover, external evaluation reports include a school context section that describes the school’s socioeconomic context, student population and geographical location. However, these reports do not systematically use standardised data to describe school context nor do they give comparative benchmarks, such as schools in similar contexts or national averages.
Policy issues
Serbia has some of the foundational processes and practices of a strong school evaluation system. It has comprehensive school quality standards focused on the indicators that are most related to improvement in teaching and learning, its external school evaluation process is overall well-designed and schools carry self-evaluations annually using the school quality standards as a reference point. However, external and self-evaluations have, so far, a limited impact on school improvement. The external school evaluation process continues to be perceived mostly as a summative process rather than formative. This is due to both the lack of clear recommendations for improvement feedback from the inspection visit and the limited follow-up and technical support provided to schools. Similarly, while most schools carry school self-evaluations, these processes remain disconnected from the process of school planning and improvement. Serbia needs to improve the developmental aspect of both external and self-evaluation processes and provide schools with the necessary capacity and training to make the most of evaluation results. To do so, the improvement of the school evaluation process recommended by this review should be embedded in the larger context of a school improvement strategy.
Policy issue 4.1: Develop external evaluation into a meaningful process for school improvement
On paper, Serbia has one of the most advanced external school evaluation systems in the Western Balkans. The evaluation process developed with the help of SICI and the Dutch Inspectorate reflects many of the features of mature systems in OECD countries. It has a clear reference framework focused on school quality indicators that cover important practices related to effective teaching and learning. The evaluation process itself triangulates various sources of evidence, such as classroom observation, stakeholder interviews and reviews of school documents. However, despite these strong design features, the practice of external school evaluation in Serbia remains somewhat peripheral to policymaking and important reforms, such as the new curriculum or new national assessment of student learning outcomes. The process has also done little to trigger improvements within schools, in part because of the perception of the process as a purely summative exercise and the lack of quality feedback, but also because of limited school resources and capacity, combined with a lack of external technical and expert support.
Serbia is about to start a second round of external school evaluations (2019-24) and is introducing changes to key aspects of the process. The quality standards were refined in 2018 based on feedback from advisors and other external evaluators following the first round of evaluation. A new core of licensed teacher-evaluators has been created to help advisors carry out the evaluations and address the shortage of qualified staff. These licensed teacher-evaluators assist advisors and accompany them in school visits. However, these changes, while important, are not sufficient to address the disconnect between school evaluation and improvement. Moreover, some of the changes, such as the increase in the number of standards, go in the opposite direction of current trends in mature school evaluation systems and risk overburdening schools by making the process more of an administrative check rather than a focused evaluation on key factors of school quality. If external evaluation is to serve as a real catalyst for change in school practice and policy, not only will the independence and authority of the IEQE need to be reinforced, both the capacity and expectation for schools to act upon evaluation results will need to be strengthened significantly.
Recommendation 4.1.1: Institutionalise and invest in capacity for external evaluation
Responsibility for external school evaluation in Serbia is divided between the IEQE, a semi-independent agency which develops evaluation resources, and the advisors in RSAs, who are responsible for implementation. This leads to a situation in which there is no entity leading external evaluation and ensuring its overall quality and effectiveness. Moreover, advisors have overlapping, conflicting responsibilities and limited capacities to carry out all their duties. An agency with full responsibility for external school evaluation and for reviewing the staffing of the evaluation team to ensure its independence must be created to give external evaluation more visibility and influence, and improve confidence in the process. The consolidation of responsibility for school evaluation within a dedicated institution will also free up school advisors’ time to focus on providing support to school development and improvement.
Create an independent national institution in charge of external evaluation
Creating an independent agency responsible for overseeing and implementing external school evaluation in Serbia would help to strengthen the integrity of the evaluation process and provide a more independent perspective on the effectiveness of national policies. Across OECD countries, a growing number of countries (13 out of 36) have set up independent external evaluation agencies or inspectorates responsible for the whole school evaluation process (see Figure 4.4). The Centre for Quality Assurance of Educational Institutions should become the sole authority responsible for school evaluation in Serbia. The centre, originally created with this intent in mind, has thus far focused mostly on the development of tools and guidelines. Its mandate should be expanded to include selecting and providing continuous training to the evaluation teams that will carry out the school visits. The evaluators should be accountable directly to the Centre for Quality Assurance of Educational Institutions. Serbia can learn from the experience of Romania, for example, which set up an independent agency responsible for external school evaluation in 2005. The Romanian Agency for Quality Assurance in Pre-University Education (ARACIP) is responsible for developing the tools for evaluation and selecting and training evaluators. The agency was also successful in ensuring the independence of its evaluation and thus gradually building trust in the fairness of its evaluation among schools (see Box 4.1).
Serbia should also consider making the Centre for Quality Assurance an agency independent from the IEQE and the ministry, and, at a minimum, make sure that it has its own dedicated, sustainable funding stream to carry out its mission. This includes funding to develop and refine the tools (e.g. school quality standards, evaluation grids, feedback template, etc.) and to provide training to evaluation teams. Currently, the centre’s budget falls under the overall budget of the IEQE, which is also responsible for carrying out other significant reforms, such as the introduction of the new end-of-upper-secondary examination (Matura) and a new national assessment. The lack of a dedicated budget line for school evaluation risks perpetuating the current situation of chronic underfunding, as other urgent reforms may take budgetary priority. Establishing the centre as an independent entity would create pressure for adequate funds, but also elevate school evaluation as a core function within the education system and give its leadership more voice in policy discussions.
Figure 4.4. Number of OECD countries by type of school evaluation agency
Source: OECD (2013[1]), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, http://dx.doi.org/10.1787/9789264190658-en.
Box 4.1. The Romanian Agency for Quality Assurance in Pre-University Education (ARACIP)
The Romanian Agency for Quality Assurance in Pre-University Education (hereafter the agency) was created in 2005 by the Quality Assurance Law, which provided the basis for the current school evaluation system in Romania. The agency is a permanent external school evaluation body, separate from the Ministry of National Education and Scientific Research (MNESR), with its own legal status and budget.
The agency’s main function is external evaluation and it is responsible for developing national quality standards and performance indicators. After an evaluation, the agency advises the ministry of education whether a school should be granted provisional authorisation, initial accreditation or recurrent evaluation.
Other than external evaluation, the agency also provides guidelines and a template model for school self-evaluation and makes recommendations to the government on issues of quality education. The agency publishes an annual activity report and releases another on the state of quality in education at the national level every four years.
As in other European countries, the agency works with evaluators with significant teaching experience to carry out their external evaluations. Candidates must have experience in the evaluation domain and, once selected, must follow an 89-hour training programme in order to assume their positions.
Source: Romanian Agency for Quality Assurance in Pre-University Education (n.d.[15]), Country Profile Romania, http://www.sici-inspectorates.eu/getattachment/1cbc0561-c91b-4c71-a71c-6b9369ecad61 (accessed on 7 June 2019); Kitchen, H. et al. (2017[16]), Romania 2017, https://doi.org/10.1787/9789264274051-en; European Commission/EACEA/Eurydice (2015[6]), Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe, http://dx.doi.org/10.2797/678.
Develop a wider pool of licensed external evaluators
The persons in charge of evaluating schools should not be the same individuals responsible for helping schools implement the recommendations or for providing other forms of ongoing technical support to school staff. This creates a conflict of interest and undermines the credibility of the evaluation process. Moreover, it has shown to be very difficult for the advisors to carry out both sets of responsibilities given their limited numbers (100 advisors across 17 RSAs). Instead of relying on the advisors from a school’s RSA to carry out school evaluations, the Centre for Quality Assurance should train, certify and contract individuals of various profiles to join the evaluation team. This is a common practice in many well‑established evaluation agencies in OECD countries, such as Ofsted in England, United Kingdom, the Swedish Schools Inspectorate and the Dutch Inspectorate of Education (European Commission/EACEA/Eurydice, 2015[6]; DICE, 2015[17]).
In Serbia, the evaluation team could draw upon:
Contracted teachers and principals: Serbia should further develop the practice of contracting teachers for external school evaluation. It should make sure that selected teachers go through a certification process to ensure they are qualified for the role and should extend this scheme to school principals. It is common across European and OECD countries for inspection teams to include school principals and teachers from other schools who have been trained and licensed as evaluators (OECD, 2013[1]). This new responsibility should be recognised in the teacher career structure.
Experts: The Centre for School Quality Assurance should also consider contracting experts or researchers in specific fields in which certified teachers, school principals and advisors lack the expertise to assess the quality of school practice. Such fields include health, nutrition, violence prevention or inclusive education. Experts should be invited to join the external evaluation team on specific occasions, depending on the focus of the evaluation.
Audit inspectors: As argued below, this review recommends that the school audit carried out by municipal inspectors be integrated more fully within the external evaluation process. Thus, inspectors would join the external evaluation team if an initial review of school documents shows a risk of noncompliance with regulations.
Selecting and training new evaluators in sufficient number may take some time and would require additional funds. As this new pool of licensed evaluators is being introduced, the centre should immediately stop using advisors from the same RSA as the school being evaluated and rely instead on advisors from other RSAs to carry out school evaluations. This is important to ensure the independence of the evaluation process. However, as their role in teacher appraisal for promotion and advice to school increases in the medium term, advisors will not have time to dedicate to external evaluations and their participation should thus become the exception rather than the rule.
Streamline the regular audit and better integrate within the external school evaluation
The audit process needs to be streamlined to avoid overburdening schools. The current practice of having two separate procedures is both costly for Serbia, as there are not enough audit inspectors to visit all schools, and creates an unnecessary burden on schools, which need to prepare one set of documents annually for municipal inspection and a different set of documents for the external evaluation every five years. Most importantly, the audit processes do not inform the external school evaluation process and function almost completely in parallel. To make the most of the available human resources of audit inspectors, this review recommends that:
Document checks are streamlined: It is very uncommon in OECD countries to check compliance with regulations during an audit visit. In a 2011 survey of audit practices, no OECD countries had regular school visits as the main means to check compliance. The majority of countries require schools to send relevant documents to the auditing agency or the inspection agency, with some countries, such as the Slovak Republic, having set up a follow-up inspection process if initial document checks show discrepancies. A growing number of countries have introduced e-platforms to simplify the submission of documents from schools (OECD, 2013[1]).
Regular audits carried out annually by audit inspectors should be discontinued. Schools should be asked to upload all relevant documents online to a platform managed by the Centre for Quality Assurance of Educational Institutions. Information from these documents should be reviewed both by audit inspectors at the municipal level to determine if schools are at risk of noncompliance and by the school evaluation team in preparation of the school visit. Schools should only be asked to update documents when necessary, instead of annually for every document.
Audit inspectors visit schools when issues are identified: If major risks of noncompliance were determined by reviewing the material uploaded by the school or in the event of complaints from school staff, parents or students, audit inspectors would visit the school to check the situation and draft a report on the necessary measures to be taken. Thus, the school audit visit would become the exception instead of the rule. This will free up inspectors’ time to contribute more actively to the external school evaluation.
Audit inspectors are trained and asked to join external evaluation teams: As the number of school audits per year will be reduced to a minimum, inspectors can join external school evaluation teams on school visits to review administrative and regulatory compliance. Audit inspectors will also need to receive the same training given to advisors and contracted teachers in order to join school visit teams in addition to their training on school audits.
Recommendation 4.1.2: Review how evaluation results are reported and used to support school improvement
If school evaluation is to help lead to improvement, schools need to have the capacity and incentives to take action in response to the issues identified. They also need to receive clear guidance from evaluators on where and how they might improve. At present, the feedback provided to schools is limited to numerical scores (grades) and a description of performance against each indicator. Given the limited human and financial resources available, external follow-up support from advisors is very rare, even for schools identified as “weak” or “very weak”. This means that schools are left very much on their own to determine what they can do to improve. The quality of the six school-improvement plans the review team examined was extremely variable. While some plans were very detailed and listed concrete actions to be taken by the school, others were quite general and would not provide adequate direction to school actors on how to improve the quality of their practices. The ministry of education, the Centre for Quality Assurance and the Institute for Improvement of Education (IIE) need to provide schools with more useful feedback as well as more support to plan and implement change in response to evaluation results. They also need to provide appropriate support and accountability structure to schools, in particular to those that do not meet the minimum level of quality.
Revise the school report template to include recommendations for improvement
The written school report on the results of the external evaluation does not include recommendations for improvement. While this model of reporting with no explicit recommendations is common in some OECD countries, such as the Netherlands, and leaves schools the freedom to choose how to address the identified issues, it is not effective in contexts with limited in-school capacity for analysis and planning and resources for improvement, such as in Serbia. Additionally, the review team observed that schools visited tended to focus more on the overall numerical score (grade) rather than their strengths and weaknesses. To shift this focus and help schools use the feedback from external evaluation for improving their practices, the Centre for Quality Assurance should consider revising the school report template as follows:
Introduce a one-page summary of the report: This one-pager should include the identified strengths and main recommendations for improvement. Recommendations should be as precise as possible and focus on areas under the direct control of the school. The centre can look at the example of Education Scotland’s inspection reports, which combine a detailed report summarising key findings for each core indicator and a brief note summarising the main strengths and recommendations for improvement (see Box 4.2).
Replace numerical scores with qualitative descriptors: At present, the written report only includes numerical scores (grades). This practice reinforces a summative view of the evaluation and focuses schools on the overall score rather than the quality of the underlying practices across core areas of their work. Introducing descriptors (e.g. very good, good, weak, and very weak) and examples of practices that illustrate the judgement will give the score more meaning and help inform the next steps.
Include contextualised performance data with benchmarks: Providing schools with data and benchmarks is useful to anchor the report in evidence and give schools materials for their own self-reflection on their practices. This includes quantitative data such as student learning outcomes and soft “data” like surveys and interviews (NCSL, n.d.[18]). The written report should, for example, include school performance data such as students’ average marks in the end-of-basic-education exam and new State Matura (examination at the end of upper secondary) as well as completion and enrolment rates. This data should be contextualised by including the average performance of schools with similar socio-economic background. Similar to the Scottish example (see Box 4.2), an annex can be added with the aggregated results of student, parent and staff surveys.
Box 4.2. School inspection reports in Scotland
Education Scotland, the agency under the Scottish government charged with supporting quality and improvement in national education, publishes three types of school inspection reports for each of the evaluated schools.
The summarised inspection findings
The summarised version of the report provides brief contextual information about the inspected school and its main findings. It presents the inspection’s conclusions and some recommendations based on core indicators and a summary of evaluation results regarding each theme covered by the referred indicator. The school’s evaluation result (for each indicator) is given under a qualitative descriptor format ranging from “very weak” to “very good”.
The inspection report
The inspection report is relatively short and addresses students’ parents or carers. It states the school’s strengths followed by the school’s areas for improvement in bullet point format. The last page is reserved for a table presenting the descriptive evaluation (from “very weak” to “very good”) of the quality indicators used to evaluate the school.
The additional inspection evidence
This evidence report serves to inform the public of the methodology used to carry the evaluation process. It presents the questionnaire given to parents/carers, school partners, staff pupil support, staff school support, staff teachers and young people (actors can vary according to the education level being evaluated). A summary of the answers given to each question is also presented. Answers are portrayed in percentages and are also shown in the form of graphs in order to facilitate its visualisation and understanding.
Source: Education Scotland (2019[19]), Reports, https://education.gov.scot/reports-by-date (accessed on 7 May 2019).
Develop the school boards’ capacity to monitor the quality of school planning and programmes
The school board is responsible for validating the school action plan that school principals are required to develop, with input from the school community following an external evaluation. This role should help keep school staff accountable for implementing the recommendations of the external evaluation. However, in practice, the school boards’ function in Serbia is limited to a mere administrative check that an action plan has been developed, instead of a genuine review of its quality. School board members receive no guidance from central authorities or the RSAs on how to evaluate the quality of a school plan or monitor its implementation.
The Centre for School Quality Assurance can take several actions to help school boards take on a more active role in monitoring the quality of action plans and supporting school improvement. The centre can provide school boards with examples of good action plans to help guide their judgement. The centre also needs to make training available to school boards on school management and planning to help them with their monitoring role (e.g. school funding, the planning cycle, etc.) and encourage school boards to take up this training. This can be done by sending letters to newly elected school board members detailing the training available. In many OECD countries such as Estonia, training on school management is available for both school leadership and school boards which are tasked with overseeing the schools (European Commission/EACEA/Eurydice, 2015[6]). Most importantly, Serbia will need to make sure that school principals are adequately prepared in how to plan for school improvement and how to engage and motivate the school community behind collective follow-up actions. This needs to be reflected in their initial training and through continuous training (see Recommendation 4.2.2.).
Introduce a risk-based approach to follow-up support
Given the limited human and financial resources at both the national and regional levels, the ministry needs to focus its technical and financial support on schools that did not meet the minimum level of quality during the external evaluation, i.e. schools that received a score of “weak” or “very weak”. This is particularly important to ensure more equity in the system as schools with low quality tend to be schools in more disadvantaged areas. In Serbia for example, between-school difference in socio-economic background explains about 40% of the variation in low performance in mathematics, suggesting significant concentrations of low performers in particular schools (OECD, 2016[20]). To do so, the ministry should consider:
Focusing advisors’ follow-up on the schools evaluated as “weak” or “very weak” in the external evaluation: As advisors will not be responsible for evaluating schools in their own region, there will be fewer risks of conflict of interest. They will also gradually be less involved in external evaluation, freeing up time they can use to help schools develop and implement their action plan. Given their limited number, advisors should focus on schools that scored “weak” or “very weak” on their region’s last evaluation. Advisors should work with these school to develop an action plan that addresses key areas of improvement. They should identify needs for training and external support the schools will require for this (i.e. coaching opportunities, participation in peer learning, etc.). Advisors should also make sure that these schools have the necessary budget to implement the improvement plan. To carry out this function effectively, the ministry and the IIE should make sure that advisors have easy access to the most up-to-date information about available training opportunities and funding sources.
Making sure that low-performing schools receive the financial resources needed to improve: As part of the national school improvement strategy recommended by this report (see Policy issue 4.3), the ministry needs to make sure that low-performing schools have access to additional financial and technical support to improve their quality. This support can be in-kind through participation in peer-learning programmes facilitated by the Centre for School Quality Assurance or additional financial resources or grants targeted at professional development.
Implement the differentiated approach to school evaluation to incentivise and reward improvement, including in high-performing schools
The Centre for School Quality Assurance has introduced a differentiated approach to school evaluation in the revised process for school evaluation introduced in 2019. The new process requires schools with very weak performance to be evaluated again after three years. This can help make sure that Serbia’s weakest schools are closely monitored and given the opportunity to demonstrate improvement. A similar approach is used in a growing number of European countries which are introducing a risk-based approach to external evaluation, by which schools that are at risk of not meeting minimum quality standards are evaluated more frequently than others (Gray, 2014[21]). For example, Ofsted, the inspection agency in England, requires schools that received a “required improvement” mark during the regular inspection to undertake a new inspection two years after the original inspection (Gray, 2014[21]). This change is positive and should thus be continued and implemented.
The compressed timeline for low-performing schools will also help incentivise schools to show progress to obtain a higher score and improve their reputation in the local community. The experience of the Netherlands shows that schools that receive a “weak” score prefer to have a short cycle of evaluation to demonstrate their progress and discard the “weak school” label (see Box 4.3). The new differentiated approach should also target high-performing schools. Schools that performed “good” or “very good” in the evaluation should be rewarded by a longer evaluation cycle (every five years for example). This differentiated approach gives high-performing schools public recognition for their practices and shows trust in their ability to drive their own improvement agenda without close external monitoring.
Box 4.3. A risk-based approach to school evaluation in the Netherlands
A risk-based approach to school evaluation in the Netherlands
The school evaluation system in the Netherlands relies on the availability of a rich set of data on schools and mechanisms for monitoring, collecting and analysing school performance. The agency responsible for external school evaluations in the Netherlands is the Dutch Inspectorate of Education. The inspectorate uses a highly developed process to conduct evaluations and its approaches are constantly revised to meet emerging needs. For example, in 2008, the Dutch Inspectorate introduced a risk-based approach to school evaluation. Schools identified as “at risk” of underperformance are evaluated more comprehensively and with more frequency than those that perform well by comparison. In this model, schools are classified into two different categories:
“At-risk” schools are identified based on a variety of data including school-level student performance data, documents sent by schools to the inspectorate as well as “failure signals”, such as media news and external complaints. Every school goes through the risk-analysis process. If risks are identified, the inspectorate must conduct a follow-up inspection. This inspection is based on a framework of quality criteria covering key aspects of pedagogical and organisational processes that may have an impact on students’ outcomes. Schools must then develop an action plan and programme for improvement. A quality inspection is carried to assess the completion of the improvement process which leads to a final inspection report responsible for assigning the school to a different inspection regime.
Schools “to be trusted” have reached basic quality after the risk-based inspection. Under this classification, schools are visited only once every four years for a “basic inspection”. This kind of inspection checks, for example, legal compliance and special needs provision but does not evaluate the whole set of aspects that impact the teaching-learning process of a school. The difference in this approach is that it relies on publicly available information, instead of a school’s own evaluation documents.
There are indications that the risk-based approach reduces the number of schools providing a weak or unsatisfactory quality of education in the country. Since the introduction of the risk-based approach, the inspectorate has reported that between 2009 and 2012, the proportion of weak schools decreased from 7.4% to 2.9%, and from 10.9% to 9.4% in primary and secondary education respectively. Studies confirm that there is indeed a positive impact of risk-based inspections on weak and unsatisfactory schools; however, doubts remain about the nature of the impact on other schools (i.e. impact on preventing new schools from entering the weak and unsatisfactory categories).
Source: Nusche, D. et al. (2012[22]), OECD Reviews of Evaluation and Assessment in Education: Netherlands 2014, http://dx.doi.org/10.1787/9789264211940-en.
Focus school principals’ accountability on their leadership role rather than the schools’ overall performance
Serbia is considering introducing new measures to make school principals accountable for following up on the results of evaluation, with the possibility of removing a principal in “very weak” schools that do not show improvement in two consecutive external evaluations. There are many risks associated with this approach. Most notably, school principals might be unfairly made accountable for factors outside of their control. For example, lack of funding is often a strong factor in a school’s inability to improve. Rather than making school principals accountable for their schools’ overall outcomes, Serbia may consider instead developing school principals’ accountability for demonstrating good leadership competencies. School principals that have strong administrative and instructional leadership capacities are important for improving the quality of school practices (Pont, Nusche and Moorman, 2008[23]). Leadership competencies should be part of the core areas evaluated during the external school evaluation and school principals with weak or very weak leadership competencies should receive coaching and support (see Recommendation 4.2.2). If no improvement is observed over several evaluation cycles, the ministry can put in place a process for changing the principal in the school.
Policy issue 4.2: Support schools to develop a culture of self-evaluation and learning
While school self-evaluation has been mandatory in Serbia for almost two decades, it has not yet led to the development of a culture of continuous learning and improvement in schools. This is in large part because of the limited guidance and support that schools receive on how to engage in meaningful self-evaluation, as well as the limited instructional leadership capacity in schools and lack of financial resources to implement improvement activities. While there was a strong push in the early 2000s to develop school capacity to reflect on quality and use such analysis to inform planning, this effort has been more or less stopped as resources at the IEQE and IIE were directed towards other policies, such as external school evaluation and teacher professional development. The Centre for Quality Assurance in Education Institutions should provide schools with clearer guidance, tools and training opportunities on how to establish effective school self-evaluation and planning processes. School leadership in schools needs also to be developed and school principals provided with more feedback on how to improve their leadership competencies.
Recommendation 4.2.1: Provide schools with guidance on how to evaluate quality and use the results to inform development plans
Schools in Serbia have some autonomy and flexibility in terms of how they carry out self-evaluations and use the results to inform their planning and day-to-day practices. Schools set their own annual self-evaluation plan and determine the domains to be evaluated. While all schools must base their self-evaluation on the same standards and indicators, they can choose to develop additional indicators for this process. This is positive, as research shows that effective school self-evaluation needs to be appropriated and adapted by schools to fit their distinct needs and aspirations (OECD, 2013[1]). However, as the capacity for self-evaluation and planning in most schools is low, more national guidance and support is needed regarding what schools can do in practical terms to kick-start a meaningful reflection on quality. The limited national guidance on how to conduct effective school self-evaluation and use the results to inform planning leads to the varying and low quality of self-evaluation across schools. It also limits the extent to which the results of self-evaluation can contribute to improving school quality. For example, Education Scotland, the external evaluation body in Scotland, has set up a central web-based resource to help schools improve their self-evaluation capacity. Its package of resources, known as Journey to Excellence, is constantly growing. It provides guidance for improvement in school planning and examples of school quality indicators (OECD, 2013[1]). The Centre for Quality Assurance of Educational Institutions should provide schools with clearer direction as to what a quality learning and school environment look likes, give them the tools to evaluate their practices in relation to standards and benchmarks, and make sure that school self-evaluation is truly embedded in schools’ improvement culture.
Create a new self-evaluation manual and encourage schools to use it
The 2005 manual of self-evaluation provides comprehensive guidelines to schools on how to conduct a meaningful self-evaluation. It includes a simple definition of what self-evaluation is and how it can be used to inform school planning. It lists the indicators that schools can use to evaluate their practices and provides a clear description of what schools need to demonstrate in order to score 4 (highest level on the scoring scale) and 2 (weak). It also provides templates for teacher and student surveys and scoring grids for each quality area. However, this manual is outdated and does not reflect the new school quality standards introduced in 2011 and updated in 2018, nor does it reference the National Education Strategy and key reform priorities, such as improving the quality of professional development. As such, schools do not use this manual in their self-evaluation practices. The Centre for School Quality Assurance should create a new manual as follows:
Review schools’ experience with self-evaluation: The centre should lead a review to understand how schools are using self-evaluation and what practical changes to the self-evaluation manual and process would help make it more useful for them. This could be done by sending schools a short survey about their practices and sending a team of experts to observe the school self-evaluation process in a sample of schools. In the Netherlands, although there is no mandatory requirement for school self-evaluation, it is the responsibility of the Dutch Inspectorate of Education to evaluate a school’s internal quality care policy, which includes self-evaluation processes if these are in place (OECD, 2013[1]).
Provide schools with a list of simple prompting questions: Research and experience suggests that self-evaluations should aim to answer simple questions focused on improving teaching and learning, such as: “how good is our school?”; “how can we make it better?”; “are teachers’ skills being put to good use?”; and “how good is learning and teaching in our school?” (Riley and MacBeath, 2000[24]). The new manual should include a shortlist of such prompting questions to focus school evaluation on essential elements of school practice.
Get rid of numerical scores and highlight core quality areas to evaluate: Getting rid of numerical marks will help shift school actors’ focus from the score (grade) to the quality of practices. It will also diminish the perception of stakes that some schools may associate with the mandated school self-evaluation. The manual should include the six core key indicators used in the external school evaluation. These indicators, which focus mostly on teaching and learning practices as well as school planning, measure the most important elements related to school quality. The updated manual should include indicator descriptors for each, as well as benchmarks of quality.
The Centre for School Quality Assurance should also play an active role in encouraging schools to use the manual to inform their self-evaluation practices. The centre can, for instance, distribute copies of the newly revised manual to every school in Serbia. It should also ask advisors in RSAs to use the manual as a reference point in discussion with schools about their improvement plans.
Provide schools with indicators and tools to measure their performance against some key national targets
The Centre for School Quality Assurance should work with the data analysis team in the ministry to provide schools with contextualised benchmarks of schools’ performance on key indicators such as enrolment and completion rate by different student categories (i.e. gender, socio-economic background, ethnic group, SEN). Standard measures of school quality help schools understand how their practices and results compare to that of other schools and national goals (OECD, 2013[1]). Without such information, self-evaluation is limited to a reflection about practices compared to the school quality standard, with no sense of how the school compares in practice to the average Serbian school or schools facing similar contexts.
This will most likely take time as the data analysis team is currently underfunded and understaffed and will require an improvement of its capacity to provide reliable data to schools (see Chapter 5). In the medium term, the centre should also make sure that contextualised benchmark results of the school Matura exam (the planned end-of-upper-secondary-school exit exam) and the national assessment are provided to schools. This data should be granular enough to allow schools to compare their students. These contextualised indicators should also be used in the external school evaluation, which currently lacks standardised measures of student learning outcomes.
Ensure that schools have access to training and technical support
The Centre for School Quality Assurance should provide opportunities for schools to learn about how they can improve their self-evaluation process. While this is already part of its mandate, the centre has not been able to provide training on school self-evaluation for over a decade. The centre should be provided with sufficient funding and human resources to ensure that schools have access to the technical support that they need. Countries that succeed in developing a real culture of self-improvement in schools, such as Scotland, have heavily invested in providing schools with sufficient technical support (OECD, 2013[1]). This technical support should combine both seminars to explain self-evaluation and why it matters, and continuous support through coaching:
Training seminars: In collaboration with the IIE, the Centre for School Quality Assurance should make sure that school principals, teachers and pedagogues have the opportunity to improve their understanding and practice of key elements of school self-evaluation, such as collecting evidence, analysing information and providing recommendations for improvement. These seminars should be included in the IIE training catalogue and their design should be informed by needs in training identified during the external evaluation.
Coaching by a licensed evaluator: School self-evaluation teams should be given the opportunity to request the technical support of a coach in carrying out their self‑evaluation. For example, in Poland, schools can request support for school self‑evaluation from coaches at the teacher education and counselling centres. Similarly, in Belgium (German-speaking Community), schools can request support and coaching services on self-evaluation free of charge from the school development council within the ministry of education (European Commission/EACEA/Eurydice, 2015[6]). These coaches could be drawn from the external evaluators licensed by the Centre for School Quality Assurance (e.g. teachers or school principals) and trained in how to provide support for school self-evaluation. Such coaching could also be made mandatory for schools that did not meet the quality standards in the area of school planning in the external evaluation.
Given limited resources, the Centre for School Quality Assurance should prioritise providing the training programmes discussed above to schools identified as weak by the external school evaluation. This should be part of the support package provided to low-performing schools as part of the school improvement strategy (see Recommendation 4.1.2).
Encourage peer learning and sharing of experiences in self-evaluation
Schools need more opportunities to learn from each other’s experience in implementing self-evaluation activities. Disseminating good practices – for engaging the whole school, undertaking classroom observations or analysing data for example – provides schools with inspiration about how they can improve their own practices. Highlighting good practices also provides important recognition to encourage schools to improve. To support this, the centre should create an online platform where schools can exchange templates for surveys and other instruments for collecting evidence. The external evaluation team can be tasked to identify good practices and produce short video interviews with the school self‑evaluation teams to explain how and why the process was successful and what other schools can learn from it. Locally, the RSAs can also pair up schools in the same region based on identified needs to share experiences and foster peer learning.
Review schools’ capacity for improvement through the external school evaluation
The external school evaluation in Serbia does not assess the school’s capacity to set a meaningful self-evaluation process. The school quality standards do not include an indicator of the quality of self-evaluation. This is a lost opportunity to signal to schools the importance of school self-evaluation and ensure that a self-reflection process is used to inform school policies and practices. In a growing number of OECD countries, the school’s capacity to self-reflect on its practices to continuously improve is a central piece of external school evaluation. In New Zealand, for example, the Education Review Office evaluates schools’ self-evaluation capacity, describing a school with “very good” self-evaluation capacity as a school that convincingly demonstrates a rigorous culture of self-review and critical reflection to sustain positive performance and continuous improvement (OECD, 2013[1]). Nationally, the 2018 quality standards for pre-school institutions are a good example to follow. The pre-school standards include an indicator of self-evaluation and improvement culture: “Standard 3.4: The institution is a place of continuous change, learning and development”. The Centre for School Quality Assurance should adapt the school quality standards to include a core indicator on self-evaluation (see Box 4.4). In the medium term, once schools have developed a stronger capacity for improvement, school self-evaluation should become the main source of information for the external school evaluation (OECD, 2013[1]).
Box 4.4. Indicators used for reviewing the quality of self-evaluation carried out by schools
In New Zealand, the Education Review Office (ERO) adopted an evaluation approach based on schools’ self-evaluations and performance in 2009. An important aspect the office takes into account when deciding on the frequency under which a school is going to be reviewed is based on a school self-evaluation capacity.
The quality of school self-review is evaluated by the office as part of its reviews of individual schools. In schools where self-reviews are well established, the office simply confirms and validates the results of the evaluation. As for schools where self-reviews are considered less well established, the external review team needs to carry a further investigation into school quality. The office assesses schools’ capacity for self-evaluation based on the following indicators:
evaluation leadership
a learning-oriented community of professionals that demonstrates agency in using evaluation for improvement in practice and outcomes
opportunity to develop technical evaluation expertise (including access to external expertise)
access to, and use of, appropriate tools and methods
systems, processes and resources that support purposeful data gathering, knowledge building and decision-making.
In Portugal, school inspection carried by the General Inspectorate of Education also covers the evaluation of school self-evaluation. The criteria used to assess self-evaluation and improvement include:
coherence between self-evaluation and action for improvement
use of results of the external evaluation in the preparation of improvement plans
involvement and participation of the educational community in the self-evaluation
continuity and scope of self-evaluation
impact of self-evaluation in planning, organisation and professional practices.
Sources: OECD (2013[1]), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, https://doi.org/10.1787/9789264190658-en; Nusche, D. et al. (Nusche et al., 2012[22]), OECD Reviews of Evaluation and Assessment in Education: New Zealand 2011, http://dx.doi.org/10.1787/9789264116917-en; Education Review Office (2016[25]), School Evaluation Indicators: Effective Practice for Improvement and Learner Success.
Recommendation 4.2.2: Develop school leadership for improving the quality of their schools
The effectiveness of any evaluation, internal or external, depends largely on the ability of schools to act upon the results. In Serbia, this will require more attention to developing overall leadership capacity. The ministry has already taken important steps to strengthen the professional independence of principals and make them more accountable for school quality. The appointment of school principals is now based on a review of competency and all principals already in service are required to take a certification examination by 2021. New principals must acquire their certification in the two years following first appointment in schools. However, school principals’ instructional leadership skills, such as setting a vision for the school and monitoring progress towards achieving this vision, remain low. The majority of school principals in Serbia never received any training in instructional leadership before or after becoming school principals (see Figure 4.5). Indeed, monitoring and planning are the main areas where school principals reported needing training in an IIE survey in 2017 (IIE, 2018[26]).
Improving school principals’ initial and continuous professional development is thus important to make sure that Serbian schools are able to act upon the recommendation of the school self-evaluation. Moreover, school principals are given very little external support to develop their competencies and performance once they are on the job. While advisors are responsible for conducting regular school principals’ appraisal (so-called “regular supervisions” in Serbia), these are rarely conducted due to their limited numbers.
Figure 4.5. School principals’ access to formal training on instructional leadership
Note: Average in this figure corresponds to the participating countries in OECD TALIS 2013.
Source: OECD (2014[8]), TALIS 2013 Results: An International Perspective on Teaching and Learning, https://dx.doi.org/10.1787/9789264196261-en.
Set up a leadership academy in charge of school principals’ training
The master’s degree in leadership introduced in 2016 in the Universities of Kragujevac and Novi Sad is a step in the right direction towards improving school principals’ preparedness for the job. However, the lack of public funding for this programme and the fact that participation is not taken into account in selecting and appointing school principals limit its appeal and have led to low take-up rates in recent years. Moreover, ways to ensure the quality and relevance of this master’s programme remain limited. To improve school principal competencies, the ministry should consider focusing as a priority on providing free and mandatory training for new principals based on the competency standards for school principals:
Introducing free and mandatory initial education: To improve school principal’s leadership capacity, the IIE should consider providing free mandatory practical training for all newly certified school principals. It is common practice among European and OECD countries to provide initial training to school principals on their key responsibilities. For example, in the TALIS 2013 survey, 70% of school principals in Finland and 60% in Poland reported having received training on school administration prior to taking up their position as school principal (OECD, 2014[8]). This is done either through bachelor’s or master’s programmes before selection or more commonly through specialised training once the candidate has been selected to be a school principal. The initial education should be of sufficient length to cover all areas of responsibility of school leadership and provide school leaders with practical training in some key areas. Twenty-one European countries require specific training before or after the appointment of a school principal, with the length of this initial training varying to a great extent among them (European Commission/EACEA/Eurydice, 2013[27]). For example, in the Czech Republic and France, headship training takes place after the appointment, with a duration of 100 hours for the former and of 1 year for the latter (European Commission/EACEA/Eurydice, 2013[27]).
Introducing a mentorship programme for new school principals: School principals with experience can serve as mentors to new principals and provide support and guidance on how to meaningfully undertake instructional leadership duties and provide regular feedback. Similar to teacher mentors, school principals’ mentorship role should be recognised and compensated and selected mentors adequately trained on how to provide guidance and feedback. In Estonia, coaches are selected among school principals with at least five years of experience. They also need to demonstrate a high level of motivation and pass a mandatory training course on communication, needs analysis, coaching and feedback skills (European Commission, 2017[28]).
To drive these changes and improve school principals’ professional ownership of the new competency standards, the ministry should consider setting up a new school leadership academy (either an independent agency under the authority of the ministry or affiliated to a public university) in charge of school principals’ initial training, certification, continuous professional development and research to improve leadership practices. This academy will help give more visibility to school leadership and strengthen the professionalisation of school principals. Many OECD countries such as Austria, Ireland and Slovenia have set up similar leadership academies (see Box 4.5). The school leadership academy should also develop training and resources for school principals working in a school cluster with one or several satellite schools. Making sure that school principals are able to lead a school cluster with multiple locations, often in isolated rural areas, is important given that 10% of students in Serbia attend satellite schools in basic education.
Box 4.5. School leadership academies in Austria, Ireland and Slovenia
In Austria, the Leadership Academy founded in 2004, provides training to improve the qualifications of executive-level personnel in schools, targeting leaders, directors and managers of and within educational institutions in the country. The training focuses on several elements of leadership, including leadership for learning, dialogue, shared leadership, innovation and the capacity to improve the quality of education.
Every year a new “generation” participates in a series of four three-day fora and works toward meeting certain certification criteria to be admitted into the Leadership Academy network. These criteria range from participating in the four fora to leading a participative development project in their home educational institution. The programme has been noted for its high degree of engagement among participants and its positive impact on leadership practice, particularly in the areas of providing direction, demonstrating strength of character and community-building and creating a culture of achievement.
In Ireland, The Centre for School Leadership (CSL) was established in 2015 under a partnership between the Department of Education and Skills (DES), the Irish Primary Principals’ Network and the National Association of Principals and Deputy Principals. Among its functions, the centre supports, leads, co-ordinates and delivers leadership professional development programmes for primary and post-primary schools, which includes a programme for newly appointed principals, coaching for active principals and others. Additionally, the centre was to develop a strategic framework for a continuum of leadership professional development and a quality assurance framework for its provision, as well as to advise DES on leadership professional development policy.
In Slovenia, the National School of Leadership in Education (NSLE) was established in 1995 and is dedicated to the training of headteachers and their professional development in the country. Its initial head teacher training equips participants with leadership and management skills and contributes to their personal and organisational efficiency. All Slovenian head teachers are required to participate in such training. Participants in the one-year programme include recently appointed principals or aspiring candidates. The training is implemented in small groups and consists of six modules: i) introductory module; ii) organisational theory and leadership; iii) planning and decision-making; iv) head teachers’ skills; v) human resources; and vi) legislation. The national school issues its call for applications for the programme once a year. The school also provides support for head teachers in their first year in position by offering a mentoring programme.
Sources: Pont, B., D. Nusche and H. Moorman (2008[23]), Improving School Leadership, Volume 1: Policy and Practice, https://doi.org/10.1787/9789264044715-en; BMBWF (2018[29]), Leadership Academy, https://www.leadershipacademy.at/, (accessed on 20 June 2019); Schratz, M. and M. Hartmann (2009[30]), Innovation an Schulen durch Professionalisierung von Führungskräften [Innovation in Schools Professionalisation of Leaders], https://www.leadershipacademy.at/downloads/LEA_Kurzfassung_Studie.pdf, (accessed on 20 June 2019); Fitzpatrick Associates (2018[31]), School Leadership in Ireland and the Centre for School Leadership: Research and Evaluation, https://cslireland.ie/images/downloads/Final_CSL_Research_and_Evaluation_Final_Report__Feb_2018_.pdf, (accessed on 20 June 2019); European Commission (2017[28]), Teachers and School Leaders in Schools as Learning Organisations: Guiding Principles for Policy Development in School Education, https://ec.europa.eu/education/sites/education/files/teachers-school-leaders-wg-0917_en.pdf, (accessed on 20 June 2019); NSLE (n.d.[32]), The National School of Leadership in Education, http://en.solazaravnatelje.si/index.html, (accessed on 20 June 2019)..
Use the external school evaluation to provide formative feedback to school principals
Advisors rarely appraise school principals in practice as their time is mostly spent on external school evaluations. Even though advisors should gradually stop being involved in the external evaluation (see Recommendation 4.1.1.), they will need to prioritise other tasks such as providing technical support to “weak” schools for which their geographic proximity and relationship with schools is more strategically needed. Rather than relying on regular supervision to provide feedback to school principals on their performance, Serbia should make use of the external evaluation process to identify leadership capacity gaps and recommend professional development opportunities. Many OECD countries use external school evaluation to appraise school principals in a formative manner (OECD, 2013[1]). For example, Austria relies solely on external school evaluation to assess the quality of school leadership. Other countries, such as the Czech Republic, Poland or Sweden make use of both the individual appraisal process and external evaluation to better identify school leaders’ areas of strengths and weaknesses and recommend adequate professional development (OECD, 2013[1]). The Centre for School Quality Assurance should include school leadership capacity as a core indicator for the external evaluation and ensure that school principals are provided with feedback on their leadership capacity at the end of the evaluation visit.
The centre should also consider setting up a process that triggers a more in-depth appraisal by licensed evaluators if the school leadership in the school is identified as “weak” or “very weak”. In this case, the appraisal should focus on identifying areas where more support is needed and point the principal towards professional development opportunities.
Policy issue 4.3: Put school improvement at the centre of the National Education Strategy
For school evaluation to contribute meaningfully to the improvement of students’ learning nationwide, it needs to be part of a broader national effort to build schools’ agency for learning and improvement by aligning external support, funding, and monitoring and accountability systems. In Serbia, policies for school improvement are carried out by separate agencies with limited co‑ordination and follow-through at the local and school levels. In addition, while schools in Serbia have a fair amount of flexibility in how they allocate human and financial resources, they are not able to make the most of this autonomy to improve their performance due to chronic underfunding and limited support to build their capacity to self-reflect and lead change.
Recommendation 4.3.1: Develop a national strategy for school improvement
While a lot has been done in Serbia to help improve school quality, these policies have been for the most part fragmented and, in some cases, only partially implemented. For example, the master’s programme on school leadership, a promising policy for professionalising the school principal role, had a low take-up due to limited funding and articulation with school principals’ recruitment and accountability processes. The Ministry needs to create a national framework for school improvement to make sure that there is much more coherence and continuity in the design and implementation of policies related to school improvement. This national framework should build on ongoing initiatives and programmes such as the SHARE peer-learning project to help schools develop a stronger culture of improvement. Research shows that effective school improvement policies focus primarily on building in-school capacity and agency for improvement by creating a culture of collaboration and developing support systems and networks for learning, elevating the role of school leaders and monitoring improvement in schools to orient and guide policies and practices (Fullan, 1992[33]).
Put school improvement at the centre of the National Education Strategy
The ministry needs to put developing school agency for improvement at the centre of the new Education Strategy 2020-30 currently being developed (see Chapter 5). The strategy should clearly state the role that schools and school actors should play in improving the quality of education in Serbia and the type of national-level support that will be provided to schools to help them improve. Particular attention should be given to transforming the roles of school principal, deputy principal and pedagogue into a professional leadership team with clearly defined responsibilities and capacity to drive improvement in schools. For example, the Costa Rican strategy for school improvement, Quality Schools as the Axis of Costa Rican Education, focused on developing school leadership capacity and ensuring that school leaders have the agency to act and innovate in their schools (Consejo Superior de Educación, 2008[34]). For a school-centred education policy to work, it needs to be accompanied by an action plan with budgeted programmes and projects (see Chapter 5). This action plan should group policies on professional learning and development for school staff such as the development of a school leadership academy and a mentorship programme for school leaders as well as policies for developing school-level data to help schools monitor progress and the reinforcement of school evaluation policies discussed in this chapter. Each project should be costed and assigned a budget to ensure sustainability of implementation over time and accountability of the ministry and other agencies.
Set up a school improvement hub
The ministry, the IIE and the IEQE should work together to develop one unique platform for all resources for school improvement. This can be done by transforming the IIE’s National Learning Portal for Education (e-learning platform for teachers) into a hub of resources for schools. Such a “hub” would make it easier for schools to access the tools and training needed to improve their practices and will gradually help to create a better understanding among school actors of the interlinkage between school evaluation, school planning and teaching and learning practices. For example, Education Scotland has set up a platform called the “National Improvement Hub” where school actors can access examples of classroom practices with proven positive impact on student learning, materials and templates to develop effective self-evaluation processes as well as summaries of recent research evidence on what works to improve teaching and learning (see Box 4.6). Serbia should set up a similar platform including the e-materials recommended by this review, such as the IIE’s teacher e-learning platform, materials and examples of student assessment, the school self-evaluation manual and templates as well as briefings about research.
Box 4.6. National school improvement hub in Scotland, United Kingdom
The Scottish government has an online platform for collaboration and sharing school-level good practices called the National Improvement Hub. The hub includes research articles on what works in schools, official documents and guidelines such as the school evaluation framework, teaching and assessment resources, exemplars of good practices selected by school practitioners. School staff is encouraged to use the hub and give feedback for improvement, as well as to participate in occasional workshops, organised both on line and at various locations across Scotland.
Effective practices on teaching and learning are compiled into the “teaching toolkit” for teachers to use as reference material in designing their classroom practice. The practices in the toolkit focus on issues most schools in Scotland face such as extending school time, peer tutoring, school uniform, etc. For each practice, the toolkit identifies its impact as measured by impact evaluations and its cost.
Source: Education Scotland (2019[35]), National Improvement Hub, https://education.gov.scot/improvement (accessed on 27 May 2019).
Strengthen and expand school networks for quality improvement
School peer-learning networks provide school practitioners with the space to learn from other schools’ practices, and discuss and solve common challenges (Pont, Nusche and Moorman, 2008[23]). Many OECD countries are actively resourcing and encouraging school actors to take part in peer-learning networks as a way to help spread tested and proven good school practices across the education system (OECD, 2015[36]). While Serbia has already some experience setting up school networks through the SHARE programme (see Box 4.7), this only covers a limited number of schools. The ministry should task the RSAs with encouraging schools under their responsibility to collaborate and exchange ideas. This can be done, for instance, by creating regular events for school principals to meet and discuss current issues, and by encouraging school staff to visit other schools to observe teaching and learning practices. The SHARE programme should also be continued and systematically implemented for schools scoring “weak” or “very weak” in the external evaluation, counting on strong co‑ordination and support coming from the national level.
Box 4.7. The SHARE programme
The SHARE project, a joint project of UNICEF, the ministry of education, the Centre for Education Policy (a research centre) and Institute for Education Quality and Evaluation (IEQE), is the first initiative in Serbia aiming to create learning communities and peer learning between schools. SHARE aims to improve the quality of education by developing horizontal learning between schools and developing schools’ and teachers’ agency to learn and lead change in the education system. The initial phase of the project took place between 2015 and 2017 with 20 schools, 1 080 teachers and 12 665 students participating across Serbia. The project paired 10 schools that performed very well in the external school evaluation (score of 4), known as “model schools”, with 10 schools that performed weakly (score of 2 or 1), known as “SHARE schools”.
The project used a reflective approach combining classroom observation and feedback on observed practice. Following the selection of participating schools, classroom visitations are planned to support reflective practice. During this step, teachers, school principals and support staff from the SHARE schools observed between 10 to 15 hours of teaching at the model schools.
Based on a pairing system, the majority of discussions between schools focused on classroom management, lesson planning, teaching techniques, student support, teamwork and preparing for external evaluation. To give constructive feedback during these peer-to-peer sessions, staff in the model schools received training on how to articulate, document and share their success with their paired schools. During the final school visits, SHARE schools were also given the opportunity to present their experience and examples of best practices, thus motivating self-reflection.
The SHARE project initiated and established mutual exchange of knowledge and best practices between schools. It provided schools with hands-on experience through its peer-to-peer learning component. In addition, as a way to enhance the sustainability and long-term benefits of the project, a learning portal was created and shared amongst educators in Serbia. Moreover, 100 practitioners were trained to provide support for quality improvement in low-performing schools, creating a network of facilitators who have been integrated into the ministry of education as educational advisors linked to school administrations around the country.
The first phase of the project had a positive impact on the 20 participating schools and show scope for growth and scaling up. A majority of participating schools have seen an improvement in six out of seven areas of quality measured by the external school evaluation. This improvement was mostly seen in the areas of teaching and learning, school ethos and organisation of work and leadership. More broadly, the project introduced participating staff to the concept of horizontal learning and encouraged teachers to work together without the fear of being judged by their peers. It also allowed them to practice new teaching methods and play a more active role in shaping their classroom and school practices.
Sources: UNICEF (n.d.[37]), Dare to Share: Empowering Teachers to be the Change in the Classroom; European Commission (2017[38]), Networks for Learning and Development across School Education, https://www.schooleducationgateway.eu/downloads/Governance/2018-wgs5-networks-learning_en.pdf (accessed on 10 June 2019).
Recommendation 4.3.2: Make sure that schools are provided with sufficient financial resources to implement their improvement plans
Underfunding – in particular for professional development activities – severely constrains Serbian schools’ capacity to implement improvement plans. Central government does not provide funding for professional development and some municipalities allocate very little funds for professional development (see Chapter 1). Serbia has thus a high level of school principals and teachers’ out-of-pocket spending on professional development as shown by the OECD TALIS survey (OECD, 2014[8]). More broadly, while schools in Serbia have some autonomy in managing their budget, they are limited in their use of this autonomy due to lack of funding. Indeed, they rarely have enough funding to implement their school development plan without help from parents, non-governmental organisation or the private sector. For example, many school principals met by the review team reported having to fundraise or collect contributions from parents to implement activities in their school development plan. This creates risks of inequity as schools in more affluent areas have better opportunities to leverage funds. Providing schools with sufficient financial resources and support to use these resources should be a central component of the school improvement strategy recommended in this review. A central targeted grant for school improvement should be put in place to help schools, in particular those that are struggling to implement their improvement plans. Regular funding of schools can also be used more efficiently to make sure that schools, in particular those in disadvantaged areas have sufficient resources to provide quality learning to their students. Currently, Serbia has no financing scheme to address the needs of low-performing schools.
Review schools’ funding mechanisms to make sure that funds are distributed equitably
The ministry has considered introducing a per-capita funding formula for schools; however, after more than a decade of debate on this issue, the policy has never been formally implemented. Serbia needs a plan to ensure that funds are distributed to schools equitably and efficiently. A more efficient allocation of school funding will help Serbia invest more in school improvement activities such as training programmes for teachers and school leadership staff (World Bank, 2012[39]). Most OECD countries have introduced funding formulas to ensure that school funding is responsive to schools’ contexts and needs (OECD, 2017[40]). Similar to practices in OECD countries, the per-capita funding formula should take into account schools’ socio-economic context to ensure that those in socioeconomically disadvantaged areas are provided with sufficient resources to meet the needs of their students.
Provide central grants to schools to implement their school development plans
To implement their development plan activities, schools in Serbia receive funds from the local authorities for professional development activities or raise funds from the local community, businesses or donor organisations. Such a system leaves many schools with very limited funds to invest in improving their practices. The ministry should consider allocating a central grant to schools for implementing their school development plan. Priority should be given to schools that performed poorly in the external evaluation and schools in low socio-economic areas. Many OECD countries, such as England, do provide discretionary funds to schools to invest in improvement activities based on the schools’ performance and socio-economic background (see Box 4.8). These funds are usually accompanied by external support and monitoring to guide schools in using the funds to develop effective strategies in improving student learning. In Serbia, advisors should be responsible for this follow-up and monitoring function as recommended above (see Recommendation 4.1.1.).
Box 4.8. Pupil premium in England, United Kingdom
In England (United Kingdom), the Department for Education has established an additional funding scheme provided to schools attending disadvantaged students (pupil premium). Pupil premium funds are provided on a per-student basis and schools have autonomy on how these resources are spent. Schools are expected to spend these resources on strategies that better support learning for disadvantaged students and close the achievement gap between disadvantaged and advantaged students. Since 2012, schools are required to publish online information about how the pupil premium is used and the interventions they are implementing to address the needs of disadvantaged students as well as the impact they are having.
Schools receiving the pupil premium are required to monitor and report the achievement of all students and to report specifically the achievement of disadvantaged students. Ofsted, the English inspection agency, monitors closely the attainment and progress of disadvantaged students and how schools are addressing the needs of disadvantaged students. If the inspection identifies issues regarding the provision for disadvantaged students, then a more thorough review (the pupil premium review) is conducted. The purpose of this review is to help schools to improve their pupil premium strategy so that they “spend funding on approaches shown to be effective in improving the achievement of disadvantaged pupils”. The Department for Education uses information reported by schools to highlight and reward those schools reaching good results for disadvantaged students.
Source: OECD (2017[40]), The Funding of School Education: Connecting Resources and Learning, https://doi.org/10.1787/9789264276147-en.
Table of recommendations
References
[29] BMBWF (2018), Leadership Academy, https://www.leadershipacademy.at/ (accessed on 20 June 2019).
[34] Consejo Superior de Educación (2008), El Centro Educativo de Calidad como Eje de la Educación Costarricense [Quality Schools as the Axis of Costa Rican Education], Consejo Superior de Educación de Costa Rica, https://www.mep.go.cr/educatico/el-centro-educativo-de-calidad-como-eje-de-la-educacion-costarricense (accessed on 10 June 2019).
[17] DICE (2015), Policies and Approaches to School Evaluation in Europe, Database for Institutional Comparisons in Europe, CESifo, https://www.cesifo-group.de/ifoHome/facts/DICE/Education-and-Innovation/Education/Organisation/policies-approaches-school-evaluation-Europe/fileBinary/policies-approaches-school-evaluation-Europe.pdf (accessed on 20 June 2019).
[9] DuFour, R. (2004), “What is a professional learning community?”, Educational Leadership, Vol. 61/8, pp. 6-11, http://www.ascd.org/publications/educational-leadership/may04/vol61/num08/What-Is-a-Professional-Learning-Community%C2%A2.aspx (accessed on 20 June 2019).
[25] Education Review Office (2016), School Evaluation Indicators: Effective Practice for Improvement and Learner Success, Education Review Office, Wellington, https://www.ero.govt.nz/assets/Uploads/ERO-15968-School-Evaluation-Indicators-2016-v10lowres.pdf (accessed on 20 June 2019).
[35] Education Scotland (2019), National Improvement Hub, https://education.gov.scot/improvement (accessed on 27 May 2019).
[19] Education Scotland (2019), Reports, https://education.gov.scot/reports-by-date (accessed on 7 May 2019).
[38] European Commission (2017), Networks for Learning and Development across School Education, Directorate-General Education, Youth, Sport and Culture, Schools and Multilingualism, Brussels, https://www.schooleducationgateway.eu/downloads/Governance/2018-wgs5-networks-learning_en.pdf (accessed on 10 June 2019).
[28] European Commission (2017), Teachers and School Leaders in Schools as Learning Organisations: Guiding Principles for Policy Development in School Education, European Commission, https://www.schooleducationgateway.eu/downloads/Governance/2018-wgs4-learning-organisations_en.pdf (accessed on 20 June 2019).
[10] European Commission/EACEA/Eurydice (2018), Teachers’ and School Heads’ Salaries and Allowances in Europe 2016/17, Publications Office of the European Union, Luxembourg, http://dx.doi.org/10.2797/24212.
[6] European Commission/EACEA/Eurydice (2015), Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe, Publications Office of the European Union, Brussels, http://dx.doi.org/10.2797/65355.
[27] European Commission/EACEA/Eurydice (2013), Key Data on Teachers and School Leaders in Europe, Publications Office of the European Union, Luxembourg, http://dx.doi.org/10.2797/91785.
[31] Fitzpatrick Associates (2018), School Leadership in Ireland and the Centre for School Leadership: Research and Evaluation, Fitzpatrick Associates, https://cslireland.ie/images/downloads/Final_CSL_Research_and_Evaluation_Final_Report__Feb_2018_.pdf (accessed on 20 June 2019).
[33] Fullan, M. (1992), “Getting reform right: What works and what doesn’t”, Phi Delta Kappan, Vol. 73/10, http://search.proquest.com/openview/90f371562121c3949aece5d25131b48f/1?pq-origsite=gscholar&cbl=41842 (accessed on 16 February 2018).
[21] Gray, A. (2014), Supporting School Improvement: The Role of Inspectorates across Europe, Standing International Conference of Inspectorates, Brussels, http://www.sici-inspectorates.eu/getattachment/5caebee9-84c1-41f0-958c-b3d29dbaa9ef (accessed on 20 June 2019).
[12] IEQE (2017), Izveštaj o spoljašnjem vrednovanju kvaliteta rada obrazovno-vaspitnih ustanova u školskoj 2016/2017 [Report on the External Evaluation of the Quality of Work of Educational Institutions in the School Year 2016/2017], Institute for Education Quality and Evaluation, Belgrade.
[26] IIE (2018), Izveštaj: Ispitivanje Potreba za Stručnim Usavršavnjem [Report: Examining the Needs for Professional Improvement], Institute for Improvement of Education, Belgrade.
[16] Kitchen, H. et al. (2017), Romania 2017, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264274051-en.
[14] MoESTD (2019), Pravilnik o Vrednovanju Kvaliteta Rada Ustanove [Rulebook on Evaluation of the Quality of Work of the Institution], Ministry of Education, Science and Technological Development, Belgrade.
[11] MoESTD (2018), OECD Review of Evaluation and Assessment: Country Background Report for Serbia, Ministry of Education, Science and Technological Development.
[18] NCSL (n.d.), School Self-evaluation: A Reflection and Planning Guide for School Leaders, National College for School Leadership, Nottingham, http://www.ncsl.org.uk (accessed on 20 June 2019).
[32] NSLE (n.d.), The National School of Leadership in Education, http://en.solazaravnatelje.si/Activities.html?#Mentoring (accessed on 20 June 2019).
[22] Nusche, D. et al. (2012), OECD Reviews of Evaluation and Assessment in Education: New Zealand 2011, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264116917-en.
[40] OECD (2017), The Funding of School Education: Connecting Resources and Learning, OECD Reviews of School Resources, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264276147-en.
[20] OECD (2016), Low-Performing Students: Why They Fall Behind and How To Help Them Succeed, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264250246-en.
[3] OECD (2016), PISA 2015 Results (Volume II): Policies and Practices for Successful Schools, PISA, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264267510-en.
[2] OECD (2015), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/eag-2015-en.
[36] OECD (2015), Schooling Redesigned: Towards Innovative Learning Systems, Educational Research and Innovation, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264245914-en.
[8] OECD (2014), TALIS 2013 Results: An International Perspective on Teaching and Learning, TALIS, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264196261-en.
[13] OECD (2013), PISA 2012 Results: What Makes Schools Successful (Volume IV): Resources, Policies and Practices, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264201156-en.
[1] OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264190658-en.
[23] Pont, B., D. Nusche and H. Moorman (2008), Improving School Leadership, Volume 1: Policy and Practice, OECD Publishing, Paris, https://doi.org/10.1787/9789264044715-en.
[24] Riley, K. and J. MacBeath (2000), Evaluating School Performance Tools and Approaches: Putting Self-Evaluation in Place, World Bank, Washington, DC, http://documents.worldbank.org/curated/en/414941468765037597/Evaluating-school-performance-tools-and-approaches-putting-self-evaluation-in-place (accessed on 20 June 2019).
[15] Romanian Agency for Quality Assurance in Pre-University Education (n.d.), Country Profile Romania, The Standing Conference of Inspectors, http://www.sici-inspectorates.eu/getattachment/1cbc0561-c91b-4c71-a71c-6b9369ecad61 (accessed on 7 June 2019).
[5] Rudduck, J. (2007), “Student voice, student engagement, and school reform”, in International Handbook of Student Experience in Elementary and Secondary School, Springer Netherlands, Dordrecht, http://dx.doi.org/10.1007/1-4020-3367-2_23.
[7] Schleicher, A. (2015), Schools for 21st-Century Learners: Strong Leaders, Confident Teachers, Innovative Approaches, International Summit on the Teaching Profession, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264231191-en.
[30] Schratz, M. and M. Hartmann (2009), Innovation an Schulen durch Professionalisierung von Führungskräften [Innovation in Schools Professionalisation of Leaders], Leadership Academy, Innsbruck, https://www.leadershipacademy.at/downloads/LEA_Kurzfassung_Studie.pdf (accessed on 20 June 2019).
[4] SICI (2003), Effective School Self-Evaluation, Standing International Conference of Inspectorates, Belfast.
[37] UNICEF (n.d.), Dare to Share: Empowering Teachers to be the Change in the Classroom, UNICEF.
[39] World Bank (2012), Serbia School Finance: SABER Country Report 2012, World Bank, Washington, DC, http://hdl.handle.net/10986/17672 (accessed on 20 June 2019).
Note
← 1. Serbia did not participate in the last cycle of TALIS in 2018.