OECD Reviews of Evaluation and Assessment in Education: Bosnia and Herzegovina

2. Supporting student learning through assessment

Abstract

Education systems in Bosnia and Herzegovina’s (BiH) have taken steps to introduce new competence-based curricula. However, assessment policies and practices in many of the competent education authorities covered by this review do not yet reflect the types of instructional practices that support student learning. For example, teachers often lack resources and support to implement formative assessments in their classrooms and the use of standardised assessment in the country is very limited. This chapter sets out recommendations for how education authorities can address these challenges and use student assessment as a tool for helping all young people to develop core competences.

Introduction

Student assessment helps focus attention towards what matters most in education: what learners know, what they are capable of doing and how they can improve. Through assessment, educators can identify specific learning needs before they develop into more serious obstacles, and help students make informed decisions about the next step in their education. Quality assessment policies and practices are especially important in light of the school disruption caused by the COVID‑19 pandemic, as they can help teachers and education systems as a whole address learning losses and other concerns. While Bosnia and Herzegovina’s (BiH) education systems have taken steps to introduce new competence-based curricula, assessment policies and practices in many of the competent education authorities do not yet reflect the types of instructional practices that support student learning. This is partly related to the lack of resources and supports for teachers to develop their classroom assessment literacy, as well as the very limited use of standardised assessment in the country and, with this, the absence of clear expectations and reliable measures of achievement. Addressing these challenges will be crucial not only to support curricula implementation but also to improve education outcomes and skills in the country by focusing actors on the types of inclusive, student-centred pedagogies that help students learn.

Student assessment in Bosnia and Herzegovina

Along with the roll out of the Common Core Curriculum Based on Learning Outcomes (CCC) and an overall emphasis on competence-based education, most of the competent education authorities covered by this review have introduced some changes to student assessment policies, such as the use of qualitative descriptors to accompany quantitative scores or diagnostic “check-in” tests to establish an initial benchmark of student performance. However, these policy reforms have not led to real changes in classroom practices, in part because they have not been accompanied by tools and support for teachers. As a result, teachers’ classroom assessments remain off-balance, with a strong emphasis on simple, summative forms of assessment and a lack of attention to formative methods or the assessment of more complex, higher-order competences (Box 2.1). Moreover, while Republika Srpska (RS) and some cantons have established external measures of student learning, the majority face capacity and resource constraints that prevent them from developing and using standardised assessments to improve the reliability of teachers’ marking and set clear expectations for students. These factors, coupled with limited co‑operation between administrative units have prevented competent education authorities from developing standardised assessment practices at the state level (Table 2.1). Lack of political co-operation has also prevented BiH from securing regular participation in international assessments. Addressing these challenges and leveraging the educational value of assessment will be key to raising student learning outcomes across the country.

Box 2.1. Purposes of assessment

Summative assessment – assessment of learning summarises learning that has taken place in order to record, mark or certify achievements.

Formative assessment – assessment for learning identifies aspects of learning as they are still developing in order to shape instruction and improve subsequent learning. Formative assessment frequently takes place in the absence of marking. For example, a teacher might ask students questions at the end of the lesson to collect information on how far students have understood the content and use the information to plan future teaching.

Source: (OECD, 2013[1]), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, https://dx.doi.org/10.1787/9789264190658-en

Table 2.1. Overview of student assessment in the administrative units of Bosnia and Herzegovina

Type of assessment	Reference document	Body responsible	Process	Frequency	Use
Classroom assessment	A) Common core based on learning outcomes (state-level) B) Individual curricula developed by competent education authorities (entity and canton level)	Teachers	Specific processes vary by competent education authority but generally consist of: - descriptive marks for early years of primary - numerical marks (on a scale of 1 to 5) for older cohorts - marks for student behaviour.	Typically occurs throughout the semester, with a final mark at the end of each term; depends on competent education authority	- evaluate student achievement (mostly summative) - partly used for selection into secondary and tertiary* education.
Examinations	Individual curricula developed by competent education authorities (entity and canton level)	Competent education authorities	- Tuzla and Sarajevo cantons have external examinations. - RS is piloting an external exam in Grade 9	Varies; typically occur at the end of primary or secondary school	- certification of completion of primary or secondary school. - sometimes used as one of the criteria for selection into higher levels of education
System-wide external assessments	No state-level system-wide external assessments exist at the BiH state level
	Expert instruction defines subject areas and assessment procedure	Republican Pedagogical Institute of RS	RS entity has an external assessment of primary students	Annual	Assessments are marked by a commission of subject teachers; results are reported at the school level, not for individual students
International assessment	PISA framework	OECD	OECD Programme for International Student Assessment (PISA): mathematics, science and reading	Every three years - first and only participation in 2018**	APOSO implements assessment and develops state-level reports to inform policy
	TIMSS framework	International Association for Evaluation of Educational Achievement (IEA)	IEA Trends in International Mathematics and Science Study (TIMSS): mathematics and science	Every four years - first and only participation for Grade 8 was 2007 and for Grade 4 was 2019**

Note: APOSO: Agency for Pre‑school, Primary and Secondary Education

*Tertiary institutions in BiH often administer a university-led entrance exam to select students into their study programmes, the results of these exams are typically considered alongside a student’s classroom marks.

**BiH will not participate in upcoming rounds of PISA (2022) and TIMSS (2023).

Source: (BiH, 2021[2]), Country Background Report for the OECD Review of Evaluation and Assessment in Education: Bosnia and Herzegovina, Unpublished.

Overall objectives and policy framework

High-performing education systems successfully align curriculum expectations, subject and performance criteria, as well as desired learning outcomes (Darling-Hammond and Wentworth, 2010[3]). Having a set of clear expected outcomes, expressed through qualifications frameworks, curricula and learning standards can help establish an education culture whereby teaching and assessment supports student learning. BiH has taken important steps to establish a competence-based curriculum framework at the state level (2018); however, the extent to which competent education authorities have integrated and aligned their local curricula and expectations for learning to this framework varies. Moreover, while it is positive that competent education authorities have rulebooks to guide student assessment policy, many rulebooks do not provide for a balanced set of assessment instruments and remain very general, making it difficult for teachers to understand how to teach and assess competences. This context creates considerable gaps between the intended curriculum and the taught curriculum across BiH, signalling a need for greater co‑ordination and implementation support within and across the country’s administrative units.

A framework curriculum based on learning outcomes was recently adopted at the BiH-level

After an intensive six-year consultation led by the Agency for Pre‑school, Primary and Secondary Education (APOSO), and with support from international partners, administrative units in BiH collectively developed the Common Core Curriculum Based on Learning Outcomes (CCC) in 2018 and now use the document as reference to build their own curricula. The CCC represents a significant shift from a content-based curriculum towards one that focuses on developing the 21st Century competences that students need for success in work and life. The competences set out in the CCC build on the European Parliament and the Council of Europe’s Recommendation on Key Competences for Lifelong Learning (updated 2018), with the addition of creativity-productivity and physical and health competences. In addition to the ten key competences, the CCC defines eight specific learning areas, for which APOSO has already started developing student achievement standards in key grades covering these areas (see Table 2.2). Other decentralised education systems in OECD countries have taken a similar approach to balancing local curricula and common learning goals. For example, in Switzerland, the Conference of Cantonal Ministers of Education sets learning goals for compulsory schooling across four fundamental subject areas in key grade levels (EDK, n.d.[4]). While BiH has not yet developed achievement standards for all subjects and grade levels, doing so is a top priority for APOSO since these materials can help education ministries and other actors design interventions to improve teaching and learning.

Table 2.2. Learning areas where BiH has developed student achievement standards

Learning area	Subject	Grade 3 (age 8‑9)	Grade 6 (age  11‑12)	Grade 9 (age 14‑15)	End of secondary (age 18 or 19)
1. Language and Communication	Mother tongue	●	●	●	●
	Foreign language
3. Mathematics		●	●	●	●
4. Natural Science	Physics	-	-	●	●
	Chemistry	-	-	●	●
	Biology	-	●	●	●
	Geography	-	●	●	●
	Nature, society and my environment	●	-	-	-
5. Social Science and Humanities	History	-	●	●	●
	Civic education
7. Technics and Information Technology
8. Art
9. Physical and Health Education*
10. Cross-Curricular area

Note: APOSO has also developed standards for pre‑school education, which are not addressed in this chapter. Cells marked with “– “are not covered at the particular grade level. *At the time of this review APOSO was in the process of developing achievement standards for this area.

Source: (APOSO, 2021[5]), Standards of student achievement, https://aposo.gov.ba/hr/standardi-ucenickih-postignuca-poj/ (accessed on 3 December 2021).

Competent education authorities develop their own curricula but the extent to which these align with the Common Core Curriculum Based on Learning Outcomes varies

Competent education authorities in BiH have full autonomy to determine school curricula in their jurisdictions, making local leadership central to improving instructional practice. There is broad consensus among stakeholders covered by this review that school curricula are outdated and overemphasise the memorisation of academic knowledge at the expense of applied learning and higher-order cognitive functions. The overall goals of the CCC are, therefore, broadly shared, and many competent education authorities and their pedagogical institutes were involved in developing the state-level CCC document. However, despite a common interest to introduce more competence-based approaches to education that focus on learning outcomes, the process of changing the culture of instructions at the local level has been slow and uneven.

A recent study of legislation across administrative units in BiH revealed that the key competences are often either omitted or included merely as a declaratory statement in official documents (OSCE, 2021[6]). When competent education authorities’ curricula do mention learning outcomes, these frequently focus on memorising facts rather than demonstrating higher-level cognitive and behavioural competences (ibid). Many curricula in BiH also lack links to a clear vision for the education system and teachers have little support or incentive to adopt the more student-centred teaching and assessment practices that underpin the CCC (see Chapters 3 and 5). This disconnect between the intended goals of the CCC and the implemented curricula across BiH continues to impede improvements in the quality of teaching and learning.

Student assessment rulebooks exist at the entity, canton and district levels but rarely mention formative assessment and do not link to learning standards

Most competent education authorities in BiH have specific rulebooks that address student assessment policy. These rulebooks generally set out different ways to measure student learning, mainly in the classroom (e.g. through oral, written, practical assessments), the frequency and timing that assessments should be administered, as well as procedures for marking and reporting feedback to students and their parents. Positively, many of the rulebooks include references to key assessment principles, such as the importance of using a mix of numerical and descriptive marking, as well as conducting “initial checks” or diagnostic tests at the beginning of the school year. However, the start-of-year diagnostic tests are not applied consistently and very few administrative units specify how teachers’ assessments might be standardised (e.g. through moderation practices, the use of external instruments or resources such as grading criteria or marked examples of student work) to promote more reliable and consistent judgements of student achievement. These challenges are further exacerbated by the fact that in general, assessment rulebooks do not explain how assessment practices can help reinforce curricula and standards of student achievements (OSCE, 2020[7]). As a result, it is difficult for teachers and other stakeholders (e.g. students and parents) to expand their understanding of assessment beyond grading and formal qualifications, that is, they are not aware of how a fuller range of assessment practices can support learning and the mastery of key competences.

While some of the practices set out in the rulebooks have the potential to support student learning, most do not clearly explain the concept of formative assessment. In fact, only the RS entity rulebook (developed recently to guide assessment during COVID‑19) and some of the Brčko District’s rulebooks explicitly refer to formative assessment as a tool for improving student learning (see Table 2.3). This differs from most EU countries and other Western Balkan systems, which have policy documents that clearly distinguish between assessment of learning (i.e. summative) and assessment for learning (i.e. formative). Many of these systems have also developed guidelines and support materials to help teachers implement formative assessments. North Macedonia, for example, created a Formative Assessment Manual in 2015 to guide professional development and encourage teachers to use formative assessment by explaining how to diagnose student achievement compared to expected learning outcomes and use this information to help differentiate and tailor instruction to individual student needs (Raleva, 2021[8]; OECD, 2019[9]). No such support materials exist in BiH currently.

Table 2.3. Features of student assessment rulebooks in select BiH administrative units

Administrative unit	Key guiding documents and legal regulations on student assessment	Formative assessment	External instruments	Initial checks	Reporting requirements
FBiH (entity)	Responsibility over student assessment is not within the competence of FBiH entity ministry; it is the responsibility of cantonal ministries
Sarajevo Canton	Ordinance on monitoring, evaluation and assessment of primary and secondary schools (2018)			●
	Ordinance on the administration of the external exam at end of primary (2018)		●		●
	COVID‑19 Guidelines for digital development educational content and monitoring, evaluation and student assessment (2020)
West Herzegovina Canton	Rulebook on the manner of monitoring and evaluating students in primary and secondary school (2018)			●
Central Bosnia Canton	Rulebook on evaluation and evaluation of students in secondary school (2012)			●	●
RS (entity)	Rules on evaluation of students in primary school (2012)			●
	Rulebook on evaluation of students in teaching and taking high school exams (2019)				●
	On monitoring, evaluation and evaluation of students during distance teaching (2020)	●			●
Brčko District	Rulebook on monitoring and evaluating secondary students (2008; amended 2010)	●		●
	Rulebook on monitoring and evaluating and descriptive evaluation in primary school (2010)			●	●
	Rulebook on evaluation of vocational secondary students (2010)	●

Source: Student assessment rulebooks and legislation provided in the background report and by competent education authorities from BiH to OECD review team.

Classroom assessment

Ongoing and regular identification and interpretation of evidence about student learning is a key component of effective instruction (Black and Wiliam, 2018[10]). In BiH, however, classroom assessment is often viewed by teachers, students and society as a summative validation and selection exercise, rather than an integrated part of the learning process. While competent education authorities and pedagogical institutes provide rulebooks on student assessment policy, these documents generally lack balance between formative and summative role of assessments (see Box 2.1). Moreover, teachers across the country lack adequate support and training to implement valid and reliable assessments, as well as how to use assessments formatively to give students feedback on their learning and inform their own teaching practices. These findings suggest a clear need to strengthen the educational value of classroom assessments in BiH, which could have a significant impact on improving student attitudes to learning and their outcomes.

Students in the early years of primary school receive descriptive marks, while older students generally receive numerical marks for each subject and overall performance

While there are some variations in how teachers are expected to conduct classroom assessment across BiH administrative units, policies are generally similar and aim to monitor, check and record student learning. For example, all three cantons covered by this review, as well as the RS entity, encourage the use of diagnostic tests at the beginning of the academic year to determine students’ entrance level of performance and identify areas for support. Some of the rulebooks examined in this review also set out requirements for the frequency and timeline of reporting results from classroom assessments. For example, continuous classroom assessments and final examinations (at the class or school level) in each subject are commonly used to calculate grades for the end of term and end of school year (i.e. grade point average, GPA). In line with other Western Balkan systems, including Serbia, Albania and North Macedonia, student grades in BiH are usually expressed as descriptive marks for younger students and numerical scores using a five-point scale for older groups of students (see Table 2.4) (OECD, 2019[9]; Maghnouj et al., 2020[11]; Maghnouj et al., 2020[12]).

It is positive that numerical grades for older students are associated with qualitative descriptors (e.g. insufficient to excellent), as this helps contextualise performance levels. Some education authorities go further to provide definitions for each descriptor. This is the case in Central Bosnia Canton, where the assessment rulebook defines the lowest grade “1” as “insufficient,” for students who do not acquire basic knowledge and skills and do not reach a satisfactory level of achievement of standards (Official Gazette Central Bosnia Canton, 2012[13]). However, many competent education authorities, including the RS entity and Sarajevo Canton, have not provided definitions for the descriptors associated with student numerical grades, leaving teachers, students and parents to interpret the meaning of different grades, such as “good” (3) or “very good” (4). Moreover, the language typically used for descriptors in BiH is at odds with competence-based approaches to education, which situate learning on a continuum (e.g. accomplished, developing or emerging), rather than a summative judgement of achievement. A more balanced understanding of assessment is further hindered by the lack of subject- and task- specific guidance that teachers receive on how to provide feedback from assessments to help students progress.

Another feature of student assessment in BiH is that teachers across the country assess student behaviour at school. Behaviour grades are usually descriptive but their implications vary. While grades for behaviour do not affect the general success of students in the RS entity or the West Herzegovina Canton, they are included in the GPA of students in the Central Bosnia Canton. Traditionally, this practice of including behaviour as part of the GPA is a disciplinary measure (i.e. to penalise students for misbehaving in class). Most OECD countries have moved away from this exclusionary practice by separating grades for academic subjects from grades for classroom behaviour and participation (OECD, 2012[14]).

Table 2.4. Grading scales across BiH

Administrative unit	Grade	Type of marking
Cantons within FBiH	Grade 1 and first semester of Grade 2 in some cantons	Descriptive, according to learning outcomes set out in the curriculum
	Grades 2-9 (primary) and 1-4 (secondary)*	Numerical: 1 (insufficient) to 5 (excellent)
RS	Grades 1-3	Descriptive: a) very successful; b) successful; or c) participates
	Grades 4-9 (primary) and 1-4 (secondary)	Numerical: 1 (fair) to 5 (excellent)
Brčko District	Grades 1-3	Descriptive
	Grades 4-12	Numerical: 1 (insufficient) to 5 (excellent)

Note: *Pupils with “mild” or “severe” learning difficulties within FBiH cantons are assessed as: a) Outstanding; b) Good or c) Satisfactory.

Source: (BiH, 2021[2]), Country Background Report for the OECD Review of Evaluation and Assessment in Education: Bosnia and Herzegovina, Unpublished.

Teachers determine their own criteria for student assessment

Rulebooks on student assessment in BiH typically set out a range of techniques that teachers and schools should use to evaluate students, including oral and written tests, project-based assessments, notebook and homework checks or practical exercises. However, teachers and schools are responsible for developing specific assessment criteria to measure student achievement and not all administrative units have a clear set of learning standards to benchmark student performance or use the learning standards of student achievements developed by APOSO. For example, the RS entity curriculum defines expected learning outcomes by subject and grade level, but Brčko District, Central Bosnia Canton and West Herzegovina Canton have no criteria for what students are expected to achieve and Sarajevo Canton is developing standards of student achievements for some subject areas (BiH, 2021[2]).

Without specific assessment criteria or marked exemplars to help determine a student’s level of achievement – and in some cases the outright lack of common learning standards – many teachers in BiH must rely on their own interpretations of the curriculum and knowledge of assessment to determine student’s strengths and areas for improvement. Interviews undertaken by the OECD review team revealed that the extent to which teachers of the same subject or grade levels collaborate to develop common assessment criteria within their school depends largely on the initiative of individual teachers or school leaders and is not systematic (see Chapters 3 and 4). In one school, for example, teachers said it is possible for students in one class to take a 5-question test in 15 minutes, while their peers with a different teacher for the same subject and grade level take a 15-question test within 45 minutes. As a result, student grades are not comparable within and across schools in BiH.

End-of-year grades typically determine student selection into secondary schools

Secondary education in BiH is provided by separate institutions that offer either general, technical, vocational or arts and religious study programmes (see Chapter 1). Selection into these different pathways is largely based on academic performance. While many OECD countries with differentiated secondary systems use academic results to inform placement, the process of selection in BiH stands out in several respects. In particular, it appears that education authorities rely more strongly on achievement data than on other criteria, such as student choice or location of residence. Data from PISA 2018 reveals that in BiH, 73% of secondary students (ISCED 3) attend a school where admission is contingent upon academic performance, compared to only 45% across OECD countries (OECD, 2020[15]).

The stakes for secondary placements in BiH are particularly high, because of the lack of permeability between educational tracks in BiH (i.e. it is difficult for students to move from one educational orientation to another) and the low quality of vocational options. Moreover, the academic performance data used for selection is largely from school-based assessments. While there is considerable variation across the OECD, when teachers’ judgements are used as the primary source of information to determine student pathways, these judgements need to be fair and extremely reliable. In BiH, various factors signal that this is not the case: there are risks of grade inflation in classroom marks, with teachers lacking external benchmarks and under pressure from parents and students to provide grades that enable acceptance into their programme of choice. This context can lead to decisions that reflect student background more than ability. For example, data from PISA reveal that socio-economically disadvantaged students in BiH are around four times more likely to attend a VET secondary school (ISCED 3) than a general one (OECD, 2020[15]). After Serbia, this is the largest difference in the Western Balkans.

In some cases, education system in BiH use external measures to inform the allocation of students into different secondary schools (ISCED 3), which can help reduce the potential bias and inconsistences in decisions. For example, specialised schools (e.g. for arts and music) in West Herzegovina Canton administer entrance exams to select students into their programmes and in Sarajevo Canton, the ministry organises an external examination, which is one of the criteria (alongside GPA) used to select students into either vocational or general secondary schools (ISCED 3). The RS is also developing a similar pilot examination to help inform selection into secondary education (ISCED 3). However, these examples are mainly exceptions: most administrative units in the country draw on students’ classroom grades to make admissions decisions.

Teachers are very experienced in using summative classroom-based assessment; however, there is a need to balance this with other assessment purposes

Teachers in BiH have a strong understanding of summative assessment, or practices that summarise learning in order to record, mark or certify achievement (OECD, 2013[1]). This aligns with the traditional role of teachers in BiH, whereby instruction serves to transmit knowledge, rather than work with students to develop their abilities and interests. Importantly, these assessment practices are norm-based, meaning students are graded based on their performance relative to other students in the class. This is partly because curricula of competent education authorities are often content-oriented and do not provide specific criteria based on state-wide or jurisdictional achievement standards, leaving teachers to develop their own assessment criteria. As a result, there is little room for criterion-based assessments, whereby teachers assess students based upon their mastery of competences set out in the curriculum; independently of how other students in the class perform.

While the start-of-year diagnostic tests encouraged by many BiH competent education authorities has the potential to serve as a formative assessment tool, these are not consistently applied. Other countries that mandate diagnostic assessments (e.g. Bulgaria, Romania and Serbia) face similar challenges (Maghnouj et al., 2020[12]; Kitchen et al., 2017[16]) (Guthrie et al., forthcoming[17]) . Moreover, some of rulebooks define continuous assessment as “summative assessment done more often” or as practice for a final summative assessment, whereas in reality, continuous assessment can serve both summative and formative purposes (Muskin, 2017[18]). The general lack of understanding and support for using assessment to support student learning has led to resistance in some parts of the country where policymakers have tried to move towards more formative and competence-based approaches to education. For example, when descriptive marking was introduced for the early grades, authorities in Brčko District reduced the policy’s coverage by one grade level because of major pushback from teachers and parents.

Most assessment rulebooks covered by this review lack clear comparative definitions of summative and formative assessment. Shifting the BiH assessment culture from teacher judgements on performance to an exercise that supports student learning will require clarity on the main types and forms of assessment, as well as support for teachers and other stakeholders to understand concepts included in student assessment frameworks. Clarifying the distinctions between key concepts, such as summative and formative assessment, as well as the main types of assessment (normative, criterion and mixed) would be a useful way to support teachers in BiH who report needing more support and training to effectively use a wider range of assessment practices in their classroom teaching.

Teachers lack support and training to develop their assessment literacy

Aside from the assessment rulebooks, teachers in BiH receive little, if any, support on how to develop valid, reliable and age-appropriate assessments. For example, most competent education authorities or pedagogical institutes do not provide examples of marked student work or facilitate moderation of grading within or between schools; such resources and practices are important to support effective classroom assessment (OECD, 2013[1]). Professional learning on assessment is also limited in BiH and reportedly based on theory from textbooks rather than practical techniques that teachers can apply in their daily practice. These challenges are exacerbated by the lack of attention given to student assessment during initial teacher education (ITE) programmes in BiH (see Chapter 3).

Despite heterogeneity in the content and quality of ITE programmes, there is a consensus that new teachers in BiH do not receive adequate preparation in assessment. Moreover, it is rare, if not unheard of, for teacher candidates to engage in programme modules explicitly dedicated to formative assessment. In fact, only one teacher that spoke with the OECD review team reported receiving training in student assessment as part of a graduate studies programme. The only assessment resource mentioned to the OECD review team (other than the assessment rulebooks) were test writing guidelines a teacher had received during an ITE programme undertaken in a neighbouring Western Balkan country. The lack of attention to key concepts and theories behind a balanced assessment framework, as well as the lack of practical experience and support for using different types of assessment in the classroom, risks holding back efforts to introduce a more student-centred, competence-based curriculum and, through this, to improve student learning outcomes.

External assessments and examinations

At present, there is no standardised testing at the state-level in BiH as competent education authorities are responsible for determining student assessment policy. Eleven out of fourteen administrative units in BiH lack standardised data on student learning (Table 2.6). As a result, teachers’ classroom assessments are often the only source of information about student achievement, making it very difficult to measure learning objectively and reliably.

Broadly speaking, there are two main testing instruments governments can use to collect standardised evidence about learning outcomes (OECD, 2013[1]). The first is an external examination, which has formal consequences for students and usually serves to certify achievement or inform selection into higher levels of education. There are currently three education authorities within BiH that conduct external examinations: Tuzla Canton (at the end of basic and secondary school; ISCED 2 and 3, respectively); Sarajevo Canton (at the end of basic school; ISCED 2) and RS (a pilot exam at end of basic education; ISCED 2). The second standardised testing instrument is an external assessment, which can take place at the international, national or local level(s) but do not carry consequences for students. Instead, external assessments provide information on learning, principally for system monitoring purposes. Within BiH, the RS is the only jurisdiction that has a low-stakes external assessment (in Grade 5), although other administrative units have experimented with this type of test. Overall, the very limited use of external tests within administrative units and at the state-level (compared to other Western Balkan and OECD economies) represent missed opportunities for education authorities in BiH to support more competence-based approaches to student assessment (Table 2.5).

Table 2.5. External examinations and assessments in the Western Balkans and select OECD economies

	External assessment (levels at which the assessment is administered)	External examination (levels after which the examination is administered)		Body responsible for developing the assessment/examination
		Lower secondary (ISCED 2)	Upper secondary (ISCED 3)
Albania	Primary	●	●	Educational Services Centre (specialised central agency)
Austria	Lower secondary and Primary		●	Federal Institute for Education Research, Innovation and Development of the Austrian School System
Canada	Lower secondary		Varies	Central education council (Council of Ministers of Education, Canada)
Czech Republic	Upper secondary; Lower secondary and Primary		●	Central education authority or government (Ministry of Education); special institute of the Ministry of Education (Centre for the Evaluation of Educational Achievement)
Bosnia and Herzegovina		Varies* (only Sarajevo and Tuzla Cantons and RS entity)	Varies (only Tuzla Canton)	Pedagogical Institutes or equivalents of competent education authorities
Germany	Lower secondary and Primary	●	●	Institute for Educational Quality Improvement
Kosovo1	-	●	●	Minister of Education, Science, Technology and Innovation
Mexico	Upper secondary; Lower secondary; Primary; Pre‑primary	Varies	●	National Institute for Educational Assessment and Evaluation
Montenegro	-	●	●	Examination Centre
North Macedonia	Under development		●	The National Examination Centre
Serbia**	Under development	●	●	Serbia’s Institute for Education Quality and Evaluation
Switzerland	Lower secondary and Primary		●	The Swiss Conference of Cantonal Ministers of Education
United States	Upper secondary; Lower secondary; Primary		●	National Centre for Education Statistics

Notes: *Primary education = ISCED 1 (primary education in BiH); lower secondary education = ISCED 2 (primary or basic education in BiH); upper secondary education = ISCED 3 (secondary education in BiH).

**Serbia’s current examination at the end of upper secondary education is school-based, meaning it is developed, administered and marked in each school and is thus not standardised across the system. Serbia will introduce a new centralised examination starting in 2023/24. Policies in some economies differ according to internal governance structures.

Source: (OECD, 2015[19]) Education at a Glance 2015: OECD Indicators, http://dx.doi.org/10.1787/eag-2015-en; (OECD, 2020[15]) Education in the Western Balkans: Findings from PISA, https://doi.org/10.1787/764847ff-en; (OECD, 2013[1]), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, https://dx.doi.org/10.1787/9789264190658-en.

Participation in international assessments is limited compared to other European countries

Positively, BiH participated in the Programme for International Student Assessment (PISA) in 2018 and Trends in Mathematics and Science Study (TIMSS) in 2019, which provided the country with comparable data on learning outcomes for the first time in over a decade (see Chapter 5). Results from these international assessments provide an important benchmark for education in BiH, revealing that while students tend to perform similar to their peers in other Western Balkan economies, overall performance remains below the European Union (EU) and OECD averages and large shares of students do not master basic competences (see Chapter 1). Participation in international assessments also gives BiH stakeholders exposure to competence-based assessments and experience analysing the data they produce, which can aide competent education authorities in implementing competence-based curricula and using assessment to support learning. In BiH, international assessments also fill an important information gap given the country’s decentralised governance structure and the lack of mechanisms to collect standardised learning data across all administrative units. For example, international assessments can provide comparable cross-country information on learning disparities between school types and student gender, language group and socio-economic background. Despite this, BiH’s participation in international assessments is very limited compared to other European countries that now have decades of trend data available. Moreover, BiH recently missed the deadline to implement in PISA 2022, preventing education stakeholders from enjoying the benefits associated with regular participation in international assessments.

Despite some experimentation with using external standardised assessments, the RS entity is the only jurisdiction that currently has an external standardised assessment

Among the jurisdictions covered in this review, there has been some experimentation with using external standardised assessments that do not have consequences for students. In 2011, for example, APOSO implemented a sample-based external assessment to students in third and sixth grades from across the country to measure achievement standards in local languages, science and mathematics (APOSO, 2021[5]). In 2013, another sample-based external assessment was administered to measure local languages and mathematics at the end of primary education (ISCED 2). However, these instruments were developed prior to the Common Core Curriculum and were never updated or used after their initial implementation. This is mainly because APOSO did not receive funding to continue administering the assessments.

In Sarajevo Canton, authorities introduced a pilot assessment in 2018 but this never materialised into a regular standardised assessment, partly because of student absenteeism and lack of reliable marking procedures (BiH, 2021[2]). At present, the RS entity is the only administrative unit in BiH with an external assessment that generates data on the learning outcomes of Grade 5 students. This sample-based assessment is administered annually and primarily supports RS as a system monitoring tool. It also generates school-level results. While external assessments cannot on their own provide a full measure for school quality, many countries use these results alongside other contextualised information (e.g. socio-economic background) to more equitably and meaningfully benchmark performance. Notably, the RS external assessment does not seem to provide feedback to inform teaching and learning practices, limiting its ability to have a positive impact on student outcomes.

Some school networks use external standardised assessments developed by private international companies

While it is not common practice, some individual schools in BiH purchase external assessment instruments to help regulate the quality of classroom assessments within their school networks. These assessment tools are often produced by private international companies (e.g. the Cambridge Progression Tests from the United Kingdom or International Baccalaureate to collect valid and reliable information about student learning to inform pedagogy and improve system management. This demonstrates a clear demand for external learning assessments among school leaders, teachers and parents within BiH. However, such instruments appear to be only available to international and/or elite schools that typically serve the country’s most advantaged students.

A small number of administrative units in BiH administer external examinations

A small number of education authorities in BiH use standardised examinations, or standardised tests that have consequences for students. These exams provide their respective jurisdictions with valuable information about learning that can complement teachers’ professional judgements. For example, Sarajevo Canton administers an external examination for students in Grade 9, the results of which translate to “points” that students use alongside their GPA to compete for places in secondary schools (ISECD 3). The RS entity has a similar external examination that the entity introduced as a pilot in 2018. Known as the RS Matura, this exam certifies learning at the end of basic education (Grade 9; ISCED 2) and helps allocate students to secondary schools (ISECD 3). Both of these exams were postponed in 2020/21 because of the COVID‑19 pandemic and the RS Matura will only become mandatory for all students starting in the 2022/23 school year. Tuzla Canton, which this review did not study in-depth, administers the country’s only external examination to certify completion of secondary education (ISCED 3). None of the other administrative units in BiH have standardised external examinations.

The use of external examinations in BiH differs from many OECD and EU countries. In particular, external exams at the end of basic school (e.g. in Sarajevo Canton and RS) have become less common internationally as policymakers seek to remove barriers to progression and reduce early tracking (Maghnouj et al., 2020[11]). The general lack of external examinations at the end of secondary education (ISCED 3) - except for Tuzla Canton - is another notable difference in the way that BiH education systems use examinations. Most OECD and EU countries use external exams at this level to help incentivise learning, certify student achievement at the end of formal schooling and/or support more equitable progression to tertiary education (OECD, 2015[19]). Moreover, BiH lacks the co-ordination or moderation systems that exist in exam and qualifications systems in other federal countries. For example, in the United States, high school graduation diplomas are granted locally but higher education institutions often use standardised examinations, such as the Scholastic Assessment Test (SAT) and the ACT2, as part of their entry criteria for admissions. While requirements vary, several Australian states and territories, as well as the majority of German Länder (i.e. states) have also introduced external examinations and/or moderation procedures to ensure the rigour and comparability of secondary qualifications (OECD, 2020[20]; OECD, 2011[21]).

Table 2.6. External standardised assessment and examinations developed within BiH

Administrative unit	Type of instrument	Eligibility	Item development	Subjects	Purpose
Sarajevo Canton	Exam	All students in Grade 9 (est. 2012/13)	Pedagogical Institute	1) Bosnian, Croatian, Serbian Language and Literature; 2) Mathematics; and 3) First Foreign Language	Results account for 30 points to qualify for enrolment in secondary school, other points are based on overall classroom marks and achievement in specific subjects relevant to the type of secondary school to which they are applying
Sarajevo Canton	Assessment (pilot 2018)*		Pedagogical Institute		To evaluate student knowledge
Tuzla Canton**	Exam	All students at end of lower and upper secondary education (i.e. after ISCED 2 and ISCED 3; referred to as primary and secondary education in BiH)
West Herzegovina Canton	No external standardised assessments or examinations
Central Bosnian Canton	No external standardised assessments or examinations
Brčko District	No external standardised assessments or examinations
Republika Srpska	Assessment	All students in Grade 5 (est. 2013/14)	Republican Pedagogical Institute	1) Serbian language and 2) Mathematics	- to determine achievement of learning outcomes - to support teachers in delivering the curricula.
Republika Srpska	Exam (pilot)	All students in Grade 9 (piloted in 2018/19; to be implemented in 2022/23)	Republican Pedagogical Institute	1) Serbian language and 2) Mathematics

Notes: *The 2018 Sarajevo pilot never materialised into a regular standardised assessment, partly because of student absenteeism and lack of objectivity in teachers responsible for marking the assessments.

**Tuzla Canton was not covered in the sample of this review but is included in the table because it is the only other canton besides Sarajevo that has implemented a standardised examination.

Source: (BiH, 2021[2]), Country Background Report for the OECD Review of Evaluation and Assessment in Education: Bosnia and Herzegovina, Unpublished.

There are concerns with external examinations in jurisdictions where they exist

Stakeholders reported to the OECD review team that external examination instruments that exist within certain jurisdictions face several challenges, including lack of alignment with competence-based curricula (e.g. they remain content based and focus on reproducing knowledge rather than measuring higher-order competences), and weak marking procedures. These challenges undermine overall trust, as well as the validity and reliability of the exam instruments, which is crucial since these tests have high stakes for students. They also present a risk that the exams will have a negative backwash effect on their respective education systems, since teachers may continue using more traditional methods to teach and assess students based on the current exam instruments, rather than helping students develop the higher-order and transversal competences that students need for success in the 21^st Century.

Most administrative units lack the capacity and resources to develop their own external standardised assessments

Several of the education authorities that participated in this review cited challenges related to staff capacity, limited funding and lack of political will as the main barriers to developing and implementing standardised testing instruments in their jurisdictions. Considering the costs and resources needed to implement large-scale standardised tests, BiH education systems could increase efficiencies and the quality of testing instruments by collaborating to develop standardised assessments. Sharing the costs of standardised testing would be beneficial to smaller jurisdictions, like Brčko District, which would likely struggle to develop and implement such instruments independently. It would benefit other jurisdictions as well by strengthening the quality and relevance of local standardised assessments. Other countries have raised financial and technical resources to support standardised assessment by leveraging support from international and donor agencies. For example, an NGO in North Macedonia implemented an assessment of learning outcomes in the early grades in 2016 (OECD, 2019[9]) and beyond the Western Balkans, in Kazakhstan, the National Testing Centre received support from the World Bank to improve the technical infrastructure for external assessment (OECD, 2020[22]). Despite multiple offers in the last decade, BiH officials have not found an effective arrangement that would allow them to use support from donors to establish external assessments or examinations at the state or local levels.

Despite several attempts BiH has not yet developed a state-level external examination of student learning

External examinations indicate student achievement, can help motivate students and arguably provide a fairer basis for taking decisions when opportunities are constrained, especially in contexts dealing with challenges related to grade inflation (OECD, 2013[1]). In the past two decades, there have been several attempts to introduce a common external examination, or standardised Matura, at the end of secondary education (ISCED 3) across BiH. However, these efforts have been unsuccessful, mainly because of a lack of political collaboration and resource constraints. This situation has implications for the education sector, as teachers face pressure to inflate student grades and there is no way for students to objectively benchmark their learning. Some of the previous and ongoing attempts to establish a BiH Matura include:

• 2005 – EU Project on the Reform of General Education in BiH. As part of a broad EU project to support education reform in BiH, the Ministry of Civil Affairs worked with EU officials to develop a “Framework Matura.” The document set out guidelines for implementing a Matura exam, standards for proficiency in mother tongue language and mathematics, as well as standards for graduation from secondary school (ISCED 3) at the state-level. The Framework Matura was shared with competent education authorities for their consideration and adoption; however, there was a lack of consensus on the terms of the instrument and BiH never implemented this instrument.

• 2011 – EU Project on Strengthening APOSO’s Institutional Capacity. In 2011 APOSO worked with a Slovenian consortium consisting of the State Examination Centre and the Institute for Education and the School for Principals to develop guidelines for establishing a BiH examination at the end of upper secondary education (ISCED 3, secondary education in BiH) (Slovenian Expert Group, 2011[23]). The guidelines built on Slovenia’s nearly 20 years of experience implementing its own State Matura and included many features of international best practice. Namely, setting up a working group to determine the purpose of the exam, steps for developing the instrument and how officials might adapt the general Matura to assess the competences of students enrolled in vocational education and training (VET) programmes. The guidance document also included considerations specific to the BiH context, such as establishing regional assessment centres to help develop the Matura, with co-ordination led by APOSO. However, some competent education authorities argued this arrangement would undermine their autonomy to set graduation requirements with their jurisdiction. Since establishing a BiH State Matura depends exclusively on the agreement of competent education authorities, the lack of political support prevented this interested parties from developing a formal BiH Matura instrument.

• 2014‑2017 "Development of the Qualifications Framework for General Education" project developed under the European Union’s Instrument for Pre-Accession Assistance. This initiative attempted to establish standards and moderation procedures for an external Matura in BiH. It also set out a roadmap for implementing the examination, with cost estimates and the required technical capacity to conduct this activity. While the project led to the creation of guidelines on teaching and assessment practices to promote better student learning outcomes, it did not lead to an external BiH Matura. Current and ongoing projects under the Instrument for Pre-Accession Assistance could represent yet another important opportunity for competent education authorities to collaborate in developing an external Matura in BiH.

Agencies and actors with responsibilities for student assessment

Capacity to develop and support student assessment policies across BiH is very weak. The only state-level agency with real technical experience in this area is APOSO, which has an over-stretched mandate and faces significant capacity challenges. Moreover, APOSO is primarily an implementation, standard-setting and evaluation body that operates independently but in co-operation with competent education authorities. This context results in education authorities and their pedagogical institutes or their equivalents often trying to provide their own tools to support student assessment. While some of the larger administrative units have the capacity to do this successfully, some lack the resources and expertise needed to develop the range of materials and trainings that can support high-quality student assessment.

The Agency for Pre-Primary, Primary and Secondary Education faces capacity limitations that prevent it from supporting more comprehensive student assessment

APOSO is a state-level technical agency with a mandate to “collect, consolidate, process and publish” evidence on the quality of BiH education systems. Among its various activities, the agency is responsible for developing and conducting extensive surveys and evaluations, such as managing BiH’s participation in large-scale international student assessments (see Chapter 5). As an expert body, APOSO could be well placed to consolidate expertise in the area of student assessment and examinations. However, the agency faces major human and financial capacity constraints. For example, around 25 employees work across APOSO’s three branches (located in Banja Luka, Mostar and Sarajevo), none of which are full-time information and communications technology (ICT) experts or psychometricians. Reportedly, only three APOSO staff member have considerable experience conducting large-scale standardised assessments.

To supplement the agency’s expertise and experience in the area of evaluation and assessment, APOSO sometimes contracts external experts with support from the international donor community. Drawing on external expertise can help mobilise and strengthen assessment capacity. However, to support more comprehensive student assessment frameworks across BiH, APOSO needs adequate and sustainable resources to develop its institutional memory, as well as its internal expertise in relevant fields.

Competent education authorities define student assessment policies but few have capacity to support their implementation

Competent education authorities, alongside their respective pedagogical institutes, are responsible for defining curricula, learning standards and student assessment policies. However, these actors generally have very limited capacity to support quality classroom assessments and develop external tests that can support education goals in their jurisdictions. For example, Brčko District has not updated its student assessment rulebook since 2010, leaving stakeholders with outdated assessment policies that do not reflect a competence-based approach to education. Moreover, competent education authorities covered by this review do not produce resources to help teachers and schools navigate their curriculum and assessment rulebooks, hindering the implementation of more formative assessment policies like diagnostic assessments.

Policy issues

Despite efforts at the state-level and among competent education authorities, student assessment policies and practices in BiH remain weak and fragmented compared to other Western Balkan economies. While this is partly caused by the lack of co‑operation at the state-level, there are major gaps in terms of the assessment tools available to help measure student learning, improve instruction and strengthen the overall quality and equity of education at the level of administrative units and the state. With few exceptions, teachers have no support to create authentic and valid assessment tasks, nor guidance on how to use results to inform instruction and provide feedback in a way that can help students progress in their learning. In many cases, they also lack clear learning expectations for many subjects and grade levels, and must establish their own assessment criteria without any structured moderation processes or reliable benchmarks. The lack of shared assessment criteria within administrative units creates opportunities for parents and caregivers to pressure teachers to increase student grades, contributing to grade inflation, undermining the rigour of certifications and diminishing the fairness of selection into higher levels of education.

These challenges are exacerbated by the fact that most education systems in BiH lack reliable external data on student learning. And despite several previous attempts to develop a standardised test at the state level or within some CEA’s, the grades students receive from their teachers continue to play a major role in determining their future opportunities in most parts of the country. To use assessment as a tool for supporting student learning, the country’s administrative units will need to work collaboratively with the teaching profession, teacher education providers and schools to shift the culture of assessment in BiH towards one that provides constructive feedback that can help each student to develop the competences needed for success, regardless of what pathway they choose after secondary school (ISCED 3). Given the shortage of technical expertise and capacities, progress in these areas will likely be difficult unless accompanied by greater collaboration across administrative units and at the state-level.

Policy issue 2.1. Strengthening the educational value of student assessment

In the last 5 years, international and state level actors in BiH, as well as competent education authorities have been working to implement more competence-based approaches to teaching and learning. These efforts have led to a greater emphasis on the key competences that students need for success in further studies, work and life. However, while BiH is very experienced with summative assessments that measure knowledge, there is a need for more balanced assessment frameworks that advance a student-centred and competence-based learning agenda. This includes much more focus on how to assess learning in relation to specified outcomes and standards, and how to integrate assessment results and feedback into the teaching and learning process. At present, teachers in BiH are generally left on their own to develop assessment criteria, receive limited professional development on formative assessment practices and have few, if any, resources available to help strengthen their overall assessment literacy. To help teachers appropriate more effective assessment practices, BiH authorities will need to work with the teaching profession, education experts and other actors to develop the resources, training and professional networks needed to foster a new assessment culture in BiH from the bottom-up. Involving parents and the wider society in these changes will also be crucial: without their understanding of why and how changes to assessment practices can benefit their children, there is likely to be resistance to reforms.

Recommendation 2.1.1. Take steps to shift the culture of learning and assessment

The introduction of new competence-based curricula in many parts of BiH provides an opportune moment to strengthen the link between assessment and learning. Many of the administrative units covered by this review already have elements within their student assessment frameworks that can support this link, notably descriptors to accompany quantitative marks and start-of-year diagnostic tests. However, many teachers continue to struggle in understanding how to implement these formative practices and with few exceptions, there are no resources or training opportunities that help them do so. Changes to how teachers assess student learning in BiH are also confronted with pressure for accountability in the form of grades and rankings. In addition to parental pressure, there is also political and public pressure from teacher unions and the wider society to maintain more traditional forms of assessment and avoid elements like external assessments. While attention to results is a positive feature of education systems, an over-emphasis on these may have a negative impact, narrowing the focus of learning and undermining both student agency and the formative role of assessment (OECD, 2013[1]). To enhance the learning value of student assessment, competent education authorities will need to adjust their rulebooks to emphasise a more balanced set of assessment practices and ensure that students receive feedback on how they can improve. Government authorities will also need to communicate the value of formative assessment to all interested stakeholders to build buy-in and support for the new, learner-focused culture of assessment.

Adjust the student assessment rulebooks of competent education authorities to emphasise a more balanced set of assessment purposes

Existing policy documentation in BiH often focuses on logistical and organisational aspects of student assessment, emphasising the role of summative and normative assessments. To promote a more balanced assessment framework that supports student learning, competent education authorities should adjust their respective assessment rulebooks to clearly define the various components and instruments included in their assessment frameworks, as well as the different purposes of these assessment types, their added value, and how they work together. To support local authorities in adjusting their rulebooks, APOSO could develop or commission a reference document that outlines key assessment principles based on international research. Together, such efforts can help build a new assessment culture in BiH that more closely aligns with the learning outcomes and competence-based approach that education systems in the country seek to implement. Specifically, revised local assessment rulebooks should:

• Clearly reference standards of student achievement. Local assessment rulebooks should set a clear expectation that teachers measure student achievement against a defined set of learning standards that state what students should know and be able to do at each level of schooling (see below). Competent education authorities can use the Common Core Curricula based on Learning Outcomes developed by APOSO. They may also add their own standards for certain subjects or curricular elements. In Germany, common educational standards for primary education (ISCED 1) apply to all Länder but local curricula reveal concrete and binding competence expectations for specific subjects (Eurydice, n.d.[24]).

• Emphasise use of assessment for different purposes. Rulebooks should provide comparative definitions of formative and summative assessment. While these purposes of assessment are synergic and cannot be sharply separated (Black and Wiliam, 2018[10]), for teachers working in a system under transition, clarification around the two approaches and how they relate would be useful. Rulebooks could also define other key assessment techniques and topics, such as reliability and validity, which would provide a reference and base for teachers to strengthen their assessment literacy.

• Adjust how summative assessments are given. There is also room to adjust the current qualitative descriptors used in the grading systems to reflect progression towards mastery of competences, rather than using labels such as “good” or “poor.” Instead, competent education authorities might express student results using language that can be more motivational for low performers, such as exemplary (5) or under-developed (1). Defining these descriptors in local assessment rulebooks, as Central Bosnia Canton has already done, can also help teachers communicate the meaning of the marks they assign to students more effectively.

• Collect and monitor examples of classroom assessments. To help create tools for supporting teachers’ assessment literacy (see Recommendation 2.1.2), student assessment rulebooks might require that teachers provide a certain number of their classroom assessments as samples to pedagogical institutes or their equivalent in their ministry. This would allow local governments to monitor the quality of classroom assessments, identify strong examples of assessment to share with the teaching profession more broadly, and help identify areas where teachers may need support to strengthen their assessment practice.

Make reporting students’ assessment results more conducive to learning

Changing specific reporting practices can help close implementation gaps between competence-based curricula and teachers’ classroom assessment practices. In OECD countries where summative scoring has tended to weigh heavily, such as France, revisions to student reports has been a particularly effective way to communicate and embed new expectations for classroom assessment. At present, schools and teachers in BiH have discretion on the criteria for assessing students and reporting on results, which can lead to inconsistencies in the type of feedback students receive and risks leaving them with little information on how to improve. For assessment to have a greater impact on learning in BiH, competent education authorities should require teachers to regularly provide feedback to students beyond the existing qualitative descriptors (Table 2.7). This feedback should consist of written feedback to individual students at least once a semester and oral feedback on other occasions.

Developing report card templates that make space for descriptive and formative feedback, as well as the summative grades that currently dominate assessment practices is another way to facilitate more formative feedback (Box 2.2). For example, while students get final grades for the end of each semester, only the end‑of‑year grade is included in the official certification of completion. This creates an opportunity to transform the final grade of the first semester into a more formative report that helps teachers plan and guides students on how they can improve in the following semester. As part of these templates, education authorities should consider requiring older students to provide input on their own learning targets and reflections about the marks they receive, as these elements can support student agency.

Table 2.7. Examples of teacher feedback that can support learning

Feedback type	Examples
Identifying errors	Underline or circle words | “?”
Explaining misunderstandings	This data is out of date | Don’t forget | Recent data shows
Demonstrating correct practice	Inserting corrections | New sentence
Engaging students in thinking	Why? | Is this logical? | Does this follow? | Is there an alternative interpretation?
Suggesting further study	“See…for information” | “Try reading… to develop your thinking further.”
Justifying marks	“I could not award a higher mark because of xxx” | “This analysis made a strong contribution to your grade”.
Suggesting approaches to future work	“In future assignments I recommend…” |”Try to develop your…”
Aligning progress from previous attainment	“I can see how you have developed this”. | “You have made progress here”.

Source: Adapted from Orsmond and Merry (2011[30]), “Feedback alignment: effective and ineffective links between tutors’ and students’ understanding of coursework feedback”, Assessment & Evaluation in Higher

Education, Vol. 36/2, pp. 125-136, https://dx.doi.org/10.1080/02602930903201651.

Box 2.2. Enhancing the recording and reporting of student assessment data in Denmark

Since 2006, all primary and lower secondary schools (ISCED 1 and 2, respectively) in Denmark must provide Individual Mandatory Student Plans (IMSPs) tracking student progress. These include a summary of students’ results and qualitative feedback on how these will be followed up. For national assessments, formative comments on student performance are included but not marks. The IMSPs are not a simple report card or performance tracker, but rather a working tool for teachers, forming the basis of discussions between students and teachers, as well as with parents. They also provide a record of student achievement that, throughout compulsory education, eases transitions between grades. Denmark’s IMSPs continue to evolve, including conversion to digital format to make them more accessible to students, parents and teachers. The digital platform enables teachers to collate information on progress, goals and student assessments, as well as recording the specific goals for the individual student, a progress status in relation to the goals and a monitoring section describing how and when to follow up.

Source: (Shewbridge, 2011[25]) OECD Reviews of Evaluation and Assessment in Education: Denmark 2011, https://doi.org/10.1787/9789264116597-en; (OECD, 2020[26]), Education Policy Outlook: Denmark, http://www.oecd.org/education/policy-outlook/country-profile-Denmark-2020.pdf (accessed on 18 August 2021).

Providing feedback that is conducive to student learning can significantly add to the non-teaching workload of teachers. Therefore, key considerations for competent education authorities will be keeping report cards and other feedback expectations simple, as well as developing guidance materials to explain how teachers should use these tools. School principals and teachers will also need preparation to explain learning progress to students and parents. Sharing best practices for communication (e.g. phone calls, email, videoconference, and in-person) and the circumstances under which each mode is most pertinent, as well as the frequency of communications can be helpful in this regard. Competent education authorities will also need to address factors that may make it difficult for teachers in BiH to gain confidence and skill in using the new approaches to assessment.

Communicate the value of assessment through a dedicated digital platform as a means to support teaching and learning

In addition to clearly defining assessment principles in local rulebooks, government authorities in BiH can establish a website specifically dedicated to promoting competence-based curricula and associated assessment practices. For example, when Portugal introduced the Project for Autonomy and Curricular Flexibility in 2017 to support the implementation of its new curriculum, the Ministry of Education established a central website that served as a digital resource for reflection and the sharing of practices, as well as a digital library for documentation to support teachers in their curricular and pedagogical decisions (Portuguese Ministry of Education, 2021[27]). Today, Portugal’s curriculum and assessment website continues to grow and support the country’s curriculum reform by providing links to official legislation, examples of good practice, access to webinars and presentations and regularly updated news and events. In BiH, some cantons have already started piloting digital platforms to support their curricula and assessment policies, which could be scaled up to include a wider range of assessment tools. APOSO could also host a website (potentially with funding from donor organisations), that serves as a compendium of competence-based curricular documents from across the country, to support peer-learning. This central website could include links to the respective websites of the curricula of competent education authorities but also serve as a platform for sharing and discussing digital versions of expected learning outcomes in the CCC and support materials. Over time, this website can provide a range of student assessment resources targeted towards teachers, students, parents and the public.

Recommendation 2.1.2. Collaborate with teachers and other actors to create resources that strengthen the educational value of classroom assessments

It is positive that APOSO has already developed (and continues to develop) student achievement standards for some key grade levels and learning areas of the CCC, as this provides a reference for student assessments. However, many subject areas and grade levels still do not have clearly-defined standards, making it difficult for teachers to form a valid and reliable assessment of where students are in their learning. Finalising the development of learning standards at each grade level should therefore be a top priority for APOSO. Teachers in BiH also lack the resources to help them use learning standards as a reference for classroom teaching and assessment. Competent education authorities should take decisions about what specific supports and incentives could facilitate such implementation in their jurisdiction. For example, entity or canton authorities could require teachers to record descriptive feedback and justification for some of their marks vis-à-vis the learning standards. However, there are also opportunities for APOSO to work directly with competent education authorities, expert teachers from across the country, as well as relevant non-government organisations and other partners, to prepare core materials that can immediately help teachers appropriate the standards. Some of these materials may include examples of marked student work, assessment tasks and diagnostic assessment tools. Such resources can be powerful tools to improve the quality of teacher assessment practices and direct student learning towards the mastery of key competences.

Finalise student achievement standards for all grade levels and key subject areas at the BiH-level (to serve as an example for competent education authorities)

While competent education authorities will continue to choose their own curricula (i.e. what and how students learn and teachers teach), having a common set of learning standards can help reinforce local efforts to ensure that all students achieve basic competences needed for further education, training or careers. The United States, for example, defines common learning standards at the country level; however, these are elaborated for all grades instead of only for key curricula stages. Other countries in Southeast Europe, including Bulgaria and Serbia, have also developed learning standards for each grade level. Having a working model of what students are expected to know and be able to do in each grade can help teachers diagnose learning needs and assess progress throughout curricular cycles, rather than just at the end. APOSO should therefore finalise its current plans to develop BiH learning standards for key grades and subjects and in the future, extend these to cover all grade levels.

Provide examples of student work to help teachers appropriate the learning standards

Providing marked exemplars of student work would help demonstrate what achievement of the learning standards looks like at different performance levels. These materials should be made available on a dedicated online platform. For example, Ireland has a dedicated website that includes examples of student work illustrating three levels of achievements (at expectation, ahead of expectation or yet to meet expectation) for each of the country’s learning outcomes (NCCA, n.d.[28]). To collect exemplars in BiH, APOSO should work in co‑operation with pedagogical institutes and organised groups of expert teachers from across the country who can not only identify examples of student work but also provide commentary on how a specific piece of work demonstrates a given level of achievement. Teachers and schools can then work in subject teams to enrich the initial base of examples and discuss students’ work in relation to the standards. While these activities can be done across all grade levels and subject areas, APOSO and pedagogical institutes from competent education authorities might choose to start disseminating examples of student work at the primary level (ISCED 1) and in key subjects, to help reinforce foundational skills and knowledge early on. These efforts would help teachers develop more reliable and consistent classroom assessments, as well as give feedback to help students progress in their learning.

Make full use of start-of-year diagnostic assessments

Most competent education authorities reference start-of-year “initial checks” or diagnostic tests in their assessment rulebooks. These can be helpful to identify gaps in learning as they emerge since evidence from international assessments reveal that many students progress through BiH schools without meeting basic competences. However, teachers receive no guidance on how to conduct these checks or what the results should be used for. In several OECD and EU countries, diagnostic tests are an important type of formative assessment that help establish a baseline of students’ prior knowledge, strengths, weaknesses and learning needs and to inform teacher planning and instruction (OECD, 2013[1]). To support teachers in BiH to make full use of the start-of-year diagnostic assessments, competent education authorities should:

• Introduce reporting requirements for diagnostic assessments. Requiring teachers to share qualitative feedback from their diagnostic assessments with students and parents can provide a reference point for monitoring progress and designing individualised learning plans. Critically, reporting should not include a numerical grade, but rather focus on descriptive feedback that identifies what the student already knows and can do, as well as the knowledge or skills that need strengthening in order to achieve learning standards. Providing reporting templates and guidance on how to interpret results would help ensure that reporting supports teaching and learning.

• Continue making use of pedagogical institutes to help plan, implement and analyse diagnostic assessments. Administrative units with pedagogical institutes should use these bodies (or their education ministries) to work directly with schools and/or groups of teachers to explain and explore specific diagnostic assessment tools as part of the methodological support they provide to teachers. This would help schools and teachers benefit from diagnostic assessment tools that meet their individual needs.

Considering the time and expertise needed to develop high-quality diagnostic assessments, a number of countries have found it more efficient and effective to provide centrally developed diagnostic assessment tools. In Romania, for example, the government develops standardised diagnostic tests for key grade levels. Serbia has a similar practice but also develops templates for marking tests at the school level (Maghnouj et al., 2020[12]). In Estonia, diagnostic tools are digital and accompanied by a series of e‑tasks that enable teachers to easily individualise instruction and group students for different activities based on their performance in the tests (Innove, n.d.[29]) (OECD, 2019[30]). In BiH, APOSO and the pedagogical institutes of competent education authorities could work with other relevant actors, such as non-governmental organisations, academic researchers within higher education institutions or private assessment companies, to provide diagnostic assessments and other central tools directly to teachers through the new assessment platform (see Recommendation 2.1.1). Creating these tools will require capacity, time and resources. APOSO should therefore consider prioritising the development of diagnostic tools for early years of schooling first, as identifying and addressing learning gaps before they become problematic can have a larger positive impact on student outcomes. Competent education authorities with the technical capacity could also develop their own diagnostic assessment tools and platforms; however, for most authorities, their efforts and resources could be better spent encouraging the use of diagnostic assessment through legislation and methodological support to schools. Combining state-level, as well as RS entity, cantonal and Brčko District requirements and tools, alongside teacher-led initiatives can help establish diagnostic tests as part of a more comprehensive assessment framework that supports the development of core competence.

In the medium term, give teachers external data about student performance to benchmark their classroom assessments

External benchmarks of student achievement, such as results in standardised examinations or assessments, can support teachers in making accurate judgements about student progress because they provide a reliable reference for expected or adequate progress (OECD, 2013[1]). This information can be particularly helpful when the assessment literacy of teachers is low. As BiH builds the technical capacity to conduct and use standardised assessments (see Recommendation 2.2.2), the results of these instruments can provide such external benchmarks (see Chapter 5). To do this, APOSO, and/or actors with responsibility for standardised tests in RS, cantons of FBiH or Brčko District, should provide detailed information on the average achievement of students in relation to specific outcomes, which would allow teachers to compare their students’ performance. Test developers could also release items where students on average perform well and poorly to help orient teachers on what they might reinforce in their own classroom assessments.

Recommendation 2.1.3. Provide teachers with training and support to develop their assessment literacy

Using assessment to support learning requires changing schools and teachers’ practices, their beliefs, and the pedagogical materials they design and use. As in other countries in the region, encouraging greater use of formative assessment in BiH will require developing teachers’ assessment literacy, but also building their understanding of why it matters (Kitchen et al., 2017[16]; Maghnouj et al., 2020[11]). The quality of initial teacher education in BiH is often considered insufficient and participation in professional development is generally low (see Chapter 3). Moreover, the RS was the only administrative unit covered by this review where teachers reported participating in specific training modules on how to assess students; although some of this training was reportedly theoretical and based on textbooks, rather than providing practical experience and tools that teachers could apply to their assessment practice. This suggests a clear need for guidance and training related to student assessment.

Improve the coverage of student assessment topics in ITE programmes

Research indicates that if teachers do not learn to meaningfully apply formative assessment practices during their initial education, this will limit their ability to apply formative assessment throughout their career (Earl, 2007[31]). Teacher candidates in BiH could benefit from more explicit instruction and practice in using formative assessment as part of broader efforts by education authorities to implement more competence-based and student-centred approaches to their school systems. Without addressing this issue in initial teacher education, teachers risk replicating traditional assessment practices rather than implementing more comprehensive approaches that can better support learning. Policymakers in BiH should consider one or more of the following steps to improve the quality of initial teacher education in this area:

• Include student assessment in programme-specific accreditation criteria for ITE. Many OECD countries have introduced mandatory, programme-level accreditation criteria to set minimum standards for ITE providers (see Chapter 3). Including requirements that explicitly require ITE providers to prepare teacher candidates to use a range of student assessment practices can help, giving this topic more attention in programmes. Providers may also need guidance on how to develop courses on student assessment in order to meet these new criteria.

• Incorporate competences related to student assessment as part of the professional standards for graduate teachers. In future reviews of professional teacher standards, competent education authorities in BiH could introduce “graduate” or ”new teacher” competences that set clear expectations of what beginner teachers should know and be able to do with respect to student assessment (see Chapter 3). Experience from New South Wales (Australia) can provide insights for this type of policy approach (see Box 2.3).

Box 2.3. Beginner teacher competences that include student assessment in Australia

In New South Wales (Australia), the Board of Studies, Teaching and Educational Standards, (now the New South Wales Education Standards Authority) identified key elements in the area of assessment that describe the qualities expected of beginning teachers and used these to develop a framework for addressing assessment in initial education programmes. These elements include: knowing the purpose of formative and summative assessment, as well as how to use both in the classroom; knowing how to improve assessment reliability, such as through moderation; having sufficient data literacy to be able to use results from large-scale assessments to improve student learning; and understanding the importance of developing criteria for evaluating performance on assessments at different levels.

Source: (NESA, 2021[32]), Graduate: Standard Descriptors, https://educationstandards.nsw.edu.au/wps/portal/nesa/teacher-accreditation/meeting-requirements/the-standards/graduate-teacher (accessed 18 December 2021).

Build the assessment literacy of teachers through their initial education and practicum experience

A growing body of research suggests that grounding initial teacher education in practical learning experiences is a critical part of preparing new teachers and can have a positive impact on student learning and teacher retention (OECD, 2019[33]). Positively, most ITE programmes in BiH include a teaching practicum; however, the duration and quality of these experiences vary, which risks leaving some teachers unprepared for work in the classroom (see Chapter 3). To reinforce teachers’ assessment literacy, competent education authorities could partner with ITE providers in training a cadre of experienced mentor teachers in student assessment practices. This would help ensure that mentors have a clear understanding of the assessment components that align with competence-based curricula. Another way to ensure that experienced teachers understand the value of and are able to use a balanced range of student-centred assessment practices (e.g. diagnostic, summative and formative, etc.), is to incorporate these expectations into professional teacher standards. Teachers that demonstrate mastery in this area could then be rewarded with higher responsibilities, such as serving as assessment leads in their schools or as mentors to new teachers. The digital platform of assessment resources (see Recommendation 2.1.1) could also support teachers and mentors to emphasise student assessment during the practicum experience.

Offer training and support for peer-learning among teachers on the topic of student assessment

Promoting quality professional development on student assessment is crucial to reaching wider cohorts of teachers. This professional development can take the form of formal workshops or webinars, or as job-embedded activities that can help relate the content of training to the specific school and classroom (OECD, 2013[1]). Positively, some schools and teachers in BiH already organise internal school networks (e.g. by subject or grade level) to discuss assessment criteria and practices. However, these networks are largely dependent on individual initiatives. There are several ways competent education authorities in BiH can more actively strengthen the assessment literacy of practicing teachers, for example:

• Encourage schools to consider student assessment as a core professional competence. Student assessment practices should be reviewed and discussed as an integrated part of all in-school teacher appraisals, starting from the teachers’ practicum to more advanced levels of experience (see Chapter 3). This will not only involve including student assessment competences among professional teacher standards (see Recommendation 3.1.1) but also giving teachers opportunities to lead key improvements in this area (e.g. by serving as assessment leads in their schools). International experience also suggests that school principals have an essential role in changing a school’s culture, as well as managing social and parental pressures (OECD, 2012[34]). Therefore, school leaders will need training and support to become instructional leaders (see Chapter 4) and emphasise the links between assessment and learning in their schools.

• Build capacity for student assessment. Pedagogical Institutes or their equivalents would be well placed to facilitate between-school moderation processes whereby teachers of the same subject who work in different schools, mark each other’s assessments and discuss differences in their marking. Research suggests that moderation can help teachers build a shared understanding of criteria for marking and expectations for learning, and it is a key strategy for improving the reliability of teacher judgements and marking within and across schools (OECD, 2013[1]). This would also be a low-cost and effective way to help teachers identify learning issues early on. Moderation could be conducted at first with end-of-term assessments and then extended to other types of assessments.

• Create space for school-based discussions on assessment. BiH could formalise and scale up the existing practice of discussing student assessment among teachers in the same school. For example, schools could establish assessment teams to organise peer-learning activities, such as peer classroom observations, coaching, in-school moderation procedures and the co-creation of instructional material. These activities could form the basis for targeted and reflective discussions around improving teacher practice in the area of assessment and could be championed by lead teachers or principals (Harrison, 2005[35]; Tang et al., 2010[36]; Darling-Hammond and Rothman, 2011[37]).

Establishing student assessment as a policy priority for teacher professional development, could also help orient the international donor community in BiH to provide support in this area. There are several good examples from international experience of donor programmes that have helped develop sustainable assessment capabilities in other countries. For example, in Georgia, the United States Agency for International Development (USAID) organised peer-learning circles for teachers to discuss student achievement and ways to improve instruction (Li et al., 2019[38]). A similar arrangement could benefit professional learning networks for teachers and schools in BiH.

Policy issue 2.2. Prioritising the development and implementation of external examinations

Reforming examinations in BiH would be a strong lever to focus the country’s education systems on the importance of developing key competences, regardless of the specific curricula, what secondary track students complete or where they attended school. This is an important and widely recognised goal across European education systems. Combining an optional and standardised Matura that is open to all BiH students at the end of secondary school (ISCED 3), with graduation requirements set by competent education authorities, would send a signal that all children should be supported to complete formal schooling with a valued and rigorous certification. This certification should also be recognised both across BiH and on a par with international certificates, which would facilitate the mobility of talent across the country by creating a recognised “passport” that students have mastered foundational knowledge and skills and are well prepared for their future endeavours. Ensuring the BiH Matura leads to equal opportunities after secondary school (ISCED 3), including post-secondary VET programmes, higher education or careers, can also support the broader equity and inclusion goals.

At present, there are no standardised state-level examinations developed and implemented in BiH and only one canton (Tuzla) has a standardised external exam at the end of secondary education (ISCED 3). However, standardised examinations have been implemented at the end of basic schooling (ISCED 2) in other entities and cantons, which can help inform decisions about students’ pathways into the secondary level (ISCED 3). In reality however, many jurisdictions in BiH are unable to develop and implement such instruments on their own because they lack the required financial resources and technical capacity. As a result, teachers often carry the full responsibility for making judgements on student achievement, not only throughout school, but also at important transition points. This would be a challenge for teachers in any education system, as external exams reinforce independence, reliability and equity - factors that are critical for high stakes decisions and help protect teachers from pressures to inflate grades. This issue is particularly acute in BiH, as many teachers lack the assessment skills and the support needed to make such consequential judgements.

This situation has negative consequences for student learning: the fact that more than half of students across BiH do not achieve baseline proficiency on the PISA reading test by age 15 suggests that students are moving through the country’s school systems without a clear understanding of whether they have mastered foundational competences, such as literacy or mathematics (OECD, 2019[39]). The lack of reliable metrics and external benchmarks also has wider implications – for discriminating aptitude for higher education, for signalling skills to employers and for efforts to focus schools on developing the human capital BiH needs for broader economic development. These considerations make reform to the country’s examinations system a priority from both an educational and socio-economic perspective.

Recommendation 2.2.1. Develop an optional external examination of core competences

The majority of competent education authorities in BiH do not provide students with a chance to validate their knowledge, skills and competences through an external examination (Matura) at the end of secondary school (ISCED 3). In a country where grade inflation is a widely recognised problem, this creates a range of challenges related to the rigour and reliability of secondary school (ISCED 3) diplomas, as well as to the fairness and efficiency in how decisions about students’ future are made. To address these challenges, competent education authorities and APOSO should work together, with support from the donor community, to develop an optional external examination of core competences at the end of secondary education (ISCED 3). This new, BiH Matura should be optional for competent education authorities (at least in the beginning) and limited to an assessment of students’ core competences (e.g. literacy, numeracy and science). The results from this exam should be considered as part of a wider range of graduation requirements set by competent education authorities, which would help raise the value of secondary qualifications by certifying the mastery of core competences. It might also serve as a criteria for university selection, which would help improve the fairness of university selection and ensure admissions processes are based on merit. The development of the optional BiH Matura should be a policy priority for education authorities, as the instrument could help raise learning outcomes by communicating expectations of the CCC and sending a strong signal that all students in BiH should graduate with foundational knowledge and skills.

Establish a steering committee comprised of representatives from competent education authorities and APOSO to develop an optional BiH Matura

Previous proposals to establish an external examination that is available to all students in BiH have been blocked by political impasse. Future efforts should therefore focus on the competent education authorities that are interested and willing to collectively develop an independent and technical exam instrument to benefit their students. While broad consensus among all competent education authorities should be a long term goal, implementation of the new BiH Matura should go forward once a minimum number of education systems in the country opt to participate. To develop this type of exam, Slovenian experts previously recommended that BiH establish a steering committee with officials and representatives from competent education authorities and APOSO (Slovenian Expert Group, 2011[23]). Per their proposal, the steering committee should determine the minimum number of education systems in BiH that would need to participate in order for the new Matura to be implemented. APOSO and interested administrative units could also analyse the costs and potential funding sources for the BiH Matura in order to determine the level of participation needed to ensure its feasibility. These arrangements would help guarantee that decisions about the optional BiH Matura are both representative and technically sound.

The steering committee initially proposed by the Slovenian expert group was supposed to also conduct structured consultations with a wider range of stakeholders from across the country, notably associations of school leaders and teachers, the private sector, non-governmental organisations and assessment experts (e.g. from universities) that have relevant experience in developing and administering exams. In line with this recommendation, BiH should continue inviting international specialists to share insights about standardised assessments and examinations in other countries, perhaps targeting those with experience in decentralised systems. The Slovenian expert group also recommended that BiH develop a state-level framework, which should set out regulations about the composition and specific tasks of the steering group. This OECD review team supports these recommendations, which would provide more transparent and collaborative governance arrangements for the optional BiH Matura and any future changes to the exam system.

Draw on employers and universities to generate demand for the optional BiH Matura

Engaging influential employers can generate demand for the optional BiH Matura, as the exam would assure hiring managers that candidates who have recently completed secondary school (ISCED 3) have mastered competences in key areas (e.g. mathematics, languages, etc.). Employers who spoke with the OECD review team reported that recruitment decisions are sometimes based on where students are from (i.e. based on assumptions that students from certain entities/cantons receive higher quality education and would therefore make better employees). An optional BiH Matura would help facilitate the mobility of talent across the country and allow students to graduate with a valued certification that is recognised within BiH and on par with international certificates. The exam would also support BiH’s broader goals for the VET sector by strengthening available qualifications and helping to raise the learning outcomes of students who attend initial VET programmes because these schools would need to reinforce foundational literacy and numeracy competences so their students can succeed on the exam (BiH Ministry of Civil Affairs, 2021[40]).

Universities are another important driver of demand for the optional BiH Matura, and their involvement in developing a new examination will be important for encouraging the use of exam results as a core criteria for admissions. At present, universities across BiH set their own entry requirements, which often includes a university-led entrance test and a review of students’ secondary school grades (ISCED 3). Teachers have full responsibility for the latter (and therefore what educational pathways are available to their students), which creates pressure for teachers to inflate grades. University-led exams can also distract teachers and students from the school curriculum, fuel a “shadow” system of test preparation and, in countries with high rates of corruption, encourage distortive practices like bribery. Having a standardised measure to help select students into university could therefore help ensure the integrity, quality and equity of the country’s higher education sector. For these reasons, parents and students themselves are also potential sources of demand for the BiH Matura. Using the Matura for university selection would not only require close consultation with the tertiary sector on the new exam, but also have implications for its design (e.g. the assessed competences and marking scale), and how individual student results are interpreted for placement decisions (see below).

Create a two-part certification system that includes an optional BiH Matura

Establishing the optional BiH Matura as a certification examination at the end of secondary education would give students, universities and employers a reliable measure to determine if an individual has developed key competences needed for success in today’s world, regardless of what entity or canton they live in. Having representative and structured leadership on the steering committee and drawing on employers, universities and parents to generate demand can, over time, help build consensus for the new BiH Matura. However, creating an optional, two-part certification system could be a more practical and immediate way forward. This approach would reconcile the need for a consistent metric to certify core competences at the end of secondary school, while allowing competent education authorities to maintain their autonomy over assessment policy and graduation requirements. This two-part certification model could involve:

• Part one: A common standardised assessment of core competences that aligns with the CCC or core competences defined by the EU. This part of the BiH Matura would provide objective information about learning outcomes on a common scale in select competence areas (see below). All competent education authorities should engage in co-developing this part of the BiH Matura, with the goal of leveraging the experience of entities and cantons that have already developed external examinations in their jurisdictions. For example, Tuzla Canton (the only administrative unit that already administers its own external exam at the end of secondary education (ISCED 3)), could use its existing infrastructure to pilot the BiH Matura and share insights with other jurisdictions. However, participation should be optional, to avoid the perception that this is a centrally mandated exam and to guarantee the autonomy of competent education authorities. Such an arrangement would especially benefit jurisdictions that lack the means to develop their own external exams, as the associated costs and technical capacity needed to create the testing instrument would be shared. International donors could also provide financial and technical support in this area, creating further incentives for competent education authorities to engage in developing the test instrument and ultimately administer it.

• Part two: This part of the certification system would be developed by individual competent education authorities, creating space for them to recognise and assess learning areas that are of specific interest to their jurisdictions. Importantly, competent education authorities can choose from several student assessment policies to complement the BiH Matura. They can also determine the weights of different indicators for secondary certification, for example considering the BiH Matura results in addition to results from:

  • An external examination at the level of administrative units. For competent education authorities with the desire and capacity to implement their own standardised examination, these efforts should be supported while ensuring complementarity with the BiH Matura. Tuzla Canton can provide valuable experience in this area and should review its external examination to ensure complementary with the BiH Matura. The results of this review should be published to help other entities and cantons reflect on how their own standardised exams and the BiH Matura can be jointly used as criteria for secondary certification.

  • School-based exams. At present, several education authorities covered by this review conduct end-of-year examinations at the school level. These exams are typically prepared and administered by teachers in the school; however, there are a few exceptions and recourses for students who are unsatisfied with their final results. For example, students in RS can request to be examined by a commission appointed by their school principal instead of their direct teacher (RS, 2019[41]). In most cases however, school-based exams in BiH do not contain any externality or moderation, reducing the reliability of results. Competent education authorities that use school-based exams should therefore strengthen moderation procedures to ensure that exam results and student GPAs are more consistent and can meaningfully complement the BiH Matura.

Focus the BiH Matura on a limited selection of core competences

While curricula and learning standards vary across BiH education systems, all governments in the country aim to develop the core competences of their children. The optional BiH Matura should therefore build on this shared goal by focusing the exam on foundational competences, such as literacy and numeracy. This approach is in line with trends in OECD countries, which often require students to take mathematics and language courses throughout secondary school to ensure they achieve the basic knowledge and skills to learn and master other subject areas. Specifically, the optional BiH Matura should have two compulsory exams: language and mathematics. Other learning areas such as social sciences and humanities, or art, would be best measured by competent education authorities, through their own external or school-based examinations. Limiting the BiH Matura to the functional use of language and mathematics would help focus education systems in the country on supporting all students to master these foundational competences.

Develop a concept note to clearly define the primary purpose of the optional BiH Matura and to develop associated technical and policy documents

As a priority, the steering committee should develop a concept note for the optional BiH Matura to clearly communicate the primary purpose of the exam. In developing this concept note, the steering committee will need to take important decisions, such as:

• How will the exam reflect diversity in the provision of secondary education in BiH, which includes a large number of VET schools? For example, will the same exam be offered to all students regardless if they attend general (gymnasiums) or VET secondary programmes or will the exam be dual-level (e.g. different tests that measure either minimum expected levels or more advanced levels of performance)?

• How can BiH education systems mitigate the potentially negative risks of introducing an external state-level exam, such as narrowing local curricula or teaching to the test?

• Will results from the BiH Matura count towards admissions to higher education institutions and if so, how?

Such decisions will have implications for students, schools, and other stakeholders who need to know how the new exam will impact their various roles and responsibilities. Using the concept note to define and communicate a clear conceptual foundation for the BiH Matura can also serve as the basis for more comprehensive legal and technical documents that set out specifications on the test instrument’s development, administration and use. Competent education authorities can then adapt or develop their own policies and rulebooks on how the optional BiH Matura results will be considered in relation to requirements for secondary school graduation that are set by RS entity, cantons of FBiH and Brčko District authorities (e.g. if this will complement an external exams administered in an administrative unit and the weight of these results vis-à-vis school-based exams or grades).

Define the scoring, scaling and reporting procedures for the BiH Matura

As part of the concept of the optional BiH Matura, the steering committee will need to define clear procedures for scoring, scaling and reporting student results. If the exam is to be used for certification of secondary school and to select students into higher education, the following decisions will need to be taken:

• The scoring scale: The score scale should be defined so the universities can easily rank students based on their performance. This will require a somewhat long and quasi-continuous scale to allow for sufficient discrimination of student performance.

• A threshold for certification: The steering committee should define the minimum score needed to pass the BiH Matura and receive a certificate of completion for secondary school. This threshold should ensure that students who pass the exam have attained the “basic level” of competences, as defined by learning standards at the end of secondary education (see Recommendation 2.1.2). Testing this threshold will be important to ensure it is accessible to most students.

The optional BiH Matura would be the first locally developed standardised assessment implemented at the state-level. Given the limited familiarity with standardised tests in many parts of BiH, the steering committee should take care to avoid conflating the purpose of the Matura with other assessment functions, such as making cross-country comparisons. Since the BiH Matura will be optional, the results will not be representative of the country. Moreover, external exams are designed to provide reliable measurement of individual students at a particular moment, not measure achievement trends over time. These features make the optional BiH Matura ill-suited to support system monitoring. The steering committee should therefore provide guidance on how data from the optional BiH Matura should be used – and ways it should not be used – for other functions.

Recommendation 2.2.2. Build the technical capacity to conduct and use standardised assessments

Once the steering group has developed the concept note and supporting technical documents, the country will need to focus on building the administrative systems to implement the optional BiH Matura. This effort should include identifying the right actors to carry out the administrative tasks, such as checking the quality of test items, producing test booklets or software and ICT infrastructure, if the steering committee chooses to administer the exam via computer. Given the low levels of public trust and limited familiarity with standardised testing in BiH, the steering committee should make use of technology to administer and mark completed tests. There is also scope to more actively involve teachers in the development of the Matura, which would help them integrate the competence-based approaches to education into their classroom practices. Finally, APOSO should be tasked by competent education authorities to help them collaborate in building understanding and support for more comprehensive student assessment systems across the country.

Build sustainable administrative systems to implement the optional BiH Matura

In allocating responsibilities for the optional BiH Matura, the steering committee should draw on a range of actors to help ensure the technical integrity of the testing instrument (see Table 2.8). Given APOSO’s experience implementing large-scale student assessments, the agency would be well placed to take on some of these responsibilities, especially developing the framework for tests. However, BiH could also contract international assessment companies, or draw inspiration from existing international tests to develop other subject tests, such as foreign languages, mathematics and science. Some of these tests are already used by school networks within BiH (e.g. Cambridge English). A combined approach of international and Bosnian expertise would not only address capacity issues but also ensure the new Matura links with international standards and meets technical specifications. Teachers should also be involved in the development of the optional BiH Matura, which would help address the need for test items while also developing teachers’ assessment literacy (see below).

The optional BiH Matura steering committee should also consider setting up regional exam centres to help administer and mark the new Matura, as this could reinforce the collective ownership of the exam, rather than the perception that it is a centrally mandated instrument. Albania, for example, has five regional exam centres run by permanent staff and trained teachers who administer and mark the State Matura locally; however, the Albanian regional assessment centres do report to a central authoritative body (Maghnouj et al., 2020[11]). Such a model would need to be adapted for the BiH context (e.g. regional centres could be identified on a rotating basis among participating education authorities). Quality assurance measures will also be needed to preserve consistency and integrity in the marking process across regions.

The administrative tasks associated with implementing the optional BiH Matura will have considerable resource implications. Support from international donors will likely be needed to guarantee adequate and recurrent funding to cover the human, technical and physical resources of implementing the BiH Matura. The Ministry of Civil Affairs would be well placed to co-ordinate this type of support; however financial contributions from competent education authorities will also be required. These actions will be critical to the long term sustainability and trust in the new examination system.

Table 2.8. Suggested responsibilities for key administrative tasks for the new BiH Matura

	APOSO	International assessment companies	Ministry of Civil Affairs	Entity, canton, district authorities	Teacher working groups	School of students taking the exam
Responsibility for the design of the BiH Matura				●
Quality control	●
Item design	●	●		●
Test production		●
Registration of candidates			●			●
Test administration
Test marking and moderation		●			●
Dissemination of results	●
Addressing students’ appeals against results	●

Make use of technology to support the integrity of the optional BiH Matura

Technology can help address potential misconduct in standardised testing, increase transparency, as well as help protect the rights of students (e.g. by using digital codes instead of names to protect students’ identity from those marking and managing the test) (Bethell and Zabulionis, 2012[42]). Considering the current integrity risks in BiH education systems (see Chapter 1), the steering committee should leverage technology to support the integrity of the optional BiH Matura. This applies to both the testing mode (i.e. the format of the exam’s administration), and marking procedures. Standardised tests that are administered on computer, where students receive test materials and submit their answers digitally, reduce risks that test materials can be “leaked” before testing occurs or be printed with errors. It also helps ensure that testing procedures are followed, whereas paper-based administration and human administrators are more prone to malpractice (e.g. giving students more time to complete test sections). The optional BiH Matura could benefit from the country’s recent experience in administering the computer format of PISA. However, since PISA is a sample-based assessment, competent education authorities would need to ensure that all students eligible to sit the BiH Matura exam have access to a computer.

The benefits of computer-based assessments also have implications for the marking of the optional BiH Matura. In particular, digital marking can reduce the time between the test’s administration and reporting of results, as well as minimise the human role in marking items (Bethell and Zabulionis, 2012[42]). Electronic marking will impact the types of question items that can be included on the exam. For example, close-format and short answer questions are more objective and easier to mark with machines, whereas open-ended questions may require human marking (e.g. to review a written essay response) and moderation procedures to ensure reliability. While including open-ended items on the exam can help measure more complex competences and higher-order skills, like those set out in the CCC, this approach has been controversial in other countries where there is low trust and exams are perceived to have high stakes. In Japan, for example, long-standing plans to introduce open-ended items to the national university entrance examination were further delayed because a pilot test revealed several marking inconsistencies (Japan Times, 2019[43]). The optional BiH Matura steering committee can build trust in the integrity of the exam by minimising the use of test items that require human marking, at least in the short term.

Engage teachers in standardised testing activities to help them integrate modern assessment principles into their practice

In order for the optional BiH Matura or other standardised assessments and exams to have a positive backwash effect on education systems in the country, teachers need to understand the underlying approach used in the testing instrument, as well as the question items, then incorporate these into their classroom practice. At present, there seems to be a general lack of understanding and engagement with available standardised assessments in BiH. For example, some stakeholders who spoke with the OECD review team mentioned that certain teachers consider external tests as challenging their professional judgements, while others expressed a need for more objective information about student achievement to better understand the curricula and learning standards. APOSO and competent education authorities will need to actively engage teachers in standardised testing activities if these instruments are to help close the gap between the intended curriculum and the taught curriculum in classrooms. In terms of examinations, which have stakes for students, it is especially important that teachers are familiar with the content and framework of the testing instrument, so their students will know what to expect and have a fair chance of success.

There are several ways to involve teachers in standardised testing, such as by writing and reviewing test items, as well as marking student responses and identifying threshold scores. Teachers will need clear instructions and training on how to develop questions that assess higher-order competences. These efforts can help create a cadre of teachers who become experienced test developers and assessment experts, who can share what they have learnt with colleagues in their school. Sarajevo Canton already involves subject teachers in the design of its Grade 9 Matura. However, in many countries, responsibilities for developing external tests and examinations in particular, is built into teachers’ formal job expectations. For example, in Norway, the marking of standardised tests is considered professional development for teachers (OECD, 2013[1]). Engaging teachers in this way can help promote a better understanding of the value of competence-based assessment, as well as a reflection about how classroom assessment practices can be adjusted to better reflect the curricula and learning standards.

Strengthen APOSO’s collaboration with competent education authorities in promoting greater understanding of standardised assessment across BiH

By participating in international assessments, APOSO staff have developed valuable technical competence in implementing large-scale standardised assessments of student learning. Despite the agency’s analysis and results dissemination efforts, this experience has not translated to a broader understanding of the potential benefits and risks of standardised assessments across BiH. To promote a more comprehensive student assessment framework, with a clear role for external standardised tests, several actions should take place. Primarily, BiH should continue to participate in international assessments, such as PISA, Progress in International Reading Literacy (PIRLS) and TIMSS (see Chapter 5). The analysis of standardised assessment results that APOSO produces in state-level reports should also be given a fixed time for discussion among policy makers of RS entity, cantons of FBiH and Brčko District. For example, such reports could be a regular topic for discussion at the BiH ministerial meetings and citizens’ assemblies, which could feed into public debates and promote greater accountability for improvements in BiH education systems (see Chapter 5). APOSO could be invited to present their reports on findings from standardised assessments.

As datasets allow, APOSO should lead efforts to tailor reports for each competent education authority or administrative units to accommodate the needs of governments, as well as schools and teachers. Using disaggregated data in the reports can also allow actors to compare themselves to country averages and make contextual and relevant comparisons (e.g. Croatian speaking cantons compare with each other and FBiH and BiH averages etc.). International actors could also provide capacity-building for competent education authorities to conduct their own analysis and report on results. As part of these new reporting efforts, item-level analysis with information about how students across the country performed on different types of tasks could help support teachers’ understanding of competence-based assessments. APOSO or university researchers would be well placed to conduct such analysis, which would be especially valuable if the reporting includes concrete examples of what students should know and be able to do across the ability range, as well as analyses of common errors that students made, with suggestions on how to improve teaching of the same content in the future.

References

[5] APOSO (2021), Standards of student achievement, https://aposo.gov.ba/hr/standardi-ucenickih-postignuca-poj/ (accessed on 3 December 2021).

[42] Bethell, G. and A. Zabulionis (2012), “19:1”, in The evoluation of high-stkaes testing at the school-university interface in the former republics of the USSR, pp. 7-25, https://doi.org/10.1080/0969594X.2011.635591.

[2] BiH (2021), Country Background Report for the OECD Review of Evaluation and Assessment in Education: Bosnia and Herzegovina, Unpublished.

[40] BiH Ministry of Civil Affairs (2021), Improvement of Quality and Relevance of Vocational Education and Training in Bosnia and Herzegovina - In LIght of the Riga Conclusions for the period 2021-2030, Compiled with support from the Agency for Education and Internationalisation and the KulturKontakt Austria, http://file:///C:/Users/Guthrie_Ca/Downloads/KKA%20VET%20RIGA-ENGLESKA-4.pdf.

[10] Black, P. and D. Wiliam (2018), “Classroom Assessment and Pedagogy”, Assessment in Education: Principles, Policy & Practice, Vol. 25/6, https://doi.org/10.1080/0969594X.2018.1441807.

[37] Darling-Hammond, L. and R. Rothman (eds.) (2011), Teacher and Leader Effectiveness in High-, lliance for Excellent Education and Stanford Center for Opportunity, Washington DC, https://edpolicy.stanford.edu/sites/default/files/publications/teacher-and-leader-effectiveness-high-performing-education-systems.pdf.

[3] Darling-Hammond, L. and L. Wentworth (2010), Benchmarking Learning Systems: Student Performance Assessment in International Context, Stanford University.

[4] EDK (n.d.), “National education goals”, Conference suisse des directeurs cantonaux de l’instruction publique, Vol. accessed 30 November 2021, https://www.edk.ch/de/de/themen/harmos/nationale-bildungsziele.

[24] Eurydice (n.d.), Germany: Teaching and Learning in Primary Education, https://eacea.ec.europa.eu/national-policies/eurydice/content/teaching-and-learning-primary-education-18_en.

[17] Guthrie, C. et al. (forthcoming), OECD Reviews of Evaluation and Assessment in Education: Bulgaria, OECD Publishing.

[35] Harrison, C. (2005), “Teachers developing assessment for learning: mapping teacher change”, Teacher Development, Vol. 9:2, pp. 255-263, https://doi.org/10.1080/13664530500200251.

[31] Hawley, W. and D. Rollie (eds.) (2007), Assessment as Learning, SAGE Publications.

[29] Innove (n.d.), Diagnostic tests, https://www.innove.ee/oppevara-ja-metoodikad/digioppevara/e-kogud/diagnostilised-testid/ (accessed on 18 August 2021).

[43] Japan Times (2019), Open-ended questions for Japan’s new university entrance exams scrapped,, p. (accessed on 23 January 2020), https://www.japantimes.co.jp/news/2019/12/17/national/open-ended-questions-scrappeduniversity-.

[16] Kitchen, H. et al. (2017), Romania 2017, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/9789264274051-en.

[38] Li, R. et al. (2019), OECD Reviews of Evaluation and Assessment in Education: Georgia, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/94dc370e-en.

[11] Maghnouj, S. et al. (2020), OECD Reviews of Evaluation and Assessment in Education: Albania, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/d267dc93-en.

[12] Maghnouj, S. et al. (2020), OECD Reviews of Evaluation and Assessment in Education: Serbia, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/225350d9-en.

[18] Muskin, J. (2017), Continuous Assessment for Improved Teaching and Learning: A Critical Review to Inform Policy and Practice, UNESCO International Bureau of Education, https://unesdoc.unesco.org/ark:/48223/pf0000255511 (accessed on 18 August 2021).

[28] NCCA (n.d.), Curriculum Online (accessed 2 December 2021), https://www.curriculumonline.ie/Home/.

[32] NESA (2021), Graduate: Standard Descriptors, https://educationstandards.nsw.edu.au/wps/portal/nesa/teacher-accreditation/meeting-requirements/the-standards/graduate-teacher (accessed on 18 December 2021).

[22] OECD (2020), Developing a national assessment that supports Kazakhstan’s national education goals, OECD Publishing, p. No. 27, https://doi.org/10.1787/1c4ed074-en.

[15] OECD (2020), Education in the Western Balkans: Findings from PISA, OECD Publishing, Paris, https://doi.org/10.1787/764847ff-en.

[26] OECD (2020), Education Policy Outlook: Denmark, OECD Publishing, Paris, http://www.oecd.org/education/policy-outlook/country-profile-Denmark-2020.pdf (accessed on 18 August 2021).

[20] OECD (2020), Education Policy Outlook: Germany, OECD Publishing, Paris, https://www.oecd.org/education/policy-outlook/country-profile-Germany-2020.pdf.

[33] OECD (2019), A Flying Start: Improving Initial Teacher Preparation Systems, OECD Publishing, Paris, https://doi.org/10.1787/cf74e549-en.

[30] OECD (2019), Education Policy Outlook 2019: Working Together to Help Students Achieve their Potential, OECD Publishing, Paris, https://doi.org/10.1787/2b8ad56e-en.

[9] OECD (2019), OECD Reviews of Evaluation and Assessment in Education: North Macedonia, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/079fe34c-en.

[39] OECD (2019), PISA 2018 Results (Volume I): What Students Know and Can Do, PISA, OECD Publishing, Paris, https://doi.org/10.1787/5f07c754-en.

[19] OECD (2015), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/eag-2015-en.

[1] OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/9789264190658-en.

[34] OECD (2012), Equity and Quality in Education: Supporting Disadvantaged Students and Schools, OECD Publishing, https://doi.org/10.1787/9789264130852-en.

[14] OECD (2012), Grade Expectations: How Marks and Education Policies Shape Students’, OECD Publishing, Paris, https://doi.org/10.1787/9789264187528-en.

[21] OECD (2011), OECD Reviews of Evaluation and Assessment in Education: Australia, OECD Publishing, Paris, https://doi.org/10.1787/9789264116672-en.

[13] Official Gazette Central Bosnia Canton (2012), Rulebook on evaluation of students in secondary school, Ministry of Education, Science, Culture and Sports of Central Bosnia Canton.

[6] OSCE (2021), POLICY BRIEF: Towards Education that Creates Changes in Bosnia and Herzegovina, Organization for Security and Co-operation in Europe (OSCE) MIssion: Quality Education through Curriculum Reform Project.

[7] OSCE (2020), Towards an Education that Makes a Difference, Organization for Security and Co-operation in Europe.

[27] Portuguese Ministry of Education (2021), Inicio [Home], https://afc.dge.mec.pt/ (accessed on 27 October 2021).

[8] Raleva, A. (2021), Profile commissioned by NEPC for the Global Education Monitoring Report 2021- Central and Eastern Europe, the Caucasus and Central Asia - Inclusion and education: All means all, https://gem-report-2020.unesco.org/wp-content/uploads/2021/02/North_Macedonia.pdf.

[41] RS (2019), Rulebook on secondary assessments.

[25] Shewbridge, C. (2011), OECD Reviews of Evaluation and Assessment in Education: Denmark 2011, OECD Publishing, Paris, https://doi.org/10.1787/9789264116597-en.

[23] Slovenian Expert Group (2011), Guidelines for the Development of Legal Acts for the Implementation of Matura in Secondary Schools in BiH.

[36] Tang, S. et al. (2010), “A case study of teacher learning in an assessment for learning project in Hong Kong”, Professional Development in Education, Vol. 36:4, pp. 621-636, https://doi.org/10.1080/19415250903554087.

Notes

← 1.  This designation is without prejudice to positions on status, and is in line with United Nations Security Council Resolution 1244/99 and the Advisory Opinion of the International Court of Justice on Kosovo’s declaration of independence.

← 2.  Originally the abbreviation of American College Testing.