Promoting Education Decision Makers' Use of Evidence in Flanders

1. Introduction to the case study

Abstract

This chapter provides a brief introduction to the OECD case study on the introduction of standardised tests in Flanders.

Introduction

In Belgium, the constitution guarantees ‘freedom of education’. Flemish schools enjoy the highest levels of autonomy among OECD countries in all aspects of education, including assessment practices. Flemish schools affiliate with umbrella organisations, many of which offer curricula and assessment supports via pedagogical advisory services. Schools can choose whether to use these services, including student assessments. There are no central examinations for students at the end of compulsory education in Flanders – an approach shared by only a handful of OECD countries (OECD, 2016[1]).

Flanders does administer some sample-based student assessments. In OECD countries, the use of standardised assessments in primary and secondary education is commonplace and over the past twenty years, the majority of OECD countries have chosen to administer census-based assessments, that is, with all students and schools participating (OECD, 2015[2]). Flanders introduced central assessments in 2002 and chose to assess only a sample of schools in each assessment. The sample-based central assessment (the ‘peilingen’) tests 1500 to 3000 students in different learning areas based on the expected level of learning, as specified in the central attainment targets. Each assessment round tests different learning areas on a rotating basis. All Flemish schools can choose to administer a parallel version of the central assessments (paralleltoets), but the uptake is low.

Just over ten years ago, OECD data from PISA indicated that standardised tests were not very present in Flemish secondary schools. In PISA 2009, three-quarters of the participating Flemish students were in schools that never used standardised tests, according to reports by their school leaders – and virtually none in schools providing parents with information on how their child’s performance compared to central benchmarks (OECD, 2010[3]). In 2010, an OECD review on school evaluation in Flanders found broad agreement in the educational field that central standardised tests would not be valuable for Flemish schools (Shewbridge et al., 2011[4]).

However, over the past ten years, there is some evidence that the reality in the educational field has evolved. In PISA 2015, over half of the participating Flemish students were in schools that reported using some form of standardised tests, and in the majority of cases these were used to compare school performance to other schools or to a performance benchmark at a local or central level (OECD, 2016[1]). Research in 2016/17 found an opening of minds to the potential benefits of standardised tests in Flemish schools (Penninckx et al., 2017[5]). The idea of using student assessments (all types) to monitor school progress from year to year has also taken root: two-thirds of participating Flemish students in PISA 2018 were in schools where this was reportedly the case (OECD, 2020[6]).

Since 2018/19, primary schools are required to administer tests to students in Grade 6 and must choose from a toolkit of validated tests, which includes the central parallel tests and tests developed by two of the umbrella organisations for schools. Already in 2015/16, the vast majority of affiliated schools chose to administer tests developed by these umbrella organisations (Janssen et al., 2017[7]).

Despite these evolutions, the Flemish education inspectorate still finds the need to improve the reliability and objectivity of school quality assurance processes in both primary and secondary education (Chapter 5). There is room to improve the availability and use of data at the school level. The current government plans to introduce central standardised tests in 2024. The proposal to introduce standardised tests was included in part of the political manifesto. The government launched a feasibility study in September 2020. The feasibility study looked to map out different scenarios for the design and use of standardised tests, including the key design features of purpose, reporting and administration.

Considerations of the different scenarios for standardised tests are ongoing. The Minister established a high-level forum in May to facilitate communication and feedback among stakeholders on policy decisions at key stages of the development of standardised tests. At the end of September 2021, the high-level forum received a preparatory text with proposals, based on the feasibility study. The first administration of standardised tests will be in May 2024, in Grades 4 and 8. In May 2023, there will be a field trial to evaluate the difficulty level of each item in the tests. The schedule is to introduce standardised tests in Grade 6 for May 2026 and in Grade 12 for May 2027.

The Flemish Department of Education and Training invited the OECD strategic education governance team to consult with stakeholders on their perceptions, hopes and concerns about the introduction of standardised tests. This reflects the department’s priority in 2021 to engage in communication efforts and dialogue with stakeholders. The OECD case study involved two main data collection channels: a series of structured discussions with key stakeholders in February and March, and a stakeholder reflection seminar in June.

This report presents feedback gathered from stakeholders as part of the OECD case study. It uses a research-based framework for strategic education governance (Chapter 2), that was the anchor for structured discussions with stakeholders and presents results as follows:

• Stakeholder involvement (Chapter 3): The OECD case study documents feedback from stakeholders on how they perceived their involvement in the early stages of the project to develop standardised tests and how they can contribute going forward.

• Strategic thinking and whole-of-system perspective (Chapter 4): Taking a step back from their daily responsibilities, stakeholders identify and comment on concerns and opportunities for the Flemish education system overall and the role that standardised tests might play in addressing these.

• Capacity and knowledge governance (Chapter 5): Stakeholders provide feedback on more concrete aspects regarding the introduction of standardised tests. This includes considerations about existing capacity for testing and quality assurance in the educational field. Stakeholders also identify the opportunities that the introduction of standardised tests may bring for their work and the necessary preparations to ensure the standardised tests support educational quality.

• Accountability (Chapter 6): Stakeholders provide feedback on the current accountability mechanisms in Flemish education. They also provide input on whether and how standardised tests may complement these.

• Lessons from the OECD case study (Chapter 7): This maps out the key points identified in the OECD case study against the research-based framework. In doing so, it provides some lessons for the further development of the standardised tests.

References

[7] Janssen, R. et al. (2017), Validering van IDP en de OVSG-toets : Eindrapport, Final report on validation of the IDP and OVSG tests, KU Leuven, https://data-onderwijs.vlaanderen.be/documenten/bestand.ashx?id=7760.

[6] OECD (2020), PISA 2018 Results (Volume V): Effective Policies, Successful Schools, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/ca768d40-en.

[1] OECD (2016), PISA 2015 Results (Volume II): Policies and Practices for Successful Schools, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264267510-en.

[2] OECD (2015), Education at a Glance 2015: OECD Indicators, OECD Publishing, Paris, https://dx.doi.org/10.1787/eag-2015-en.

[3] OECD (2010), PISA 2009 Results: What Makes a School Successful?: Resources, Policies and Practices (Volume IV), PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264091559-en.

[5] Penninckx, M. et al. (2017), Zicht op leerwinst. Scenario’s voor gestandaardiseerd toetsen (Scenarios for standardised tests), Acco, Leuven.

[4] Shewbridge, C. et al. (2011), OECD Reviews of Evaluation and Assessment in Education: School Evaluation in the Flemish Community of Belgium 2011, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264116726-en.