This Chapter presents a brief overview of the analytical underpinning for the strategic education governance case study and the methods used.
Promoting Education Decision Makers' Use of Evidence in Flanders
Educational Research and Innovation
2. Strategic education governance and case study methods
Abstract
Strategic education governance
The OECD case study is anchored in the strategic education governance analytical framework (Figure 2.1). This draws on a body of work by the Centre for Educational Research and Innovation (CERI) that included research and case studies documenting the complexity of modern education systems and providing insights to evidence on effective governance processes. It is published in two main volumes Burns and Köster (2016[1]) and Burns, Köster and Fuster (2016[2]).
The complexity of education systems arises from the multitude of different stakeholders involved and their various interactions. This stems from a desire to enable greater responsiveness to diverse local demands, multi-level governance arrangements and an increasing number of interactions and activities taking place across different organisational layers. Authorities are joined by parents and other stakeholders in education decision making. Nevertheless, Ministries of education remain responsible for ensuring high quality, efficient, equitable and innovative education at the national level.
With the multitude of stakeholders comes a complex mix of varying perspectives on challenges, differing interpretations of reality and preferred solutions. Information is now more widely gathered than ever before, and while the growing availability of information allows new insights and approaches to shape education, it also prompts new demands and uncertainties.
OECD research identified six interdependent domains of strategic education governance to help government authorities manage the dynamism and complexity of today’s education systems while steering a clear course towards established goals (Figure 2.1).
Figure 2.1. Domains of strategic education governance
Note: Drawing on the CERI body of knowledge on effective governance processes in complex education systems.
Source: Shewbridge and Köster (2019[3]), Strategic education governance - Project Plan and Organisational Framework, http://www.oecd.org/education/ceri/SEG-Project-Plan-org-framework.pdf.
Case study methods
The Flemish Department of Education and Training (the Department) invited the OECD strategic education governance team to conduct a case study on the introduction of standardised tests in Flemish schools. Stakeholder involvement is a central element of the strategic education governance framework and the Department recognised the importance to gather feedback from stakeholders on their perspectives, expectations and motivations. A core ambition of the case study, therefore, is to give insight to stakeholder concerns. That information can guide and help the process of introducing standardised tests. To this end, the case study included three main components:
1. Individual structured discussions with stakeholders (data collection). The OECD team liaised with the Department to obtain contact details for key stakeholders. The OECD team was responsible for contacting stakeholders with an invitation to a structured discussion in English. The invitation included a quick overview of the OECD strategic education governance framework with key points for the discussion (Box 2.1). Stakeholders were not required to prepare for the discussion, but had an idea of how the discussion would be structured and the main vocabulary to be used. Each discussion was scheduled for an hour and lasted a maximum of 80 minutes (depending on the number of participants). In total, fourteen structured discussions were held (Table 2.1). All stakeholders were invited to submit written material and evidence to support their perspectives.
Table 2.1. OECD case study individual structured discussions with stakeholders
In February and March 2021
Note: Each interview lasted 60-80 minutes. All interviewees were also invited to submit written material. Follow up invitations were sent to stakeholders who had not replied to offer different time slots for discussions or at a time of their convenience.
Box 2.1. Standard text in the OECD team’s invitation letters to stakeholders
We work at the Centre for Educational Research and Innovation within the OECD. Drawing on seven years of research on effective governance in complex education systems, the OECD developed a framework for strategic education governance. This aims to help policy makers and stakeholders apply the OECD’s accumulated knowledge and to shine the light on processes that support and sustain effective change.
The Flemish Department of Education and Training has asked us to conduct a case study on the introduction of standardised tests. Applying the OECD principles of strategic education governance, we would like to learn from you how standardised tests could best support your efforts to ensure high quality education for pupils. To familiarise you with our approach, we provide a brief overview of the framework and some of the related considerations for the introduction of standardised tests in Flanders.
Accountability provides reasons to other stakeholders for one’s actions and the actions of one’s organisation. Behind this stands a legitimation purpose, which relates to complying with existing laws and regulations on the one hand, and accounting for the quality and efficiency of education on the other. Accountability can provide recognition of efforts towards providing high quality education. Accountability is central to public trust in the functioning of the education system.
In what ways could standardised tests best contribute to this?
Capacity refers to the skills, resources and other enabling factors to carry out tasks and responsibilities. This includes individual skills and organisational structures, including allocating the requisite time and other resources to do so. In schools, important areas include the capacity to collect and analyse a breadth of evidence. This includes existing tests delivered by specific networks, the central tests (peilingen and paralleltoets), as well as evidence of learning progress via formative assessment and student portfolios; within the network, capacity includes developing tests available to schools and supporting the appropriate use of the results.
What will be the implications of the introduction of standardised tests? What are the opportunities to build on existing capacity? Will there be specific needs for the standardised tests?
Knowledge governance seeks to make relevant knowledge available and promote its use. This includes putting in place feedback channels, for instance, for schools and school boards to achieve their goals and engage in quality assurance. Feedback can support accountability to the public as well as school quality assurance processes. A core focus is on fostering adequate capability, opportunity and motivation of key stakeholders (teachers, school leaders, pedagogical advisory services, Flemish Department of Education) to make use of evidence for improving the quality of student learning.
What opportunities could the standardised tests bring for your work? How could you best use the results of standardised tests?
Stakeholder involvement integrates the perspectives and knowledge of schools, school boards, networks and other stakeholders in policy making. In Flanders, this safeguards the constitutional principle of freedom of education while creating suitable feedback channels and strengthening accountability to parents and the public. It also helps to identify capacity needs and address concerns, which may otherwise be overlooked.
How can your professional knowledge and expertise contribute to the introduction of standardised tests? What concerns do you have regarding the introduction of standardised tests?
Strategic thinking links knowing where to go with strategies of how to get there – especially when contexts change. With evidence from the sample-based assessments peilingen and international assessments, there is a shared concern among stakeholders on the overall quality of education provided to Flemish pupils. Attainment targets provide ‘anchors’ to help advance educational efforts towards agreed goals for students.
How could standardised tests best complement the attainment targets as a central ‘anchor’? What would be the role of standardised tests in advancing Flanders towards a common goal of educational excellence?
A whole-of-system perspective enables a look at the “big picture” of education in Flanders. It supports better learning and efficiency for stakeholders within the system through greater alignment and co‑ordination of efforts. In this way, it seeks to maximise synergies and minimise the duplication/waste of time and effort of those involved.
How could the introduction of standardised tests be most efficient for your work? How would you align these with existing quality assurance efforts?
2. Initial reporting of feedback from individual structured discussions with stakeholders (data feedback). The OECD team gave feedback to representatives of the ministerial cabinet on 29 March 2021 regarding stakeholder perceptions of their involvement at early stages in the development of standardised tests (see Chapter 3). The OECD team presented key points of the discussions with stakeholder representatives during the first meeting of the High-level forum on 12 May 2021. The High-level forum was established as a mechanism for feedback and information flow between stakeholders regarding the introduction of standardised tests.
3. Stakeholder reflection seminar on 9 June 2021 (data feedback and collection). This seminar was organised to provide feedback to stakeholders and to challenge them to think about their roles in the introduction and use of the standardised tests. Invitations were organised by the Department, extended to all stakeholders that had been invited to individual discussions with the OECD team in February and March, plus to researchers involved with the university centre and representatives from the Flemish strategic advisory council for education and training (VLOR). In addition, the OECD team had requested that each umbrella organisation invite some school leaders to join the seminar. The OECD team presented its analysis of feedback from stakeholder discussions in February and March. Participants were assigned to four working groups, each with a moderator and note taker from the Department. The working language was Dutch. Each working group discussed two questions:
What opportunities do the standardised tests bring for you?
What do you need to prepare to get the most out of them in June 2024?
The note takers sent key points to the OECD team to allow a presentation of a summary overview from working groups (in English).
In total, 36 people participated in the stakeholder reflection seminar. This included representatives from all umbrella organisations, the inspectorate, AGODI, AHOVOKS, the ministerial cabinet, academics from Gent University, Leuven Catholic University, Antwerp University and Vrije Universiteit Brussels, and parent associations and trade unions. The OECD team regretted that due to the timing of the seminar, student representatives were not able to attend. However, the OECD team received written feedback from the Flemish Student Association (VSK) on the two questions that are included in this report. It was grateful for the active participation of all representatives and to gain feedback via the working groups from the trade union (ACOD), parent association (KOOGO) and a range of researchers with whom it had not had the opportunity to discuss in February and March.
References
[1] Burns, T. and F. Köster (eds.) (2016), Governing Education in a Complex World, Educational Research and Innovation, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264255364-en.
[2] Burns, T., F. Köster and M. Fuster (2016), Education Governance in Action: Lessons from Case Studies, Educational Research and Innovation, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264262829-en.
[3] Shewbridge, C. and F. Köster (2019), Strategic education governance - Project Plan and Organisational Framework, http://www.oecd.org/education/ceri/SEG-Project-Plan-org-framework.pdf.