Finnish Education Evaluation Centre (FINEEC)

4. Conclusions

Abstract

This chapter provides the conclusions of the evaluation of the Finnish Education Evaluation Centre (FINEEC). It highlights the importance of FINEEC’s enhancement-led evaluation approach and its contributions to schools, higher education institutions, and municipalities, and its reputation, and impartial and highly credible institution. The chapter then summarises the nine key identified areas for improvement, as well as the strategies and actions for moving forward. The chapter concludes with suggestions for strengthening FINEEC’s contribution to policy making and the wider Finnish education system.

Focus and scope of the evaluation

Since its inception in 2014, the Finnish Education Evaluation Centre (FINEEC) has played an important role within the Finnish education sector. As an institution at the forefront of educational evaluation, FINEEC has played a crucial role in monitoring and assessing Finland's education landscape. It is widely recognised as an impartial, credible and relevant institution, with its outputs serving as invaluable tools for educational institutions.

However, the Finnish education system has undergone significant transformations in recent years, mirroring broader societal and economic shifts. For example, in the last decade, Finland has experienced a significant decline in performance in international assessments like the OECD Programme for International Student Assessment (PISA), raising concerns over the future trajectory of its education system. Concurrently, the employment outcomes for vocational education and training (VET) graduates have not met expectations, necessitating urgent attention. Moreover, the advent of new technological tools is reshaping learning paradigms, while contemporary issues such as student mental health, the integration of immigrants into the education system and bullying demand a renewed focus. Amid these shifts, a pivotal question arises: To what extent is FINEEC effectively responding to these new challenges, and what improvements can be implemented?

This study was conducted with the aim of inspiring strategic enhancement to FINEEC’s contribution to policy making. It involved producing and disseminating information and evidence aimed at designing and implementing policies, which is especially useful to the Ministry of Education and Culture (OKM), the Finnish National Agency for Education (EDUFI) and legislators.

However, it is crucial to highlight that the contribution of FINEEC to the Finnish education system extends beyond the inputs it provides to policy makers, which are the central focus of this study's analysis. FINEEC's enhancement-led evaluation approach provides invaluable data, analyses and insights that benefit schools, higher education institutions and municipalities, supporting them in improving internal processes, such as quality assurance, and fostering a culture of quality. Moreover, its thematic reports shed light on educational trends, assisting institutions in understanding and adapting to changes. Additionally, FINEEC's assessments of learning outcomes offer crucial information on student performance, enabling institutions to benchmark and identify areas for improvement both locally and nationally.

However, it is important to note that this study does not directly assess FINEEC’s performance in these areas, focusing instead on the institution's contributions to policy making. The evaluation only considers these aspects to the extent that they are related to or influence FINEEC’s capacity to contribute to policy making. In addition, this evaluation aims to identify areas for improvement, and therefore, the areas where FINEEC excels are not discussed in detail.

The relevance of this study extends beyond Finland and holds international relevance for several reasons. First, Finland is widely recognised as a benchmark in education, and it is esteemed for its high standards, top performance, and innovative practices. Additionally, FINEEC is regarded as an institution of international prestige with extensive networks. However, this report highlights that even the most successful educational systems face important challenges, underscoring the importance of adaptability and continuous improvement. Second, by studying Finland's responses to these challenges, other countries can gain insights into effective strategies and practices that can be adapted to their own evaluation systems in order to anticipate and mitigate similar issues within their own evaluation systems. Finally, by sharing its experience, Finland contributes to the global educational dialogue, enabling a collaborative exploration of new strategies and improvements to common challenges in the assessment of education systems.

Summary of key findings

Based on a comprehensive analysis of evidence collected through primary literature, examination of international literature, FINEEC documentation and outputs, stakeholder consultations, international expert workshops, and detailed discussions with staff from FINEEC and the OKM, the study presents nine key findings shown in Table 4.1, Table 4.2 and Table 4.3. The following sections suggest possible pathways for future development.

The nine findings are categorised into three main areas: FINEEC’s strategy and scope of work, its operations, and its contribution to policy making. Each category encapsulates distinct elements of FINEEC’s functionality:

• Strategy and scope of work: This pertains to the overall direction, focus and range of FINEEC’s evaluation activities, assessing how well these align with evolving educational priorities and societal needs.

• Operations: This focuses on FINEEC’s internal processes, including funding, workload management, and strategic integration across various evaluation strands, evaluating operational efficiency and quality of output.

• Contribution to policy making: This area examines the timeliness, accessibility and relevance of FINEEC's evaluation results and their direct impact on educational policy and reform, particularly focusing on VET and higher education.

Table 4.1. Key findings regarding FINEEC’s strategy and scope of work

Key finding	Description
1. There is a lack of agreement on the role FINEEC should play in policy making	FINEEC's role in proposing solutions is not well defined, which may affect its impact on educational reforms.
2. There is disagreement about FINEEC's autonomy in defining the Evaluation Plan	Differences in perception regarding FINEEC's autonomy may impact its strategic planning and flexibility.
3. FINEEC’s Evaluation Plan could provide greater flexibility to adapt to evolving priorities	The four-year Evaluation Plan could be more adaptable to respond effectively to changing needs and new priorities in education.

Table 4.2. Key findings regarding FINEEC’s operations

Key finding	Description
4. FINEEC’s undertakings sometimes exceed its resources, which could negatively impact the quality of its output	FINEEC faces challenges in aligning its activities with available resources, which could affect the quality of its work.
5. Seeking greater complementarity between FINEEC’s lines of work could help to generate new insights	There is an opportunity for FINEEC to enhance integration among its evaluations to deepen understanding of the education system and its challenges.

Table 4.3. Key findings regarding FINEEC’s contribution to policy making

Key finding	Description
6. Unequal access to results of learning outcomes assessments may undermine education equality	The current model for learning assessments could be refined to ensure more equitable access for all municipalities.
7. A greater focus on producing causal evidence would substantially contribute to policy making	Undertaking more research of a causal nature could strengthen FINEEC's ability to support educational policy development.
8. Data could be better leveraged to improve understanding of the relationship between institutional-level policies and student outcomes	There is potential for better utilisation of existing data to inform the relationship between educational policies and student outcomes.
9. Strengthening learning assessments in VET could support more informed policy making	In the absence of direct measures for VET graduates’ skills, FINEEC relies on teacher-assigned grades as a proxy, acknowledging the limitations of this approach. Despite efforts to complement this with qualitative data and occasional direct assessments, there is room for improvement in ensuring reliable data on VET graduates’ learning outcomes for informed policy making.

Recommendations for moving forward

Enhancing FINEEC’s strategic alignment, adaptability and responsiveness

To address the challenges identified in relation to FINEEC’s strategy and scope of the work, a comprehensive approach focused on collaboration, clarity and flexibility is required. First and foremost, it is imperative to establish a clear, shared understanding of FINEEC’s role within the policy-making landscape, particularly in relation to its evaluative functions and formulation of policy recommendations. This involves clarifying FINEEC’s role in developing policy solutions to the challenges identified in its studies and fostering agreement on the type of evidence required to support these proposals. A consensus among all stakeholders, facilitated by structured dialogues and formal agreements, would mitigate the current ambiguity and leverage FINEEC’s contributions more effectively.

Simultaneously, FINEEC’s Evaluation Plan requires greater adaptability to respond to the dynamic nature of educational needs. This could involve establishing a reserve fund or a systematic review process at mid‑cycle, enabling the incorporation of new priorities while maintaining ongoing evaluations’ relevance. Integrating these changes would require finding a balance between the need for both autonomy and responsiveness – i.e. ensuring that FINEEC remains an impartial, independent entity while at the same time ensuring that it has the flexibility and incentives to respond to the evolving needs of Finland’s educational system.

Furthermore, reconciling differing interpretations of FINEEC’s autonomy is crucial for improving communication and alignment with the OKM. Establishing clear, regular, structured communication channels between FINEEC and the OKM, as well as between FINEEC and other educational stakeholders, would ensure that evaluations are aligned with national priorities. This collaborative approach should extend to developing and adjusting the Evaluation Plan, ensuring it remains a living document that reflects current educational challenges and priorities.

Optimising FINEEC’s operations: Balancing resources and fostering evaluation synergies

To effectively manage the challenges and maximise the potential of FINEEC’s operations, a multifaceted approach focusing on resource management and synergistic work practices is necessary. Addressing the issue of resource constraints requires a clear demarcation between tasks funded through the annual state appropriation and those projects requested by external bodies such as the OKM or educational providers. FINEEC should ensure that the projects and evaluations included in the Evaluation Plan are executed with direct allocations. Additionally, FINEEC should ensure that projects not part of the Evaluation Plan are undertaken only if they come with adequate funding to cover all associated costs, thus preventing resource dilution and ensuring that the quality of FINEEC’s core activities is not compromised.

Furthermore, FINEEC could leverage its broad range of evaluations and research to foster greater complementarity and synergy between different lines of work. This involves breaking down silos within the organisation and promoting an integrated approach where insights from various evaluations can inform and enhance each other. By adopting a more holistic approach, FINEEC can produce richer, more interconnected findings that provide deeper insights into the Finnish education system. This could be facilitated by creating platforms for regular interaction and knowledge sharing among different teams and aligning thematic areas more closely with strategic priorities.

To implement these changes effectively, FINEEC could benefit from a more strategic oversight from the Evaluation Council, which could provide guidance for integrating different evaluation streams and ensure that FINEEC’s activities are aligned with broader educational goals. This approach could improve internal efficiency and the impact of evaluations on policy and ensure that FINEEC remains a relevant, agile and responsive actor in the Finnish education sector.

Enhancing FINEEC’s contributions to policy making through robust evidence

Several actions could be undertaken to enhance FINEEC’s contribution to policy making. First, FINEEC could consider evolving its assessment strategies to ensure broader, equitable access across municipalities regardless of its administrative or financial resources while working with the OKM to provide nuanced, contextual information to counteract the negative impacts of school rankings and promote a more balanced understanding of educational outcomes.

Furthermore, when addressing questions that necessitate causal evidence, FINEEC needs to adopt alternative methodologies. This would involve enhancing the robustness of its evaluations by, for example, integrating experimental or quasi-experimental designs where feasible and clarifying methodological limitations where they exist. It is important that FINEEC and the OKM engage in dialogue to establish a shared understanding of what constitutes robust evidence for policy making.

FINEEC could enhance its capacity to inform policy making by utilising existing data more strategically. Specifically, there could be a discussion around allowing third parties like research centres and universities to access and conduct research with FINEEC’s collected data. This approach would maximise the utility of the data, add external validity to FINEEC’s research, and enrich the academic and policy discourse with fresh perspectives, new investigations and methodologies. To facilitate this, FINEEC could refine its permissions framework to allow for greater flexibility in how the results are used while respecting potential privacy concerns.

Finally, this report highlights the need for more reliable assessments of VET learning outcomes. FINEEC might consider developing standardised assessment criteria and incorporating independent evaluators to ensure the reliability and validity of VET assessments. Expanding direct, standardised assessments to cover more skills and providers, coupled with work-embedded assessments, would offer a more comprehensive picture of occupational competence and support more informed policy making in the VET sector.