Melissa Mouthaan
OECD
Nóra Révai
OECD
Who Really Cares about Using Education Research in Policy and Practice?
Developing a Culture of Research Engagement
Educational Research and Innovation
1. Building a culture of research engagement in education
Abstract
This chapter introduces the theme of the report: developing a culture of research engagement. It discusses key concepts and lessons from the knowledge mobilisation literature. In particular, the chapter briefly explains what a culture of research engagement means at the organisational and system level and identifies two cross-cutting themes in developing such a culture. The two themes – developing skills and capacity and creating stable structures and processes – are then introduced along with a set of questions the remaining chapters in this publication will be exploring. Finally, the chapter outlines the structure of the report.
Introduction
In today’s dynamic and rapidly evolving world, evidence-informed decision making has emerged as a cornerstone in guiding effective education policy and practice. Research evidence in education serves as a compass, directing stakeholders towards informed choices that maximise student outcomes and drive educational progress. Evidence-based approaches empower educational systems to address inequities, enhance teaching and learning methodologies, support the efficient use of resources, and foster continuous improvement. Ultimately, they nurture the development of well-rounded individuals equipped with the knowledge and skills needed to become responsible and ethical agents of change to reshape our 21st century society in the face of major challenges.
However, harnessing research evidence in policy making and practice poses formidable challenges despite countries’ significant investments (OECD, 2022[1]). The OECD Centre for Educational Research and Innovation (CERI) has been supporting countries in overcoming these challenges since the late 1990s. Following an expansion of actors over the past two decades with an explicit role to reinforce research impact – from brokerage agencies and what works centres to roles such as government researchers, research champions in schools and ministries – CERI renewed its effort to help understand “what works in what works” by launching the Strengthening the Impact of Education Research project in 2021. As a first step, it mapped countries’ mechanisms and barriers to facilitate research use, and the diverse set of actors that play a role in this. The first results were published in the Who Cares About Using Education Research in Policy and Practice (OECD, 2022[1]). The present report continues that discussion and delves into the question of how we can develop a culture of using education research.
Creating a culture of evidence use is often highlighted as a key ingredient for strengthening research impact. However, what it really means and how we can establish or change the culture often remains implicit. For example, the term culture is frequently used as a complementary element to structures. Structures usually refer to something concrete, like establishing a brokerage agency (OECD, 2007[2]), creating a department in a ministry or setting up a formal network (OECD, 2022[1]), while culture is an elusive concept. Yet, researchers, experts and stakeholders appear to agree about the critical importance of this intangible factor to maximise the impact of education research.
There is a consensus that more effective research mobilisation requires all actors to develop or hone certain skills (Cordingley, 2016[3]). Practitioners and policy makers must be able to access, understand and interpret research, and translate it for their context. For research evidence to be more relevant for policy and practice, researchers need to understand the context of policy and practice. Actors also need to be able to communicate with each other, which often requires skilled facilitators who can span boundaries across different communities. In addition to individual skills, the collective capacity of teams and organisations and system-level capacity are necessary to create and nurture research engagement. Such skills and capacity have been widely explored in the health sector (Davies, Nutley and Mannion, 2000[4]; Belkhodja et al., 2007[5]), but it is a more recent field of study in education.
This publication intends to open the black box of evidence use culture by exploring the following questions:
What do we mean by a culture that supports research engagement? What are the characteristics of such a culture and how can we develop these in educational organisations and at the system level?
How can we develop individuals’ and teams’ skills, and organisational and systemic capacity for the quality use of research in policy making and practice?
Which structures and processes support the development (or transformation) of an organisational and system-level culture for better research engagement?
This report addresses these questions by examining two distinct levels: system and organisational. System refers to all elements – regulations, guidelines, structures, incentives, values, standards, etc. – that characterise aspects of research production and engagement at the level of an education system in a country or sub-national entity (e.g. state, province). Organisational refers to a wide range of educational organisations: schools, school boards, inspectorates, ministries of education, executive agencies, etc. Organisations and systems are, of course, made up of individuals. Individuals’ characteristics, roles and relationships are thus key elements of both organisational and system-level cultures and are discussed across both levels.
This report presents cutting-edge research from leading experts in the field of knowledge mobilisation and draws on analyses conducted as part of the OECD/CERI Strengthening the Impact of Education Research project. This includes an analysis of policy survey data collected in 2021 (Box 1.1) from education ministries in 37 systems, insights from two expert meetings organised in 2022, and two learning seminars held in the Netherlands and Flanders in 2022 (see Box 1.2).
Box 1.1. The Strengthening the Impact of Education Research policy survey
Survey design and data
The OECD Strengthening the Impact of Education Research policy survey – conducted from June to September 2021 – collected data on the mechanisms used to facilitate research use in countries/systems. Overall, 37 education systems from 29 countries1 responded to the survey. Responses represent the perspective of ministries of education at the national or sub-national (state, province, canton, etc.) level.
The survey covered the following themes:
actors, mechanisms and relationships that facilitate the use of research in policy making and practice
characteristics of policy makers’ use of research
drivers of and barriers to research use in policy and practice; covering issues of mindset and culture, resources, skills and capacity, as well as learning opportunities
actors and mechanisms of research production, including funding for research, the accessibility and relevance of research, the involvement of actors and their incentives.
As a follow-up to the survey, six countries2 were selected for further data collection through semi‑structured interviews to ensure that survey data were correctly interpreted and to better understand good practices and challenges with regard to using research in policy and practice.
Policy makers in the survey
The Strengthening the Impact of Education Research policy survey targeted the highest level of decision making in education (ministry/department of education). In federal systems, this corresponds to the state (province, canton, autonomous community, etc.) department. Ministries were asked to co‑ordinate the response across departments.
The follow-up interviews revealed that ministries of education had various definitions of policy makers. Interviewees most commonly associated the term with high-level ministry officials such as directors, deputy directors and director generals. Overall, there was a high degree of recognition that policy makers are those with influence over the policy-making process, rather than those tasked with implementing policies. Some systems, however, took a broader view, considering all those working at the ministry of education, as well as individuals in the executive and legislative branches of government.
As a result of the different understandings, comparisons between systems in policy survey data should be made with caution.
Meaning of research
Education research in the Strengthening the Impact of Education Research project is understood as a form of systematic investigation of educational and learning processes to increase or revise current knowledge. This is consistent with most definitions of research (e.g. Langer, Tripney and Gough (2016[6])). This conceptualisation recognises that research need not necessarily be conducted within academia or by researchers only. However, this definition does not consider (raw) information and data as “research” as such – only when these are analysed and investigated for a purpose.
The follow-up interviews to the survey showed that, overall, respondents had a similar understanding of research, although some placed a stronger focus on certain types of research. For example, some had primarily the analysis of international tests and surveys in mind; others were more focused on what works type of evidence.
1. OECD countries: Austria, Belgium (Flemish and French Communities), Canada (Quebec, Saskatchewan), Chile, Colombia, Costa Rica, the Czech Republic, Denmark, Estonia, Finland, Hungary, Iceland, Japan, Latvia, Lithuania, the Netherlands, New Zealand, Norway, Portugal, the Slovak Republic, Slovenia, Spain, Sweden, Switzerland (Appenzell Ausserrhoden, Lucerne, Nidwalden, Obwalden, St. Gallen, Uri, Zurich), the Republic of Türkiye, the United Kingdom (England) and the United States (Illinois). Non-member countries: the Russian Federation and South Africa.
2. Japan, New Zealand, Norway, Portugal, Slovenia and South Africa.
Source: OECD (2022[1]), Who Cares about Using Education Research in Policy and Practice? http://dx.doi.org/10.1787/d7ff793d-en.
This chapter first discusses key conceptual lessons from the knowledge mobilisation literature. It then provides a brief account of what a culture of research engagement means at the organisational and system levels and identifies two cross-cutting themes in developing such a culture. The two themes – developing skills and capacity and creating stable structures and processes – are then introduced along with a set of questions the remaining chapters in this publication will explore. Finally, the chapter outlines the structure of the report with a brief description of its parts and chapters.
A systems approach to research engagement: Rendering the abstract concrete
The three approaches to knowledge mobilisation, linear, relational and systems, presented in Best and Holmes’ (2010[7]) seminal paper have been applied to analyse knowledge mobilisation strategies in different public sectors. Studies have made it clear that neither linear research transfer and dissemination mechanisms nor relational ones that focus on partnerships and network building are sufficient by themselves to reinforce the use of research in policy and practice (Campbell et al., 2017[8]; Langer, Tripney and Gough, 2016[6]). Complex systems characterised by multiple actors interacting at multiple levels and non-linear feedback loops (Burns and Köster, 2016[9]) require systems approaches that act on several elements simultaneously. However, linear, relational and systems approaches should not be seen as mutually exclusive but rather as embedded in one another (Figure 1.1). Generating, synthesising and disseminating evidence is still fundamental, as are relationships among actors. These should be part of a systems approach that includes co-ordination mechanisms at the system level (Maxwell, Sharples and Coldwell, 2022[10]). Yet, policy survey data showed that linear and relational mechanisms still dominate the landscape of research mobilisation in most systems (OECD, 2022[1]).
Figure 1.1. Linear, relational and systems approaches to knowledge mobilisation
Source: Adapted from Boaz (2021[11]), Push or pull: What does harnessing the use of research involve and imply?
The first report of the project suggested a few avenues to move forward. It posited that changing the terms we use to speak about knowledge mobilisation will influence the discourse and may ultimately shape our thinking and actions. Moving away from terms such as knowledge transfer, translation and dissemination may help broaden the focus from linear mechanisms. Although less conspicuous, the research production and use dichotomy also reflects linear thinking to an extent, suggesting actors’ roles are limited to these categories. Yet it is not uncommon now for different actors to be involved in some way across stages of research production. In addition, research use refers to the direct and straightforward application of research. However, in reality, research evidence can rarely be applied in such a way. More often, teachers and policy makers interpret evidence, mixing it with their contextual knowledge, values and dispositions. Applying research recommendations directly, for example, implementing an evidence-based pedagogical intervention, is only one type of research use. It is not necessarily easy to retrace the original research findings in actions – teaching practices, policy design and implementation – because it has transformed through the process of engagement. Although it is difficult to clearly distinguish between use and engagement (Coldwell et al., 2017[12]), the term engagement reflects an active role from practitioners, policy makers and other actors who draw on research in their work. In the context of educational practice, Rickinson et al. (2022, pp. 141-142[13]) define quality research use as:
… thoughtful engagement with and implementation of appropriate research evidence, supported by a blend of individual and organisational enabling components within a complex system. […] thoughtful engagement and implementation reflect critical engagement with the research evidence, shared deliberation about its meaning and effective integration of aspects of the evidence within practice.
This second report makes an effort to use terms that better reflect systems thinking and mutuality.
The first report called for unpacking what research engagement means in practitioners’ and policy makers’ contexts. It recognised that creating relationships and dialogue among actors is necessary but not enough in itself. Similarly, co-producing research holds many promises, but it will not automatically result in high‑quality research engagement and generation.
Finally, the first report also proposed a stronger focus on some key elements of systems approaches: strategic leadership, incentives and resources. While it provided a few examples, it did not show what these elements should (or can) look like to reinforce research engagement in policy and practice. Applying the embedded model of approaches to knowledge mobilisation shown above also means that the various pieces should be aligned and co-ordinated in order to develop a culture of research engagement.
This second report aims to move from abstract to concrete and from superficial or rough indicators to a deep understanding of what systems thinking for research engagement looks like. It brings together case studies and analyses that give concrete examples of strong forms of engagement, showing what creating a culture of research engagement looks like.
What do we mean by a culture of research engagement?
While the term culture in the context of research use is elusive, there is a substantive body of literature on organisational culture specifically relating to this theme studied extensively in the health sector, and more recently in education as well. However, a culture at the system level is often loosely defined or referred to implicitly.
Culture at the organisational level
To support policies, we need to consider whether culture is inherent in nature or an attribute of an organisation or system (Davies, Nutley and Mannion, 2000[4]). If we consider culture to be an attribute it suggests that culture can be transformed, so it is meaningful to identify levers of change and provide recommendations. Some argue that the truth lies in between: an organisation’s culture emerges to some extent unpredictably from elements of the organisation, which implies that while it is not fully controllable, its characteristics can be described and assessed (Davies, Nutley and Mannion, 2000[4]). From a policy perspective, it may seem desirable for culture to be something we can transform or shape towards a given goal. The various chapters of this publication will probe this question and investigate the enablers of a strong culture of research engagement.
Organisational culture is often described as “the way things are done around here, as well the way things are understood, judged, and valued” (Davies, Nutley and Mannion, 2000, p. 112[4]). In this sense, organisational culture includes shared values, beliefs, attitudes, norms and standards, and language (Davies, Nutley and Mannion, 2000[4]; Belkhodja et al., 2007[5]). Concretely, in the context of educational practice, research use being a norm embedded in a school’s way of working can involve:
promoting research use in the school’s policy and planning documents
having an ethos that encourages staff to reflect on their practice and try research-based approaches
having a deliberate strategy to advance evidence-informed practices (Rickinson et al., 2022[13]).
These can easily be made to fit the policy context: similar norms can be interpreted for a ministry, executive agency, inspectorate or local authority. In terms of attitudes and values, a research use culture reflects openness to research: “research constitutes a preferred source of information” (Belkhodja et al., 2007, p. 392[5]). It also demonstrates “individual and organisational commitment to promote research culture”, which is reflected in the intensity of research engagement (Belkhodja et al., 2007, p. 392[5]).
Organisational culture can be analysed at the level of individuals (people), and relationships and interactions between them (Belkhodja et al., 2007[5]). People’s skills, mindsets and relationships can all be enablers of (or barriers to) quality research engagement (Rickinson et al., 2022[13]). In addition, organisational culture emerges from and is shaped not only by people and their interactions, but also by organisational structures and infrastructure, processes, leadership, and resources (Davies, Nutley and Mannion, 2000[4]; Rickinson et al., 2022[13]; Slade, Philip and Morris, 2018[14]). Individual and organisational factors all mutually influence one another. In addition, organisational culture can also be influenced by external factors: the status and identity of the profession, regulations, incentives, systemic mechanisms, etc. (Rickinson et al., 2022[13]; Davies, Nutley and Mannion, 2000[4]).
Culture at the system level
It has long been recognised that people’s and organisations’ level and quality of research engagement are strongly influenced by systemic factors (Slade, Philip and Morris, 2018[14]). Recent literature on knowledge mobilisation speaks about “evidence ecosystems” and unanimously points to the necessity of a system‑level culture that recognises the importance of generating quality evidence and actively promotes its use [e.g. (Gough, Thomas and Oliver, 2019[15]; Boaz and Nutley, 2019[16]; Sharples, 2013[17]; Maxwell, Sharples and Coldwell, 2022[10])]. A system-level culture is also reflected in policy-oriented literature, such as the work of international organisations and policy documents at the national level. To cite recent examples, the OECD/CERI’s work on Governing Complex Education Systems (Burns and Köster, 2016[9]; Burns, Köster and Fuster, 2016[18]) and Strategic Education Governance highlight the following as key elements of knowledge governance at the system level:
promoting the production of adequate evidence
mobilising produced evidence for convenient use
stimulating a culture of evidence use
nurturing evidence-related capacities (Shewbridge and Köster, 2019[19]).
The United Nations Educational, Scientific and Cultural Organization (UNESCO) also places evidence generation and use high on the education agenda (NORRAG, 2022[20]; IIEP-UNESCO, 2021[21]). Organisations with an explicit mission, and in some cases government mandate, to promote and strengthen the generation and use of research evidence also emphasise systemic factors (Education.org, 2021[22]; Maxwell, Sharples and Coldwell, 2022[10]).
However, describing the characteristics of a system-level culture and identifying the “key nodes” that have a large impact on research engagement is much more difficult. As opposed to the clearly delimited nature of an organisation, an education system consists of a large number of actors whose involvement with and in research has been expanding over the past decades (Burns and Köster, 2016[9]). Figure 1.2 represents a set of potential actors that may all influence the system-level culture. The group of actors and their roles vary across systems.
Figure 1.2. Possible landscape of educational actors influencing research engagement
Similar to organisational culture, a system-level culture includes the values, attitudes, norms and standards around research itself, its production and engagement with it. This may be reflected in explicit research policies and strategies; regulations around evidence use in policy and practice, for instance, through systems of accountability; links among research, policy and practice actors; incentives for actors, etc. (Slade, Philip and Morris, 2018[14]). The extent of alignment (coherence and co-ordination) of the various elements may also reflect the strength of a research engagement culture at the system level (Godfrey and Brown, 2018[23]; Maxwell, Sharples and Coldwell, 2022[10]).
Given the large number of actors, the question is whether we can actually speak about a system-level culture as opposed to multiple cultures. This question is relevant even within a single organisation. Martin [ (1992[24]) in Davies, Nutley and Mannion (2000[4])] talks about cultures that may be integrated (with a wide consensus about basic beliefs), differentiated (with multiple, incompatible views across groups) and fragmented (with no shared norms and large differences even within sub-groups). The diversity of actors at the level of a system likely implies differentiated cultures at best, if not fragmented.
A commonly cited obstacle to research mobilisation is, in fact, the different cultures of the research, practitioner and policy communities (Locock and Boaz, 2004[25]; Shewchuk and Farley-Ripple, 2022[26]; Cooper, 2014[27]). These cultures involve different understandings of research evidence itself, its relevance and use (Lomas, 2007[28]; Ward, House and Hamer, 2009[29]; OECD, 2022[1]). Motivations for producing research and engaging with it also differ and the time frames for corresponding processes are not aligned, or even incompatible (OECD, 2022[1]). Various strategies have been suggested and tested to bridge this cultural gap, ranging from creating formal intermediary organisations and roles that span boundaries and mediate between the different communities to collaborative research production involving researchers and practitioners/policy makers. While some of these initiatives show promising results (Nutley, Walter and Davies, 2009[30]; Wiggins et al., 2019[31]; Gu et al., 2021[32]; Langer, Tripney and Gough, 2016[6]), there is a general lack of understanding of their effectiveness and impact (Torres and Steponavičius, 2022[33]; Oliver et al., 2022[34]).
Strengthening a system-level culture of research engagement thus requires understanding the variety of subcultures and finding ways in which these can be bridged. Notably, the various efforts and debates have laid bare the absence of adequate incentives for researchers to engage policy makers and vice versa, while prompting systems to consider how they can better incentivise processes of mutual exchange and fruitful engagement. Clearly, national policies of education, but also policies on science, research and innovation, play an important role in this. Other actors, such as brokerage organisations, policy networks and research councils, can also be system leaders.
Two emerging themes
Two broad cross-cutting themes seem to be key for both an organisational and system-level culture of research engagement. These themes emerged from the 2022 OECD learning seminars (Box 1.2) specifically with respect to research engagement in policy, but they can be applied to the practice context too. The first relates to human resource strategies to build individual and collective competences for better research engagement and provide appropriate professional learning and development. The second theme was identifying, describing, creating and maintaining stable structures and processes to support the development of a culture of research engagement.
Box 1.2. Strengthening the Impact of Education Research learning seminars
The Strengthening the Impact of Education Research learning seminars are part of the CERI learning seminars series. They are in-person, 1.5-day events with ~25-30 participants from ~5-7 countries that provide extensive opportunities for reflection, critical enquiry and exchange.
Two learning seminars were held in 2022 that focused on empowering civil servants and policy organisations to use research systematically and well in policy. The host countries, the Netherlands and Belgium (Flanders), identified similar challenges to the systematic use of education research in policy making and decided to work together and focus on civil servants’ skills and collective civil service professionalism to address these. Finland, Ireland and Norway participated as peer countries.
Two evidence journeys in policy making
The Netherlands analysed how it used evidence in the curriculum revision process using the “Quality Use of Research in Education” (QURE) framework.
Flanders presented the evidence use journey in the process of introducing standardised testing using the OECD Strategic Education Governance framework as an analytical lens.
In both countries, a diverse panel of stakeholders provided insight into the reality of research use from different perspectives.
Two themes for creating a culture of evidence use
Two themes emerged in the first seminar and were explored in depth in the second:
human resource strategies to build collective civil service professionalism
stable structures and processes for a cumulative knowledge base.
How can we develop skills and capacity for a systematic and quality engagement with research?
Building a culture of research engagement requires investing in people to ensure the right skills and capacity, in the right measures, within and across organisations of practice, and policy to systematically and thoughtfully engage with research. There are promising developments in this area and thus ample scope for mutual peer learning – both across countries and between policy- and practice-oriented organisations.
Within ministries, there has been renewed debate on civil service professionalism focusing on the individual and team competences needed for civil servants to engage well with research, and to use research evidence thoughtfully in decision making. In many cases, this has started with improving research literacy among policy makers. As Oliver and Cairney (2019[35]) highlight, a strong understanding of research evidence encompasses an understanding of the quality and limitations of evidence and research tools, and what research(ers) can (and can’t) deliver. A good understanding of different research tools can better equip civil servants to understand what type of research they require, or how to apply existing research to serve their needs. This includes understanding the limitations of knowledge produced by research: often, knowledge doesn’t accumulate to the extent of providing concrete “proof” that a specific policy option should be pursued – nor does it remove the need for deliberation (Cairney, 2019[36]; Topp et al., 2018[37]). Increasingly, systems and organisations are advocating for a shift in how we view the (education) policymaking profession in the 21st century, with varying degrees of research literacy increasingly considered to be a necessary individual or team competence.
In tangent, the debate on research “impact” beyond traditional academic measures has gained a strong foothold in research funding bodies, universities and other research institutions. There is now a more general acceptance that evidence-informed policies require a more “policy-literate research community” in addition to a research-literate policy community (Tilley and Laycock, 2000, p. 213[38]). Engaging a wider range of audiences – including policy makers and practitioners – is now increasingly expected of researchers in many systems and at different stages of research cycles. To achieve this type of engagement, researchers require the skills to understand the contexts in which their research can be applied and to communicate their findings to relevant audiences, including in formats and through venues that are not traditional to academic cultures. Increasingly, researchers may also be expected to collaborate with policy makers and practitioners and co-produce research.
Many systems are striving to find ways to support practitioners in developing their capacity to engage with research. Practitioners need appropriate skills to be able to engage in structured conversations around evidence and to be willing to challenge their existing assumptions explicitly in these conversations. By developing such a skillset, practitioners can feel safe to expose gaps in their own knowledge and become more confident in applying evidence to their practice and trialling new approaches that might emerge from these conversations around evidence.
Finally, it is becoming increasingly clear that closing the gaps between research production and its use in policy and practice requires those who have the skills to act as brokers, research translators, leaders and boundary spanners (Cooper, 2014[27]; Shewchuk and Farley-Ripple, 2022[26]). Research knowledge intermediaries are positioned between different communities of research, policy and practice and carry out the types of activities that connect actors between these groups, and that connect evidence with actors, thus bridging different communities. A culture of research engagement will require understanding who the intermediaries are and what skills are needed for intermediary activities.
The chapters in this publication explore the individual and collective skills the various actors – researchers, policy makers, practitioners, brokers and leaders – need to promote a culture of research engagement and develop capacity. They provide examples of tools and learning opportunities that can develop and harness these skills; and of schemes that promote mutual learning and understanding between different communities and actors.
Which structures and processes support a culture of research engagement?
As highlighted above, embedding a research culture in organisations requires a “regulatory environment, governance and organisational structures” but also “systems, tools, resources and time” (Slade, Philip and Morris, 2018, p. 9[14]).
What do we mean by stable structures and processes? We define stable structures as arrangements and mechanisms put in place at the system or organisational level. We emphasise “stable” because creating or adopting a culture of research engagement over time requires long-term mechanisms that are resistant to organisational, staff or leadership changes and political shifts that commonly occur in policy or practice organisations. At the same time, stable does not mean rigid and unchangeable. These structures and processes need to be flexible enough to adapt to changing circumstances, such as emerging needs related to technological progress and other societal phenomena.
Structures and processes are necessary to support the development of successful human resource strategies detailed in the previous section. In many cases, building the skills of different actors to engage with research and with other actors in the education research production landscape is not straightforward, and depends on the structures and processes that exist within organisations and systems that enable such engagement. For example, practitioners need structures within their schools that allow them to participate in conversations around evidence, such as school networks aimed at professional development and mutual learning. Researchers need structures and processes that enable the development of both the skills and incentives for them to engage broader audiences with their work. Structures and processes are also needed to support the development of a robust knowledge base in education, where such processes could include a long-term research strategy for education research or dialogue platforms for stakeholder engagement.
The Strengthening the Impact of Education Research policy survey allowed us to first take stock of the various mechanisms that scale up research use in OECD systems, as well as the commonly reported barriers to the use of research in policy and practice (OECD, 2022[1]). This analysis concluded that despite some systems reporting a reasonable number of mechanisms to support the generation of research relevant for policy and practice, facilitating interactions between actors and supporting their engagement with research, there remain several barriers to generating research and engaging with it (Torres, 2022, pp. 111-112[39]). This implies that something is still lacking, these mechanisms do not (yet) work, are not sufficiently co-ordinated or are simply not enough in themselves. This publication builds on these findings to explore the different types of structures and processes that can support skills and capacity development among different actors for research engagement, improve the accessibility of education research, and allow a better co-ordination of education research production.
The various chapters in this publication explore the structures and processes that support the development of research engagement in different types of educational organisations and at the system level. They help elicit the key characteristics of such structures and processes, and organisational and systemic conditions for them to successfully facilitate research engagement.
Purpose and structure of this report
This report builds on the initial findings of the OECD publication Strengthening the Impact of Education Research (OECD, 2022[1]) and further develops the analysis in a few specific ways. First, as with the first publication, a number of leading experts in knowledge mobilisation (evidence-informed policy/practice) present their cutting-edge research. Second, further analysis of the OECD Strengthening the Impact of Education Research policy survey is reported. While the first publication provided an initial mapping of structures, processes, relationships and barriers that relate to evidence use in policy and practice, this report seeks to move from the abstract to the concrete. It does so primarily by deepening the analysis of issues, and to present concrete examples and case studies from organisations of policy and practice. The report is structured as follows.
Part I: Conceptual landscape
The first part sets the scene for the report with one introductory chapter.
The current Chapter 1 presents the rationale for the work and the questions explored in the report.
Part II: A culture of research engagement at the system level
The second part focuses on culture at the system level and includes six chapters.
In Chapter 2, Melissa Mouthaan and Mykolas Steponavičius examine aspects of the generation of education research drawing on the OECD Strengthening the Impact of Education policy survey. The authors present policy mechanisms that can help build a high-quality, robust knowledge base in education for policy makers and practitioners, as well as the associated challenges. The chapter discusses different types of education research, their relevance to policy makers and their accessibility. It also looks into the promises of collaborative research to incentivise knowledge mobilisation, outlining the range of open questions about its benefits and drawbacks.
Chapter 3 investigates dimensions of a culture of research engagement in OECD countries drawing on the Strengthening the Impact of Education policy survey. Jordan Hill and José Manuel Torres analyse the key characteristics of a research engagement culture, and policy makers’ and practitioners’ levels of skills and learning opportunities. The chapter identifies systemic enablers of culture and skills and illustrates promising practices that help strengthen a research engagement culture.
In Chapter 4, Rien Rouw and Quirine van der Hoeven present the use of knowledge in the Dutch Ministry of Education, Culture and Science in a policy process: curriculum revision. The authors use the Quality Use of Research Evidence framework to analyse the evidence use journey. This includes the knowledge infrastructure and types of knowledge that the different stages of the revision process drew upon, and various aspects of the culture of knowledge use within the ministry. The authors assess the appropriateness of knowledge and the levels and quality of policy makers’ engagement with the evidence.
In Chapter 5, Jeroen Backs, Bieke de Fraine, Miekatrien Sterck, Jonathan D’haese and Katrijn Ballet examine the use of evidence in the policy process of implementing standardised testing in the Flemish Community of Belgium. The authors use the OECD framework of Strategic Education Governance to analyse the evidence use journey. The chapter first presents the actors and their roles in the use of evidence throughout the process. It then analyses the use of evidence from a governance perspective, considering knowledge governance, capacity in the system, stakeholder involvement and a whole‑of‑system perspective.
Chapter 6 reflects on the value of analysing evidence use journeys. Nóra Révai provides a brief comparative analysis of the Dutch and Flemish evidence use journeys presented in the previous chapters. The chapter explores how the two frameworks enable understanding the culture and skills around evidence use. The chapter also reflects on the analytical exercise itself and describes the Dutch and Flemish analyses as policy action research.
Chapter 7 is a case study of research use in education policy in Norway. Melissa Mouthaan, Håkon Kavli, Elisabeth Buk-Berge and Kari-Elisabeth Vambeseth Skogen explore how research has been used to govern and develop the education sector, mapping the landscape of stakeholders and strategies in the policy arena. The chapter presents policy makers’ perceptions of the strengths and remaining gaps when it comes to integrating knowledge and evidence in policy. It presents the Norway public sector PhD scheme as a promising initiative aimed to facilitate research use in policy.
Part III: A culture of research engagement within and across research, policy and practice-oriented organisations
The third part focuses on culture within and across research, policy and practice-oriented organisations.
In Chapter 8, Chris Brown and Cindy Poortman examine the potential role of learning conversations, a type of collaborative learning among teachers, in improving teaching and student outcomes. The chapter describes the approach taken in learning conversations to enable practitioners to engage with research evidence, systematically generate and test ideas, and respond to problems or situations that practitioners report. The analysis also reflects on the effectiveness of learning conversations for educational improvement.
In Chapter 9, Amanda Cooper, Michelle Searle, Stephen MacGregor and Tiina Kukkonen discuss arts‑informed approaches as a collaborative method used in the context of research-practice partnerships. Drawing insights from three arts-based initiatives in research-practice partnerships in Canada, they argue that such approaches offer a novel way for practitioners and researchers to engage collectively with research while also advancing equity in schools. The chapter proposes a model to assess the principles, strategies and impacts of research-practice partnerships.
In Chapter 10, Elizabeth Farley-Ripple, Hilary Mead and Katherine Tilley present insights drawn from their analysis of four case study schools in the United States classified as “deep users” of research. The chapter explores how these schools enable deep research engagement in their individual contexts. The authors propose that the synergies between culture, leadership, processes and structures are key to the deep use of research in these schools.
Part IV: Forging connections across the organisational and system levels
The fourth part focuses on creating a culture of research engagement across organisations and systems.
Chapter 11 presents five case studies on leadership and culture within and across organisations in Chile, England, Ireland, Norway and Wales. The chapter explores how a culture of thoughtful engagement with knowledge and evidence is created within policy and practice-oriented organisations and the role of networks and leadership in this process. It starts with a short overview of the development of the leadership approaches in such organisations. This is followed by five case studies written by Toby Greany, Georgina Hudson, Jørn Pedersen, Inger Sofie Berge Hurlen, Huw Morris, John O’Connor and Mauricio Pino‑Yancovic. The chapter concludes with an analysis of three common themes that emerge across the case studies while also noting where there is scope for further research in a few key areas.
Chapter 12 draws together the lessons learnt from the previous chapters. Nóra Révai and Melissa Mouthaan highlight the overall messages that emerged from the research presented in the report. The chapter notes how these messages contribute to a better understanding of effective knowledge mobilisation and takes stock of the remaining challenges for building a culture of research engagement. It ends with a description of how this agenda can be advanced in the coming years.
References
[5] Belkhodja, O. et al. (2007), “The extent and organizational determinants of research utilization in Canadian health services organizations”, Science Communication, Vol. 28/3, pp. 377-417, https://doi.org/10.1177/1075547006298486.
[7] Best, A. and B. Holmes (2010), “Systems thinking, knowledge and action: Towards better models and methods”, Evidence & Policy: A Journal of Research, Debate and Practice, Vol. 6/2, pp. 145-159, https://doi.org/10.1332/174426410X502284.
[11] Boaz, A. (2021), Push or pull: What does harnessing the use of research involve and imply?, Presentation given at the OECD Strengthening the Impact of Education Research project meeting on 27 April 2021.
[16] Boaz, A. and S. Nutley (2019), “Using evidence”, in Boaz, A. et al. (eds.), What Works Now? Evidence-Informed Policy and Practice, Policy Press, Bristol.
[9] Burns, T. and F. Köster (eds.) (2016), Governing Education in a Complex World, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/9789264255364-en.
[18] Burns, T., F. Köster and M. Fuster (2016), Education Governance in Action: Lessons from Case Studies, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/9789264262829-en.
[36] Cairney, P. (2019), “Evidence and policy making”, in Boaz, A. et al. (eds.), What Works Now? Evidence-informed Policy and Practice, Policy Press, Bristol.
[8] Campbell, C. et al. (2017), “Developing a knowledge network for applied education research to mobilise evidence in and for educational practice”, Educational Research, Vol. 59/2, pp. 209-227, https://doi.org/10.1080/00131881.2017.1310364.
[12] Coldwell, M. et al. (2017), Evidence-informed Teaching: An Evaluation of Progress in England, Sheffield Hallam University, University College London and Durham University, https://www.gov.uk/government/publications/evidence-informed-teaching-evaluation-of-progress-in-england.
[27] Cooper, A. (2014), “Knowledge mobilisation in education across Canada: A cross-case analysis of 44 research brokering organisations”, Evidence & Policy, Vol. 10/1, pp. 29-59, https://doi.org/10.1332/174426413x662806.
[3] Cordingley, P. (2016), “Knowledge and research use in local capacity building”, in Burns, T. and F. Köster (eds.), Governing Education in a Complex World, OECD Publishing, Paris, https://doi.org/10.1787/9789264255364-9-en.
[4] Davies, H., S. Nutley and R. Mannion (2000), “Organisational culture and quality of health care”, Quality in Health Care: QHC, Vol. 9/2, pp. 111-119, https://doi.org/10.1136/QHC.9.2.111.
[22] Education.org (2021), Calling for an Education Knowledge Bridge: A White Paper to Advance Evidence Use in Education, Education.org, https://whitepaper.education.org/download/white_paper.pdf.
[23] Godfrey, D. and C. Brown (2018), “How effective is the research and development ecosystem for England’s schools?”, London Review of Education, Vol. 16/1, pp. 36-51, https://doi.org/10.18546/lre.16.1.12.
[15] Gough, D., J. Thomas and S. Oliver (2019), “Clarifying differences between reviews within evidence ecosystems”, Systematic Reviews, Vol. 8/170, https://doi.org/10.1186/s13643-019-1089-2.
[32] Gu, Q. et al. (2021), The Research Schools Programme in Opportunity Areas: Investigating the Impact of Research Schools in Promoting Better Outcomes in Schools, Education Endowment Foundation, Millbank, http://www.educationendowmentfoundation.org.uk.
[21] IIEP-UNESCO (2021), IIEP in Action, UNESCO International Institute for Educational Planning, https://unesdoc.unesco.org/ark:/48223/pf0000379692.locale=en.
[6] Langer, L., J. Tripney and D. Gough (2016), The Science of Using Science: Researching the Use of Research Evidence in Decision-Making, EPPI-Centre, Institute of Education, University College London, London, https://eppi.ioe.ac.uk/cms/Portals/0/PDF%20reviews%20and%20summaries/Science%202016%20Langer%20report.pdf?ver=2016-04-18-142701-867.
[25] Locock, L. and A. Boaz (2004), “Research, policy and practice – Worlds apart?”, Social Policy and Society, Vol. 3/4, pp. 375-384, https://doi.org/10.1017/s1474746404002003.
[28] Lomas, J. (2007), “The in-between world of knowledge brokering”, BMJ, Vol. 334, pp. 129-132, https://doi.org/10.1136/bmj.39038.593380.AE.
[24] Martin, J. (1992), Culture in Organizations: Three Perspectives, Oxford University Press, Oxford.
[10] Maxwell, B., J. Sharples and M. Coldwell (2022), “Developing a systems‐based approach to research use in education”, Review of Education, Vol. 10/3, https://doi.org/10.1002/rev3.3368.
[20] NORRAG (2022), Strategic Review of Global and Regional Evidence and Knowledge Initiatives, Networks and Platforms in Education, Network for International Policies and Cooperation in Education and Training, http://www.norrag.org.
[30] Nutley, S., I. Walter and H. Davies (2009), “Promoting evidence-based practice: Models and mechanisms from cross-sector review”, Research on Social Work Practice, Vol. 19/5, pp. 552-559, https://doi.org/10.1177/1049731509335496.
[1] OECD (2022), Who Cares about Using Education Research in Policy and Practice?: Strengthening Research Engagement, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/d7ff793d-en.
[2] OECD (2007), Evidence in Education: Linking Research and Policy, OECD Publishing, Paris, https://doi.org/10.1787/9789264033672-en.
[35] Oliver, K. and P. Cairney (2019), “The dos and don’ts of influencing policy: A systematic review of advice to academics”, Palgrave Communications, Vol. 5/21, https://doi.org/10.1057/s41599-019-0232-y.
[34] Oliver, K. et al. (2022), “What works to promote research-policy engagement?”, Evidence & Policy, Vol. 18/4, pp. 691-713, https://doi.org/10.1332/174426421x16420918447616.
[13] Rickinson, M. et al. (2022), “A framework for understanding the quality of evidence use in education”, Educational Research, Vol. 64/2, pp. 133-158, https://doi.org/10.1080/00131881.2022.2054452.
[17] Sharples, J. (2013), Evidence for the Frontline: A Report for the Alliance for Useful Evidence, Alliance for Useful Evidence, London.
[19] Shewbridge, C. and F. Köster (2019), Strategic Education Governance: Project Plan and Organisational Framework, OECD, Paris, https://www.oecd.org/education/ceri/SEG-Project-Plan-org-framework.pdf.
[26] Shewchuk, S. and E. Farley-Ripple (2022), Understanding Brokerage in Education: Backward Tracking from Practice to Research, Center for Research Use in Education, University of Delaware, Newark, DE, http://www.research4schools.org/wp-content/uploads/2023/04/Backward-Tracking-P2R.pdf.
[14] Slade, S., K. Philip and M. Morris (2018), “Frameworks for embedding a research culture in allied health practice: a rapid review”, Health Research Policy and Systems, Vol. 16/1, https://doi.org/10.1186/s12961-018-0304-2.
[38] Tilley, N. and G. Laycock (2000), “Joining up research, policy and practice about crime”, Policy Studies, Vol. 21/3, pp. 213-227, https://doi.org/10.1080/01442870020019507.
[37] Topp, L. et al. (2018), “Knowledge management for policy impact: The case of the European Commission’s Joint Research Centre”, Palgrave Communications, Vol. 4/87, https://doi.org/10.1057/s41599-018-0143-3.
[39] Torres, J. (2022), “Facilitating research use: Scary barriers (and super mechanisms)”, in Who Cares about Using Education Research in Policy and Practice?: Strengthening Research Engagement, OECD Publishing, Paris, https://doi.org/10.1787/cd4e8487-en.
[33] Torres, J. and M. Steponavičius (2022), “More than just a go-between: The role of intermediaries in knowledge mobilisation”, OECD Education Working Papers, No. 285, OECD Publishing, Paris, https://doi.org/10.1787/aa29cfd3-en.
[29] Ward, V., A. House and S. Hamer (2009), “Knowledge brokering: The missing link in the evidence to action chain?”, Evidence and Policy, Vol. 5/3, pp. 267-279, https://doi.org/10.1332/174426409X463811.
[31] Wiggins, M. et al. (2019), The RISE Project: Evidence-Informed School Improvement, Education Endowment Foundation, London, https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/the-rise-project-evidence-informed-school-improvement.