Building Capacity for Evidence-Informed Policy-Making

Annex A. Mapping of interventions, strategies and tools onto the skills framework for EIPM

 

Skill	Definition	Types of Interventions	Examples of Interventions
Understanding	Policy makers with this skill will understand the role of evidence and its place in the policy making cycle, as well as the challenges and opportunities which come with the use of evidence. This will be underpinned by knowledge of different research methods and their purpose, as well as the fundamentals of statistical and data literacy (including big data, machine learning and artificial intelligence).	Diagnostic tools of individual capacity: Used to determine individual capacity and motivation to use evidence in their work.	Australia’s Staff Assessment of enGagement with Evidence (SAGE) – a tool that aims to provide an evaluation of current levels of research engagement and use. SAGE combines an interview and document analysis to concretely assess how policy makers engaged with research, how research was used and what barriers impacted the use of research in relation to a specific policy product.
			Australia’s Seeking, Engaging with and Evaluating Research (SEER) – a practical tool that assists policy agencies in assessing their capacity to use research, and evaluates the success of initiatives designed to improve evidence use in policy making.
		Senior Civil Service programmes: These programmes work to encourage senior civil servants to use evidence and trains the senior civil service in how to create a culture within their organisations of evidence use.	Finland’s Public Sector Leadership training – a programme that strengthens the ability of public-sector leaders to handle challenges and support the public sector in the reform of its social role.
Obtaining	Policy makers with this skill be able to gather existing evidence in their own policy area and know who to turn to as sources of evidence synthesis. They will be able to recognise where there are evidence gaps and commission high quality evidence to fill these gaps using a range of project management techniques.	Access to research through online databases: Providing policy makers with access to research articles or syntheses via an online database aims to maximise access to specific types of research and increase policy makers’ confidence in accessing and using such content.	Campbell Collaboration – promotes positive social and economic change through the production and use of systematic reviews and other evidence synthesis for evidence-informed policy and practice.
			Cochrane Library – is global independent network of researchers, professionals, patients, carers and people interested in health. The Cochrane Library contains systematic reviews of medical and healthcare interventions.
		Disseminating tailored syntheses of evidence: Increasing the ease of access through having relevant research evidence synthesized, tailored for specific users and sent directly to policy makers, in order to increase policy makers’ use of evidence.	Argentina’s Health Policy Research briefs – the WHO runs a programme where they product policy briefs for policy makers on research evidence tailored to the policy makers’ needs.
			UK What Works Centres produce a range of policy briefs to disseminate key messages to its target audience, including policy makers.
		Commissioning research and reviews: Ensuring policy makers are able to commission evidence when there is gaps in the research. This can lead to policy makers increasing their engagement with and control of the research which in turn would increase the relevance and applicability of the research.	UK’s Policy Reviews Facility – a place where policy teams, government analysts and academic experts from three universities work closely together to determine the focus of systematic review products to best meet the needs of policy work.
			Australia’s ‘Evidence Check’ – assists Australian policy makers in commissioning high quality reviews of research with knowledge brokers who assist to formulate and refine the scope of and questions addressed by the review.
			The US’s Office of Management and Budget has developed grant review and support structures to assess the quality of evidence being commissioned by policy makers and government agencies.
		Seminars to present research findings Presenting policy makers with relevant research. Seminars are generally well received by participants and preferred to reading reports by many.	The Joint Research Centre’s lunchtime science lecture series – The seminars features JRC scientists and researchers, as well as external guest speakers. The seminars are also web streamed so participants can join from anywhere.
Interrogating and Assessing	Policy makers with this skill will make use of a set of holistic, systemic and critical thinking tools. They will be able to assess the provenance, reliability and appropriateness of evidence. They will have an ability to interrogate evidence by critically assessing its quality and context, using a range of techniques to challenge assumptions and biases.	Intensive skills training programmes: Increasing policy makers’ capacity to engage with and understand the more technical sides of research findings. Training programmes can be very effective when they are learner centred and participatory, ideally embedded within long-term strategies for professional development.	INGSA’s capacity building initiatives – a collaborative platform for policy exchange, capacity building and research across diverse global science advisory organisations and national systems. The initiative provides workshops, conferences, tools and guidance.
			The UK’s Alliance for Useful Evidence – runs an Evidence Masterclass where policy makers can learn about how to use evidence in their policy work and can practice their new skills through simulations.
		Knowledge brokers (organisations) Knowledge brokers can help to facilitate policymakers’ access to research evidence by helping them to navigate research material that may be unfamiliar.	Poland’s Centre for Evaluation and Analysis of Public Policies – a research centre that is part of a university. The Centre aims to work together with public administration in the field of evaluation and analysis of public policies, and its methodology.
			Top Institute of Evidence-Based Education Research in the Netherlands – a research centre belonging to an academic institution. The Centre produces research for policy makers to use in their work.
			Australian Institute for Family Studies (AIFS) – a government agency in the Department of Social Services. Fills the knowledge broker function while being within the government.
			The Research and Evaluation Unit Department of Children and Youth Affair in Ireland – working within the government as a knowledge broker unit, providing evidence to the government Department.
			The Haut Conseil à l’Enfance, la Famille et l’Âge in France – the Department integrates knowledge broker functions into its day-to-day operations.
Using and Applying	Policy makers with this skill will understand their own policy context and recognise possible uses of evidence in the policy cycle. They will be proficient in knowledge management and understand the role of innovation, with an ability to assess and manage risks and challenges. They will be familiar with and know when to use innovative techniques like behavioural insights, design thinking, policy labs and foresight.	Intensive skills training programmes: Training programmes geared towards policy makers can provide them with the necessary skills to increase the use of evidence in their work. Training programmes can be very effective when they are learner centred and participatory, ideally embedded within long-term strategies for professional development.	OECD/Mexico’s Capacity Building for RIA – in partnership with the OECD, Mexico’s Ministry of Economy hosted a four day seminar of how to produce and analyse impact assessments.
			In Canada, the Executive Training in Research Application (EXTRA) programme – provides support and development for leaders in using research. Participants learn how to use evidence in their decision making and are then able to train their co-workers.
			Public Sector Training in Finland – SITRA, the Finnish Innovation Fund has a programme for policy makers to increase their skills and capacity in their role, which includes a module on putting new practices and lessons learned from experiments into practice.
			Portugal’s National Institute of Public Administration – on an annual basis, all public service organisations inform the National Institute of Public Administration of employees’ training needs, which then feeds into the development of an annual training programme.
			UK’s Behavioural Insights Team – runs many trials across the world related to increasing knowledge of behavioural insights, which are accompanied with corresponding capacity building of local civil servants.
		Knowledge brokers (individuals): Individuals as knowledge brokers can present relevant research directly to policy makers. Many governments have official positions like Chief Science Advisor whose role is to present evidence to parliament.	New Zealand’s Chief Science Advisors – the Government has Chief Science Advisors for both the Prime Minister and the Cabinet as well as individual Ministries. They work to improve the use of evidence in policy development and provide advice.
			In the UK, the Government Chief Science Advisor’s (GCSA) role is to advise the Prime Minister and Cabinet on science, engineering and technology. The GCSA reports directly to the Cabinet Secretary and works closely with the Science Minister, and other ministers and permanent secretaries across Whitehall.
			In Australia, the GCSA advises the Prime Minister and Cabinet, and also holds the position of Executive Officer of the Commonwealth Science Council to identify challenges and opportunities for Australia that can be addressed using science.
		Mentoring: Mentoring can provide policy makers individualised training where they can learn the skills from their peers and ask questions and get feedback on using evidence in their work.	South Africa’s mentoring programme for policy makers – through this programme, policy makers are trained in evidence use and then are paired up with colleagues who have not received training in order to mentor them one-on-one.
Engaging with Stakeholders	Policy makers with this skill will have strong engagement and communication skills, including ability to create effective evidence based messages for different types of audiences and to engage and inspire variety of stakeholders. They will be able to manage and facilitate evidence-informed debate with policy makers and citizens, and maintain collaboration with the evidence community. They will have a good grasp of co-creation, co-production and participatory methodologies.	One-off or periodic interactive forums: Interventions and approaches that bring together policy makers and researchers. This approach aims to build mutual interest, trust, respect as well as promoting learning about each other’s values, contexts, constraints and practices.	Joint Research Centre’s Evidence and policy summer school – aims to help junior to mid-career policy makers to use evidence for policy solutions. The summer school focuses on the tools and approaches to inform the policy making process through evidence.
			Australian Primary Health Care Research Institute (APHCRI) ‘Conversations’ – which is a regular programme of roundtable presentations held at the Department of Health to facilitate exchange between APHCRI Network researchers and Department policy makers.
		Platforms for ongoing interactivity: Platforms for ongoing interactivity can include communities of practice, formal networks and cross sector committees. Repeated face to face contact permits the development of trust, respect and ease of communication between policy makers and stakeholders.	The Global Preventing Violence Across the Lifespan Network (PreVAil) – an integrated knowledge translation network to support effective partnerships between its members as well as joint research and application in the area of family violence prevention.
			Policy Liaison Initiative for improving the use of Cochrane systematic reviews – an ‘Evidence-Based Policy Network’ that facilitates knowledge sharing between policy makers and researchers, alongside seminars by national and international researchers in the field of evidence synthesis and implementation.
		Partnership projects: Partnership projects include various schemes to bring policy makers into contact with individual scientists, through collaborating in the development of research projects.	The Netherlands Academic Collaborative Centres – a virtual infrastructure for long-term collaborations between a regional Public Health Service (PHS) and a university research department.
			UK Pairing Programme – links policy makers with a range of experts through ‘Policy Fellowships’. The programme aims to facilitate more dialogue between the two communities, to make research more accessible, and to increase policy makers’ use of evidence in their work.
			The Australian Institute of Family Studies (AIFS) – a government body that has developed an ‘Expert Panel’ which gathers experts in research, practice and evaluation, who support practitioners in the implementation of policies, measuring outcomes, trying new policy approaches and conducting research and evaluations.
			Finland’s ‘Hack for Society’ – brings together academics, NGOs as well as national and local government to develop co-creative teams to work on service design, co-creation and societal trials.
			In the US, The National Poverty Research Center – a partnership between the US Department of Health and Human Services and the University of Wisconsin-Madison, providing research, training and dissemination to inform policy and practice.
			The US’ Quality Enhancement Research Initiative –provides timely, rigorous data analysis to the government to support the development of policy by bringing together stakeholders including practitioners, researchers, policy makers, service users and the general public to inform research and policy recommendations.
Evaluating	Policy makers with this skill will understand different evaluation approaches and tools, and know how to use comparative examples to inform EIPM. They will understand that evaluation should be built in the policy cycle and should serve to inform and improve EIPM. They will know and use qualitative and quantitative indicators of successful evidence use.	Diagnostic tools of organisational capacity for evidence use: Diagnostic tools can measure organisation’s capacity to access, interpret and use research findings in order to identify strengths, areas for improvement and assess the impact of capacity building initiatives.	Canada’s Evidence Literacy diagnostic tool – a self-assessment tool to help service managers and policy organisations understand their capacity to acquire assess, adapt and apply research
			The US’s ‘Norm of Evidence and Research in Decision-making’ (NERD) – a tool that can be used across organizational and functional settings to assess evidence based management practices within an agency.
			Organisational Research Access, Culture and Leadership (ORACLe) – a tool that assesses multiple dimensions of organisational capacity including the systems, supports and tools that organisations have in place to use research, as well as the values placed on research within an organisation.