Improving policy coherence and efficiency has become an important goal across OECD governments in recent decades. This chapters analyses two separate mechanisms that have been used by OECD countries to address this challenge: policy co‑ordination and the implementation of Monitoring and Evaluation systems. First, both vertical and horizontal forms of policy co‑ordination are particularly relevant for cross-cutting, multidimensional policy challenges and fragmented administrative structures within national governments. Furthermore, several OECD countries have established dedicated teams to monitor and evaluate public sector delivery, which is a major step towards an agile and responsive government that is well equipped to address complex policy challenges in a more effective and efficient manner.
Evolving Family Models in Spain
8. Improving policy coherence and efficiency
Abstract
Facilitating co‑operation across different policy areas and government levels
In recent decades policy co‑ordination to achieve greater policy coherence has become particularly relevant for many OECD countries, mainly due to the emergence of cross-cutting, multidimensional policy challenges and the subsequent automisation of administrative structures illustrated by the exponential growth of agencies and other autonomous bodies (Alessandro, Lafuente and Santiso, 2013[1]). Vertical co‑ordination across different levels of government within the same policy sector and horizontal co‑ordination across different sectoral unities (ministries, agencies) are both relevant.
Vertical co‑operation in policy making could be facilitated by the creation of “super-ministries”, which are expected to provide vertical co‑ordination among the large number of administrative units existent within a policy sector. An example of this sort of mechanism is the Conference for childhood and youth policy in Switzerland (Conférence pour la politique de l’enfance et de la jeunesse, CPEJ). In Switzerland, cantons, cities and municipalities have the lead competence over all matters related to childhood, while the federal government (or Confederation) oversees certain aspects of child policy. The CPEJ is made up of the cantonal contact services for childhood and youth policy and is responsible for co‑ordinating childhood and youth policy at the inter-cantonal level. The CPEJ is a technical conference for the Conference of cantonal directors for social affairs (Conférence des directrices et directeurs cantonaux des affaires sociales, CDAS) and manages the implementation of children rights as well as the development of childhood and youth policy in the country. The CPEJ co‑ordinates actively with the federal government, and advises and informs all organs of the CDAS on matters related to childhood and youth. Indeed, the federal law on the promotion of childhood and youth mandates for the increasing collaboration and exchange of information between the Confederation and the cantons on these issues (Federal Social Insurance Office, 2019[2]).
A similar approach could be followed to encourage horizontal co‑ordination, by establishing inter-ministerial committees that focus on broad policy areas (social, economic, environmental), or on specific topics or issues. For example, in Mexico, the 2013 launch of the National Crusade against Hunger required the co‑ordination among 19 national-level agencies, 31 states and 400 municipalities. To succeed in this endeavour, the central government equipped an inter-ministerial commission with the authority to select programmes and modify their budgets; while commissions at the state and local level were supposed to share relevant information. The overall programme also had a technical secretariat housed in the Ministry of Social Development, and was evaluated by Mexico’s National Council for the Evaluation of Social Development Policy. The evaluation showed that the federal-level commission was the most successful at achieving true co‑ordination, while success at lower levels varied from state to state (Cejudo and Michel, 2017[3]).
In the case of Spain, the state government reference authority for family policies, the General Directorate for Family Diversity and Social Services of the Ministry of Social Rights and Agenda 2030, already works with an inter-territorial commission, which includes family support representatives from the Autonomous Community administrations, and with an inter-ministerial commission, which includes representatives from relevant ministries in the family policy making arena.
Setting up a Monitoring and Evaluation (M&E) system
Evidence‑informed policy making can be defined as the process of consulting different sources of information (including statistics and the best available research evidence and evaluations) are consulted before making a decision to design, implement, and (where relevant) change public policies and programmes (OECD, 2020[4]). Encouraging an evidence‑informed approach to policy making is a critical step towards a government that is well equipped to address complex policy challenges in a more effective and efficient manner.
As a result, a number of OECD countries have established dedicated teams to monitor and evaluate public sector delivery, whilst ensuring that the government makes use of the evidence that is generated throughout that process. For example, in the United Kingdom, a dedicated team within the Cabinet Office supports the government’s ‘What Works Initiative’, which is aimed to improve the way government and other institutions generate, share, and use high quality evidence for decision making. In the United States, a dedicated Evidence Team within the Office of Management and Budget (OMB) acts a central hub of expertise across the federal government, working with other OMB offices in order to set research priorities and ensure the use of appropriate evaluation methodologies in federal evaluations. In Italy, the Office for the Programme of Government of the Prime Minister’s Office monitors and assesses progress on the implementation of the government programme, while in Korea, the government Performance Evaluation Committee is responsible for evaluating the policies of central government agencies on an annual basis. Mexico has a decentralised public body called the National Council for the Evaluation for Social Development Policy (CONEVAL), with the autonomy and technical capacity to generate objective information and evaluations of social policy, while Colombia created a National Monitoring and Evaluation system within the Ministry of Planning.
In the case of Spain, evaluations at the central government level are carried out by individual ministries as well as by the Institute for the Evaluation of Public Policies, a sub-directorate within the Ministry for Territorial Policy and Public Function. The Institute for the Evaluation of Public Policies was created in 2017 and is responsible for the drafting of guides on good practices in evaluation, which are expected to serve as support for different agencies in charge of designing and implementing evaluations. In addition, this body promotes M&E training for public employees, advises on evaluability of the plans and programmes during the planning stage, and it is in charge of several strategic plans to carry out ex-post evaluations.
Promote regular monitoring and evaluation involving different actors
Despite the advances made in developing an M&E culture in OECD countries, the latest progress report from the Global Partnership for Effective Development Co‑operation (OECD/UNDP, 2014[5]) concludes that further efforts are needed as important challenges remain, such as limited technical capacity, lack of resources, poor data availability and quality, and underuse of results information by policy makers.
A potential approach to tackle the latter challenge is to encourage the design and implementation of regular M&E activities (as opposed to one final independent deliverable) in order to provide insights that can shape programmes even while they are being implemented. A regular monitoring system can be established to track progress towards the expected changes, while the evaluation work stream can serve to provide explanations on the advances and reasons behind such progress. The results information produced from both systems can be consolidated and then used for internal management and/or external reporting.
As an example, Poland designed and established a regular evaluation system of the implementation of the Act of 4 February 2011 on Childcare Services for Children, which triggered several adjustments of the legislation, including the extension of the catalogue of entities that may create care institutions and the adaptation of requirements for the establishment of ECEC facilities. This evaluation system is considered to have been one of the main factors contributing to the rapid increase in ECEC coverage that Poland has experienced in the past few years (European Commission, 2021[6]).
Strengthen data infrastructure
The quality and availability of data (big data, open data, statistical data, programme monitoring data, etc.) is a key factor in how easily a policy can be evaluated (OECD, 2019[7]; OECD, 2020[8]). Similarly, the quality of data has an important influence on the rigorousness of the resulting evaluation. In order for data to meet the quality criteria to be used for evaluation, it needs to be accurate, verifiable and documented.
Consequently, evidence‑informed policy making can be hindered by the lack of available adequate data and the capacity gaps among government departments and agencies to generate it in a format that can be used. Such challenges include understanding what data and datasets currently exist across institutions and how they can be used for policy analysis. Evaluators and analysts are not necessarily aware of all the data that exists nor do they necessarily have access data across agencies, which may be especially true of external evaluators.
Evidence from the OECD OURData Index suggests that the countries achieving better results in evidence‑based policy making are those that clearly assign the responsibility to co‑ordinate open data policies. For example, in the United States, the federal government sought to increase the use of evidence in policy making across all federal agencies, acknowledging that some agencies were already excellent at using evidence while others lacked the skills or capacity necessary. In 2019, the Foundations for Evidence‑Based Policy-Making Act was approved. The law pushes agencies to adopt stronger evaluation practices in order to generate more evidence about what works and what needs improvement, and establishes that any data collected should be made accessible across agencies and to external groups for research purposes.
Disseminate evaluation results
Finally, a key step to promote the use of evidence in policy making is that the results of M&E activities are made available to their intended users – simply put, that evaluation results are communicated and disseminated to stakeholders (OECD, 2020[9]). Making evaluation results public is an important element to ensure impact and thus increase the use of evaluations.
Evaluation results are increasingly made public by OECD countries, through increased openness and transparency. In Poland, for example, all evaluations commissioned, including those concerning the implementation of EU funds, must be made accessible to the public. To facilitate this task, and with respect to the evaluations related to Cohesion Policy, a national database has been created and all evaluations are published on a dedicated website.1 This platform shares the results of more than a thousand studies conducted since 2004, as well as methodological tools aimed at evaluators.
Norway’s evaluation portal is also a publicly accessible web service that gathers all the findings of evaluations carried out by the central government.2 This database is operated by the Directorate for Financial Management and the National Library of Norway. It contains evaluations carried out on behalf of government agencies from 2005 until today, as well as a selection of central evaluations from 1994 to 2004. Evaluation reports are registered in the database as soon as they are made available to the public. Moreover, the portal provides evaluation guidelines, a calendar of the key activities in the evaluation area, news and professional papers.
Similarly, the European Platform for Investing in Children (EPIC) is an evidence‑based online platform which consolidates information on policies for children and their families in Europe. The platform serves as a tool for monitor activities implemented across member states triggered by the Recommendation for Investing in Children. It also helps as a repository for sharing the best of policy making for children and families and to foster co‑operation and mutual learning in the field.
It is important to highlight that the way evidence is presented and disseminated should be strategic and driven by the evaluation’s purpose and the information needs of intended users (Patton, 1978[10]). Tailored communication and dissemination strategies that increase access to clearly presented research findings are very important. These strategies include the use of infographics, online seminars, and the dissemination of ‘information nuggets’ and parts of storytelling through social media and other entertainment platforms. A good example of tailoring evidence results to reach a large audience is the Australian series Life at, which was filmed in conjunction with the long-term scientific study, Growing Up. Life At is an observational documentary series that chronicles the lives of a select number of children as they grow from infancy to adolescence, aiming to show what it takes to give a child the best chance in life. As the lives of the sampled children unfold, the series rigorously examines the complex question: what it is that makes us all thriving, independent, and creative human beings?
References
[1] Alessandro, M., M. Lafuente and C. Santiso (2013), The Role of the Center of Government - A Literature Review, Inter-American Development Bank, Washington, D.C.
[3] Cejudo, G. and C. Michel (2017), “Addressing fragmented government action: coordination, coherence, and integration”, Policy Sciences, Vol. 50/4, https://doi.org/10.1007/s11077-017-9281-5.
[6] European Commission (2021), Study on the economic implementing framework of a possible EU Child Guarantee Scheme including its financial foundation, https://ec.europa.eu/social/main.jsp?catId=738&langId=en&pubId=8382&furtherPubs=yes (accessed on 23 August 2021).
[2] Federal Social Insurance Office (2019), Which services deal with issues related to children and youth?, https://www.bsv.admin.ch/bsv/fr/home/politique-sociale/kinder-und-jugendfragen/grundlagen-gesetze/zustaendige-stellen.html (accessed on 14 January 2022).
[9] OECD (2020), Building Capacity for Evidence-Informed Policy-Making: Lessons from Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/86331250-en.
[4] OECD (2020), Mobilising Evidence for Good Governance: Taking Stock of Principles and Standards for Policy Design, Implementation and Evaluation, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/3f6f736b-en.
[8] OECD (2020), “The OECD 2019 Open Useful Reusable Data (Ourdata) Index”, OECD, Paris, https://www.oecd.org/gov/digital-government/ourdata-index-policy-paper-2020.pdf.
[7] OECD (2019), The Path to Becoming a Data-Driven Public Sector, OECD Digital Government Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/059814a7-en.
[5] OECD/UNDP (2014), Making Development Co-operation More Effective: 2014 Progress Report, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264209305-en.
[10] Patton, M. (1978), Utilization-focused evaluation, Sage, Beverly Hills.