This chapter analyses Nuevo León’s current institutional system related to strategic planning, monitoring and evaluation. It calls for clarification on the respective responsibilities of strategic planning of the Nuevo León Council for Strategic Planning and of the centre of government (notably the Executive Office of the Governor and the Secretariat of Finance). In particular, it recommends strengthening Nuevo León Council’s role as an advisory body and strategic knowledge broker, focusing on planning and evaluation, and moving it away from implementation and monitoring. Furthermore, it calls for strengthening the centre of government mandate and capacities for policy coordination and monitoring. Finally, it suggests that a whole-of-government strategy for policy monitoring and evaluation could contribute to better policy coherence.
Monitoring and Evaluating the Strategic Plan of Nuevo León 2015-2030
1. Toward a Sound Institutional System for Planning, Monitoring and Evaluation in Nuevo León
Abstract
Introduction
There is increasing interest in long term planning (Máttar and Cuervo, 2017[1]), whether in national, supra-national or sub-national governments. This demand is driven by two main factors:
at the international level, governments are pursuing the UN’s 2030 Agenda for Sustainable Development Goals.
at the national level, governments have realised the need to ensure continuity in public policies beyond electoral cycles (see Box 1.1).
Box 1.1. Comparative approaches to long term strategic planning
A survey of international practices reveals a wide selection of long-term strategic planning tools. Below are some examples of these long-term plans and their structure.
The 2030 Agenda for Sustainable Development, adopted by all the United Nations members, provides a shared blueprint for peace and prosperity for people and the planet, now and into the future. The agenda is composed of seventeen Sustainable Development Goals framing a path to achieve the associated targets. Moreover, each target has one or several indicators to monitor its performance at local, national and global level. Given the nature of the SDGs, the goals that the Agenda includes could be considered impact or outcome level objectives.
The Europe 2020 Strategy is the European Union's agenda for growth and jobs for the current decade, used as a reference framework for activities at EU, national and regional levels. The Strategy proposes a set of targets for the EU countries to consider. The targets, which are set at the outcome level, are monitored through the annual reports of the national reform programmes.
In Colombia, the 1991 Constitution provides the framework for strategic planning at the national and subnational level. It requires the enactment of a National Development Plan (NDP) and of subnational development plans. At the subnational level, the most important planning tool is the Department Development Plan, which is the blueprint for the Governor’s term. Strategic planning in Colombia is also supported by various vertical and horizontal co-ordination mechanisms, such as the “Pactos Territoriales” used to carry out multi-level initiatives that contribute to the objectives of the NDP.
Sources: (United Nations, 2020[2]), Sustainable Development Goals, https://sustainabledevelopment.un.org/; (European Commission, 2010[3]), Europe2020: A strategy for smart, sustainable and inclusive growth, https://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-%20Europe%202020%20-%20EN%20version.pdf; (Scottish Government, 2020[4]), National Performance Framework, https://nationalperformance.gov.scot/
Strategic planning helps governments to cluster policy initiatives, prioritise areas of interest, and define a small number of integrated policy priorities (OECD, 2018[4]). The development of these priorities should be driven by an anticipation of, and preparation for, future needs and trends, considering the future costs of present-day actions and managing current and future risks (OECD, 2010[5]). Furthermore, strategic planning can help align governance structures in such a way that governments can effectively implement and deliver their policy priorities. This alignment includes ensuring coherence across time-horizons by linking short, medium and long-term priorities, as well as guiding government entities and units in policy implementation (OECD, 2018[4]). Finally, strategic planning, can facilitate the measurement and communication of progress, both internally and externally (OECD, 2018[4]).
A sound strategic planning system requires the presence of multiple elements, such as a thorough review of problems based on evidence, consultation and engagement with relevant stakeholders, development of objectives and definition of relevant indicators, as well as the elaboration of an action plan and calculation of costs. Those tasked with strategic planning as well as external stakeholders involved in the process need sufficient time to comprehensively tackle these elements (Vági and Rimkute, 2018[6]).
A robust M&E system is key to ensuring effective planning
A robust monitoring and evaluation (M&E) system is essential to the success of middle and long-term objectives. A sound M&E system generates evidence on what has worked, why, and for whom. When the information generated by M&E efforts is fed back into the decision-making and planning process, it can help to improve the design of public policies and programmes. Furthermore, sound M&E can help identify challenges to policy implementation and ways to address them, based on lessons learned (OECD, 2019[7]). In addition to policy learning, M&E can foster transparency and accountability by providing performance information to citizens on progress in achieving government objectives (Vági and Rimkute, 2018[6]).This is particularly relevant for subnational governments, who given their proximity to their constituents, are considered the ‘coal-face’ of public service delivery (Masuku and Ijeoma, 2015[8]).
Though interconnected, monitoring and evaluation are distinct practices (as outlined in Table 1.1). Monitoring corresponds to a routinized process of evidence gathering and reporting to ensure that resources are adequately spent, outputs are successfully delivered, and milestones and targets are met (OECD, 2020[9]). Policy evaluation, on the other hand, is a structured and objective assessment of an ongoing or completed initiative, its design, implementation and results. The goal of policy evaluation is to determine the relevance and fulfilment of objectives, efficiency, effectiveness, impact and sustainability, as well as the worth or significance of a policy (OECD, 2020[9]).
Table 1.1. Comparing policy monitoring and policy evaluation
Source: Adapted from McDavid, J.C. and Hawthorn, L.R.L. (2016[10]), Programme evaluation and performance measurement, an introduction to practice, Thousand Oaks, California: Sage, in (OECD, 2019[7]), Open Government in Biscay, https://doi.org/10.1787/e4e1a40c-en.
Nuevo León’s M&E system ranks just above average in comparison with other Mexican states.
Mexico is one of the pioneers of monitoring and evaluation in Latin America, having transitioned from limited and scattered practices of M&E to a whole-of-government M&E system. While the history of M&E in the country dates back to the mid-1970s, M&E and performance-based management reforms began to be integrated into the federal public administration in the late 1990s (Castro, Lopez-Acevedo and Busjeet, 2009[11]). For example, in 1997, the government of Mexico carried out its first rigorously planned programme evaluation of the country’s social assistance programme (Progresa/Oportunidades). In 1998, congress began requiring government programmes providing subsidies to prepare rules of operations (ROPs), which outlined the programmes’ design, objectives, performance indicators and operations, thereby improving the quality and quantity of performance data. Finally, in 1999, congress required all programmes with ROPs to be evaluated annually by external evaluators.
This push towards the institutionalisation of M&E received impetus from the two financial crises. These crises, in conjunction with an extreme poverty rate of 20%, led to questions regarding the effectiveness of social spending (Castro, Lopez-Acevedo and Busjeet, 2009[11]). Answering these questions required a sound monitoring and evaluation system. The main outcome was the creation of the National Council for Evaluation of Social Policy (CONEVAL) in 2005 (see Box 1.2). This body, given its independence, technical capacities, and mandate, spearheaded the development of an M&E system in Mexico’s social sector, as well as the federal government, in general (Castro, Lopez-Acevedo and Busjeet, 2009[11]).
Box 1.2. The National Council for Evaluation of Social Policy (CONEVAL)
The CONEVAL was created in 2005 as a decentralised body of the federal administration with the mission of evaluating social development policy, its programmes, funds and actions, and of measuring poverty at national, state and municipal level. CONEVAL’s mandate encompasses between 100 to 130 federal programmes from year to year, all of which are required to execute internal evaluations governed by CONEVAL’s guidelines. The organisation also directly oversees over a dozen evaluations per year. Results from the evaluations are influential. In 2013-2014, half of the evaluated programmes were substantially refocused and 41% of programmes underwent corrections regarding activities or operational aspects (Lázaro, 2015[12]).
CONEVAL is the main vehicle towards the institutionalisation of policy evaluation within Mexico through initiatives that clarify, cement, and advance M&E processes. The 2007 issuance of the mandatory General Guidelines for federal programme evaluations provided definitions, regulations, principles, and requirements for components of the monitoring and evaluation system. In 2008, a tracking system for evaluations was implemented and subsequent efforts were made to develop digital platforms for this system to make it accessible to the wider public. Training seminars were also organised for programme managers (Gaarder and Briceño, 2010[13]). These efforts may have the long-term effects of structuring evaluation practices and increasing capacity, even outside of CONEVAL affiliated entities— fostering an evaluation culture (Lázaro, 2015[12]).
Sources: (Lázaro, 2015[12]), Comparative study on the institutionalisation of evaluation in Europe and Latin America, http://sia.eurosocial-ii.eu/files/docs/1456851768-E_15_ENfin.pdf ; (Gaarder and Briceño, 2010[13]), Institutionalisation of government evaluation: balancing trade-offs, https://www.tandfonline.com/doi/full/10.1080/19439342.2010.505027
CONEVAL also advises subnational governments on the implementation of their M&E systems and generates guidance on successfully conducted, good-quality programme evaluations. Since 2011, it has prepared an assessment of the M&E systems of Mexican states with an aggregated index (see results Figure 1.1. Monitoring and Evaluation Index by state (2017). This index is composed of two principal dimensions. The first dimension covers the ‘normative’ instruments; these refer to the regulations issued by the states to govern their M&E practices. The second dimension (i.e. the ‘practical’ dimension) focuses on how M&E practices are effectively implemented (CONEVAL, 2017[14]). Thus, to construct the index, CONEVAL gathers relevant information from the states and assigns them a score (based on pre-established criteria and good practices) for each of these two dimensions, the sum of which reflects the overall performance of the state’s M&E system.
Over the last decade, this assessment has identified an across-the-board improvement in subnational governments’ M&E systems in Mexico. For example, in 2017, 21 states were classified as “medium-high” or “high” in terms of their level of advancement, compared to 10 in 2011 (CONEVAL, 2017[14]). In 2017, Nuevo León was classified as slightly above average with a combined index score of 77.8 (see Figure 1.1), which corresponds to the medium-high category. Despite its advanced socio-economic standing and strong performance in the “practical” dimension of the index (ranked 6th among 32 Mexican states), Nuevo León’s middling position in the combined index is explained by the fact that it performs poorly on the ‘normative’ dimension (ranked 23rd among 32 Mexican states). Some examples of criteria for which CONEVAL has given Nuevo León a low score are:
State regulations establish that evaluation results should be published.
State regulations establish performance and management indicators for social development policy and programmes.
State regulations establish the responsibilities of those responsible for co-ordinating and carrying out the evaluation of social development policy in the state.
Figure 1.1. Monitoring and Evaluation Index by state (2017)
Note: Red dashed line represents the average
Source: (CONEVAL, 2017[14]), Diagnóstico del avance en monitoreo y evaluación en las entidades federativas [Diagnosis of the progress of monitoring and evaluation in Mexican states], https://www.coneval.org.mx/InformesPublicaciones/Documents/Diagnostico_mye_estados_2017.pdf
Building a sound long-term planning system in Nuevo León
The Nuevo León Council reflects a whole-of-society effort towards the long term development of the state
In 2010, Hurricane Alex hit numerous cities of Nuevo León including the state capital Monterrey, destroying infrastructure, water and electricity facilities, and homes. Facing the most important natural disaster in its history, the state needed to launch reconstruction and development initiatives, which resulted in the creation of the State Council for the Rebuilding of Nuevo León (2010-2013). The creation of this intersectoral forum of exchange, and the innovative collaboration with the private sector which resulted from its functions, led state authorities to identify the need to improve the quality of life of citizens. This created the impetus for the government and private sector to establish the Nuevo León Council for Strategic Planning (the council, hereafter). Formally established in 2014, this state consultative body aims to support the executive in developing a long-term vision through the Strategic Plan for the State of Nuevo León 2015-2030 “Nuevo León Mañana” (SP) (State of Nuevo Leon, 2019[15]).
The council was created and is governed by the Strategic Planning Law for the State of Nuevo León (Ley de Planeación Estratégica del Estado de Nuevo León) and its corresponding guidelines (Reglamento De La Ley De Planeación Estratégica Del Estado De Nuevo León). It is an advisory/consultative body of the state executive on strategic planning and its evaluation (article 7 of the State Strategic Planning Law). By design, the council is a non-partisan public body with a ‘trans-sexennial’ mandate; thus, while it advises the executive branch, it is intended to be guided by a long-term strategy beyond electoral cycles (OECD, 2018[16]).
The Nuevo León Council’s mission is to promote a long-term vision for the sustainable development of Nuevo León and the well-being of all its citizens, in particular through the planning and evaluation of public policies. To this end, the council also seeks to encourage a performance culture inside the state and across its administration. According to the Strategic Planning Law, the council is responsible, inter alia, for the development of the Plan, for defining strategies to include society in the efforts for its implementation, for defining criteria to develop indicators (article 9), as well as for monitoring and evaluating the Plan’s implementation and results (article 19) (see Box 1.3). The mandated responsibilities reiterate the council’s intended commitment to a whole-of-society approach to long term development, as they include elements pertaining to effective stakeholder engagement (e.g. citizen engagement, dialogue across levels of government, etc.) and policy and strategic coherence (e.g. alignment between planning instruments, sectoral alignment, etc.)
In addition, the council has to “know, propose and give an opinion on the State Development Plan”. As explained in chapter 2, the State Development Plan (SDP) is a strategic planning document that each state administration must elaborate upon entering office. Therefore, the Nuevo León administration elaborated an SDP in 2016, specifying its long-term objectives until 2021. The public policy priorities of the Federal Government are established in a National Development Plan, which sets-out the national goals and objectives over a planning horizon of six years (NDP 2019-2024). All SDPs must be designed in accordance with the NDP.
Box 1.3. . Mandated responsibilities of the council
The council is legally mandated to have the following responsibilities and powers:
draw up the Strategic Plan
promote a culture of honesty in medium and long term planning, performance measurement and accountability
outline strategies that will enable the different sectors of society to align their efforts with the set goals and objectives
know, propose and give an opinion on the State Development Plan and its strategic projects and priority programmes, including any revisions, all in accordance with the guidelines of the Strategic Plan
establish criteria for the elaboration of quantifiable indicators in the economic, social, environmental and other relevant areas, in order to measure and evaluate progress towards the objectives and goals of the Strategic Plan and the State Development Plan
communicate to citizens the objectives, indicators, and progress of the Strategic Plan and the State Development Plan
promote dialogue and the necessary agreements among the three levels of government, companies, social organizations and educational institutions, in order to jointly carry out actions aimed at achieving the established objectives
be aware of the progress in relation to strategic projects and priority programmes, as well as the progress in the indicators measuring the economic and social development of the State
prepare a long-term forecast of the evolution of the state’s income, current and investment expenditure, and debt, in order to have a reference for the objectives of the Strategic Plan
issue procedural manuals for the development of the Strategic Plan
be aware of the budget allocations pertaining to the development of the Strategic Plan
approve the creation of commissions to deal with specific policy issues
Source: Adapted from Article 9 of (Council of Nuevo León, 2016[17]) Ley de Planeación Estratégica del Estado de Nuevo León [Strategic Planning Law for the State of Nuevo León]
The council also seeks to foster transparency and stakeholder engagement by virtue of its composition; its members represent the public and private sectors, academia and civil society. This is in line with OECD standards on open government, which recommend that stakeholders should be engaged “in all phases of the policy cycle, service design and delivery, contributing to source ideas and co-create solutions” (OECD, 2017[18]). Effective stakeholder participation and public engagement enlarges the range of inputs provided at every stage of policy design and delivery, which results in a better understanding of nuanced public needs (OECD, 2019[19]). This, in turn, can lead to more robust evidence-informed decision-making and more effective public policies (OECD, 2009[20]). Furthermore, since public understanding and support is essential to policy implementation, public engagement is an important component of effective policy implementation and of reducing non-compliance (OECD, 2019[19]). Box 1.4 illustrates examples of meaningful government-citizen partnership and stakeholder engagement.
Box 1.4. Fostering a government-citizen relationship
Citizen participation has long been acknowledged as a powerful tool to enhance public policies, especially at the subnational level. It can improve the quality of policies and their responsiveness to the population’s needs and citizens’ proposals can solve development challenges.
The experience of Alberta in Canada and the “What we heard” report
In Canada, the province of Alberta offers an interesting example of stakeholder involvement at the early stages of policy-making. In order to ensure a needs-tailored design and implementation of its “Health Policy Framework”, the government sought the opinion of health system stakeholders, health care professionals, unions, municipal leaders, educators and community organisations through letters, meetings, online expression of opinions etc. The results of this consultation process were subsequently used to further improve approaches to integrate citizens’ opinions in policy-making.
Involving citizens in the definition and monitoring of State Development Plans in Mexico
In Mexico, the state of Aguascalientes created a Citizen’s Consultative Committee in charge of participating in the preparation of the state’s plans and programmes as well as monitoring and evaluating the state planning system. It is composed of representatives from the public, private, social and academic sectors.
In the state of Jalisco, there are four sectorial citizen councils for planning, measurement and evaluation. The goal of the councils is, amongst other things, to promote citizen participation in the monitoring of the State Development Plan’s indicators. The councils are composed of representatives from civil society and public institutions, selected according to their professional expertise along four thematic areas: social development, economic development, territorial development and security.
A multi-sectorial commission to monitor the OGP in Peru
In Peru, a permanent multi-sectorial commission is in charge of monitoring the implementation of the Open Government Partnership National Action Plans. Participants in this commission include several government ministries, the judiciary as well as several CSO and private sector organizations. Civil society representatives were highly involved in the preparation of the OGP, they reviewed the commitments, offered suggestions and participated in the consultations.
Sources : Based on (OECD, 2016[21]), Open Government: The Global context and the Way Forward; https://www.oecd-ilibrary.org/docserver/9789264268104-en.pdf?expires=1583850456&id=id&accname=ocid84004878&checksum=E6A7AD508FAABD7F84466B00395192D4; Open Government Partnership, (2019[22]), Designing and Managing an OGP Multistakeholder Forum, https://www.opengovpartnership.org/wp-content/uploads/2016/09/Multistakeholder-Forum-Handbook.pdf; Secotrial citizen councils for planning, measurement and evaluation website (https://seplan.app.jalisco.ob.mx/evalua/consejo-evalua/consejo-ciudadano)
In addition to its multi-stakeholder nature, the council also reaches across levels of government. As shown in Figure 1.2, the council includes a representative from the federal government. Furthermore, the council can also include, as special guest advisors, public servants from federal, state and municipal government entities as well as national and international experts to contribute to the analysis of specific issues (article 7 of the Strategic Planning Law).
Figure 1.2. Breakdown of the Nuevo León council
Notes: The representative of state congress is preferably the president or a member of the State Finance Committee
Source: Adapted from Article 7 of (Council of Nuevo León, 2016[17]), Ley de Planeación Estratégica del Estado de Nuevo León [Strategic Planning Law for the State of Nuevo León]
The council is composed of 16 members and a technical secretary, as illustrated in Table 1.2. It is led by the president of the council, who is the Governor of the state. Meanwhile, the executive president is chosen from among the six citizen councillors of the council, while the remaining five eventually serve as the presidents of its five thematic commissions. Each commission also includes a technical secretary, a member of the executive branch entity whose area of expertise corresponds to that of the commission. Among the six commissions, there are five thematic commissions, which correspond to the main themes of the Strategic Plan 2015-2030. These themes include economic development, human development, sustainable development, effective government and transparency, and security and justice. Other commissions can be created as well. For instance, in 2018, a public finance commission was also created in order to ensure greater alignment between the Plan and the state budget. The council is funded in equal parts by the government of the state and the private sector.
Table 1.2. Mandated roles and responsibilities of the council members
Source: Adapted from Article 10 of (Council of Nuevo León, 2016[17]) Ley de Planeación Estratégica del Estado de Nuevo León [Strategic Planning Law for the State of Nuevo León]
The fact that citizen councillors chair both the council and the commissions reinforces citizens’ engagement in the work of the council. In particular, as shown in Table 1.2 in the case of the executive presidency, citizens are involved in overseeing the overall functioning of the council by calling and presiding over (when needed) meetings, requesting relevant information from government entities, and ensuring internal compliance. Likewise, in the case of commission presidencies, citizens have voting and speaking privileges, the ability to present substantive proposals, and carry out citizen consultations.
However, there are no clear rules/procedures to decide on membership/participation in the council and commissions. This is echoed in the assessment of the Nuevo León council by GESOC (Gestión Social y Cooperación), a civil society organisation focused on governance issues, which acknowledges the lack of a competitive process for the appointment of management positions (GESOC, 2017[23]). When interviewed by the OECD, several stakeholders also expressed that its representativeness is perceived as unbalanced across sectors of civil society. In particular, citizens appear to have limited voice/influence, in contrast to the influence of the private sector. In addition, several stakeholders stressed the need to involve universities more actively in the council’s work.
Clear methods for stakeholder engagement are important, inter alia, to incorporate underrepresented or vulnerable populations and their needs into the council decision making process. This element will be further addressed in chapter 5.
The council offers an inclusive platform to reflect on key policy initiatives
The council offers a space to discuss new initiatives that were not explicitly set out in the Strategic Plan. There appears to be a strong commitment on the part of many stakeholders, both to the plan and to dialogue and debate within the council. The members of the council (the chairperson, president, commissions’ presidents, state representatives, academics, the federal government representative and the secretary general) each have a voice and a vote. This enables engagement and dialogue between civil society and the executive (Council of Nuevo Leon, 2019[24])
In particular, the creation of sectoral commissions has stimulated strategic thinking in specific policy areas. This is the case of the Hambre Cero Nuevo León initiative (see Box 1.5), an inter-institutional effort derived from the Strategic Plan to eradicate extreme food poverty and food waste in the state. This initiative requires the concerted and co-ordinated action of the government, the private sector, the civil society and the public.
Box 1.5. Hambre Cero / Zero Hunger initiative
The Hambre Cero is a multi-stakeholder initiative with the objective of eradicating hunger in Nuevo León by reducing food loss and waste, as well as by developing capacities for food self-sufficiency of its target population. The initiative reflects the United Nations Sustainable Development Goals (Goal 2: Zero Hunger) and the Strategic Plan 2030 (thematic area of Human Development). The target population of the initiative is the share of the population living below the minimum welfare line (as determined by CONEVAL), which is the segment of the population whose entire income is less than the cost of a basic food basket. The objectives are organised around three strategic pillars, as outlined below:
Food aid and counselling: Provide food aid and guidance to contribute to food security; improve the nutritional health of poor and vulnerable populations; and periodically assess progress in reducing food shortages.
Reduction of food loss and waste: Establish policies and mechanisms to reduce the food waste in the food supply chain with a focus on identifying, capturing, transforming and distributing potential food waste to target populations.
Food and family self-sufficiency: Develop individual, family and community capacities for self-management and self-improvement, and enable complete social inclusion – through inter-institutional co-ordination and collaboration.
In addition to the strategic pillars, the initiative is also informed by four transversal axes that further specify its aims and modalities, as described below:
Social innovation: Generate innovative practices to meet social needs, create collaborative schemes to enable desired transformation, and employ a scientifically supported process to replicate experiences of a community elsewhere.
Communication: Use participatory processes to motivate all members of the society to contribute to the initiative’s objectives, for example, by highlighting the contributions of key stakeholders involved in the initiative.
Regulations/rules: Address legal gaps through the creation of regulations and other legal instruments to help achieve the initiative’s goals and objectives.
Evaluation: Institute a scientifically and methodologically rigorous process that monitors each stage of the initiative so that timely adjustments can be made to improve implementation, optimise use of resources and maximise the impact and durability of the initiative.
Source: Adapted from Hambre Cero Nuevo León, (2020[25]), Hambre Cero Nuevo León, https://www.hambreceronl.mx/index
The overall planning system in Nuevo León
The Organisational Law of the Public Administration for the State of Nuevo León (Ley Orgánica de la Administración Pública para el Estado De Nuevo León) organises and regulates the functioning of the public administration of the state (OECD, 2018[26]). The chief of the public administration and the principle holder of executive power in the state of Nuevo León is the Governor of the state (article 2), who is elected every six years (article 84 of the Nuevo León constitution). The head of the executive is supported by agencies and administrative units of the state public administration, as outlined in article 18 of the Organisational Law (see Figure 1.3).
Among these entities, the centre of government’s (CoG) institutions play a central role in whole-of-government planning. The CoG is “the body or group of bodies that provide direct support and advice to Heads of Government and the Council of Ministers, or Cabinet” (OECD, 2014[27]). The strategic role of CoG has been expanding over the course of the last decade due to the increasing complexity of policy-making; the emergence of whole-of-government strategy-setting and implementation; strategic monitoring of government performance over the medium term; and strategic issues management (Alessandro, Lafuente and Santiso, 2013[28]). The CoG is mandated “to ensure the consistency and prudency of government decisions and to promote evidence-based, strategic and consistent policies” (OECD, 2020[9]).
Figure 1.3. Actors in the state public administration
Source: Adapted from (Secretary of the State, 2009[29]), Ley Orgánica de la Administración Pública para el Estado De Nuevo León [Organisational Law for the State of Nuevo León]
CoG institutions vary from one country to another, depending on the constitutional order, the political system, the administrative structure of the country, contextual and historical actors. Expanded definitions of the CoG can include institutions or agencies that perform core cross-cutting governmental functions, such as finance or planning ministries. In Nuevo León, the CoG capacity (in the broad sense) is mainly distributed across the following institutions:
The Executive Office of the Governor. This administrative unit is in charge of planning, coordinating and reporting to the Head of the Executive on the execution of plans and programmes by public agencies and entities, in accordance with the State Development Plan. It also aims at facilitating communication between public entities and the Governor.
The Secretariat of Finance and General Treasury is in charge of the financial, fiscal and tax administration of the state's public treasury. It prepares the yearly budgeting process.
In addition, the Government’s General Secretary is in charge of conducting the state internal affairs and of coordinating the relationships between the executive and the federal government.
Additional secretariats play an important role in supporting cross-government policy co-ordination, such as the Secretariat of Economy and Labour and the Secretariat of Social Development.
In regard to strategic planning functions, article 5 of the Strategic Planning Law states that the council and the head of the executive are the main actors in the strategic planning of Nuevo León. The responsibilities of the two (as they pertain to the strategic planning instruments) are outlined, as follows: whereas the Executive Office of the Governor is responsible for issuing the manual for the development of the State Development Plan and its derived programmes, as well as integrating them (Article 16, Strategic Planning Law), the council is tasked with consultation and advice on matters of strategic planning, as well as evaluation (article 7). This means that their functions are complementary.
The regulations for the Strategic Planning Law outline the strategic planning responsibilities of the heads of specific entities within the state public administration. Article 14.V of the regulations defines the responsibilities for the heads of all dependencies of the state public administration. In addition to this, article 14.I to article 14.IV (inclusively) pinpoint specific and specialised roles for the following entities:
Head of the Executive Office of the Governor: They coordinate the strategic planning process, and monitor the implementation of the initiatives of the State Development Plan. This office also absorbed, de facto, most of the functions of the former Technical Coordination, Planning Evaluation and Government Innovation Cabinet, dissolved in 2020. The technical coordination was responsible for monitoring the tasks of the planning process, as well as the implementation of the State Development Plan. It was responsible for preparing procedural manuals for the elaboration of the State Development Plan and its initiatives. It also had a monitoring function, as it collaborated with the Secretariat of Finance to measure the achievement of annual operational programmes and with the Secretariat of Finance, State General Treasury and Office of Government Comptrollership Transparency to supervise the assessment of planning instruments. Finally, it used to coordinate research and training activities (related to strategic planning) for entities of the state public administration.
The Secretariat of Finance and the General State Treasury: It is responsible for preparing and proposing to the head of the executive a financial plan, as well as the initiatives pertaining to the state’s expenditure law and its revenue law; it does so in accordance with the State Development Plan. Likewise, the Secretariat coordinates with the head of the executive the planning and organisation of the governments’ budget, respecting the State Development Plan. Finally, in conjunction with the Government Comptrollership and Transparency, the Secretariat monitors the use of financial resources in accordance with the State Development Plan and its programmes.
Heads of the entities and dependencies of the state public administration: In accordance with the guidance of the executive branch, they participate in the strategic planning and process, and subsequently elaborate and execute the programmes and projects of the Strategic Plan and State Development Plan within the scope of its purview (Article 14 v.b.). They also have M&E functions, as they are tasked with proposing indicators for the monitoring of public policies and the compliance of their programmes and budget, and local actions with the State Development Plan. Lastly, they act as the technical secretaries of the council’s commissions that deal with their area of expertise.
Evidently, the scope of these functions sometimes pertains to the ‘planning process’ as a whole, which can be understood as both the Strategic Plan and State Development Plan (article 15 of the regulations of the Strategic Planning Law). For example, the head of the Executive Office of the Governor is responsible for coordinating the tasks of the planning process. In other cases, the functions are specific to the State Development Plan. For example, the head of the Executive Office of the Governor is responsible for monitoring the execution of the State Development Plan. The only explicit mention of the Strategic Plan is in article 14.V.b, which mandates dependencies of the state public administration to elaborate and execute the programmes and projects of the Strategic Plan and State Development Plan within the scope of their purview (Article 14, v.b.). Article 14 v.e. also mandates the monitoring of programmes and budget, in compliance with the objectives and guidelines of the State Plan and the applicable legislature, in order to favour the attainment of the greatest social and economic benefits.
In addition to the Strategic Planning Law and its regulations, the internal regulations of the entities of the state public administration are part of the legal framework governing the strategic planning system in Nuevo León. These regulations define the goals and functions of the secretariat, the legal provisions to which it must adhere, and its organisational structure. For example, the Secretariat of Sustainable Development is governed by Reglamento Interior De La Secretaría De Desarrollo Sustentable. Other line ministries and dependencies of the executive have analogous legal documents.
A complex interaction between the council and the state public administration
The interaction between the council and the state public administration manifests in several ways. For example, the work of each of the council’s commissions is supported by the staff and public servants from the relevant state public administration entities and departments (article 9 of the regulations of the Strategic Planning Law). To that end, the technical secretary for each commission is a representative from an entity in the executive branch, whose area of expertise is relevant to the commission.
Likewise, in preparing its studies and reports, the executive president of the council has the legally mandated ability to request relevant information from the state public administration of Nuevo León (as well as other levels of government) so that the council may carry out its evaluations/policy studies (article 10 of the Strategic Planning Law). This mutual dependence means that the coordination between the council and the executive is paramount to a sound strategic planning system in Nuevo León. To date, the Executive Office of the Governor carries out this function.
Beyond the executive, the legislative branch has a role in the strategic planning system of Nuevo León. As dictated by article 22 of the Strategic Planning Law, the council is required to deliver the evaluation results of the Strategic Plan and the State Plan to the state congress. The executive is correspondingly mandated to inform the state congress of its decisions in response to the evaluation results of the Strategic Plan and the State Development Plan. However, the legal framework does not dictate how or if the legislative branch should follow up once it receives the evaluation results of the Strategic Plan. The state congress also has a role in monitoring the public expenditure directed towards the Strategic Plan and the State Development Plan (article 3.XV of the Strategic Planning Law). Furthermore, through the inclusion of a representative from the state congress in the council, the legislative branch of Nuevo León is incorporated in the strategic planning system.
While the council is an advisory/consultative body of the state executive on strategic planning and its evaluation, it works, de facto, as a body operating at arm's-length from the executive. Its composition includes representation from the legislative branch of government, the state’s judicial branch, academia, civil society, the federal government and the private sector. In addition, it benefits from a funding mix, which gives it certain operational autonomy from the core state public centre of government has limited capacities for strategic planning, coordination and monitoring in Nuevo administration.
Responsibilities for strategic planning are blurred between the Executive Office of the Governor and council
The strategic planning law contains several articles that have led to some conceptual confusions regarding the council’s responsibilities on monitoring and evaluation. While article 7 gives the council responsibilities only on planning and evaluation, article 18 of the same law states that the council is also in charge of monitoring the implementation of the Plan. In addition, it does not specify how this monitoring function should be carried out.
Moreover, the council is currently bound by law to produce a yearly report on the implementation of the Strategic Plan, which is officially described as an “evaluation”, but in reality is a monitoring report. The publication of this yearly report tends to create tension with the government, first, as it highlights implementation gaps in achieving-long term objectives; this is because the strategic plan is in essence long-term and ambitious, and in several areas progress cannot be perceived within a given year. In fact, international good practice suggests that evaluation should be selective and targeted, and not necessarily a yearly requirement and, more importantly, that it needs to go more in depth than just a monitoring exercise to understand what works or not and why, to suggest helpful corrective action (Vági and Rimkute, 2018[6]). Second, the monitoring report seems to focus mostly on the strategic projects, which are output objectives rather than the long term outcome and impact goals that correspond to the Council’s mandate.
This situation has generated a slightly misplaced perception of the council as a sort of independent “watchdog” of the government, whereas it is in fact a forward-looking part of the state governance apparatus. The perception by the state public administration is that the council is a body that controls their performance instead of supporting them in the achievement of long-term goals, which does not help. This situation is partly fuelled by the lack of clarity concerning the council’s role in strategy setting, implementation, monitoring and evaluation.
Since its creation, the council has seen some expansion of its actions from a policy standpoint. This has created tensions with the state public administration and resulted in the council devoting comparatively less attention to long term planning – a function which is also intrinsically linked to foresight and evaluation. Moreover, this situation has created an incompatible mix of functions: some require strong political influence and commitment (e.g. monitoring the implementation of plans) while others benefit from increased independence and technical legitimacy (e.g. evaluation).
In order to resolve this discrepancy, the council could consider focusing on its strategic and policy advisory role and moving away from implementation and monitoring. OECD research has found that clearly distinguishing the roles of policy advisory bodies from those of policy decision makers is essential to an effective policy advisory system (OECD, 2017[30]). This delineation involves allowing advisory bodies to develop different policy options and to argue in favour of certain among them, but ensuring that the task of ultimately deciding on policy choices is given to policy makers. To do so, international good practice suggests halting the policy advisory process once the policy advice is given (OECD, 2017[30]). The Australian Productivity Commission provides a good illustration of this delineation (see Box 1.6).
Box 1.6. Australia’s Productivity Commission: an autonomous government body
Australia’s Productivity Commission, located in the Government’s treasury portfolio, serves as an independent research and advisory body to the country’s government; it covers an array of economic, social and environmental policy issues affecting the welfare of Australian citizens. At the request of the Australian Government, it provides independent and quality advice, and information on key policy and regulatory issues. It also conducts self-initiated research to support the Government in its performance reporting and annual reporting, and acts as a secretariat under the council of Australian government for the inter-governmental review of government service provision.
The policy advisory function of the Commission is well defined and clearly demarcated from policy decision making. It does not have executive power and does not administer government programmes; rather, its core contribution is centred on informing policy formulation and public debate by providing high quality, independent advice to government entities. Specifically, the Commission exhibits the following characteristics:
Independence: it operates under its own legislation, and its independence is formalised through the Productivity Commission Act. Moreover, it has its own budget allocation and permanent staff working at arm’s length from government agencies. Even if the commission’s work programme is largely defined by the government, its results and advice are always derived from its own analyses.
Transparent processes: all advice, information and analysis produced and provided to the government is subject to public scrutiny through consultative forums and release of preliminary findings and draft reports.
Community-wide perspective: under its statutory guidelines, the commission is required to take a view that encompasses the interests of the entire Australian community rather than particular stakeholders.
Source: Adapted from Australian Government, (2020[31]), About the Commission, https://www.pc.gov.au/about
Another interesting example is France Stratégie, which is a strategic office attached to the Prime Minister’s office, but operating at arms’ length from day to day decision making and the rest of the French CoG (see Box 1.7).
Box 1.7. The relationship between advisory bodies and CoG : the example of France Stratégie
Created in 2013, France Stratégie is an independent advisory body affiliated with the French Prime Minister’s Office. It conducts studies shedding light on medium and long-term policy issues that fall within its fields of competence (economics, social policy, labour, employment and skills, sustainable development and digital technology).
While the Prime Minister’s Office and line ministries are in charge of implementing and monitoring public policies, France Stratégie coordinates ex post policy evaluation. It also fosters dialogue on policy-making with various stakeholders (academia, government bodies, unions, international organizations and civil society) and recommends policy orientations and reforms. Finally, it oversees a network of seven specialized public bodies that conduct studies, such as the Council of Economic Analysis (CAE) and the Centre for Forecasting and International Studies (CEPII).
To carry out its research, France Stratégie draws on a team of multidisciplinary experts, analysts and scientific advisers. Its autonomy is guaranteed as it is solely responsible for its publications and communicates independently. It also carries out its work in a non-partisan manner, in interaction with different political parties, unions as well as social and regional entities.
Each year, France Stratégie defines its programme of work in accordance with policy priorities. For instance, sustainable development was a common theme to all studies launched in 2019. At the request of the Prime Minister, it can also provide forecasts and recommendations on major government reforms.
Sources: Adapted from France Stratégie, (2018[32]), France Stratégie 2018, https://www.strategie.gouv.fr/
Deciding the distribution of functions is in itself a political choice but ought to be done in a way that ensures policy coherence and efficiency. Giving the council a strategic function in long-term planning, focusing on generation of evidence and planned evaluation rather than the yearly monitoring would serve such a purpose. This will also provide institutional space for the development of high-level policy documents. Additionally, it would reinforce the council’s role as a knowledge broker and policy advisor (see chapter 5).
Meanwhile, the implementation and monitoring functions are best suited to governance structures close to power. This is well-supported by OECD analysis, which has found that the centre of government (CoG) is ideally placed to monitor the implementation of government-wide policies and programmes, to remove obstacles and implementation bottlenecks when performance is hindered, and to manage political considerations to facilitate the approval and implementation of government policies (OECD, 2017[30]). Finally, the current situation has been exacerbated by a conceptual confusion between monitoring and evaluation that has affected the organisation and the dynamics of the whole M&E ecosystem. Again, article 18 of the Strategic Planning Law gives a single definition for both monitoring and evaluation, while the two practices are distinct and require different kinds of professional background and expertise (Table 1.1). This has led to several misunderstandings. For example, the council’s annual reports have been called “evaluation reports” although in reality these are monitoring reports (see chapters 3 and 4).
The centre of government has limited capacities for strategic planning, coordination and monitoring in Nuevo Leon.
From an OECD perspective, the emergence of cross-cutting, multi-dimensional policy issues requires the set-up of appropriate structures, to overcome silos, ensure policy coherence and provide functions for policy coordination both horizontally across administrative units and vertically across levels of government. Such functions are traditionally located within the centre of government. Although the institutional peculiarities of CoG structures vary across OECD countries, they share common functions in spearheading whole-of-government coordination. This includes driving evidence-informed, inclusive and timely decision-making; policy coordination across government; medium-term strategic planning; monitoring and evaluating the implementation of government policy; strategic communication; and leading transition planning (OECD, 2020[33]).
In Nuevo León, there is currently no body with the explicit mandate of co-ordinating and monitoring the implementation of the Strategic Plan across government as the council has a responsibility limited to monitoring but not to coordination. Article 3 of the Strategic Planning Law does indicate that the state has to coordinate with municipal and federal governments in order to implement joint actions and programmes that promote the attainment of objectives that are included in different planning instruments. However, this lacks clarity in terms of who should coordinate the implementation of the Plan and the way in which it shall be done.
Another challenge is that there are insufficient resources and mechanisms within the administration dedicated to implementing the Plan’s objectives and projects. OECD research suggests that planning processes should always include “the proper calculation of the costs of resources needed for its implementation. In addition, steps must be taken to ensure that the identified resources are budgeted, that is, they are set aside in the annual and medium-term budget(s)…” (Vági and Rimkute, 2018[6]).
Although the Strategic Plan’s opportunity areas were broken into strategic lines and initiatives, these were not directly linked to the use of inputs (human, financial or material resources) with clear responsibility for their implementation (OECD, 2018[34]). For instance, implementing a Strategic Plan requires some form of costing exercise and alignment to a budget and short-term objectives, which is lacking in the case of Nuevo León. In addition, this Strategic Plan has not been properly articulated with the State Development Plan, which itself should be linked to the budget process.
To address the need for strategic coordination, the state of Nuevo León first created the state public administration’s Executive Coordination in 2015, whose main function was to coordinate the integration and operation of the government cabinet, liaise with civil society and guide the policies, plans, programmes and actions of the public administration. Nevertheless, this unit had limited mandate and capacities to align and co-ordinate the implementation of the different planning mechanisms (Strategic Plan, State Development Plan, and budget) across the administration and was ultimately dissolved in 2020.
The Executive Office of the Governor (EOG) absorbed the responsibilities of the former Executive Coordination. These recent changes represent an opportunity to strengthen the mandate and capacities of this office to further align them to OECD practices on centre of government, in particular fulfilling the full functions of planning and monitoring. In order to strengthen the CoG functions on strategic planning, Nuevo León could consider formally transferring the former functions of the Executive Coordination office to the Executive Office of the Governor. This can be done, in the short term, through revising the Internal Guidelines of the EOG. “OECD country experience also suggests that the centre is relatively more involved in assuring funding for overall national strategies and assessing whether the overall programme respects the boundaries of the country’s fiscal framework; just over half (54%) of respondents claimed overall responsibility for this” (OECD, 2018[34]). This could surely apply to the case of Nuevo León. Based on OECD country experiences, “most monitoring regimes are based on requirements for departments to report on progress against their own individual departmental plans – including financial, human resources and administrative performance - which the centre then collates” (OECD, 2018[35]).
Recommendations
In light of the above analysis, and taking into account the results of international experiences and good practices, this report recommends for Nuevo León public authorities to:
Clarify the council and the centre of government’s respective responsibilities on strategic planning (notably the Executive Office of the Governor and the Secretariat of Finance) with a view to strengthening capacities for the design, implementation, monitoring and evaluation of policy priorities.
In particular, the centre of government should be coordinating the various implementation and monitoring activities of the Strategic Plan and of the State Development Plan. To this end, the CoG has to guarantee that the elaboration manual of the SDP include a link to the Strategic Plan This should be coordinated with the Secretariat of Finance, as stated in the current legal framework, to ensure coherence with financial resources. The release of a communication leaflet on the monitoring of the Strategic Plan, coordinated by the Executive Office of the Governor, should be done at mid-political term and at the end of the mandate, that is, every 3 years. Chapter 3 will further analyse the monitoring process and the role of the different actors, including the council.
Strengthen Nuevo León council’s role as an advisory body and strategic knowledge broker (see chapter 5), moving it away from implementation and monitoring (see chapter 4). This requires building technical capacities within the council to:
Provide objective and timely policy advice to the government in key priority areas, based on the collection, analysis and facilitation of evidence. This can include practices such as strategic foresight (long-term), identifying knowledge gaps, conducting evidence synthesis, etc.
Select and discuss a limited number of policy priorities that require the concerted action of the state government, civil society and the private sector, for which the council would prepare focused evaluations with clear recommendations. One specific example is the “Hambre Cero” initiative.
Promote policy evaluations in various areas.
Strengthen the Nuevo León’s centre of government, with a specific mandate and resources to enable its function in terms of coordination, monitoring as well as the evaluation of the programmes derived from the State Development Plan. In order to strengthen the CoG functions, Nuevo León could consider formally transferring the former functions of the Executive Coordination office to the Executive Office of the Governor. This can be done, in the short term, through revising the Internal Guidelines of the EOG. The centre of government should be given the functions, mandate, responsibilities and resources to:
Ensure effective whole-of-government coordination, such as the creation of specific roundtables and follow up instruments.
Carry out objective setting and prioritization exercises, which is the ability to translate political commitments into clear and measurable policy objectives and actions plans. This includes practices such as strategic design, logic modelling and challenges and prices, and should not be limited to the implementation of the State Development Plan.
Publish regular monitoring results. Instead of a yearly report, this could be done on a biannual basis for the strategic plan. It could include the definition of key performance indicators and in particular the capacity to strategically integrate and manage performance information for the development, coordination and implementation of key policy priorities.
Conduct strategic internal and external communication, particularly through use of social media to outline key progress, success stories, bottlenecks and possible areas for improvement.
Equip the administration with the tools, human and financial resources, and institutional structures/arrangements to disclose and share key performance information with relevant stakeholders.; and
Pursue efforts to implement a coherent strategy for monitoring and evaluation policy for Nuevo León, drawing on the respective strengths of the council, the Executive Office of the Governor and of the Secretariat of Finance. Realigning the functions will strengthen the monitoring and evaluation apparatus as a whole. Overall, the apparatus requires:
A clear definition of the concepts “monitoring” and “evaluation” for the entire state government (including the council).
Specific capacities and provisions to monitor and evaluate both the Strategic and the State Development Plan.
Clarification of the council’s functions in terms of planning, policy advice and evaluation.
References
[28] Alessandro, M., M. Lafuente and C. Santiso (2013), The Role of the Center of Government: A Literature Review | Publications, Inter-American Development Bank, https://publications.iadb.org/publications/english/document/The-Role-of-the-Center-of-Government-A-Literature-Review.pdf (accessed on 5 March 2020).
[31] Australian Government (2020), About the Commission, https://www.pc.gov.au/about (accessed on 6 March 2020).
[11] Castro, M., G. Lopez-Acevedo and G. Busjeet (2009), Mexico’s M&E System: Scaling Up from the Sectoral to the National Level, http://www.worldbank.org/ieg/ecd (accessed on 30 January 2020).
[14] CONEVAL (2017), “Diagnóstico del avance en monitoreo y evaluación en las entidades federativas”, https://www.coneval.org.mx/coordinacion/entidades/Documents/Diagn%C3%B3stico_2017/Diagn%C3%B3stico_ME_2017.pdf (accessed on 16 October 2019).
[36] Consejo Nuevo León (2018), Manual de operación: Consejo Nuevo León para la Planeación Estratégica.
[24] Council of Nuevo Leon (2019), Nuevo León Council | About us?, https://www.conl.mx/quienes_somos.
[37] Council of Nuevo León (2017), Análisis y replanteamiento de los ingresos, gastos y deuda del Estado de Nuevo León, Centro de investigación económica y presupuestaria.
[17] Council of Nuevo León (2016), “Strategic Plan for the State of Nuevo León 2015-2030”.
[3] European Commission (2010), Europe 2020: A strategy for smart, sustainable and inclusive growth, https://ec.europa.eu/eu2020/pdf/COMPLET%20EN%20BARROSO%20%20%20007%20-%20Europe%202020%20-%20EN%20version.pdf (accessed on 6 March 2020).
[38] France Stratégie (2019), France Stratégie official website, https://www.strategie.gouv.fr/ (accessed on 3 February 2020).
[32] France Stratégie (2018), France stratégie, plaquette 2018.
[13] Gaarder, M. and B. Briceño (2010), “Institutionalisation of government evaluation: balancing trade-offs”, Journal of Development Effectiveness, Vol. 2/3, pp. 289-309, http://dx.doi.org/10.1080/19439342.2010.505027.
[23] GESOC (2017), Consejos de Monitoreo y Evaluación de Política Social, http://www.coneval.gob.mx/Medicion/MP/Paginas/Pobreza_2014.aspx.
[25] Hambre Cero Nuevo León (2020), Hambre Cero.
[12] Lázaro, B. (2015), Comparative study on the institutionalisation of evaluation in Europe and Latin America, Eurosocial Programme, Madrid, http://sia.eurosocial-ii.eu/files/docs/1456851768-E_15_ENfin.pdf (accessed on 9 July 2019).
[8] Masuku, N. and Ijeoma (2015), A Global Overview of Monitoring and Evaluation (M&E) and its Meaning in the Local Government Context of South Africa.
[1] Máttar, J. and L. Cuervo (2017), Planificación para el desarrollo en América Latina y el Caribe: Enfoques, experiencias y perspectivas, http://www.cepal.org/es/suscripciones.
[10] McDavid, J., I. Huse and L. Hawthorn (2016), Program Evaluation and Performance Measurement: An Introduction to Practice.
[9] OECD (2020), Improving Governance with Policy Evaluation: Lessons From Country Experiences, OECD Publishing, https://doi.org/10.1787/89b1577d-en.
[33] OECD (2020), Policy Framework on Sound Public Governance - OECD, OECD Publishing, https://www.oecd.org/governance/policy-framework-on-sound-public-governance/ (accessed on 25 November 2020).
[19] OECD (2019), Governance as an SDG Accelerator : Country Experiences and Tools, OECD Publishing, Paris, https://dx.doi.org/10.1787/0666b085-en.
[7] OECD (2019), Open Government in Biscay, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/e4e1a40c-en.
[39] OECD (2019), Toward a sound monitoring and evaluation system for the Strategic Plan of the State of Nuevo León 2015-2030.
[34] OECD (2018), Assessment of Nuevo León’s Strategic Plan 2015-2030.
[35] OECD (2018), Centre Stage 2- The organisation and functions of the centre of government in OECD countries, https://www.oecd.org/gov/centre-stage-2.pdf.
[16] OECD (2018), OECD Integrity Review of Nuevo León, Mexico: Sustaining Integrity Reforms, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264284463-en.
[4] OECD (2018), OECD Public Governance Reviews: Paraguay: Pursuing National Development through Integrated Public Governance, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264301856-en.
[26] OECD (2018), Public procurement in Nuevo León, Mexico: Promoting efficiency through centralisation and professionalism.
[30] OECD (2017), “Policy Advisory Systems - Supporting Good Governance and Sound Public Decision Making”, https://www.oecd.org/governance/policy-advisory-systems-9789264283664-en.htm (accessed on 21 October 2019).
[18] OECD (2017), Recommendation of the Council on Open Government, http://acts.oecd.orgRECOMMENDATIONPUBLICGOVERNANCE (accessed on 26 February 2020).
[21] OECD (2016), Open Government: The Global Context and the Way Forward, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264268104-en.
[27] OECD (2014), “Centre Stage Driving Better Policies from the Centre of Government”, http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=gov/pgc/mpm(2014)3&doclanguage=en (accessed on 23 September 2019).
[5] OECD (2010), Finland: Working Together to Sustain Success, http://www.sourceoecd.org/governance/9789264085893www.sourceoecd.org/9789264085893 (accessed on 17 February 2020).
[20] OECD (2009), Focus on Citizens: Public Engagement for Better Policy and Services, http://www.sourceoecd.org/governance/9789264048867 (accessed on 18 February 2020).
[22] Open Government Partnership (2019), Designing and Managing an OGP Multistakeholder Forum.
[29] Secretary of the State (2009), Organic Law of Public Administration from the State of Nuevo Leonn, http://sgi.nl.gob.mx/Transparencia_2015/Archivos/AC_0001_0002_0167618-0000001.pdf (accessed on 27 February 2020).
[15] State of Nuevo Leon (2019), Consejo Estatal de Transporte y Viabilidad: Responsabilidades de la dependencia, http://www.nl.gob.mx/dependencias/cetv/responsabilidades (accessed on 8 April 2020).
[2] United Nations (2020), Sustainable Development Goals Knowledge Platform, https://sustainabledevelopment.un.org/ (accessed on 6 March 2020).
[6] Vági, P. and E. Rimkute (2018), “Toolkit for the preparation, implementation, monitoring, reporting and evaluation of public administration reform and sector strategies: Guidance for SIGMA partners”, SIGMA Papers, No. 57, OECD Publishing, Paris, https://dx.doi.org/10.1787/37e212e6-en.