OECD Integrity Review of Mexico City

Chapter 2. Developing a strategic approach to public integrity in Mexico City

Abstract

This chapter assesses Mexico City’s current monitoring and evaluation framework for integrity policies. It provides an overview of how the recently established Local Anti-corruption System can formulate a monitoring and evaluation framework. Evidence generated through monitoring and evaluation makes it possible to assess the performance and the progress of the anti-corruption system. From this data, lessons can be drawn that can be used to strengthen the system.

2.1. Introduction

Setting strategic objectives and priorities for the public integrity system, based on evidence aimed at mitigating public integrity risks, can help develop a strategic approach to strengthening public integrity (OECD, 2017[1]). Such an approach shifts the focus from ad hoc integrity policies to a behavioural, risk-based approach to strengthening integrity that is better equipped to adapt to the context on the ground.

The sustainability of integrity policies as governments come and go is not always a given. It may be constrained by vested interests, a lack of strategic planning and co-ordination, the absence of institutional arrangements, insufficient financial and human resources and inadequate risk management processes, all of which severely dilute the impact of integrity policies. In addition, integrity systems often fail to build in a monitoring and evaluation mechanism to measure the outputs and outcomes of their policies over time.

Until now, Mexico City has not had an explicit integrity strategy, or integrity policies based on a strategic approach based on indicators and data. One of the priority areas for action of the General Programme of Development 2013-2018, however, is effectiveness and accountability in the fight against corruption. In this area, six opportunity areas have been identified: 1) better regulation and administrative simplification; 2) planning, evaluation and results-based budgeting; 3) use of information and communication technology (ICT); 4) transparency and accountability; 5) strengthening of public finances; and 6) professionalisation of the public service. While the inclusion of anti-corruption as a separate pillar is positive, Mexico City has not developed an action plan defining the sequencing of measures or detailing institutional responsibilities for implementing the strategy.

By creating the Local Anti-corruption System (Sistema Local Anticorrupción, or SLAC-CDMX), Mexico City has created a strong institutional framework for strengthening integrity, with clear institutional arrangements (see Chapter 1). The mandate of the Co-ordination Committee to approve, design, promote and evaluate integrity policy is an opportunity for Mexico City to build a strategic approach to integrity by setting objectives and priorities for the system based on relevant data. This should be reflected in the annual action plan drafted by the Co-ordination Committee.

Many integrity policies and strategies fail thanks to a lack of communication between those who design the strategies and those assigned to implement the measures. Public institutions with the mandate to implement specific preventive anti-corruption measures need to feature prominently at the design stage and to be held responsible for the results. This means that the political leadership of the country needs to demand this actively from its public sector chief executives, and hold them accountable for progress (Hussmann, 2007[2]).

In Mexico City, the representation of all the government entities responsible for integrity policies and civil society in the Co-ordination Committee create the basis for a broad coalition of support for the strategy. This can help reduce the vulnerability of the approach to leadership changes. Through their role in the Co-ordination Committee, the institutions can take an active part in shaping a strategic approach to integrity. This is a precondition for creating ownership for the strategy and for the subsequent implementation of the policy.

2.2. Developing a strategic approach to public integrity

2.2.1. The Co-ordination Committee could conduct a preliminary diagnosis of priority areas, by leveraging the data provided by the National Institute for Statistics and Geography and other expert assessments of corruption.

The first step in designing a coherent and comprehensive public integrity strategy is to analyse information on the current extent and nature of the problem. Typical questions would include, for example: Which sectors are most affected? What kind of harm is the lack of integrity causing? In which government entities is corruption most prevalent? Using an extensive knowledge and information base, the strategy and subsequent action plan can be focused and practical, help design policies and policy instruments for priority areas and ensure the correct prioritisation and sequencing of action. This information base can also ensure monitoring and assessment of the strategy, creating a baseline against which progress can be measured. The diagnostic can include detailed surveys and public opinion polls, expert corruption assessments, focus group discussions and sectorial assessments (Box ‎2.1).

Box ‎2.1. Diagnosing the problem in Lithuania

In Lithuania, an initial diagnostic was made before drafting the anti-corruption strategy. Chapter II of the current anti-corruption strategy provides a preliminary diagnosis of Lithuania’s anti-corruption environment, including a review of sociological surveys, analysis of the previous anti-corruption plan, summaries of proposed sectors with the highest risk of corruption, and a review of political and legal factors. Lithuania uses a range of national sociological and factual surveys. The data deals with sector-specific information on such subjects as health care, the judiciary, police and municipalities (the sectors still judged to be the most corrupt). They actively track opinions and data from the business community on different aspects of corruption. This might include reporting corruption, experiencing corruption, opinions on who is to blame for the corruption, opinions on the most effective remedies, and whether corruption is increasing or decreasing.

Source: (Pyman, M. et al., 2017[3]), “Countries curbing corruption: Research comparing 41 national anti-corruption strategies – Insights and guidance for leaders”, Norton Rose Fulbright.http://www.nortonrosefulbright.com/knowledge/publications/147479/countries-curbingcorruption

Mexico’s National Institute for Statistics and Geography (Instituto Nacional de Estadística y Geografía, or INEGI) has created a wealth of information on the extent of corruption as measured by citizens’ perception. This includes the perception of which government entity and sector is most corrupt, how often citizens were asked to pay a bribe and for which action, and an estimate of the cost of corruption. Internal staff surveys and data, for example, on penalties, staff cost or staff fluctuations and lessons from earlier policy approaches could provide additional information for a diagnostic. The review and recommendations would also be essential input for the initial diagnostic. Leveraging this knowledge base could help the Co-ordination Committee gain a detailed picture of the strengths and weaknesses of the current approach and could help conduct a systematic corruption risk assessment. For example, data on high staff turnover could present a higher risk of corruption, which would need to be taken into account in developing policy.

This risk assessment could help guide the overall goal of the policies and priority areas. Armed with this diagnostic, the Co-ordination Committee would be in a position to design a strategic approach to integrity, combining mutually reinforcing reforms that ultimately build into a comprehensive programme. This strategic approach should clearly define the impact that the Co-ordination Committee intends to achieve. General statements, e.g. combating corruption, should be disaggregated as much as possible into clear and measurable outcomes, to facilitate the subsequent monitoring and evaluation of the policies. Where possible, these should be translated into policy goals for the overall anti-corruption system and the government entities. The Co-ordination Committee could develop the annual work plan on this basis, with clear responsibilities assigned for each identified policy goal.

2.2.2. The Action Plan would need to be translated into a plan tailored to each government entity and included in its Annual Operational Programme.

The cornerstone of the success of the SLAC-CDMX will be to make sure that the design and purpose of integrity policies are clearly defined and implemented at the administrative level. While general overarching policies will be designed by the Co-ordination Committee, the organisational policies, measures and guidelines will require commitment from each government entity. To ensure this commitment, the responsibilities should be clearly spelled out in the Action Plan. In addition, the plan needs to include strategic objectives of the policies and information on the chain of results, resources needed for each measure by entity, and the timeline for implementation.

The Action Plan could also be complemented by organisational-level strategies addressing the specific integrity risks unique to individual organisations. The local Co-ordination Committee could oblige public sector entities to adopt their own risk-based approach towards integrity by conducting both fraud and corruption risk-mapping exercises and corresponding integrity plans of individual public sector organisations. This is the case in Latvia, where each ministry develops its own corruption prevention plan, or in the United States, where each government entity has its own ethics programme (Box ‎2.2).

Box ‎2.2. Corruption prevention plans at the institutional level

Several OECD member and partner countries require that individual line ministries or departments prepare corruption prevention plans that are tailored to their organisation’s specific internal and external risks. Every organisation is different, and risks for fraud and corruption thus vary, depending on mandate, personnel, budget, and infrastructure or IT use. For example, line ministries responsible for transferring social benefits face higher risks of fraud; likewise departments with higher public procurement spending (such as health or defence) may face risks of corruption associated with procurement. In addition to ensuring that prevention policies are developed on a risk-based basis, such plans also help guarantee that, where relevant, organisations’ anti-corruption efforts are aligned with national and sectorial strategies.

Some countries thus complement national anti-corruption plans with organisational level strategies. In Latvia, for example, each ministry has a corruption prevention plan, with oversight of the national anti-corruption agency, the Corruption Prevention and Combating Bureau (Korupcijas noversanas un apkarosanas birojs, known as the KNAB).

In Lithuania, the Special Investigation Service (SIS), an independent anti-corruption law enforcement body, is responsible for monitoring the implementation of the National Anti-corruption Programme, along with the Interdepartmental Commission on Fighting Corruption, led by the Department of Justice. The SIS co-ordinates risk-management activities throughout the public sector. Each public institution is required to design its own risk map, which is submitted to the SIS for review. The SIS provides guidance and comments to improve these plans.

In Slovenia, the Commission for the Prevention of Corruption assists organisations in developing unique integrity plans, which identify, analyse and evaluate risks and propose appropriate mitigation measures. The Commission urges departments to adopt an inclusive approach in the development of their plans, since it was found that they offered an opportunity to effectively communicate values and enhance a shared understanding of integrity. The commission provides guidance, such as sample integrity plans, on its website.

In the United States, the Office of Government Ethics (OGE) conducts reviews on government agencies’ ethics programmes about once every four years. These Ethics Programme Reviews are OGE’s primary means of conducting systematic oversight of the executive branch’s ethics programme. The Compliance Division’s Programme Review Branch conducts ethics programme reviews at each of the more than 130 executive branch agencies. This helps ensure that the ethics programme complies consistently and sustainably with established executive branch ethics laws, regulations and policies, and provides recommendations for meaningful programme improvement. Individual reviews identify and report on the strengths and weaknesses of an agency’s ethics programme by evaluating 1) agency compliance with ethics requirements, as set forth in relevant laws, regulations, and policies; and 2) ethics-related systems, processes and procedures for administering the programme.

In Colombia, individual organisations are required to institute their own risk maps and anti-corruption plans. The Anti-corruption Statute directs public entities of all kinds to produce a strategy at least annually to combat corruption and improve citizen service. These plans are based on the criteria defined by the Secretariat of Transparency of the Presidency of the Republic.

Sources: (OECD, 2017[1]), OECD Recommendation of the Council on Public Integrity; (OECD, 2017[4]), OECD Integrity Review of Colombia; OECD accession report of Lithuania (unpublished), OECD accession report of Latvia (unpublished), for the Office of Government Ethics, www.oge.gov/web/oge.nsf/Program%20Review.

In Mexico City, the Directorate for General Co-ordination for Administrative Modernisation (Coordinación General de Modernización Administrativa, or CGMA) in the Administrative Office of the Government of Mexico City, in co-ordination with the city’s School of Public Administration, has also developed a methodological guide for developing the institutional and sectorial programmes derived from the General Development Programme (Guía Metodológica para el Desarrollo de los Componentes de los Programas Derivados del PGDDF 2013-2018).The entity programmes translate the content of the sectorial programme, derived from the General Development Programme, into objectives and aims in the medium and long term. It also contains indicators to assess the institutional targets. The following steps are mandatory for developing the institutional programme:

1. Taking into account the diagnostic assessment of the sectorial programme;

2. Taking up the objectives of the sectorial programme corresponding to the institution;

3. Establishing quantifiable institutional targets based on the sectorial targets;

4. Elaborating indicators;

5. Determining policies on the institutional level that address the areas of opportunities of the General Development Programme;

6. Presenting the Institutional Programme to the head of government for approval, and publishing it in the Official Gazette.

While the framework seems well developed, it emerged during the interviews with stakeholders for this review that there was relatively little awareness of the existence of this monitoring and evaluation programme and in particular a limited degree of technical knowledge. This raises doubts to how effectively the General Development Programme is applied on an institutional level.

In Mexico City, the integrity plans of each government entity should form part of the Annual Operational Programme. For example, one of the risk factors identified might be an absence of rules and procedures that promote ethical behaviour and transparency. If this leads to ethical misconduct, one line of action in the Annual Operational Programme should be promoting ethical conduct in the entity. For this line of action, clear measures should be identified in the plan. A code of ethics should be introduced, or if one exists, it should be better publicised. The Supreme Audit Institution of Mexico City (Auditoría Superior de la Ciudad de México) and the Office of the Comptroller-General of Mexico City (Contraloría General de la Ciudad de México) could assess the quality of maps and plans on the basis of their prior audits (OECD, 2017[5]).

2.2.3. The Co-ordination Committee could develop a sub-strategy for sectors identified in the initial diagnostic as presenting a particular integrity risk.

A whole-of-government approach to integrity requires a broad local anti-corruption plan that covers integrity measures outside the public sector. The action plans should also specifically address, and urgently update, public sector integrity measures that may warrant a separate sub-strategy of their own. These should be tailored to the specific integrity risks of sectors, organisations and officials, which may require the creation of public integrity subsystems (OECD, 2017[5]). Sectors that could benefit from a specific integrity strategy would include the health sector and public procurement (see Chapter 7). For example, the Public Services and Procurement Canada 2017-18 Departmental Plan identified fraud and corruption as a key risk factor that could affect the effectiveness of the plan. To mitigate this risk, Public Services and Procurement Canada has devised an Integrity Programme that provides the government of Canada services and programmes that support sound and ethical management. Integrity was thus incorporated into the annual operational plan on public procurement. Furthermore, clear performance indicators were created to measure how effective the programme was at achieving the desired outcome (Table ‎2.1).

Table ‎2.1. Measuring the effectiveness of integrity programmes in public procurement in Canada

Expected results	Performance indicators	Targets	Deadline for achieving target	Actual results
				2015-16	2014-15	2013-14
High-quality and timely integrity programmes and services that support fairness, openness and transparency in government operations.	Number of government departments that apply the integrity regime.	140	31/03/2018	N/A**	N/A**	N/A**
	Percentage of simple reliability screenings processed within 7 business days	85%	31/03/2018	71%	67%	68%

** “Integrity regime” is a new performance indicator that was not reported in previous years.

Source: Public Services and Procurement Canada 2017-18 Departmental Plan, https://www.tpsgc-pwgsc.gc.ca/rapports-reports/pm-dp/2017-2018/pm-dp-04-eng.html#a4.

2.3. Monitoring and evaluation

2.3.1. The Executive Commission of the Local Anti-corruption System could invite the General Co-ordination for Administrative Modernisation to meetings to help draft the monitoring and evaluation framework.

As noted in the OECD Recommendation (2017[1]), a strategic approach to public sector integrity is based on evidence, and aims to identify and mitigate public integrity risks. This can be achieved by careful planning and by setting strategic objectives and priorities following a risk-based approach. It further involves developing benchmarks and indicators and gathering credible, relevant data on the level of implementation, performance and overall effectiveness of the public integrity system (OECD, 2017[1]). A monitoring and evaluation system can act as an assurance that integrity policies follow an evidence-based strategic approach, enabling continuous learning. The monitoring and evaluation programme will measure both the success of anti-corruption initiatives as well as the effects of failure. Evidence from monitoring or evaluation can also help target and guide current and future policies. It also makes it possible to detect challenges and problems in the implementation of the policy (OECD, 2017[6]). Effective monitoring and evaluation create a feedback mechanism for policy design. On the one hand, they help focus on mainstreaming the public integrity system’s strategic goals. On the other hand, they feed back information from the implementation level to the policy design stage and allow for effective steering, informed decision-making and improved policy design (OECD, 2017[6]).

Monitoring and evaluation strengthen accountability in the public integrity system, by making efforts and results measurable (Box ‎2.3). Determining whether the efforts have been successful, and benchmarking the different public entities can create public pressure to encourage integrity. Making the results public can create additional leverage to promote integrity policies (OECD, 2017[6]).

Box ‎2.3. The difference between monitoring and evaluation

Monitoring refers to the process of collecting and analysing information on a policy’s direct and intermediary outputs. Outputs are the direct results in the sphere immediately affected by the policy. What functions is the policy expected to implement? This question is typically answered at the output level. In some cases, outputs of a policy are self-evident to the degree that monitoring them becomes redundant. More information may then be obtained by monitoring the intermediate output. Intermediate outputs result from the policy at the first step of corollary inference. This means that they don’t automatically result from the policy, but are likely to occur if the policy is implemented as intended. Often, the usage or uptake of an output is a valuable intermediate output to observe.

Evaluation, in turn, asks for a policy’s mid- and longer-term outcomes. Outcomes are the indirect results of a policy in the final sphere of desired impact. They are indirect, since these outcomes are affected not only by the policy, but also by a range of other variables outside the control of the implementation process. They tend to capture the effect of a policy on social, economic or organisational variables. Due to the multiple factors influencing the desired outcome variable, the causal link between the specific policy and the observed outcome is usually not straightforward (i.e. an attribution gap occurs). While monitoring is often a continuous function, evaluation is a measurement endeavour specifically set up to investigate a certain policy’s effect, with a causal attribution.

Source: (OECD, 2017[6]), Monitoring and Evaluating Integrity Policies, OECD Publishing, Paris; (Mathisen et al., 2011[7]), How to monitor and evaluate anti-corruption agencies: Guidelines for agencies, donors and evaluators, U4, Issue No. 8, Chr. Michelsen Institute, Bergen, Norway.

In Mexico City, the CGMA is responsible for the design and co-ordination of monitoring and evaluation policies in the public administration. The Guidelines for Monitoring and Evaluation of the Public Administration of Mexico City (Lineamientos de Monitoreo y Evaluación de la Gestión Gubernamental de la Administración Pública de la Ciudad de México), issued in 2016, define the criteria and procedures that each entity in the public administration is required to observe. On the online platform Monitoreo CDMX, government entities can upload the indicators for each entity. The CGMA develops the Monitoring Report of the General Development Programme according to the five focus areas, the area for opportunities and the type of programme for each government entity. The Monitoring Report then serves as the basis for the Evaluation Report. The reports are presented to the Mayor of Mexico City, the Comptroller’s Office, the Ministry of Finance and the heads of the respective entities, which take the appropriate measures to ensure or improve fulfilment of the objectives.

Although a general monitoring and evaluation programme seems to exist, no specific monitoring and evaluation exercises appear to be focused on integrity policies. While the annual operational plans can include some integrity measures, which are evaluated to some degree, there is no overarching monitoring and evaluation framework for integrity. Mexico City could leverage the introduction of the SLAC-CDMX to establish such a framework. Under the Law of the Local Anti-corruption System of Mexico City, the Executive Commission will be responsible for drafting a proposal on the methodology for measuring the impact of the integrity policies, based on quantifiable indicators. It will also draft the annual progress and results report on integrity policies and programmes. Given the related mandate of the CGMA, the two bodies will need to collaborate closely to avoid any overlap. As CGMA oversees the general monitoring and evaluation framework, CGMA could be responsible for ensuring that the specific integrity monitoring and evaluation framework is embedded in and aligned with the general framework. To this end, the Executive Commission could invite CGMA to the sessions in which the methodology for measuring the progress of the integrity policies is designed.

2.3.2. In developing the monitoring and evaluation framework, the Executive Commission could call on the technical expertise of the Executive Secretariat.

Each integrity policy typically has one or several goals. A goal reflects the change the policy wants to encourage. A policy could, for example, have the goal of promoting merit-based recruitment in a public administration unit. The first step of any measurement process is to identify the final goals and translate them into intermediate objectives. Objectives define the implications of a goal in a specific context. Each objective summarises one aspect of a goal positively and unambiguously in a single sentence. Ideally, they provide the who, when, what and where of a goal.

Goals, objectives and indicators can be defined at output as well as outcome levels. They can also be designed to assess certain qualities of an output or outcome, e.g. the value in relation to an input (see Box ‎2.4) (OECD, 2017[6]).

Box ‎2.4. Example for outputs, intermediate outputs and outcome for an Integrity Code policy

Principle 4 of the 2017 OECD Recommendation of the Council on Public Integrity calls for “high standards of conduct for public officials” to be set by “including integrity standards in the legal system and organisational policies (such as codes of conduct or codes of ethics), to clarify expectations and serve as a basis for disciplinary, administrative, civil and/or criminal investigation and penalties, as appropriate”. One possible action for achieving this is the introduction of an Integrity Code for public officials. This table presents some potential goals, objectives and indicators that an Integrity Code could have on output and outcome level:

 

	Output	Intermediate Output	Outcome
Goals	Existence of a useful Integrity Code	Establish Integrity Code	Establish integrity as an organisational value
Objectives	Integrity Code: • exists • covers all relevant topics • is feasible	Public officials: • know the Integrity Code and have been trained in using it • are initiating discussions on grey areas and ethical dilemmas • suggest solutions • Managers use the Code as a management tool, e.g. in interviews of candidates for positions in their team, or performance evaluation interviews.	Public administration staff change their behaviour and make decisions based on the rules and principles of the Integrity Code.
Example indicator	• Identified risk areas are covered by the code • Staff at all managerial levels have participated in focus groups developing an Integrity Code	• Number of integrity-related suggested improvements • Percentage of staff working in risk areas who have received risk-specific integrity trainings • All applicants to a vacant position are introduced to the Integrity Code before proceeding in the selection process	• Integrity is measured in a staff survey

Source: (OECD, 2017[6]), Monitoring and Evaluating Integrity Policies, OECD, Paris.

In order to set up a measurement methodology, the Executive Commission could identify the goals in the Annual Plan and translate them into objectives. For example, if the recommendation given in Chapter 6, “Implement a systematic risk management framework to strengthen the internal control framework”, is incorporated into the Annual Plan, this would be a goal on the output level, at the level of the government entity. The objective would be for each entity to develop a systematic risk-management framework. The longer-term outcome of the policy would be to identify integrity risks and combat corruption through targeted action. This outcome would have the long-term impact of reducing corruption in risk areas (for further details, see ‎Annex 2.A). It is important that each objective translate the policy goal into a concrete action (OECD, 2017[6]), both for the overall anti-corruption system and for the government entities.

2.3.3. The Executive Commission could enlist the technical expertise of the Executive Secretariat to develop indicators for the Annual Plan.

Indicators provide measures that attempt to analyse the objectives and measure whether an objective has been fulfilled. One objective can have several indicators. Carefully chosen indicators are at the heart of any monitoring and evaluation strategy. Ideal indicators are specific, measurable and realistic. An ideal indicator measures only one variable unambiguously and involves only a reasonable amount of effort to monitor. In addition, indicators can be used to specify and measure certain qualities of an objective, such as cost-effectiveness or the sustainability of the policy (Box ‎2.5).

Box ‎2.5. Indicators to monitor the performance of the Corruption Prevention and Combating Bureau of Latvia (KNAB)

The Corruption Prevention and Combating Bureau of Latvia (KNAB) was established in 2002 and has 142 staff members today. Performance indicators for the bureau are available in its Progress and Results in Preventing and Combating Corruption in Latvia. The following list of indicators can be derived from its narrative report:

• number of legislative proposals in the area of anti-corruption

• number of draft legal regulations presented

• number of new laws successfully introduced

• number of amendments to existing laws successfully adopted by parliament

• amount of money discovered to have been spent illegally on political party financing

• amount of money spent illegally on political party financing reimbursed by parties

• number of asset declarations analysed

• amount of money earned by people breaking the laws on conflict of interest and additional employment and discovered by the Association of Chartered Accountants (ACA)

• amount of money recovered by the ACA

• number of people convicted as a result of investigations by the ACA

• number of criminal proceedings forwarded to the prosecutor’s office

• number of public servants trained on issues of conflict of interest, ethics and internal anti-corruption measures

• number of international requests for the ACA to provide its expertise abroad

• number of hosted delegations from ACAs abroad.

Source: (Mathisen et al., 2011[7]), How to monitor and evaluate anti-corruption agencies: Guidelines for agencies, donors and evaluators, U4 Issue No. 8, Chr. Michelsen Institute, Bergen, Norway.

The Executive Commission would need to define indicators to measure whether the objective has been reached. Several indicators can be defined for each objective. The indicators transpose the objectives to the operational level. Using the earlier example of promoting merit-based human resources, one indicator might be the share of positions filled during the last six months that were available on the website for at least eight days prior to the application deadline (OECD, 2017[6]).

In developing the indicators, the Executive Commission could invite government entities and experts to help identify relevant indicators. Stakeholder consultation improves the quality of the indicators and helps ensure that, at the end of the process, the evaluation findings will be considered credible by the parties involved. Through stakeholder involvement, the Executive Commission could also ensure that the indicators are measurable. Involving the members of the Citizen Participation Committee in the Executive Commission would help to establish a social control and to avoid setting the standard of the measurement indicators too low to be effective (OECD, 2017[6]).

Throughout the monitoring process, the Executive Secretariat would act as the impartial monitoring unit, allowing for the interlinkage of the planning and the implementation level. The Executive Commission could also undertake an examination of available data sources, to assess their relevance, applicability, validity and reliability. It could consider collecting the data centrally through the anti-corruption system’s Digital Platform. The Executive Secretariat would set the standard for data collection, upon which it could draw for its conclusions for the monitoring report. In its capacity to monitor the anti-corruption system, the Executive Secretariat would need to ensure that the government entities have the necessary capacity and knowledge to measure the data objectively and coherently.

Each government entity, in turn, would break these goals into specific objectives and indicators in its Annual Operational Programme, the organisational integrity plan. The entities would be required to report the results of the indicators to the Executive Secretariat of the SLAC-CDMX, which would use the data to create a Monitoring Report reporting to the Co-ordination Committee, as stipulated in the Law on the Local Anti-corruption System. Based on the monitoring report, the Co-ordination Committee could formulate recommendations for the entities on how to improve their integrity system.

Taking the earlier example of the recommendation “Existence of a systematic risk management framework”, an example indicator would be the relative number of all public entities in Mexico City’s public sector with a systematic risk management framework, and the existence of a central risk-mapping identifying the public institutions in Mexico City most at risk (for further details, see ‎Annex 2.A).

2.3.4. The Co-ordination Committee could use the public annual report on the Anti-corruption System to report on progress of the Action Plan.

Using public praise or criticism, the Co-ordination Committee could attempt to harness the power of public opinion to encourage the government entities to implement the Action Plan. In its annual report on the progress and results of anti-corruption system, the Co-ordination Committee could publish reports on the level of implementation of integrity policies in the entities. This could include, for example, the introduction of a code of ethics at the entity level, the percentage of staff aware of ethical dilemmas and the guidelines on corruption, and other such issues. Ideally, this should be tied to the indicators used to monitor the Action Plan. Public reporting on the progress made in implementing integrity policies might encourage these government entities to improve their performance. Cultivating broad-based support for the local anti-corruption system and its implementation among the public, the media and civil society organisations will also help with the mission of the Co-ordination Committee.

In Korea, the Anti-corruption and Civil Rights Commission (ACRC) develops a tiered ranking of how institutions perform according to performance groups (from 1 to 5, with 1 being the best-performing category). The tiered ranking of each institution and a consolidated report is released to the public. The results receive significant media attention, and institutions that score well enhance their organisational reputation. ACRC also provides each government entity with a specific report that includes suggestions for improvements (Lee and Lee, n.d.[8]).

2.3.5. Mexico City’s Council for Evaluation of Social Development could be charged with developing the evaluation methodology of the impact of the Local Anti-corruption System, so that the Co-ordination Committee can draft binding recommendations for improvement.

To assess what the impact of policy measures are, a clear and measurable methodology for evaluation will need to be developed. In this way, the positive change a policy has created towards a policy goal and impact can be measured and assessed. Objectives and indicators for evaluation require the same qualities as for monitoring, but are defined on the outcome level (OECD, 2017[6]).

Within the SLAC-CDMX, the Co-ordination Committee has the mandate to determine the methodology for impact evaluation. In the legislative proposal, this refers specifically to the work of the Internal Control Organs in evaluating the policies’ impact. However, when developing a specific evaluation framework, the Co-ordination Committee could consider adopting a more strategic approach. From the outset, it could set the priorities to evaluate certain policies whose effectiveness needs to be assessed, such as measuring the impact of whistle-blower protection on staff well-being. This would mean developing overarching objectives and indicators at the outcome level. Those outcomes would be long-term and involve the social and economic impact of the policy in relation to the long-term strategic goals of Mexico City’s General Development Programme. The objectives and indicators for the evaluation should be developed according to the same criteria as those for monitoring, but on the outcome level. For example, to evaluate a whistle-blower policy, one would look at the outcome level and assess whether a culture of integrity and accountability had been established, and whether individuals were confident in reporting fraud, misconduct and corruption.

To ensure the accountability and independence of the process, the methodology for the evaluation should be developed externally. The Council for Evaluation of Social Development of Mexico City (Consejo de Evaluación del Desarollo Social de la Ciudad de México, or EvalúaCDMX), a decentralised public entity in charge of the external evaluation of social policies, could develop the evaluation methodology. The entity is working with a network of external evaluators from civil society and academia to conduct the evaluation of social programmes. In addition, EvalúaCDMX could publish its recommendations for the evaluation methodology, which would create public pressure to comply with such recommendations. This is the reason why it is not advisable for EvaluáCDMX to be included in the Co-ordination Committee, as noted in Chapter 1, since it needs to be independent if it is to conduct the evaluation without bias.

Proposals for action

The Local Anti-corruption System of Mexico City has the potential for developing a coherent integrity system. To assess the goals of the system to combat corruption, a strong monitoring and evaluation framework will need to be developed. To this end, the OECD recommends that Mexico City consider taking the following actions:

Developing a strategic approach to public integrity

• To develop a targeted strategic approach to integrity and to the annual plan, the Co-ordination Committee could conduct a preliminary diagnostic of the priority areas, leveraging the data provided by National Institute for Statistics and Geography and other expert assessments of corruption.

• Taking the resources and capacity of each government entity into account, the Action Plan would need to be translated to an entity-specific integrity plan included in the entity’s Annual Operational Programme.

• The Co-ordination Committee could consider developing a sub-strategy for sectors that present a specific integrity risk, as identified in the initial diagnostic.

Monitoring and evaluation

• The Executive Commission, which is responsible for developing the monitoring and evaluation framework of the Local Anti-corruption System, could invite the General Co-ordination for Administrative Modernisation to meetings on drafting the framework, co-ordinating its efforts with the general public policy monitoring framework.

• To develop the monitoring and evaluation framework, the Executive Commission could call on the technical expertise of the Executive Secretariat, to translate the Annual Plan’s goals into objectives on the output and outcome level.

• To measure whether the objectives of the Annual Plan have been fulfilled, the Executive Commission, using the technical expertise of the Executive Secretariat, could develop indicators in consultation with the government entities and experts.

• The Co-ordination Committee could use the public annual report on the Anti-corruption System to report on progress that the government entities and agencies are making to encourage implementation of the Action Plan.

• Mexico City’s Council for Evaluation of Social Development could be the government entity responsible for developing the evaluation methodology for the Local Anti-corruption System, and support the Co-ordination Committee in drafting binding recommendations for improvement.

References

[2] Hussmann, K. (2007), “Anti-corruption policy making in practice: What can be learned for implementing Article 5 of UNCAC?”, U4 report 2, http://www.u4.no/publications/anti-corruption-policy-making-in-practice-what-can-be-learned-for-implementing-article-5-of-uncac.

[8] Lee, J. and A. Lee (n.d.), Introduction to Korea’s Anti-corruption Initiative Assessment – A Tool to Evaluate Anti-Corruption Efforts in the Public Sector in the Republic of Korea, http://www.undp.org/content/dam/uspc/docs/USPC%20ACRC%20final.pdf (accessed on 04 October 2018).

[7] Mathisen, H. et al. (2011), “How to monitor and evaluate anti-corruption agencies: Guidelines for agencies, donors, and evaluators” U4 Issue No. 8, https://www.u4.no/publications/how-to-monitor-and-evaluate-anti-corruption-agencies-guidelines-for-agencies-donors-and-evaluators-2/.

[6] OECD (2017), “Monitoring and Evaluating Integrity Policies”, https://one.oecd.org/#/document/GOV/PGC/INT(2017)4/en?_k=vpoh5x.

[4] OECD (2017), OECD Integrity Review of Colombia: Investing in Integrity for Peace and Prosperity, OECD Public Governance Reviews, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264278325-en.

[5] OECD (2017), OECD Integrity Review of Mexico: Taking a Stronger Stance Against Corruption, OECD Public Governance Reviews, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264273207-en.

[1] OECD (2017), OECD Recommendation of the Council on Public Integrity, http://www.oecd.org/gov/ethics/Recommendation-Public-Integrity.pdf.

[3] Pyman, M. et al. (2017), Countries curbing corruption: Research comparing 41 national anti-corruption strategies – Insights and guidance for leaders, Norton Rose Fulbright, http://www.nortonrosefulbright.com/knowledge/publications/147479/countries-curbing-corruption.

Annex 2.A. Developing a measurement methodology for concrete policy goals

Annex Table ‎2.A.1. Suggested measurement methodology for the recommendation “Implement a systematic risk management framework to strengthen the internal control framework”

Goals	Existence of a systematic risk- management framework	Implementation of a systematic risk- management framework to strengthen internal control framework	Integrity risks are effectively identified and corruption is countered by targeted action in these areas.	Reduction of corruption by addressing risk areas
Objectives	In each government entity, management has developed a systematic risk management framework.	In each entity, managers are aware of the relevant systematic risk management framework and make use of it in regular risk assessments.	Each government entity can clearly identify functions and decisions bearing a high integrity risk within their organisation.	Each government entity can combat internal corruption, thanks to awareness of the risk areas.
Example indicator	• Relative number of all public entities in Mexico City’s public sector that have instituted a systematic risk- management framework • Existence of a central risk- mapping, identifying the public institutions in Mexico City most at risk	Relative number of senior managers who report being aware of and applying the risk- management framework	• Management’s perception of the suitability of the risk assessment • Number of projects that meet their objectives within the specified period • Percentage of identified risks that were mitigated effectively • Reduction in perceived level of corruption	Randomised comparison of number of corruption cases in government entities with and without risk assessment
Example dataset	Internal records	• Internal records • Staff survey	Surveys	Encuesta Nacional de Calidad e Impacto Gubernamental (ENCIG) survey by INEGI