The OECD has defined six evaluation criteria – relevance, coherence, effectiveness, efficiency, impact and sustainability – and two principles for their use. These criteria provide a normative framework used to determine the merit or worth of a development intervention (policy, strategy, programme, project or activity). They serve as the basis upon which evaluative judgements are made.
The criteria are one part of a broader set of evaluation norms. They should be used in conjunction with the OECD Quality Standards for Development Evaluation, Principles for Evaluation of Development Assistance, and other applicable ethics standards and institutional guidelines.
They are not a methodology or approach but lay out the questions to be answered about an intervention – either as part of an evaluation or during the design, implementation or monitoring phases.
The criteria should be viewed as a set of lenses through which one can understand and analyse an intervention. They provide complementary perspectives, giving a holistic picture of the intervention, its implementation and results. Together they describe the desired attributes of all development interventions (policies, programmes and projects): they should be relevant to the context, coherent with other interventions, achieve results in an efficient way and have positive, lasting impacts for sustainable development.
The OECD Development Assistance Committee (DAC) adopted a new set of evaluation criteria in 2019, following an extensive consultation process. The Evaluation Criteria were first set out by the DAC in 1991 (Principles) and later defined in the first edition of the Glossary of Key Terms in Evaluation and Results Based Management. Originally intended for use in international development co-operation, they are now used widely across the public and private sectors and are enshrined in the OECD legal instrument on Evaluating Public Policy.
Since reducing inequalities and leaving no one behind are global policy goals, interventions are increasingly aimed at equity, and there is strong interest in evaluating whether particular interventions are helping to achieve those goals or not. Aspects of equity are covered in evaluations as part of relevance, effectiveness and impact – as well as under sustainability, coherence and efficiency.
Some criteria definitions were changed in 2019 to encourage better analysis of equity effects. Specifically:
Relevance is defined as responding to the equity conditions in context.
The definition flags the possibility of conflicts between different stakeholders’ priorities and needs (with marginalized groups often having been excluded from official processes to set priorities). The priorities of individuals affected by the intervention are given particular attention in the relevance criterion.
The definition of effectiveness includes “differential results across groups”, a significant change that encourages deeper investigation of exclusion and power dynamics
The 2021 guidance explains how to apply an equity lens to each criterion and provides examples.
Equity needs to be considered beyond the criteria: it should inform the entire evaluation process, including decisions about what gets evaluated, how and by whom. The OECD and EvalNet members continue to work to develop equity-informed approaches to evaluation.
No. The term “intervention” is used throughout the definitions because it is a broad, neutral term that can be applied to any evaluand (subject of evaluation) in different contexts. The criteria are commonly used for evaluating all kinds of public and private sector activities and policies, including strategies, thematic areas, peacebuilding activities, programmes, etc.
No. The evaluation criteria define the key questions evaluations will answer and are a key part of the OECD’s normative framework for evaluation. However, they are not themselves an evaluation approach or methodology and can be used for any type of evaluation. The criteria guidance should be read in conjunction with Evaluation Quality Standards and Principles.
The criteria are useful for evaluating efforts (whether national, sub-national or international) aimed at achieving the Sustainable Development Goals as set out in Agenda 2030 and national contributions to the Paris Agreement. These criteria can be used both to evaluate the extent to which environment and climate change are well-integrated into development interventions (mainstreaming), and to evaluate specific climate mitigation or adaptation interventions.
The revision of the evaluation criteria strengthens the emphasis on environmental effects in several ways, and will be useful to push for better evaluation of Paris-related commitments, as well as for applying a climate lens to evaluations of development and humanitarian activities. References to environmental effects have been added to the definitions of Relevance, Impact and Sustainability. Furthermore, Coherence was added as a new criterion to encourage better responses to the climate emergency, including analysis of potential synergies and trade-offs between development and climate priorities.
Yes and no. Yes, the criteria can be used to understand how different interventions are (or are not) creating transformational changes. The potential transformative effects of an intervention are captured under the new definition of impact. Evaluations should look more and more at questions related to transformation, reflecting the growing interest in understanding transformational change and the imperative for transformation described in both Agenda 2030 and the Paris Agreement. The evaluation methods needed to answer these questions are still emerging.
However, the evaluation criteria are intended to be used in evaluating an intervention (some intentional effort to create a change in the world), they are not a particularly useful tool for descriptive analysis of transformative change or systems change.
The criteria are useful for evaluating national or international SDG-related efforts, and they will also remain relevant and useful when policy priorities and goals change. The criteria have stood the test of time, and in today's context are widely used to evaluate interventions aimed at achieving goals set out in Agenda 2030.
However, the evaluation criteria are intended to be used in evaluating an intervention (some intentional effort to create a change in the world), they are not a particularly useful tool for descriptive analysis of transformative change or systems change.
