Monitoring Progress towards a Resource-Efficient and Circular Economy

Annex C. OECD approach to environmental indicators

Abstract

This Annex presents key elements of the OECD approach to environmental indicators developed with the active support of OECD Member countries.

General approach

For more than 40 years, the OECD has prepared harmonised international data and sets of indicators on the environment, and assisted countries in improving their environmental information systems.

The main aims of this work have been:

• to measure environmental progress and performance.

• to monitor and promote policy integration, in particular, the integration of environmental considerations into policy sectors, such as transport, energy and agriculture, and into economic policies more broadly.

• to help monitor progress towards sustainable development and green growth, including by measuring the extent of decoupling of environmental pressure from economic growth.

The OECD approach to indicators is based on the view that:

• There is no unique set of indicators; whether a given set of indicators is appropriate depends on its use.

• Indicators are only one tool among others and generally should be used with other information to draw robust conclusions.

• OECD environmental indicators are relatively small sets of indicators (core sets) that have been identified for use at the international level, and that should be complemented by national indicators when examining issues at national level.

The work builds on agreement by OECD members:

• to use the pressure-state-response (PSR) model or its variants as a common reference framework.

• to identify indicators on the basis of their policy relevance, analytical soundness and measurability.

• to use the OECD approach and adapt it to their national circumstances.

The development of environmental indicators in OECD has been grounded in the practical experience of OECD countries. It has benefited from strong support from member countries and their representatives in the OECD Working Party on Environmental Information and its predecessor bodies. OECD work on indicators also benefits from close cooperation with other international organisations, notably the United Nations Statistics Division (UNSD) and United Nations regional offices (including UNECE), the United Nations Environment Programme (UNEP), the World Bank, the European Union (including Eurostat and the European Environment Agency), and international research institutes.

Definitions and selection criteria

Box ‎C.1. Functions and definitions of environmental indicators

Functions

The OECD terminology points to two major functions of indicators:

• They reduce the number of measurements and parameters that normally would be required to give an exact presentation of a situation.

   As a consequence, the size of an indicator set and the level of detail contained in the set need to be limited. A set with a large number of indicators will tend to clutter the overview it is meant to provide.

• They simplify the communication process by which the results of measurement are provided to the user.

   Due to this simplification and adaptation to user needs, indicators may not always meet strict scientific demands to demonstrate causal chains. Indicators should therefore be regarded as an expression of "the best knowledge available".

Definitions

• Indicator: a parameter, or a value derived from parameters, which points to, provides information about, describes the state of a phenomenon/environment/area, with a significance extending beyond that directly associated with a parameter value.

• Index: a set of aggregated or weighted parameters or indicators.

• Parameter: a property that is measured or observed.

Source: OECD (1993), OECD Core Set of indicators for environmental performance reviews, Environment Monograph N.83, https://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=OCDE/GD(93)179&docLanguage=En.

Box ‎C.2. Criteria for selecting environmental indicators

As indicators are used for various purposes, it is necessary to define general criteria for selecting them and validating their choice. Three basic criteria are used in OECD work: policy relevance and utility for users, analytical soundness, measurability.1

 

POLICY RELEVANCE AND UTILITY FOR USERS	An environmental indicator should: Provide a representative picture of environmental conditions, pressures on the environment or society’s responses; be simple, easy to interpret and able to show trends over time; be responsive to changes in the environment and related human activities; provide a basis for international comparisons; be either national in scope or applicable to regional environmental issues of national significance; have a threshold or reference value against which to compare it, so that users can assess the significance of the values associated with it.
ANALYTICAL SOUNDNESS	An environmental indicator should: be theoretically well founded in technical and scientific terms; be based on international standards and international consensus about its validity; lend itself to being linked to economic models, forecasting and information systems.
MEASURABILITY	The data required to support the indicator should be: readily available or made available at a reasonable cost/benefit ratio; adequately documented and of known quality; updated at regular intervals in accordance with reliable procedures.

1. These criteria describe the “ideal” indicator; not all of them will be met in practice.

Source: OECD (1993), OECD Core Set of indicators for environmental performance reviews, Environment Monograph N.83, https://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=OCDE/GD(93)179&docLanguage=En.

Guidance for use

Box ‎C.3. Guiding principles for using environmental indicators

The OECD has accumulated practical experience not only in developing, but also in using environmental and related indicators in its policy work. The indicators are used as a specific tool for evaluating environmental performance and contribute to monitor sustainable development and measure progress with green growth.

When using environmental indicators in analysis and evaluation, the OECD applies the following commonly agreed upon principles:

Only one tool

Indicators are not designed to provide a full picture of environmental issues, but rather to help reveal trends and draw attention to phenomena or changes that require further analyses and possible action.

Indicators are thus only one tool for evaluation; scientific and policy-oriented interpretation is required for them to acquire their full meaning. They need to be supplemented by other qualitative and scientific information, particularly in explaining driving forces behind indicator changes which form the basis for an assessment. One should also note that some topics do not lend themselves to evaluation by quantitative measures or indicators.

The appropriate context

Indicators’ relevance varies by country and by context. They must be reported and interpreted in the appropriate context, taking into account countries’ different ecological, geographical, social, economic, structural and institutional features.

Inter-country comparison and standardisation

Most OECD indicators focus on the national level and are designed to be used in an international context. This implies not only nationally aggregated indicators, but also an appropriate level of comparability among countries.

There is no single method of standardisation for the comparison of environmental indicators across countries. The outcome of the assessment depends on the chosen denominator (e.g. GDP, population, land area) as well as on national definitions and measurement methods. It is therefore appropriate for different denominators to be used in parallel to balance the message conveyed. In some cases absolute values may be the appropriate measure, for example when international commitments are linked to absolute values.

Reference values

Relating the indicators to reference values (benchmarks, thresholds, baselines, objectives, targets), helps users better understand the significance of the indicator values, and enables comparison between data that are otherwise not easy to compare.

However, the choice of the initial level of an environmental pressure and of the time period considered can affect the interpretation of the results, because countries do proceed according to different timetables.

Level of aggregation

Within a country a greater level of detail or breakdown may be needed, particularly when indicators are to support sub-national or sectoral decision making. This is important, for example, when dealing with river basin or ecosystem management, or when using indicators describing drivers which are relevant at the local level. It is also important when national indicators hide major regional differences.

The actual measurement of indicators at these levels is encouraged and lies within the responsibility of individual countries. At these levels, however, comparability problems may be further exacerbated.

Source: OECD (1993), OECD Core Set of indicators for environmental performance reviews, Environment Monograph N.83, https://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=OCDE/GD(93)179&docLanguage=En

Box ‎C.4. Quality criteria for aggregated environmental indices

Policy makers and the public at large need reliable and well‑synthesised information about the environment without getting lost in detail. By combining the information contained in two or more indicators, aggregated indices make it possible to convey simple messages about complex environmental issues. Among their strengths is the potential to simplify the public communication process and to reach audiences that receive little environmental information at all. However, reducing the number of indicators by condensing information also runs the risk of misinterpretation because users are not always aware of the scope and limitations of the index methodology, and because the message conveyed may be distorted by data gaps.

At highly aggregated level, it is therefore appropriate for indicators to be used in conjunction with other indicators to gain a more balanced picture. It is also important that the aggregation process itself satisfies specific quality criteria.

Aggregation has been defined as “the process of adding variables or units with similar properties to come up with a single number that represents the approximate overall value of its individual components” (UNDESA, 2000). It requires a series of steps, that usually involve more or less subjective choices and judgements.

 

Aggregation steps

Selection of variables that are representative of the topic, policy issue or phenomenon of interest Transformation into a common metric. Weighting of the constituent variables. This is the process of judging the relative importance of various components of an index. Weighting can be carried out in different ways. Valuation. This means comparing the indices with a pre-determined classification of what constitutes good or poor values. It is commonly used to present air quality indices to the public.

The aggregation process itself should thus satisfy specific quality criteria keeping in mind the intended use of the resulting index, and keeping in mind the quality criteria of the indicators: policy relevance, analytical soundness and measurability. Among the quality criteria are:

• The aggregation process must be completely transparent, i.e. every step in the process should be traceable. Users should be aware of all assumptions and choices.

• The variables to be aggregated should be independent, i.e. not show cause-effect relationship.

• The variables to be aggregated should be situated at the same step in the cause-effect chain. This rule excludes the aggregation of pressure and state indicators.

• All components of an index should be part of the problem and amenable to change in response to human intervention (e.g. although temperature is an important factor in ozone formation, it is not a valid component of an air quality index).

• All components of an index should show about the same order of magnitude.

• The conversion (transformation) of indicators prior to their aggregation with other indicators should follow certain explicit rules. Also, the rules for comparing the results should be defined before selecting an aggregation method (because the choice of aggregation method affects the message conveyed).

• Never combine objective (i.e. by way of accepted methods used by the natural and social sciences) and subjective weighting methods in the same step of aggregation.

• An index should be tolerant to inconsistencies arising from aggregation and valuation.

• As for other indicators and depending on their purpose, additional information and interpretation in context is required for aggregated indices to acquire their full meaning.

Source: OECD (2002), Aggregated Environmental Indices – Review of Aggregation Methodologies in Use.

See also OECD/European Union/EC-JRC (2008), Handbook on Constructing Composite Indicators: Methodology and User Guide, OECD Publishing, Paris, https://doi.org/10.1787/9789264043466-en.