The Short and Winding Road to 2030

Annex A. Methodology

With the aim of helping its Member countries to implement the 2030 Agenda, and at their request, the OECD has developed a unique methodology for measuring the distance that OECD countries have to travel to achieve SDG targets. Since 2016, a series of reports has shown OECD average and country-level distances from the SDG targets based on indicators from UN and OECD databases. These reports also presented the current data gaps, identifying areas where statistical development would be critical to assess whether OECD governments are meeting the commitments they made when signing the 2030 Agenda in 2015.

Beyond providing a static snapshot of where countries stand today, this edition develops new tools to assess progress towards the SDGs over time, including a trend assessment (i.e. whether the trend, based on current policies, has been upwards, stable or downwards) and projections relying on stochastics methods to assess the likelihood of meeting the 2030 targets.

Selecting Indicators

The starting point of this report is the global indicator framework,1 developed by the IAEG-SDGs2 and adopted by the UN General Assembly. This choice (which also applied to previous editions of this report) was made following consultation with delegates to the OECD Committee on Statistics and Statistical Policy and reflected a number of considerations. First, the role played by the statistical community in monitoring the UN process. Second, the status of the global indicator framework as the only framework that has been internationally endorsed for the monitoring of the Sustainable Development Goals. Therefore, the indicators included in that framework are considered by the statistical community as the best choice to monitor SDG targets across countries, given the state of available information. Third, adhering as closely as possible to the global indicator framework limits the scope for additional judgements and interpretations of the SDG targets.

While the SDGs and the global indicator framework apply to all countries, as acknowledged by the 2030 Agenda, the targets (and therefore indicators) are aspirational and global and may need to be adapted to national contexts:

“Targets are defined as aspirational and global, with each Government setting its own national targets guided by the global level of ambition but taking into account national circumstances. Each Government will also decide how these aspirational and global targets should be incorporated into national planning processes, policies and strategies.”

In this spirit, and while recognising the need for comparability among OECD Member countries, the present report goes beyond the global indicator framework in a few cases, in particular, for:

• Monitoring indicators and targets for which no comparable data are currently available. For example, Target 11.3 on sustainable urbanisation is meant to be monitored by the “ratio of land consumption rate to population growth rate”. Yet data series on this indicator are not currently included in the SDG Global Database. This report thus relies on OECD series on the average annual change in built area per capita (see Haščič and Mackie (2018[1]) for more detail).

• Tailoring the analysis to the policy challenges confronting OECD countries, as reflected by the different work streams of the Organisation. For instance, focusing on mobile coverage to keep track of Target 9.a on connectivity would be inconsistent with the work carried out by the OECD working party on Communication Infrastructures and Services Policy that recognises the important interaction between fixed and mobile connectivity. Therefore, in this report, the monitoring of Target 9.a is complemented by a measure of fixed broadband subscriptions.

Choosing between different data sources

This report uses data from both the SDG Global Database and OECD sources to populate the global indicator framework. Nevertheless, neither of these sources provide an “off the shelf” solution for SDG monitoring in OECD countries. This implies that considerable data processing is needed to support the exercise undertaken in this report.

UN Data

The SDG Global Database compiles data provided by the UN System and other agencies (including the OECD) acting as “custodians” of specific indicators.3 This database primarily aims at feeding the UN Secretary-General's annual report on "Progress towards the Sustainable Development Goals". As of October 2021, OECD countries were covered in this database by 513 unique data series4 that allow keeping track of progress towards 154 of the 169 SDG targets (for 216 of the 247 indicators proposed by the Inter-Agency and Expert Group on SDG Indicators)5 over a period that can extend up to 60 years. This database is fully aligned with the global indicator framework, meaning that each data series included in the database is associated with one of the 247 indicators identified by the IAEG-SDGs.

A number of steps were taken to structure the database to support the analysis in this report:

• First, some variables were transformed to make them usable for the analysis, for instance by converting monetary variables into constant PPPs or by attributing specific numerical values to data expressed as ranges (e.g. for most OECD countries, the Proportion of population with primary reliance on clean fuels and technology (Indicator 7.1.2) is “>95”, for the purpose of this report, it became 97.5).

• Second, systematic controls and quality checks were run to identify possible inconsistencies in data series.

• Third, all data series were carefully reviewed to discard those that do not directly measure the achievement of SDG Targets.6

• Finally, some data series refer to different population groups (e.g. by gender, age or disability status) but also by mode of transport, types of product, etc. The UN database is structured to allow identifying the “main” population, with additional data series being considered as “disaggregations” of the main one. In most cases, the choice of the most suitable series for this report was obvious. For instance, the proportion of fatal occupational injuries per 100 000 employees (indicator 8.8.1) is available by migratory status and gender but also for the total population, which was here selected as the main data series. However, in other cases, it was not possible to consider a specific data series as more representative than others. For example, the number of deaths attributed to non-communicable diseases (3.4.1) is available in the UN database for four different diseases (cardiovascular disease, cancer, diabetes and chronic respiratory disease). For these data series, all the different indicators were considered separately.

Following these adjustments, 658 unique data series from the SDG Global Database are used for this report, each of them associated with a specific “SDG Indicator” (730 data series when taking into accounts data series associated to more than one SDG indicator).

OECD Data

In some cases, the degree of harmonisation and quality of the data used in this report was enhanced by using data from OECD sources pertaining to the global indicator framework. This allows tailoring the analysis to the policy challenges confronting OECD countries, as reflected by the different work streams of the Organisation.

The selection of OECD sources rested on an extensive consultation with other OECD directorates and affiliated bodies (such as the OECD Development Centre, the International Energy Agency or the International Transport Forum), which allowed to identify the most relevant and up-to-date sources. There are at least three main justifications for considering additional OECD data in this report:

• First, OECD data often complement the SDG Global Database. OECD data generally follow strict standardisation procedures, validated by Member countries, which facilitates cross-country comparison. The rigorous processes used by the OECD to collect and disseminate data allow meeting high statistical standards, thus providing higher quality and consistency than some of the data included in the SDG Global Database. For instance, under target 8.2, the indicator for productivity growth agreed by the IAEG-SDGs is “8.2.1 Annual growth rate of real GDP per employed person”. While this indicator is available in the SDG Global Database, OECD databases also include measures of productivity based on the number of hours worked, which provides a better assessment of the total quantity of labour inputs used in production (OECD, 2001[2]).

• Second, OECD data allow mirroring specific conditions from OECD countries. For instance, while mortality rates included in the OECD and the SDG Global Database are both based on the same original source (the WHO Mortality Database), the former are age-standardised (by the Secretariat) based on the structure of the OECD population in 2010. This ensures that countries’ comparisons are not unduly influenced by differences in the age structure of the population between different countries.

• Third, OECD sources usually provide a wider country coverage of Member countries, longer time series and more up-to-date data, while remaining close to the spirit of the 2030 Agenda. Analysis included in OECD (2019[3]) showed that, the numerical values of indicators based on OECD sources strongly correlate with those from the SDG Global Database.

The consultation conducted with other OECD Directorates allowed identifying 88 OECD data series that complement the 730 data series from the SDG Global Database. These OECD data cover 77 targets and span all 17 goals.

Restrictions

Together, UN and OECD sources comprise 818 data series, but not all of them are included in the analysis. While these data are deemed by the statistical community to be accurate, i.e. “they address the purposes for which they are sought” (OECD, 2011[4]) and relevant, i.e. they “correctly describe the quantities or characteristics they are designed to measure” (OECD, 2011[4]), in order to support a comparative benchmarking exercise, data also need to be broadly available among OECD countries and over time.

Minimum country coverage

Data series need to cover a minimum set of countries. Including indicators with a limited country coverage would weaken the robustness of the analysis. As the methodology underpinning this report uses a comparative approach to gauge a country’s performance on SDGs, a limited distribution of data across countries is likely to affect the results. Both the normalisation method used in this report – which uses the standard deviation measured among countries’ performances at a given point in time – and (part of) the target-setting – with some end-values based on the best performance(s) observed across OECD countries – are comparative in nature and can thus be affected by a limited country coverage.

Yet as country coverage grows, target coverage falls. Figure A A.1 shows that there is a clear trade-off between the minimum number of countries included in the analysis and the number of available data series. While partial country coverage undermines the robustness of the analysis, a partial coverage of an indicator limits its comprehensiveness. Setting a high minimum threshold for country coverage would prevent a comprehensive assessment of Member countries’ performance on the 2030 Agenda, as for some targets no indicators may be available to support our analysis.

Half of the data series feeding this report cover 30 OECD countries or more (Figure A A.1). However, in practice, some of the data series are available for only a much smaller number of OECD countries. For instance, around one in ten data series cover six OECD countries or fewer. Conversely, less than one in four data series cover all 38 OECD Member countries. This report arbitrarily sets the minimal threshold for country coverage at 20, as using a higher threshold would drastically reduce the number of data series considered in this report.

Figure A A.1. Distribution of data series by minimum number of OECD countries covered

Statlink

Beyond minimal country coverage, an additional criterion for data selection is that the series should ensure a sufficient global coverage. The OECD has 38 Member countries spanning the globe, from North and South and from four world regions (America, Europe, Asia and Oceania). Therefore, an additional requirement for inclusion in this report is that a data series should cover at least three of these world regions.

Minimum length of time series

A dynamic assessment of countries’ performances on SDGs raises additional data challenges, related to the availability of robust time-series information.Two different concepts allow gauging the “length” of the available time series: the time-span (i.e. the number of years between the first and last available data points) and the number of observations within that time-span. When estimates are produced annually, the time-span equals the number of observations, but this is not the case when observations are available at irregular intervals. As a threshold, the methodology used in this report requires at least three observations (see the section Measuring countries’ performances over time). Yet the more observations (and the longer the time-span), the better is the assessment of the dynamics of the data series.

As shown in Figure A A.2, the number of available data series falls sharply when the average number of observations increases. For instance, while some data series may have 50 data points or more, only 25% of the series used in this report have more than 10 data points. Wherever possible, data series are tracked for the last two decades. However, in practice, to accommodate the fact that some of the available time series are much shorter, the minimum requirement for inclusion in this analysis is that at least three observations should be available over a five-year period.

Figure A A.2. Distribution of data series by average number of observations

Note: Estimates of the average number of observations include countries with no data (number of observations had been set at 0).Therefore the average length of observation may be below 1.

Statlink

Additional limitations

This study applies a standardised methodology to measure the distance between OECD countries’ current performances and where they should be in 2030. As detailed in the section Setting Target, the methodology rests on three elements: i) selecting indicators and data; ii) setting end-values for the indicators; and iii) normalising the values to a common basis, in order to allow assessing distances across different fields.

Therefore, while some data are available and meet the selection criteria mentioned above, they may not support the analysis in this report. For instance, end-values could not be set for a subset of these indicators, which are useful only to contextualise or complement other indicators. These indicators, while still included in this report when informative of the context of a specific issue, typically lack a clear normative direction (i.e. to judge what is good performance and what is bad). While no end-value is specified by the target for the recycling rate (indicator 12.5.1), there is a clear normative direction (the more, the better). Therefore, even when there is no clear target to be reached, it is possible to benchmark outcomes to top-performing countries. Conversely, forest area as a share of total land (indicator 15.1.1) in countries with a desert climate will never be as high as in countries such as Finland or Japan, where more than two-thirds of total land is covered by forest. In these cases, structural differences and circumstances will never allow matching the achievement of the best performers.

In addition, indicators that can take only a binary (yes or no) form, such as indicator 16.10.2 (assessing whether “countries adopted and implemented constitutional, statutory and/or policy guarantees for public access to information”) are considered only for assessing current performance, but not for progress over time.

The dataset supporting this report

In total, this report relies on data for 183 of the 247 indicators listed in the global indicator framework (or for close proxies of these indicators), covering enough OECD countries to support a comparative assessment.7 These indicators cover 134 of the 169 SDG targets. Target coverage is uneven across the 17 goals. For instance, Figure A A.3 shows that all the targets pertaining to the goals on Good health and well-being (Goal 3) and Quality education (Goal 4) are covered by at least one indicator. Conversely, other goals have significant data gaps. For instance, 1 in 5 targets under the goals on Gender equality (Goal 5), Climate action (Goal 13) and Affordable and clean energy (Goal 7) is not covered by our dataset, and the same applies to 2 in 5 targets under the goals on Sustainable cities (Goal 11), Life below water (Goal 14) and Partnerships for the goals (Goal 17).

Figure A A.3. Share of the 2030 Agenda’s targets covered in this report by at least one indicator, by goal and primary source

Note: Numbers from 1 to 17 stand for the goals: 1 No poverty, 2 Zero hunger, 3 Good health and well-being, 4 Quality education, 5 Gender equality, 6 Clean water and sanitation, 7 Affordable and clean energy, 8 Decent work and economic growth, 9 Industry, innovation and infrastructure, 10 Reduced inequality, 11 Sustainable cities and communities, 12 Responsible consumption and production, 13 Climate action, 14 Life below water, 15 Life on land, 16 Peace, justice and strong institutions and 17 Partnerships for the goals. These goals are grouped under five broad themes (the “5Ps”): People, Planet, Prosperity, Peace and Partnership.

Statlink

Target coverage varies widely among OECD countries. Figure A A.4 shows that it ranges from 70% or less (i.e. 120 of 169 Targets) in Colombia, Iceland, Luxembourg, Chile, Costa Rica and Israel to 80% (i.e. 135 of 169) in Slovenia and Italy. Although this is an improvement in coverage relative to both previous editions of this report and to other SDG-related measurement initiatives, significant data gaps for all OECD countries clearly remain. In addition, it should be noted that these coverage rates reflect the OECD focus of the report, with indicator coverage being lower for countries that joined the OECD in more recently.

Figure A A.4. Indicator coverage across OECD countries

Statlink

While the data used for this report allow covering 134 SDG Targets, a distance to target could be assessed for only 112 of them (i.e. 22 SDG targets are supported only by data that lack a clear normative direction). Figure A A.5 shows that, when limiting the analysis to indicators that allow assessing distances to targets, target coverage is also quite uneven across the 17 goals. While distance to target can be estimated for more than three in four targets for 8 of the 17 Sustainable Development Goals, none of them has all targets covered. Conversely, three goals (14 on Life below water, 9 on Industry, innovation and infrastructure and 17 on Partnerships for the goals) have less than half their targets covered by data that allow estimating distances from target levels.

Data gaps become starker when looking at data series that allow measuring the distance to target. Good health and well-being (Goal 3) and Quality education (Goal 4) are the only goals for which the data series included in this report allow monitoring more than 9 in 10 targets, while for the goals on Life below water (Goal 14), Industry, innovation and infrastructure (Goal 9) and Partnerships for the goals (Goal 17), less than half of the indicators for the targets are available to support the analysis. A dynamic assessment of countries’ performances on the SDGs raises additional data challenges, related to the availability of robust time-series information. Figure A A.5 shows that, for nine goals (Goal 5 on Gender equality, Goal 11 on Sustainable cities, Goal 16 on Peace, justice and strong institutions and Goal 17 on Partnerships for the goals as well as all of the Planet Goals besides Goal 6 on Clean water and sanitation), our database lacks the data needed to gauge progress over time for more than half of the targets.

Figure A A.5. Target coverage, by type of assessment, OECD average

Note: Numbers from 1 to 17 stand for the goals: 1 No poverty, 2 Zero hunger, 3 Good health and well-being, 4 Quality education, 5 Gender equality, 6 Clean water and sanitation, 7 Affordable and clean energy, 8 Decent work and economic growth, 9 Industry, innovation and infrastructure, 10 Reduced inequality, 11 Sustainable cities and communities, 12 Responsible consumption and production, 13 Climate action, 14 Life below water, 15 Life on land, 16 Peace, justice and strong institutions and 17 Partnerships for the goals. These goals are grouped under five broad themes (the “5Ps”): People, Planet, Prosperity, Peace and Partnership.

Statlink

Setting target levels and normalisation

This report applies a standardised methodology to measure the distance between OECD countries’ current performances and where they should be in 2030.

Once data series are selected, an appropriate end-value (target level) is set for each of them in order to measure the distance between the current position and the target level to be achieved. The 2030 Agenda does not always specify the end-value to be attained. Therefore, this report relies on a four-step process for setting end-values:

• Wherever possible, the target levels specified in the 2030 Agenda were used. This is typically a fixed value identified in the wording of the target (e.g. for Target 3.1, maternal mortality ratio below 70 for every 100 000 live births) or, in a small number of cases, it is expressed as a relative improvement from current levels (e.g. for Target 1.2, reduce by at least half the proportion of people living in poverty). These are classified here as “type-A” targets.

• When no target value is identified by the text of the 2030 Agenda, target levels were drawn from existing international agreements (e.g. reduce PM_2.5 pollution to less than 10 micrograms per cubic metre, according to the WHO) or based on OECD expert judgment (e.g. water stress is considered to be low if total freshwater abstraction is below 10% of total internal renewable resources (OECD, 2020[5]). These are classified as “type-B” targets.

• When no target value could be identified from either the 2030 Agenda or expert assessments, the target level is based on the “best performance” among OECD countries observed in the most recent available observation. This is defined in this report as the average level attained by the top 10% of OECD countries (e.g. in the case of the recycling rate of municipal waste). These are classified as “type-C” targets.

• Finally, for indicators which are useful only to contextualise structural differences and circumstances or to complement other indicators – typically indicators lacking a clear normative direction such as forest area as a proportion of total land area – no target level is set and therefore no “distance from target” is measured in this report.

Finally, in order to compare performance across different targets, indicator values were normalised using a modified version of the z-score (i.e. the distance from target levels is expressed as the number of OECD standard deviations observed across countries in the most recent year). This approach is described in this report as the “standardised difference” between the country’s current position and the target end-value. The greater the distance, the further the country will need to travel to achieve its target. A zero distance means the country has already achieved the 2030 target. Negative scores mean the country already exceeds the target and, in this report, are reported as zero (i.e. countries are not rewarded for going beyond the target). The distance to target is then defined as the average distance of data series that support the target (with equal weights between indicators as listed in the global indicator framework).

Measuring countries’ performances over time

Previous editions of this report, including OECD (2019[3]), mainly focused on countries’ current positions vis-à-vis the SDG targets (see the Setting target section for methodological details), rather than on the direction or pace of improvement. This static assessment does not capture the underlying path of countries’ performances. For instance, when a country is already at (or near) its 2030 target, it may slip behind if recent developments point to a worsening of its performance . Conversely, a country that is still far from its 2030 target might still be expected to reach it by maintaining the rapid progress that it has achieved in the recent past. Examining OECD countries’ recent historical performances provides a key complement to the assessment of their current positions and is therefore essential to inform priority setting.

Conceptual framework

Assessing trends is a challenging exercise. It is even more challenging in the context of the SDGs, as the 2030 Agenda includes a wide range of different indicators whose developments are to be assessed over a long period of time. In addition, while the 2030 Agenda does not apply equally to all countries, a comparative assessment needs to be based on a single procedure. Inter alia, this means that the same method should ideally be applied to different countries (irrespectively of their political, economic, social and environmental circumstances) and indicators (irrespectively of their nature).

Developing “dynamic baselines” requires both identifying past trends – which is difficult, especially when time series are short or lacunar – and predicting the future evolution of the different indicators – which requires making assumptions about the underlying drivers of change. Depending on the purpose of the exercise, different types of dynamic analysis could be carried out. These range from a simple detection of the recent trend to more sophisticated forecasting methods. Furthermore, some basic factors such as the length of the time series (i.e. the number of observations and the time-span covered) or the type of data (e.g. ordinal or cardinal) considered are likely to influence the method used. While a wide range of tools could be used, two broad types of approaches can be distinguished (Hyndman, 2011[6]):

• Explanatory models – i.e. models combining data analysis and expert judgement. In this case, models assume that the variable to be projected is linked through an explanatory relationship to one or more other variables. For instance, the OECD uses short-term economic indicators such as business sentiment, consumer surveys, industrial production, retail sales, house prices, etc., to predict near-term quarterly movements in GDP. The purpose of the explanatory model is to describe the form of the relationship between the variable of interest and its driving factors and to use it to forecast future values of that variable. While this type of analysis can provide highly reliable results, it could not be applied to forecast SDG indicators: first, it needs to be supported by in-depth evaluation both of the factors driving each data series and of contextual factors; second, it may not be appropriate to long-term time horizon projections.

• Time Series (or exploratory) models – i.e. models for which the analysis is based on observed data only and which make no attempt to uncover the factors driving the behaviour of the target variables. Within this class of models, the estimation can be parametric (e.g. linear, polynomial or exponential estimations) or non-parametric (Spearman's rho tests, modified Mann-Kendall test, Sen’s slope estimators, etc.) These models provide transparent results and can be easily adapted to different contexts; they are therefore preferred to assess trends in this report.8

All these reasons have also led most authors and international organisations to adopt rather simple exploratory models for assessing the direction and pace of recent changes. Most of the time, trends are assessed by comparing the observed change of a given variable and that required to reach the target by 2030. Some models assume linear growth (Sachs, 2020[7]), while others rely on geometric growth (Eurostat, 2021[8]; UNESCAP, 2020[9]; UNSD, 2020[10]) – for a more comprehensive review, see Gennari and D’Orazio (2020[11]). In practice, the estimations of both linear and geometric models rely on linear regressions between different observations of the same variable (e.g. the compound growth rate corresponds to drawing a line between the log-transformed values of the original variable).9 This report also adopts such a rather simple model for assessing the likely value of the different indicators by 2030. Yet, instead of making direct estimates of the value of the indicator by 2030, it models the likelihood of achieving a specific level, as detailed in Box A A.1.

Exploratory models use the inertia of the variable to estimate the value they could reach in 2030. They are quite flexible and can provide results even with short time series. However, as all models, they rely on specific assumptions. When the distribution of some indicators is unknown, when it violates some underlying assumptions or it includes outliers, the results from exploratory analysis will be less reliable. These issues are particularly important in times of great uncertainties.

Box A A.1. Using Monte Carlo simulations to estimate the likelihood of meeting a target at some future date

Monte Carlo methods encompass a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. The underlying concept is to use “randomness” to solve problems. In this specific case, by construction, the simulation will approximate the minimum mean square error forecast following a simple geometric growth model. Monte Carlo algorithms allow going beyond the average outcome by modelling a complete distribution of future events. Therefore, the share of simulations that reach or exceed the target level by 2030 allow estimating the likelihood of reaching this SDG target.

More concretely, a deterministic model would estimate a growth rate and use it to project the time series. Formally, if S_t is the level of achievement in time t, n is the final year and r is the estimated growth rate, this relationship could be expressed as:

1. $S_n=S_0{\left(1+ r\right)}^n$

In order to introduce a degree of uncertainty, Monte Carlo simulations allow for random variations of the growth rate. This allows projecting different plausible trajectories. Formally, if r is a random variable that can take different values at any point in time, defined as:

2. $r=\frac{S_{t+1}-S_t}{S_t}$

we can assume that r follows a normal distribution N(μ,σ)1 and denote as X the random variable following a standard normal distribution:

3. $\frac{S_{t+1}-S_t}{S_t}=\mu +\sigma X$

This equation can also be written as:

$S_{t+1}-S_t=\mu S_t+\sigma S_{t}X$

4. $S_{t+1}=\left(1+\mu \right)S_t+\sigma S_{t}X$

which allows to estimate a possible value of ${\left(S_t\right)}_t$ at any point in time. In order to reduce the computation time, this report estimates the value of S in time n as:

$S_n=S_0\mathrm{e}\mathrm{x}\mathrm{p}\left(\left(\mu -\frac{1}{2}{\sigma }^2\right)n+\sigma \sqrt{t}X\right)$

Finally, $S_n$ is estimated 10 000 times with different values for X. The likelihood of reaching the target is then defined as the shares of projected values that met the target level.

Note:

1. While most deterministic approaches used to estimate progress towards the SDG target do not account for the volatility of the past growth rate, using a random model allows modelling the uncertainty relating to past volatility.

In addition, instead of making explicit assumptions on the distribution of each variable, this report looks for the presence of a monotonic trend (i.e. whether the variable consistently increases, or decreases, through time). As detailed in OECD (2019[3]), trends are summarised by computing the Spearman (rank) correlation coefficient between the observed values of each data series (in their original units of measurement) and time (expressed in years). Thus, a significant positive correlation (approaching 1.0) indicated a positive overall trend of the data series over time, while a significant negative correlation (approaching -1.0) indicated a negative overall trend. Non-significant correlations (around 0) indicated that no consistent trend could be determined over the time period assessed.10 This rank-based approach has the advantage of being simple to implement. It also avoids making assumptions on the distribution of data (skewness, presence of outliers, etc.) or on the type of growth (linear or geometric) exhibited by each variable. However, the results obtained through trend detection methods need to be interpreted carefully, as the direction of the trend does not say anything about whether the pace achieved by a country would be sufficient to meet the target level by 2030.

To overcome the issues relating to both methodologies, this report combines both approaches to understand the dynamics behind the 2030 Agenda. Both methods are run independently (for instance, the coefficient correlation is not used to constrain the Monte Carlo simulation).

Details of the methodology used in the report

Combining the trend assessment with an estimation of the likelihood of reaching the target allows some flexibility. In short, rather than providing forecasts, this method allows to understand the underlying dynamics of the different indicators. Concretely, a trend can be “upwards” (i.e. improving over time), “stable” or “downwards” (i.e. deteriorating over time), while a target can be considered as “on track” (i.e. the current pace of improvement, when extended to 2030, should allow a country to reach its target value by the end of the period) or “off track” (in the opposite case). Therefore, there are six different situations, each of which is associated with one of the three cases listed below:

• “No progress or moving away from the SDG target”, when the likelihood to reach the target is below 75%, and when the recent trend cannot be classified as “progress towards the target”, i.e. the correlation coefficient11 between the indicator and the year is below 0.20 (or the coefficient is not statistically significant at the 10% level);

• “Progress is being made but is insufficient to meet the target”, when the likelihood to reach the target is below 75%, and the correlation coefficient between the indicator and the year is above 0.20 and significant at the 10% level;12

• “Target is on track to being achieved”, when the likelihood to reach the target is above 75%.

When more than one data series is available for measuring a given SDG indicator, the indicator is classified according to where most of the underlying data series stand. While these simplifications might overlook some specific situations, they provide a meaningful overall picture.

No progress or moving away from the SDG target

As mentioned above, an indicator is classified as “No progress or moving away from the SDG target” when the likelihood to reach the target by 2030 is below 75% and when the trend cannot be classified as “progress towards the target”. Concretely, there are two possible cases:

• The indicator is on a downward trend, and it is not likely that the target would be achieved by 2030. As shown in Figure A A.6, panel A, this is the case, for example, of the obesity rate in Denmark, where the share of population considered as obese has been increasing constantly over the past 20 years. In the absence of a significant change in the recent dynamic, Denmark is likely to be even further away from the target by 2030 than it is now.

The indicator does not show any specific trend and is not likely that the target would be achieved by 2030. As shown in Figure A A.6, panel B, relative poverty in Latvia has been hovering around 15% for the past 15 years. Therefore, in the absence of a significant change in this trend, Latvia is likely to stagnate around the same value, yet, given the relative volatility observed over the past 15 years, the model allows for wide variations around this average scenario. In any case, though, the relative poverty rate in Latvia is not likely to reach the target level by 2030.

Figure A A.6. Example of data series classified as “No progress or moving away from the SDG target”

Note: The horizontal line stands for the 2030 target value to be reached. Dotted lines reflect the 10th and 90th percentiles of the projected data series; dashed lines reflect the 25th and 75th percentiles of the projected data series; the continuous lines reflects the 50th percentile of the projected data series.

Statlink

Progress has been made, but is insufficient to meet the target

An indicator is classified as “Progress has been made but is insufficient to meet the target” when the likelihood to reach the target is below 75%, and the correlation coefficient between the indicator and the year is above 0.20 and significant at the 10% level. Concretely, there is only one scenario in this case:

the trend is upwards but few (or none) of the projected values will meet the target. An example is provided by Figure A A.7 on greenhouse gas emissions per unit of GDP in Chile. In 20 years, greenhouse gas emissions fell from 0.33 tonnes of CO₂ equivalent per USD in early 2000 to 0.26 tonnes in 2018. While progress is being made, unless the pace increases, it will not be enough to reach the target by 2030.

Figure A A.7. Example of data series classified as “Progress has been made, but is insufficient to meet the target”

Note: The horizontal line stands for the agreed 2030 desired value to be reached. Dotted lines reflect the 10th and 90th percentiles of the projected data series; dashed lines reflect the 25th and 75th percentiles of the projected data series; the continuous plain lines reflects the 50th percentile of the projected data series.

Statlink

The target is achieved or on track to being achieved

An indicator is classified as “being achieved or on track to being achieved” when it has a high likelihood to meet the target by 2030. In this case again, there are three different possible scenarios:

• The trend is stable and the indicator is classified as on track, as more than 75% of projected series meet the target. For instance, Figure A A.7, panel A, shows that, in Norway, the extreme poverty rate has been stable between 0 and 0.5% for the past 20 years (i.e. below the target level set at 3%); therefore, it is likely that Norway will remain below the target level by 2030 unless significant changes occur.

• The trend is worsening, but the indicator is still likely to meet the target level by 2030. Figure A A.8, panel B, shows that, in the United States (Figure A A.8, panel B), although maternal mortality has been on an upward trend, it is still significantly below the target level. Hence, even though the maternal mortality ratio may keep going up, it is quite unlikely that the United States will not meet the target by 2030.

• The trend is improving at such a rate that the indicator is likely to meet the target level by 2030. Figure A A.8, panel C, shows the dramatic improvement of infant mortality in Colombia. While Colombia is not (yet) at target level, it is on a trajectory that would allow meeting the target by 2030.

Figure A A.8. Example of data series classified as “Target is achieved or likely to being achieved”

Note: The horizontal line stands for the 2030 target value to be reached. Dotted lines reflect the 10th and 90th percentiles of the projected data series; dashed lines reflect the 25th and 75th percentiles of the projected data series; the continuous lines reflects the 50th percentile of the projected data series.

Statlink

References

[8] Eurostat (2021), Sustainable development in the European Union — Monitoring report on progress towards the SDGS in an EU context — 2021 edition, Publications Office of the European Union, https://doi.org/10.2785/195273.

[11] Gennari, P. and M. D’Orazio (2020), “A statistical approach for assessing progress towards the SDG targets”, Statistical Journal of the IAOS, pp. 1-14, https://doi.org/10.3233/sji-200688.

[1] Haščič, I. and A. Mackie (2018), “Land Cover Change and Conversions: Methodology and Results for OECD and G20 Countries”, OECD Green Growth Papers, No. 2018/04, OECD Publishing, Paris, https://doi.org/10.1787/72a9e331-en.

[6] Hyndman, R. (2011), “Forecasting: An Overview”, in International Encyclopedia of Statistical Science, Springer Berlin Heidelberg, Berlin, Heidelberg, https://doi.org/10.1007/978-3-642-04898-2_256.

[5] OECD (2020), Environment at a Glance 2020, OECD Publishing, Paris, https://doi.org/10.1787/4ea7d35f-en.

[3] OECD (2019), Measuring Distance to the SDG Targets 2019: An Assessment of Where OECD Countries Stand, OECD Publishing, Paris, https://doi.org/10.1787/a8caf3fa-en.

[4] OECD (2011), Quality Framework for OECD Statistical Activities, https://www.oecd.org/sdd/qualityframeworkforoecdstatisticalactivities.htm (accessed on 13 December 2019).

[2] OECD (2001), Measuring Productivity - OECD Manual: Measurement of Aggregate and Industry-level Productivity Growth, OECD Publishing, Paris, https://doi.org/10.1787/9789264194519-en.

[7] Sachs, J. (2020), The Sustainable Development Goals and COVID-19. Sustainable Development Report 2020., Cambridge University Press, https://www.sdgindex.org/reports/sustainable-development-report-2020/.

[9] UNESCAP (2020), Asia and the Pacific SDG Progress Report 2020, https://www.unescap.org/publications/asia-and-pacific-sdg-progress-report-2020.

[10] UNSD (2020), Sustainable Development Goals Progress Chart 2020, https://unstats.un.org/sdgs/report/2020/progress-chart-2020.pdf.

Notes

← 1.  According to the Resolution adopted by the UN General Assembly on Work of the Statistical Commission pertaining to the 2030 Agenda for Sustainable Development, the indicator framework is to be refined annually and reviewed comprehensively by the UN Statistical Commission every five years (i.e. in 2020 and in 2025). For instance, in 2020, the IAEG-SDGs proposed 36 major changes to the framework in the form of replacements, revisions, additions and deletions as part of the 2020 Comprehensive Review; these recommendations were approved by the UN Statistical Commission in March 2020.

← 2.  The Inter-agency and Expert Group on SDG Indicators (IAEG-SDGs), composed of representatives of selected national statistical offices and including regional and international agencies as observers, was created in 2015 at the forty-sixth session of the UN Statistical Commission with the goal to develop and implement a global indicator framework for the goals and targets of the 2030 Agenda. Since then, the global indicator framework developed by the IAEG-SDGs had been endorsed by the UN Statistical Commission and adopted by the UN General Assembly.

← 3.  Custodian agencies are UN bodies and other international organisations responsible for compiling and verifying country data and metadata and for submitting the data, along with regional and global aggregates, to the UN Statistics Division (UNSD). These agencies are also responsible for developing international standards and recommending methodologies for monitoring. Another responsibility of the custodian agencies is to strengthen national monitoring and reporting capacity. When country data are missing or collected using a different methodology or inconsistently reported by different sources, custodian agencies may need to produce estimates or adjust the data for specific countries (with all final data that are submitted to the UNSD then being validated and approved by the respective country).

← 4.  However, some data series are repeated under two or three different targets. Therefore, the total number of data series in the SDG Global Database is 565.

← 5.  While the SDG Global Database compiles all SDGs following the global indicator framework, these indicators may be at different stages of development, with some indicators already well developed and regularly collected and others at early stages of conceptual development and data collection. These global indicators are classified into three tiers based on their methodological development and data availability (see https://unstats.un.org/sdgs/iaeg-sdgs/tier-classification/ for further details).

← 6.  In particular, some data series in the SDG Global Database only provide additional detail to the “main” indicator. For instance, indicator 5.5.1 on gender representation in parliaments includes the total number of seats in national parliament, the number of seats held by women as well as the proportion of seats held by women. Only the latter is included in the OECD framework underpinning this report.

← 7.  UN and OECD sources include 537 data series. This means that, on average, each indicator in the global indicator framework is supported by more than one data series. For instance, 44 different data series support the assessment of SDG indicator 4.5.1: “parity indices (female/male, rural/urban, bottom/top wealth quintile and others such as disability status, indigenous peoples and conflict-affected, as data become available) for all education indicators on this list that can be disaggregated”. All the series pertaining to this indicator included in this report compare the achievement of the OECD adult population in the fields of math or reading by socio-economic status, gender, place of living, migrant status and different levels of education (pre-primary school, primary school, lower secondary, upper secondary as well as training of youths and adults). In such cases, the distance is defined as the unweighted average over all available data series that support the indicator as listed in the global indicator framework.

← 8.  Yet it is important to stress that this approach assesses only a country’s long-term trajectory in a “business as usual” scenario. As such, no policy variable is considered in the assessment. Yet, their impact may be implicitly taken into account to the extent that they influenced the recent trend.

← 9.  While these approaches are suitable in the presence of relatively short time series, Gennari and D’Orazio (2020[11]) suggest that, even in these cases, it would be preferable to estimate the slope of the regression line fitted across all the available data points (the original values of each variable vs. time in the linear case; and log-transformed values vs. time, in the case of geometric growth).

← 10.  Data series are considered as “constant” when the relative standard variation (i.e. standard deviation divided by the mean) is below 1%.

← 11.  The sign of the coefficient correlation is corrected for the normative direction so that a positive correlation is always interpreted as progress towards the target, while a negative correlation is always interpreted as a decline.

← 12.  For obvious methodological reasons, when a target is set at 0 it is statistically impossible to reach it. In most cases, the target was set slightly above the null threshold. In the few remaining cases, the target is considered to be reached when the standardised distance to the target is lower than 0.10.