States of Fragility 2018

Annex A. Methodological annex

The OECD fragility framework considers fragility to be multidimensional, measurable on a spectrum of intensity and expressed in different ways across five dimensions. It uses robust quantitative approaches to measure the magnitude of fragility, and it compares and contrasts different types of fragility descriptively. This mixed approach allows the analysis to extract the best value from the quantitative methods while also addressing the limitations of these methods through qualitative descriptions.

The methodology is based on a two-stage process that first examines contexts in each of five dimensions and then aggregates this information to obtain an overall picture of fragility. For each dimension, principal components analysis (PCA) is used to combine the risk and coping capacity indicators into two statistically derived components (Figure A A.1). Deriving two measures per dimension has distinct advantages over creating one composite index. First, using two measures allows for greater understanding of the differences among contexts that would score equally when a single measure is used. Second, using the first two principal components allows contexts to be broadly grouped based on their similarities in all of the input variables. Third, each indicator is weighted by the amount of new information it brings to the data, rather than on a set of normative judgements on their relative importance. With the components of each dimension calculated, contexts are then grouped on similarities and classified descriptively. Thus this mix of both quantitative and qualitative methods offers a more flexible approach to describing the diversity of fragility.

Figure A A.1. The 2018 OECD fragility framework methodology

Once contexts are classified into groups within each dimension, the second part of the methodology aggregates this information to arrive at an overall picture of fragile contexts. To do so, the components of each dimension provide inputs to a second aggregate PCA that is then used to produce the 58 fragile contexts in the fragility framework.

The methodology is ambitious in its objectives but has limitations. By using PCA, the range of indicators can be reduced to two core components, thereby explaining most of the variance in the original data. However, in doing so information invariably is lost. The second stage of PCA (PCA Stage 2) exacerbates this loss of information. In short, the results of this approach are a summary of the initial indicators that is then interpreted in terms of fragility. Despite these limitations, this summary is both more informative and less arbitrary than any composite index based on the initial indicators.

Aside from the technical limitations, there are also certain practical limitations to what can be captured in any quantitative approach. The unit of analysis for the OECD fragility framework is country level. As a consequence, the framework is unable to capture macro-level drivers of fragility – drivers that spill over borders – or micro-level drivers that lead to localised pockets of fragility within states. Going forward, it would be useful to find ways to draw on subnational data and to link up regional and global data. Further, while data on governance are widely available, data on informal institutions are less so. While every effort has been made to include indicators of both of these, at this point the lack of quality data is a limiting factor for the model. Finally, the calculations exclude 27 countries and territories where there was insufficient data to feed into the analysis (Box A.1).

Box A A.1. Countries and territories not included in the fragility framework

Data availability is a key issue in calculating the OECD fragility framework. As the unit of analysis is the state or territory, it is important to select indicators that are comparable across those states and territories. While statistical imputation methods can be used to fill data gaps, such an approach is best used sparingly; preference should always be given to real-world data, even if it means dropping indicators or countries and territories that otherwise would have been included. The fragility framework methodology aims to strike a balance between the number of indicators, the contexts covered and the amount of imputation that would be required to build a complete data set. A criterion for inclusion in the OECD framework was at least 70% of the required data had to be available for a country or context. As a result, only 172 contexts could be included in the calculations.

This does not mean that the excluded contexts are not fragile. Indeed, many of those excluded are small island developing states that face unique challenges. The final list also excludes two territories with UN peacekeeping missions (Kosovo and Western Sahara) and several Pacific Island countries whose high levels of interpersonal violence are well known.

Table A A.1. Countries and territories excluded in 2018 due to insufficient data

Anguilla	Kiribati	Niue	Tonga
Antigua and Barbuda	“Kosovo”	Palau	Turks and Caicos Islands
Belize	Malta	Saint Helena, Ascension and Tristan da Cunha	Tuvalu
Cook Islands	Marshall Islands	Saint Kitts and Nevis
Dominica	Mayotte	Wallis and Futuna
Federated States of Micronesia	Saint Lucia	Western Sahara
Grenada	Nauru	Tokelau
Montserrat	Saint Vincent and the Grenadines	Samoa

Indicator coverage and missing data

The choice of indicators has been driven by selection criteria in line with the OECD’s fragility concept of high risk and low coping capacity. Normal technical criteria for selecting good indicators were used, but with a particular and added emphasis on selecting indicators based on their relationship to fragility – that is, are they a cause of fragility or an outcome of fragility? Indicators that represent outcomes of fragile contexts do not offer clear guidance as to policies that can reduce fragility. For example, infant mortality is an indicator used in several fragility measures. However, infant mortality is arguably more of an outcome of a fragile health context than a cause of it. In selecting indicators, then, the following factors were considered, in keeping with OECD concept of fragility:

• Risk. Do the indicators alter either likelihood or impact ex-ante?

• Coping capacity. What indicators would stop the risk cascading ex-post if the risk occurred

Using these criteria does not eliminate the challenge of separating some indicators into the category either of a risk or a coping capacity. For example, the level of armed personnel can be considered a coping capacity for dealing with insurgencies. It can also be considered a contributor to the risk of violence. Methodological decisions have been made to account for this challenge to produce the best approximation, given the limitations.

Further, some coping capacity indicators have been used in more than one dimension, introducing an unintended issue in aggregating the dimensions to provide the final 58 contexts used in analysing flows by fragility. Using indicators more than once essentially weights these indicators more than others. Statistical measures have been used to minimise this effect, and this method ultimately was chosen as the preferred alternative to dividing some of these indicators into one, and only one, dimension. For example, government effectiveness and rule of law are important not only for the security dimension but also for the environmental dimension. Forcing them into a single dimension arbitrarily creates an incomplete picture of the interconnectedness of coping capacities. Therefore, the de facto weighting effect was considered justifiable, if not ideal, given cross-dimensional importance. Table A A.2 lists the indicators used in the 2018 fragility framework.

Table A A.2. Indicators of Fragility

Dimension	Type	Indicator name	Source	Description
Societal	Risk	Gini coefficient	V-DEM	Distribution of income expressed as a Gini coefficient
	Risk	Gender inequality	UNDP/HDI	Measures gender inequalities in reproductive health (maternal mortality ratio and adolescent birth rates); empowerment (proportion of parliamentary seats occupied by females and proportion of adult females and males aged 25 years and older with at least some secondary education); and economic status (expressed as labour market participation and measured by labour force participation rate of female and male populations aged 15 years and older)
	Risk	Horizontal inequality	V-DEM	Do all social groups (distinguished by language, ethnicity, religion, race, region or caste) enjoy the same level of civil liberties or are some groups generally in a more favourable position?
	Risk	Uprooted people	INFORM	Combination of the number of refugees, returned refugees and internally displaced persons
	Risk	Urbanisation growth	WB	Annual urban population growth
	Coping	Core civil society index	V-DEM	Provides a measure of how robust a nation’s civil society is
	Coping	Access to justice	V-DEM	Extent to which citizens enjoy secure and effective access to justice
	Coping	Voice and accountability	WGI	Reflects perceptions of the extent to which a country’s citizens are able to participate in selecting their government, as well as freedom of expression, freedom of association and free media
Political	Risk	Regime persistence	Polity IV	Number of years polity has persisted (as measured by no change in any of the Polity IV measures)
	Risk	Political terror	PTS	Levels of state-sanctioned or state-perpetrated violence (e.g. political violence such as assassinations of political challengers and police brutality)
	Risk	Perception of corruption	TI	TI Transparency International (TI) Corruption Perceptions Index (CPI) ranks countries annually by their perceived levels of corruption, as determined by expert assessments and opinion surveys”. The CPI generally defines corruption as “the misuse of public power for private benefit”.
	Coping	Decentralised elections	V-DEM	Are there elected regional governments and, if so, to what extent can they operate without interference from unelected bodies at the regional level?
	Coping	Voice and accountability	WGI	Reflects perceptions of the extent to which a country’s citizens are able to participate in selecting their government, as well as freedom of expression, freedom of association and free media
	Coping	Restricted gender physical integrity value	OECD	Measures prevalence of laws on rape and domestic violence and captures experience of violence
	Coping	Judicial constraints on executive power	V-DEM	To what extent does the executive respect the constitution and comply with court rulings, and to what extent is the judiciary able to act in an independent fashion?
	Coping	Legislative constraints on executive power	V-DEM	To what extent are the legislature and government agencies (e.g. comptroller general, general prosecutor or ombudsman) capable of questioning, investigating and exercising oversight over the executive?
Environmental	Risk	Natural hazard exposure	INFORM	Measures the exposure to disasters such as earthquake, tsunami, flood, tropical cyclone and drought
	Risk	Environmental health	Yale	Measure of health impacts, quality of air, water and sanitation
	Risk	Uprooted people	INFORM	Combination of the number of refugees, returned refugees and internally displaced persons
	Risk	Prevalence of infectious disease	GBD and CSIS	Infectious disease deaths per 100 000 population
	Risk	Socio-economic vulnerability	INFORM	Measures the (in)ability of individuals and households to afford safe and resilient livelihood conditions and well-being; combines indicators of development and deprivation, inequality and aid dependency
	Coping	Rule of law	WGI	Reflects perceptions of the extent to which agents have confidence in and abide by the rules of society, in particular the quality of contract enforcement, property rights, police and courts, and the likelihood of crime and violence
	Coping	Government effectiveness	WGI	Captures perceptions of the quality of public services; the quality of the civil service and the degree of its independence from political pressures; the quality of policy formulation and implementation; and the credibility of the government’s commitment to such policies
	Coping	Core civil society index	V-DEM	How robust is civil society?
	Coping	Food security	INFORM	Prevalence of undernourishment, average dietary supply adequacy, domestic food price index and domestic food price volatility
Economic	Risk	Resource rent dependence	WB	The sum of oil rents, natural gas rents, coal rents (hard and soft), mineral rents and forest rents
	Risk	General government gross debt	IMF	General government debt as a percentage of GDP
	Risk	NEET	ILO	Proportion of youth not in employment, education or training (NEET)
	Risk	Aid dependency	INFORM	Combination of net ODA as a percentage of GNI, total ODA per capita in the last two years and total humanitarian aid per capita in last two years
	Risk	Unemployment rate	WB	Unemployment rate
	Risk	Socio-economic vulnerability	INFORM	Combination of indicators related to development and deprivation, inequality, and aid dependency
	Risk	GDP growth rate	WB	Compound annual growth rate of GDP over the last five years
	Coping	Women in labour force	UNDP/HDI	Percentage of female participation in the labour force
	Coping	Males in labour force	UNDP/HDI	Percentage of male participation in the labour force
	Coping	Education	HDI	Measured by the mean of years of schooling for adults age 25 years and over and expected years of schooling for children of school age entering school
	Coping	Regulatory quality	WGI	Reflects perceptions of the ability of the government to formulate and implement sound policies and regulations that permit and promote private sector development
	Coping	Remoteness	EVI	Trade-weighted average distance from world markets
	Coping	Food security	INFORM	Prevalence of undernourishment, average dietary supply adequacy, domestic food price index and domestic food price volatility
Security	Risk	Violent conflict risk	INFORM	Statistical risk of violent conflict in the next 1-4 years based on 25 quantitative indicators from open sources.
	Risk	Homicide rate	UNODC	Intentional homicide rate per 100 000 population
	Risk	Level of violent criminal activity	IPD	Intensity of violent activities by underground political organisations: by criminal organisations (e.g. drug trafficking, arms trafficking, prostitution, etc.).
	Risk	Deaths by non-state actors per capita	UCDP-NS	Total of one-sided and non-state actor datasets, average per capita rate, 2013-16.
	Risk	Impact of terrorism	IEP/START	Global Terrorism Index score for a context in a given year accounts for the relative impact of incidents in the year. Four factors are counted: number of terrorist incidents; number of fatalities caused by terrorism; number of injuries caused by terrorism; and approximate level of total property damage from terrorist incidents in a given year. It is a five-year weighted average to capture lingering fear effects.
	Risk	Battle-related deaths per capita (log)	UCDP-BD	Total of battle-related deaths per capita, transformed using the logarithm function
	Coping	Police officers per 100 000	GPI	Police officers per 100 000 population
	Coping	Armed security officers per 100 000	GPI	Armed security officers per 100 000 population
	Coping	Rule of law	WGI	Reflects perceptions of the extent to which agents have confidence in and abide by the rules of society, in particular the quality of contract enforcement, property rights, police and courts, and the likelihood of crime and violence
	Coping	Control over territory	V-DEM	Over what percentage of the territory does the state have effective control?
	Coping	Government effectiveness	WGI	Captures perceptions of the quality of public services, the quality of the civil service and the degree of its independence from political pressures; the quality of policy formulation and implementation; and the credibility of the government's commitment to such policies.
	Coping	Restricted gender physical integrity value	OECD	Measures prevalence of laws on rape and domestic violence and captures experience of violence
	Coping	Formal alliances	COW	Formal alliance between at least two states that falls into the class of defence pact, neutrality or non-aggression treaty, or entente agreement

Age of data

The 43 indicators selected do not cover all contexts and imputation techniques have been used to fill in data gaps. Lack of data is the primary reason why a context may not be included. At least 70% of data for a context had to be available for it to be included in the OECD fragility framework. In 2018, this yields a list of 172 contexts. It is possible to assume that contexts missing from the dataset have a certain value for some indicators. For example, those missing from the datasets for battle deaths and deaths by non-state actors can be assumed to have a value of 0. Where no reasonable assumption could be made, data are imputed using k-nearest neighbour (KNN) imputation that uses statistical inference to fill in missing data from the k most similar contexts. In the OECD fragility framework, this has been done using the 15 most similar contexts for each missing data point (Table A A.3).

Table A A.3. Indicator coverage

Indicator	Minimum year of data used	Maximum year of data used	Number of countries with available data	Imputation technique used
Access to justice	2016	2016	169	KNN
Aid dependency	2016	2016	172	KNN
Armed security officers per 100 000	2017	2017	161	KNN
Battle-related deaths per capita (log)	2016	2016	172	Missing countries assigned 0
Control over territory	2012	2015	164	KNN
Core civil society index	2016	2016	169	KNN
Deaths by non-state actors per capita	2016	2016	172	Missing countries assigned 0
Decentralised elections	2016	2016	169	KNN
Education	2015	2015	170	KNN
Environmental health	2016	2016	169	KNN
Food security	2016	2016	172	KNN
Formal alliances	2012	2017	172	KNN
GDP growth rate	2015	2015	165	KNN
Gender inequality	1995	2015	154	KNN
General government gross debt	2008	2016	164	KNN
Gini coefficient	2012	2015	148	KNN
Government effectiveness	2016	2016	172	KNN
Homicide rate	2008	2015	172	KNN
Horizontal inequality	2016	2016	169	KNN
Impact of terrorism	2016	2016	172	KNN
Judicial constraints on executive power	2016	2016	169	KNN
Legislative constraints on executive power	2016	2016	169	KNN
Level of violent criminal activity	2016	2016	140	KNN
Males in labour force	2015	2015	171	KNN
Natural hazard exposure	2016	2016	172	KNN
NEET	1995	2016	140	Combined OECD data with World Bank and ILO datasets
Perception of corruption	2016	2016	166	KNN
Police officers per 100 000	2017	2017	161	KNN
Political terror	2015	2015	172	KNN
Prevalence of infectious disease (deaths per 100 000)	2016	2016	172	KNN
Regime persistence	2016	2016	164	KNN
Regulatory quality	2016	2016	172	KNN
Remoteness	2005	2015	155	OECD countries assigned 0, KNN imputation for the remainder
Resource rent dependence	2007	2015	171	KNN
Restricted gender physical integrity value	2014	2014	144	OECD countries given OECD average, KNN imputation for the remainder
Rule of law	2016	2016	172	KNN
Socio-economic vulnerability	2016	2016	172	KNN
Unemployment rate	2016	2016	170	KNN
Uprooted people	2016	2016	172	KNN
Urbanisation growth	2011	2016	172	KNN
Violent conflict risk	2016	2016	172	KNN
Voice and accountability	2016	2016	172	KNN
Women in labour force	2015	2015	171	KNN

Creating a time series

States of Fragility 2018 extends previous OECD work by creating a time series for deeper analysis of improvements and deteriorations in previously identified fragile contexts. To do this, all data have been imputed and extended to a period spanning 2016 and 2017. Using only 2016 or 2017 data, PCA models were generated for all five dimensions and aggregate scores. These models were applied across different years to create a time series comparable from year to year.

Cluster analysis

Fragile contexts can be grouped based on their similar characteristics. In order to identify frequently repeating patterns across these contexts and to group them based on their performances, a clustering algorithm was carried out using a hierarchical clustering procedure. Hierarchical clustering has emerged as a useful baselining tool in political science studies (Wolfson, Madjd-Sadjadi and James, 2004[1]). The clustering procedure provides two results. First, each context is grouped with other contexts that have the highest possible similarity with one another. Second, it defines the profile of an average context for each group. The profiles highlight the relevant attributes and distinct profile of each group, making it possible to quantitatively differentiate among groups. Six groups or clusters have been identified and named. The main attributes of each cluster are defined by their unique quantitative behaviour.

The OECD uses the clustering procedure as an indicative aid to assist in the qualitative assessment of different types of fragility. The severity of factors and/or combination of factors have been assessed by experts. Within each dimension, clusters have been ranked on a six level severity scale:

1 = Severe fragility

2 = High fragility

3 = Moderate fragility

4 = Low fragility

5 = Minor fragility

6 = Non-fragile

Once grouped, each cluster is compared to every other cluster to determine what characteristics best define its specific fragility. To do this, a Tukey ANOVA test is used (Hinton, 2014[2]). This method takes the means of all indicators within each cluster and conducts a difference in means test, which compares the mean of indicators in every other cluster. A statistical significance criterion has been developed to identify indicators whose levels are unique to any cluster when compared to the rest of the world. This criterion identifies indicators where the mean for any one cluster is significantly different at 95% confidence levels from at least four of the remaining clusters. Broadly speaking, this criterion can be interpreted as highlighting indicators in each cluster that are statistically different to at least 80% of the rest of the world.1

The next sections will show and describe the results of the cluster analyses for each of the five dimensions of the OECD’s 2018 fragility framework.

Economic dimension

The economic dimension aims to capture the vulnerability to risks stemming from the weaknesses in economic foundations and human capital including macroeconomic shocks, unequal growth, high youth unemployment, etc.

Figure A A.2 shows the economic dimension biplot with each cluster in a different colour. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by higher participation of men and women in the workforce. However, this cluster is weaker in regulatory quality, food security, education, socio-economic vulnerability and dependence on resource rent. The olive-coloured cluster shares the same weaknesses, with additional weaknesses in NEET and aid dependence.

Figure A A.2. Economic dimension typology

Environmental dimension

The environmental dimension aims to capture the vulnerability to environmental climactic and health risks to citizens’ lives and livelihoods. This includes exposure to natural disasters, pollution and disease epidemics.

Figure A A.3 shows the biplot with each cluster in a different colour. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by low environmental health and high prevalence of disease and socio-economic vulnerability. The violet cluster is distinguished by high exposure to natural disaster. The grey cluster is strong in rule of law and government effectiveness and has low socio-economic vulnerability and lower levels of environmental health.

Figure A A.3. Environmental dimension typology

Security dimension

The security dimension aims to capture the vulnerability of citizen security emanating from social and political violence. As such it includes indicators of citizen exposure to direct political and social violence.

Figure A A.4 shows the security dimension biplot with each cluster in a different colour. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by higher deaths by non-state actors, higher battle-related deaths and low control of territory. However, this cluster has stronger coping capacities than other clusters. The olive cluster is distinguished by higher levels of interpersonal and political violence. It is weak in coping capacities relating to rule of law and government effectiveness. The grey cluster is stronger in government effectiveness, rule of law and formal alliances, which lead to lower risk of conflict and violent crime.

Figure A A.4. Security dimension typology

Political dimension

The political dimension aims to capture the vulnerability to risks inherent in political processes, events or decisions, to its political inclusiveness (incl. elites) and transparency (corruption) and to its ability to accommodate change and avoid oppression.

Figure A A.5 shows the biplot coloured by cluster. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by having higher political terror and perception of corruption. It is also weak in coping capacities relating to voice and accountability, gender physical integrity, and constraints on executive power. Conversely, the olive cluster is weak in all but one of the coping capacities but does not have the same presence of risk factors as can be seen in the aqua cluster. The brown and grey clusters have strong coping capacities coupled with low levels of risk factors.

Figure A A.5. Political dimension typology

Societal dimension

The societal dimension aims to capture the vulnerability to risks affecting societal cohesion that stem from both vertical and horizontal inequalities (inequality among culturally defined [or constructed] groups), social cleavages, etc.

Figure A A.6 shows the biplot coloured by cluster. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by having lower coping capacities in voice and accountability and access to justice. It is also faced with high risks through urbanisation growth, uprooted people and both gender and income inequality. The olive cluster is distinguished by low levels of all coping capacities and high horizontal inequality. The grey cluster is strong in coping capacities and faces lower levels of risk.

Figure A A.6. Societal dimension typology

Aggregate fragility

The second part of the OECD methodology aggregates all of the information to arrive at an overall picture of combinations of fragilities. This second tier aggregate analysis generated the group of the 58 most fragile contexts, which are classified as extremely fragile and fragile.

The second tier PCA generated six fragility clusters that, are differentiated not only by their extent of fragility but also in the dominant characteristics of that fragility. This is shown in Figure A A.7. The first dimension of the PCA represents coping capacities. The second dimension represents the types of fragility. To arrive at the 58 most fragile contexts, two cut-offs have been selected. In order for a context to be classified as extremely fragile, it has to score less than -2.5 on the first principal component of the aggregate PCA shown in Figure A A.7. In order to be classified as fragile, a context must score between -1.2 and -2.5 on the first principal component.

The biplot of the aggregate can be split by contexts above and below the x-axis. Those above the x-axis are dominated by economic factors and the contexts below the x-axis are dominated by fragilities in the political, societal and/or security dimensions. Fragility in the environmental dimension can be found in contexts above and below the x-axis.

Figure A A.7. Biplot for aggregate fragility

References

[2] Hinton, P. (2014), Statistics explained, Routledge, https://www.routledge.com/Statistics-Explained-3rd-Edition/Hinton/p/book/9781848723122 (accessed on 26 June 2018).

[1] Wolfson, M., Z. Madjd-Sadjadi and P. James (2004), “Identifying National Types: A Cluster Analysis of Politics, Economics, and Conflict”, Journal of Peace Research, Vol. 41/5, pp. 607-623, http://dx.doi.org/10.1177/0022343304045975.

Note

← 1. For further information, also see (Lamb, 2017[145]), (Lamb, 2013[147]) and (Lamb and Mixon, 2014[146]).

This report includes 172 countries grouped into 6 clusters for each dimension. By conducting the Tukey ANOVA test at 95% confidence for all our indicators, we are comparing the indicators’ means of each cluster to the indicators’ means of the other clusters. If any mean is statistically different from the means of at least four of the remaining five clusters, it is considered a defining characteristic. Significance in this case can broadly be interpreted as indicators’ means for each cluster being statistically different to approximately four-fifths (80%) of the rest of the world.