Policy Guide on Social Impact Measurement for the Social and Solidarity Economy

Introduction

What do we mean by social and solidarity economy?

The social and solidarity economy (SSE) is an important source of employment and economic development in many OECD countries and around the world. In the European Union, social economy organisations employ over 13.6 million people, which is 6.3% of the total workforce (CIRIEC, 2017[1]). In Colombia and Mexico, the SSE accounts for 4% and 3.2% of overall employment respectively, while social enterprises, one component of the SSE, employ nearly 5.8 million people in Japan and generate USD 104 billion in profits each year (OECD, forthcoming[2]). The SSE plays an important role in making economies and societies more sustainable and focussed on the needs of people and the planet.

The social and solidarity economy encompasses a diverse array of enterprises and organisations. Historically consisting of associations, cooperatives, mutual societies and non-profit organisations, the SSE has expanded to include foundations, social enterprises as well as more spontaneous community-based initiatives emerging at the grassroots level. Other notions such as the social economy, the solidarity economy and the third sector are used alongside the SSE, with the latter being the most encompassing term increasingly used by practitioners and academics, as well as at the international level. The resulting plurality of organisations, practices, business models and legal forms that have emerged across OECD member states and beyond can represent a challenge for policy makers in identifying and adequately supporting them (OECD, forthcoming[2]).

According to the OECD Recommendation on the Social and Solidarity Economy and Social Innovation, the SSE is “made up of a set of organisations such as associations, cooperatives, mutual organisations, foundations, and, more recently, social enterprises. In some cases, community-based, grassroots and spontaneous initiatives are part of the social economy in addition to non-profit organisations (…). The activity of these entities is typically driven by societal objectives, values of solidarity, the primacy of people over capital and, in most cases, by democratic and participative governance”.1

Social and solidarity economy entities2 are under increasing pressure to demonstrate their added value to society. This is also increasingly true for the private sector, including financial and corporate players. All are seeking to plan, implement, and assess their impact. Public policies that place greater emphasis on competition for contracts, user choice and value for money call further attention to social impact measurement. External donors and investors request evidence on how resources are used and what results are achieved (Clark et al., 2004[3]). Most importantly, social and solidarity economy organisations can proactively and voluntarily embrace social impact measurement for both learning and promotional purposes (OECD, 2021[4]).

What do we mean by social impact measurement?

The term “social impact measurement” has only recently emerged and stands at the juncture of other pre-existing approaches such as performance management, evaluation and accounting (Clifford, 2014[5]; Gibbon and Dey, 2011[6]; Alix, 2015[7]; Gentile, 2002[8]). As of today, there is no official international agreement on a common standard or definition, despite increasing attention by policy makers around the world.3 The term is generally perceived to put emphasis on impact quantification, although there is growing consensus among the scientific community that mixed-method approaches, combining both qualitative and quantitative data, are preferable.

Social impact measurement aims to assess the social value produced by the activities of any for-profit or non-profit organisation. It is commonly thought of as the process of understanding how much change in people’s well-being or the condition of the natural environment has occurred and can be attributed to an organisation's activities.4

Hence, social impact measurement practices encompass all the activities put in place by an organisation (or group thereof) to understand the (individual or collective) contribution to changes observed in society and the environment. Concretely, these approaches greatly differ, owing to the strategic and methodological choices that shape them (OECD, 2021[4]), and most notably:

• The scope of the analysis: micro (i.e., on a project, product or activity); meso (i.e., on the organisation itself); and macro (i.e., comprising groups of organisations or projects, on a given territory or thematic area).

• The timing of the assessment: estimating impact before implementation, concurrent to implementation as part of continuous performance management or retrospectively evaluating impact.

• The level in the causal chain: although the term “impact” indicates “the ultimate significance and transformative (potential, assumed, and/or achieved) effects of an intervention” (OECD, 2019[9]), it is often used to encompass changes observed across the whole results chain, including immediate outputs, intermediary outcomes and long-term impacts.

For social and solidarity economy organisations, social impact measurement focuses on understanding the additional, net value generated by their activities, in the pursuit of their mission and beyond, through a more or less tailored approach. An opinion by the European Economic and Social Committee further clarifies that, as far as social enterprises are concerned, “social impact measurement aims to measure the social outcomes and impact created by specified activities of a social enterprise and not the enterprises itself” (EESC, 2014[10]; OECD/EU, 2015[11]). SSE entities are driven by social and environmental objectives, which are not always easy to track, such as well-being, soft skills, community trust and sense of belonging. Other, more easily captured, indicators may include labor market outcomes, poverty and livelihoods, access to services, attainment of education, environmental degradation, carbon footprint, land use, etc.

The impacts identified through the measurement process, at the organisational or aggregate level, can also be referred to as impact evidence. Collecting such evidence is an important step for both policy makers and SSE entities towards understanding the effects generated by SSE activities (and public initiatives around them).

Impact evidence can be defined as the available body of facts or information that can be used to judge to what extent (or not) impact has occurred. Both quantitative and qualitative in nature, the evidence can be generated by individual SSE entities, groups thereof, other stakeholders in the SSE ecosystem (e.g., impact investors) and public authorities. It can then be triangulated to evaluate the impact of an organisation or initiative, and thereby inform decision-making (OECD, 2021[4]).5

To build “robust” evidence, the available information needs to show certain qualities. It needs to be transparent, complete and accurate (see frameworks such as the Social Value Principles, Standards of Evidence, the Nesta Toolkit on Evidence of Impact6). When it relates to the SSE, the following attributes are particularly important:

• Relevant: impact measurement efforts are proportionate to the needs and timing is synchronised with the decision-making process,

• Transparent: the assumptions used to produce impact estimates are made explicit and any negative findings are included,

• Representative: all relevant stakeholders have been consulted,

Impact measurement practices help corroborate impact by an organisation, which may initially appear as limited or anecdotal. As impact measurement practices become more frequent and proficient, the available evidence on the impact of a given organisation or initiative accumulates. The emergence of a credible and publicly available body of knowledge around the impact of the SSE, and hence a more widespread recognition of it, can thus be beneficial to both its member organisations and the policy makers supporting them.

Why does the social and solidarity economy need social impact measurement?

Social and solidarity economy entities are increasingly acknowledged for their significant contribution to addressing social and environmental challenges. Their social mission and operating models, based on values of solidarity, the primacy of people over capital, and democratic and participative governance, position them as pivotal actors in the pursuit of more inclusive economies and more equal societies. As such, there is growing interest in monitoring and evaluating the impact that can be attributed to them. In fact, a more widespread uptake of impact measurement practices among SSE entities could also facilitate the emergence of an impact measurement culture among their partners, including public and commercial entities.

Social impact measurement is an on-going process and an important strategic planning tool for social and solidarity economy organisations (EESC, 2014[10]; OECD/EU, 2015[11]). Impact information is needed to feed the different stages of the decision-making process, serving both for internal learning purposes (i.e., deriving insight and strategic orientation to improve decisions) and external accountability (proving credible results that can withstand the test of independent verification). Because SSE entities primarily pursue social goals and aim to maximise their social impact, they need impact information to (OECD, 2021[4]):

• Effectively allocate resources to social value creation: identify the interventions which are helpful for the given social mission, and those which are not, to prove and improve progress on the societal problem at hand.

• Improvise, experiment and innovate: creatively adapt standards (e.g. engaging difficult-to-reach target groups, changing public perception of a product/service, promoting inclusive governance practices), particularly relevant when engaging in “social bricolage”.7

• Increase stakeholder participation and collaboration: work with diverse stakeholders across sectors to foster novel solutions (e.g., addressing HIV infections with a combination of hygiene and education stakeholders).

• Persuade: convince potential supporters (e.g., volunteers, donors, financiers) and attempt to influence stakeholders through political and public advocacy (e.g. presenting to parliament).

Infographic 1. Uses of social impact measurement at different stages of the decision-making cycle and for specific objectives of the SSE

Source: (OECD, 2021[12])

Fostering a strong impact measurement culture in SSE ecosystems could significantly unlock new opportunities for growth. These opportunities could be on the fronts of diversified sources of financing, increased capacity to explore new strategies for scaling activities and impact or improved ability to access public and private markets among others. As they attempt to access new sources of financing and income, SSE entities face multiple and diverse reporting expectations from external donors, investors and partners. Impact investing is rapidly growing as a financing modality to allocate both public, private and blended resources on projects with positive material contributions to society and the environment. Providing a quantifiable track record of the impact generated by the SSE could help attract additional investors and open new markets. Not surprisingly, both social enterprises and non-profits tend to prioritise funder requirements when defining their impact measurement approach. At the same time, some SSE entities (like foundations, charities, cooperative banks and micro-finance institutions) can act as both standard setters and recipients of impact measurement requirements (OECD, 2021[4]).

Understanding the positive contributions of the SSE through a streamlined and regular approach can also inform public policy on where societal needs lie. By pursuing their social and environmental mission, SSE entities may serve as active facilitators in the attainment of sustainable development objectives by policy makers. Existence of an evidence base on progress towards particular social and environmental objectives can signal to public policy where there is heightened need for further public investment and where market and non-market solutions can accommodate such needs.

Why social impact measurement may be particularly challenging for the social and solidarity economy?

Social impact measurement for SSE may require higher efforts to solidify and pin down specific performance indicators, given the complexity of societal issues addressed by SSE entities. It focuses on measurement of both social and environmental objectives. Some “types” of impact may be easier to quantify or capture, as seen in indicators on environmental degradation, while others such as community well-being or social inclusion may be harder to pin down. Thus, in turn, may tilt the social impact measurement practice to favour the use of particular metrics. Typically, the social outcomes and participatory models pursued by SSE entities may raise both technical and conceptual difficulties when attempting to track them in a quantitative manner. Hence, they risk being overlooked or only partially reflected in less tailored social impact measurement approaches.

Social impact measurement methodologies can appear confusing and fragmented for social economy entities. A striking feature is simply the large number of options available. Social impact measurement practices may range from identifying potential impacts, counting outputs and beneficiaries, collecting stakeholder feedback, all the way to conducting impact evaluations, which may include attempts at measuring wellbeing and monetisation. As the impact measurement approach becomes more ambitious, it will require increasing levels of knowledge, resources and skills to master the more elaborate techniques.

There has been a strong push for standardisation at the international level over the last decade, from the public and private sector alike. Despite significant progress, there is not one universally agreed methodology. Prevailing solutions are often perceived as poorly adapted to social and solidarity economy organisations since, to a large degree, the conversation has so far been shaped by private institutional investors and commercial businesses. This runs the risk of disenfranchising those working on the frontlines, by forcing upon them ill-suited frameworks and measures. In response, there is an emerging consensus that a one-sized fits all approach would be inappropriate and SSE representatives are increasingly requesting more bottom-up and flexible approaches.

Possible responses to institutional pressures for social impact measurement range from complete resistance to external demands, to proactive and voluntary use for learning and promotional purposes. Still, the lack of capacity or motivation to engage with social impact measurement on the frontlines can be a hindrance to the common agenda of both implementing entities and their financers. In practice, many factors will influence how SSE entities decide which specific indicators, tools, methods or frameworks to adopt. External drivers include funder requirements, prevailing sector practices and the availability of methodological guidance. Internal determinants can be found in the governance and organisational culture, the purpose of assessing impact, and the nature and complexity of the theory of change being pursued. The lack of dedicated resources for social impact measurement, within each social and solidarity economy organisation and in the sector as a whole, is a pervasive problem.

Infographic 2. Factors influencing how social and solidarity economy entities define their social impact measurement practices and potential responses to institutional pressures

Source: (OECD, 2021[12]).

SSE entities also confront methodological hurdles. Rigorous data collection and analysis are needed to establish credible causal links between what organisations do and the impact that is created. Measurement can be particularly arduous for intangible and subjective impacts, such as social capital, well-being, soft skills, and other psycho-cultural aspects. In a context of limited resources, social impact measurement triggers a constant tension between satisfying internal learning needs as opposed to external accountability demands. To be at once useful for the SSE entities and credible to external recipients, the process needs to foster employee buy-in and motivation, while also including diverse stakeholders in a fair and accessible manner (OECD, 2021[4]).

Why policy makers may want to promote social impact measurement for the social and solidarity economy?

In the transition towards more inclusive, green and digital economies, policy makers need more sustainable modes of production and consumption, which the SSE can spearhead. The SSE has the potential to transform the economy and our societies, including by setting a virtuous example for the for-profit sector. However, this needs to be supported by proven data on the impact of their activities. By gathering better evidence on the results achieved by the SSE, policy makers can: 1) learn how to improve existing policies in support of the SSE, 2) report to citizens on progress achieved through the SSE, 3) mobilise other actors to collaborate with the SSE e.g. as financiers or operational partners.

The Recommendation of the OECD Council on the Social and Solidarity Economy and Social Innovation8 encourages all adhering countries to develop and adopt a social economy framework that is supported at all levels of government with a view to scale the impact of their social economy for the benefit of all. One of the building blocks is to encourage impact measurement and monitoring by:

• Promoting the development of indicators and criteria for social impact measurement in public policies and programmes.

• Encouraging social economy organisations to use part of the resources they receive from public authorities in the form of subsidies or contracts for social impact measurement.

• Supporting the design and dissemination of guidance on social impact measurement methods tailored to the social economy.

• Promoting the measurement of the non-market value, in addition to market value, of social economy organisations to better analyse their performance and assess their social impact, including on well-being.

• Supporting capacity to conduct social impact measurement by offering dedicated funding or training from specialised intermediaries and by mobilising other resources, including networks of expertise.

Public authorities need to understand the pros and cons of different approaches they may take. While public support is direly needed to advance social impact measurement, it might trigger unintended consequences by increasing reporting burden, discouraging innovative practices, disadvantaging emerging or small organisations, etc. Whilst the promotion of simple, harmonised indicators could facilitate more widespread adoption, it may fail to capture more complex phenomena. At the same time, raising the bar for social impact measurement, especially without the appropriate capacity building efforts, is likely to discriminate against smaller entities that do not have the resources to develop adequate protocols. Attention is also needed when considering the scope of the measures, since social impact is not immediate but emerges over a period of time. Metrics that focus on short-term, quantitative results are therefore likely to bias funding towards more immediate solutions, rather than in support of long-term social progress (OECD, 2021[4]).

References

[7] Alix, N. (2015), Puissance et Limites des Indicateurs ou Mesures d’Impact: Objectifs, enjeux, acteurs, Confrontations Europe & Groupe Caisse des Dépôts: Institut CDC pour la Recherche, Paris, https://confrontations.org/wp-content/uploads/2016/05/2015-10-02-Indicateurs-mesure-impact-social1.pdf (accessed on 23 June 2021).

[1] CIRIEC (2017), Recent Evolutions of the Social Economy in the European Union, https://doi.org/10.2864/699.

[3] Clark, C. et al. (2004), “Double Bottom Line Project Report:Assessing Social Impact In Double Bottom Line Ventures. Methods Catalog”, Center for Responsible Business Working Paper Series, No. Paper 13, University of California, Berkeley, CA, http://repositories.cdlib.org/crb/wps/13 (accessed on 23 June 2021).

[5] Clifford, J. (2014), Impact Evaluation by Social Enterprises: Measuring the un-measurable?, https://www.slideshare.net/OECDLEED/wd-jim-clifford (accessed on 25 June 2021).

[10] EESC (2014), Opinion of the European Economic and Social Committee on social impact measurement (own-initiative opinion), Official Journal of the European Union, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52013IE6135 (accessed on 12 November 2020).

[8] Gentile, M. (2002), “Social Impact Management: A Definition”, The Aspen Institute Business and Society Program Discussion Paper Series, No. Discussion Paper II, Aspen Institute, New York, NY, http://www.CasePlace.org (accessed on 23 June 2021).

[6] Gibbon, J. and C. Dey (2011), “Developments in Social Impact Measurement in the Third Sector: Scaling Up or Dumbing Down?”, Social and Environmental Accountability Journal, Vol. 31/1, pp. 63-72, https://doi.org/10.1080/0969160X.2011.556399.

[12] OECD (2021), Social Impact measurement for the Social and Solidarity Economy, https://doi.org/10.1787/d20a57ac-en (accessed on 16 November 2022).

[4] OECD (2021), “Social impact measurement for the Social and Solidarity Economy: OECD Global Action Promoting Social & Solidarity Economy Ecosystems”, OECD Local Economic and Employment Development (LEED) Papers, No. 2021/05, OECD Publishing, Paris, https://doi.org/10.1787/d20a57ac-en.

[9] OECD (2019), Evaluation Criteria: Adapted Definitions and Principles for Use, http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=DCD/DAC(2019)58/FINAL&docLanguage=En (accessed on 12 November 2020).

[2] OECD (forthcoming), Untangling the Complexity: A Conceptual Overview to the Social and Solidarity Economy.

[11] OECD/EU (2015), Policy Brief on social impact measurement for social enterprises: Policies for social entrepreneurship, https://www.oecd.org/social/PB-SIM-Web_FINAL.pdf (accessed on 12 November 2020).

Notes

← 1.  https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0472

← 2.  Hereafter, components of the social and solidarity economy are interchangeably described as organisations, actors or entities, without prejudice to their legal form or status.

← 3.  In fact, several synonyms are frequently used interchangeably, such as (social) impact assessment, social value measurement, social performance measurement or reporting. For a more detailed discussion on the different interpretations of this term, please refer to (OECD, 2021[4]).

← 4.  https://impactmanagementplatform.org/

← 5.  https://impactmanagementproject.com/wp-content/uploads/How-do-we-know-if-impact-has-occured.pdf.

← 6.  https://www.nesta.org.uk/toolkit/evidence-of-impact/

← 7.  “Social bricolage” can be defined as the purposeful and creative recombination of ideas and resources in the day-to-day pursuit of a social mission (OECD, 2021[4]).

← 8.  https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0472. As of 2022, 44 countries have committed to encourage impact measurement and monitoring by adhering to the OECD Recommendation on the Social and Solidarity Economy and Social Innovation.