Generating and gathering impact evidence can be a dauntingly complex and costly endeavour. Yet, publicly available data on the impact of the social and solidarity economy (SSE) can support the achievement of its mission, open new opportunities for growth and ultimately benefit society as a whole. This chapters discusses what policy makers can do through specific actions, from mandating impact studies, to publicly disseminating impact information and setting up easily accessible repositories of knowledge.
Policy Guide on Social Impact Measurement for the Social and Solidarity Economy
Local Economic and Employment Development (LEED)
3. Build evidence
Abstract
Why is this important?
The generation of impact evidence is a complex and costly endeavour. In particular, if there is a lack of internal motivation or when facing a scarcity of resources, social and solidarity economy (SSE) entities may tend to forego or limit the ambition of their social impact measurement activities. By commissioning, disclosing and disseminating impact studies or creating public repositories of knowledge in national languages, policy makers can create a benchmark or even a baseline that may inspire similar efforts by SSE representatives themselves.
Publicly available impact evidence at the local, regional or national level can help confirm the impact achieved by SSE entities and present them as a virtuous example. Better understanding the SSE’s contribution to inclusive and sustainable growth, as well as to the green and digital transitions, can help mobilise support and facilitate access to finance from both public and commercial partners. Granular information on costs and benefits can also help shape policies to target the most effective and efficient interventions.
Public efforts to generate, compile and communicate impact data can incentivise SSE entities and others to add to a joint evidence base, for the benefit of society as a whole. The inclusive, bottom-up establishment of a powerful and unified voice on the importance of impact evidence is a critical factor in driving transformative change in the SSE and beyond (Buckland and Hehenberger, 2021[1]). Complementary information on the uptake and use of impact evidence and follow up on the decisions that have been taken as a result can also serve as a motivating example for SSE entities to pursue social impact measurement.
How can policy makers help?
Policy makers can produce and disseminate impact evidence on the SSE and/or specific policy areas that SSE actors are active in. This evidence can help understand the impact of SSE entities more generally or analyse their activities in specific sectors and territories. It can also serve as an inspiration to foster SSE development, by showcasing good practices emerging from the SSE and testing methodologies for impact evaluation. When such evidence becomes a common good, it greatly enhances the capacity of all actors in the ecosystems to engage in social impact measurement. Greater public awareness can further motivate uptake by facilitating access to finance and by avoiding impact-washing (OECD, 2021[2]).
By making impact evidence freely available, policy makers can lower the cost of generating or accessing information for single organisations. When the data infrastructure becomes a common good, it greatly enhances the capacity of all actors in the ecosystem to engage in social impact measurement. Greater public awareness can further motivate uptake of social impact measurement, by facilitating access to finance and preventing impact washing. This can be reinforced through the offer of support services, capacity building, guidance or other resources.
Policy makers can support SSE entities through impact evidence in three ways: (i) by commissioning or directly producing impact studies on the SSE or policy areas that SSE actors are active in, such as health, social services, education, etc.; (ii) by publicly disseminating impact evidence that has been produced by SSE entities or other sources; (iii) by creating repositories of knowledge that regroup a wide variety of evidence and other resources in an easily accessible manner.
Infographic 3.1. Success factors and pitfalls to avoid: Build evidence
Possible actions
Produce impact evidence
Statistical accounts and monitors represent an infrastructure of information that serves as a basis for impact measurement and allows to determine baseline values against which progress can be assessed. In this context, smaller scale surveys are most common, whether they apply to the local SSE ecosystem, e.g. in the Canton of Geneva in Switzerland (Après-GE, 2015[3]), or for specific types of SSE entities; e.g. social enterprises, as it happens in Australia, Canada, Europe, or a number of other countries (Centre for Social Impact/Social Traders, 2016[4]; Elson, Hall and Wamucii, 2016[5]; Dupain et al., 2021[6]; British Council/Social Enterprise UK, 2022[7]). They can be conducted at different territorial levels, with annual frequency or less regularly. Indeed, according to some SSE representatives, voluntary surveys may be preferable to mandatory impact measurement (CECOP, 2020[8]). In a more permanent fashion, statistical accounts have been introduced in some regions and countries such as Wallonia and Brussels in Belgium (Observatoire Economie Sociale, 2020[9]), France (INSEE/Flores, 2021[10]), Quebec/Canada, Italy (ISTAT/EURICSE, 2021[11]), Portugal (CASES/Statistics Portugal, 2019[12]), Spain (Ministerio de Empleo y Seguridad Social, 2019[13]). In the United States, the Los Angeles Economic and Workforce Development Department performed an impact evaluation of the Regional Initiative for Social Enterprise (LA:RISE) in 2019. The German government had also funded a study on the impact of the welfare association sector (Kehl et al., 2016[14]). These accounts often lay the groundwork for the development of more advanced impact studies, offering a backbone of information on SSE activities and more immediate outputs.
Policy makers can commission more advanced impact studies to better understand the social impacts SSE entities are making, in addition to their economic and environmental benefits. Public authorities are often the only ones capable of initiating large scale, costly and time-consuming research projects that can deliver credible scientific evidence around long-term impacts. In Portugal, for example, the government published condensed information on the effects of publicly financed interventions on “priority social problems” (i.e. social protection, education, healthcare, employment and justice) as the One Value portal.1 Where appropriate definitions, methods and granular data are available, these studies can go as far as deploying counterfactual analysis to determine the net impact of the SSE. This has been done in Spain (see Making it happen ), where the social value added was calculated in terms of how many more jobs were created by the SSE compared to commercial businesses.
Making it happen 3.1. Measuring the Social Economy’s contribution to social and territorial cohesion (Spain)
Why?
After adopting a law on the social economy in 2011, the Spanish government saw the need to gather data beyond what was publicly available. The aim of the study was to identify and quantify the distinct contributions of the social economy to better social and territorial cohesion.
What?
Published in 2019, the Analysis of the Socio-economic Impact of the Values and Principles of the Social Economy in Spain (Análisis del Impacto Socioeconómico de los Valores y Principios de la Economía Social en España) was a joint effort of the Ministry of Labour, Migration and Social Security, the Spanish Business Confederation of the Social Economy (CEPES), two prominent cooperatives, Grupo Cooperativo CAJAMAR and Corporación Mondragón, and Fundació Espriu. Using a counterfactual approach and analysing microdata, the study was able to determine the specific value added of social economy enterprises that would be lost if they behaved like profit-oriented firms. For 2017, the study found that the social economy contributed:
Employment opportunities: more than 172 000 additional jobs compared to profit-oriented firms for people with barriers to employment, of which more than 54 000 are located in rural areas, small or medium sized cities. Moreover, it creates stable employment relationships for almost 125 000 extra individuals.
Monetary benefits to society: an additional EUR 6.2 billion of annual net benefits to society, thereof EUR 3.9 billion in salary income to groups that otherwise would not have been employed and EUR 1.7 billion of direct and indirect benefits (cost savings) for the public administration. Further benefits accrue to the companies themselves, including labour cost subsidies associated to these types of workers and lower cost in terms of selection and adaptation to the job position due to lower turnover of workers.
The report offers a detailed outline of the methodology, which could be replicated in other European countries, where information about firms and workers is available in the same granularity and allows for the identification of SSE entities. The study was implemented using microdata, and a large sample of entities analysed. This allowed to develop precise control groups, apply counterfactual analysis and ensure representativeness as well as the robustness of the analysis as explained above.
Impact
As of 2020, the study has been picked up in Spanish research on equality and equal access to labour markets and the social economy. CEPES was able to use the data and findings from this study to mobilise national and international thought leaders. The report was presented to the Spanish government, the Parliament, the Senate and to national political parties. The data and the conclusions of the study underpin CEPES’ proposals and positions to the government and also to the European Institutions on the different legislative initiatives that affect social economy enterprises. The study has also attracted international attention: it has been presented to high representatives of European institutions and bodies (Commission, Parliament and European Economic and Social Committee) and at the 2019 International Conference of the United Nations’ Task Force on the Social and Solidarity Economy (UNTFSSE).
Source: (CEPES, 2020[15]).
Publicly disseminate impact information
Policy makers can increase awareness around the possibilities of social impact measurement and the impact of SSE entities by disseminating impact information publicly. When impact studies are published on a government website, discussed in Parliament, or shared at a conference hosted by public authorities, they immediately gain visibility and recognition. By shedding light on the impact of the SSE, policy makers can mobilise support for the SSE and, at the same time, motivate SSE entities to engage more actively in social impact measurement. For instance, the 2022 Social Enterprise World Forum, sponsored by the Queensland Government (Australia), featured several workshops and masterclasses around the topic of impact measurement (SEWF, 2022[16]). In India, the public agency NITI Aayog has published two editions of the SDG India Index, which documents the progress made by states and territories towards achieving the 2030 targets2. Villgro, India’s oldest and one of the world’s largest social enterprise incubators, uses the baseline report to monitor the impact of each investee. In the United States, the 2021 presidential Executive Order on Advancing Racial Equity and the Support for Underserved Communities foresaw the establishment of an Equitable Data Working Group tasked to produce better disaggregated data at the national and local level (OECD, 2021[2]). Through their online presence, policy makers can ensure the availability of, and facilitate access to, impact-related information that can be instrumental to further social impact measurement by SSE entities. The Justice Data Lab in the United Kingdom provides impact assessment support to organisations working on reintegration of previously incarcerated populations and the results are then publicly displayed on the government website (see Making it happen). Governments can create online dashboards showcasing the social outcomes of public interventions (and their monetary costs) in an easily accessible manner, as it happened in the United Kingdom3 and Portugal4. Similarly, the Korean Ministry for SMEs and Start-ups runs a database for impact measurement, currently covering assessments for 80% of Korea’s social enterprise scene (see Making it happen). This can support the design of SSE-led activities where funding is contingent on measurable outcomes, such as social impact bonds.
Making it happen 3.2. Justice Data Lab (United Kingdom)
Why?
It is essential for the Ministry of Justice to gather evidence about what works to reduce reoffending. However, it is unlikely that organisations can track the reoffending outcomes for each person they have worked with. Some may have access to reoffending data for their cohort, but this alone does not demonstrate the impact of the intervention. A suitable benchmark, such as a comparison group, is required to estimate the difference in reoffending behaviour.
What?
The Justice Data Lab (JDL) is a team of analysts at the Ministry of Justice that provides group-level reoffending information to organisations who have worked with offenders and would like to understand the impact of their intervention. They compare the reoffending outcomes of people who have received the intervention with those of a similar comparison group that did not take part in the intervention. Any organisation that has worked with offenders can contact the JDL. Previous collaborators include charities, public sector organisations, private sector organisations and educational institutions. The service is free for the submitting organisations.
Reports of the analysis conducted by the JDL are published on the government’s website. They include amongst others reoffending statistics for treatment and comparison groups, estimated impact of the intervention, illustrated graphs and guidance to interpret results as well as descriptions of the intervention, treatment group characteristics and success rates. The publication of these reports also helps develop a collaborative understanding of effective rehabilitation.
Impact
The Justice Data Lab has produced 283 analyses within 178 reports. Of these, 74 analyses showed a statistically significant reduction in re-offending on the proven re-offending rate and 15 analyses showed a statistically significant increase, whilst the remaining 194 analyses were non-significant. A wide-ranging variety of interventions are evaluated, covering employment, education, mentoring and accommodation.
Source: (UK Ministry of Justice, 2018[17]).
Making it happen 3.3. Database for Impact Measurement KOTEC (Korea)
Why?
In recent years, interest in social enterprises and their activities increased dramatically in Korea. As impact investing became more popular, decision-oriented impact performance and measurement methods began to attract attention. The Ministry of Employment and Labour of Korea developed the Social Value Index (SVI) in 2016 to objectively measure the social value and performance created by social enterprises. In a survey conducted by the Ministry in 2021, 41.8% of the companies that participated in the SVI measurement responded that they used it to reflect and improve their business performance. However, many social enterprises continue to lack resources to perform social impact measurement.
The Korean Technology Finance Corporation (KOTEC), a government-affiliated institution under the Korean Ministry for SMEs and Start-ups, was in need of a comprehensive and reliable framework for measuring social ventures performance that was aligned with global standards. The Impact Management Project (IMP) model stood out from global discussion and was flexible enough to test its adaptation to the local context.
What?
In 2020, KOTEC commissioned Impact Square, a Korean social venture accelerator, to conduct a feasibility study, identify priority social impact themes and pilot an assessment. Ten priority areas were identified based on the national survey of social enterprises regularly conducted by KOTEC. Among these, two themes were selected to be pilot tested using existing statistics and public data to set the stones to create a public data library for social impact measurement. Over three years, more than 50 impact measurement cases were developed, and baseline data was actively provided to social enterprises so they could use it to estimate their impacts.
Impact
Despite still being implemented, the three-year project has already marked a number of successes. Firstly, the long-term planning has proven useful to thoroughly investigate existing data, pilot and develop a robust model. Second, the engagement of stakeholders across government, intermediary organisations, academia and practitioners in social enterprises has been instrumental in creating a well-honed feedback loop. The basic database on the two areas of impact measurement for employment and resource circulation became available to the public in 2020. It covers all Korean social ventures working in these areas (50% of all social ventures). The addition of five additional impact themes, planned by the end of 2022, will increase this coverage to 80% of all social enterprises. It is meaningful that social enterprises not only use the database but also contribute to it, which will improve data quality over time.
To spread the use of the database and improve its data quality, ongoing capacity building efforts are needed. Impact Square continues to work with government-related institutions and social start-ups to create impact performance cases that can further a common understanding of impact performance management in the SSE ecosystem.
Source: (Dho, Yun and Baek, 2021[18])
Create repositories of knowledge
Policy makers can regroup impact studies and evaluations in the form of curated libraries. The accumulation of knowledge, on both positive and negative effects, enhances the potential for collective learning. Through these online portals, SSE entities and other stakeholders can find vetted evidence that can be used to better design interventions, as a blueprint to shape for their own social impact measurement strategy, and finally for fundraising and public advocacy purposes. This can be done in the form of What Works Centres, that gather experimental impact studies on relevant policy areas, such as the one on Well-Being in the United Kingdom.5 More targeted to the SSE space, the Impact Tank’s Wall of Solutions in France (Impact Tank, 2022[19]) displays impact stories collected by social economy entities as inspiration or templates for their peers. The Social Enterprise Evidence Space in Australia offers a well-curated catalogue of impact evidence, case studies, scientific articles, good practices and further information (see Making it happen).
Making it happen 3.4. Social Entrepreneurship Evidence Space (Australia)
Why?
With the aim to provide a more structured open research platform for social entrepreneurship, the government of the state of Victoria in Australia co-funded the creation of the Social Entrepreneurship Evidence Space (SEESpace) website together with the Lord Mayor’s Charitable Foundation. The website is designed and hosted by the Centre for Social Impact Swinburne, with contributions from centres at Griffith University, Queensland University of Technology and the University of Technology Sydney.
What?
Following the research guidelines, universities, industry bodies and community partners can submit evidence-based research including articles, publications, tools and more. The SEE Space structures the evidence into eight impact areas (community development, health and wellbeing, employment and careers, housing, economic impact, environmental impact, impact measurement, and social connection and support). Information is further grouped into collections (social enterprise, social finance and impact investing, measuring and managing impacts, buying from social enterprises, and social entrepreneurship support) as well as by target group (social financiers and funders, social start-ups, social entrepreneurs, policy makers, not-for-profit managers, and researchers and students). The information is then further classified as insight, statistic, report, academic article, infographic, tool or video. Every piece of evidence is displayed with its title, a brief description, the categories it is tagged with, the authors, a link to the resource and related research. The website also operates a powerful search engine.
On impact measurement, the website features amongst others a recent report on improving health equity of young people through social enterprises, an infographic on community enterprises, an academic article on long-term health impacts of a work integration social enterprise and statistics snippets from different reports including the 2016 Finding Australia’s Social Enterprise Sector report.
Impact
SEESpace is widely used by policy makers, practitioner networks, philanthropy, individual social enterprises and researchers in Australia as a ‘one stop’ source of empirical evidence about social entrepreneurship and its enabling ecosystem in this country. It was developed in response to a 2019 national policy and research roundtable convened by CSI Swinburne, where coordination of evidence about the field was a knowledge need prioritised by participants.
Infographic 3.2. Guiding questions: Foster evidence
References
[3] Après-GE (2015), Panorama de l’économie sociale et solidaire à Genève: Etude statistique 2015, https://apres-ge.ch/sites/default/files/Etude-statistique-web-APRES-GE_2015.pdf.
[7] British Council/Social Enterprise UK (2022), More in common: The global State of social enterpirse, British Council.
[1] Buckland, L. and L. Hehenberger (2021), Measuring Social Impact Can Help Foster a Stronger European Social Economy, Stanford Social Innovation Revies, https://ssir.org/articles/entry/measuring_social_impact_can_help_foster_a_stronger_european_social_economy (accessed on 2 July 2021).
[12] CASES/Statistics Portugal (2019), Social Economy Satellite Account 2016 and Survey on Volunteer work 2018, CASES, https://www.cases.pt/wp-content/uploads/2019/11/Livro-Conta-Satélite-Voluntariado.pdf.
[8] CECOP (2020), “Measuring the Social Impact of Industrial and Service Cooperatives in Europe A toolkit for members”, https://cecop.coop/uploads/file/vqXaL3qDRaYIZxlE39VajDsun2AZ3mVnpvgQY2Be.pdf (accessed on 15 December 2022).
[4] Centre for Social Impact/Social Traders (2016), Finding Australia’s Social Enterprise Sector 2016: Final Report, Centre for Social Impact/Social Traders.
[15] CEPES (2020), Análisis del impacto socio económico de los valores y principios de la economía social en España, https://www.cepes.es/files/publicaciones/118.pdf (accessed on 18 March 2021).
[18] Dho, T., N. Yun and H. Baek (2021), A Government data-driven approach to foster social impact measurement practice: Providing a public databse of ’Baseline’ and ’Threshold’ data by impact themes, https://cdn-assets.inwink.com/e34b7337-d151-4294-a09a-36038b91b434/76dcf20b-dcf1-4ac2-b702-c2b4a12450b1?sv=2018-03-28&sr=b&sig=lF0PxpYrgU1L8orBRBo4DpergQvbIglfr8Ne0nVxwq8%3D&se=9999-12-31T23%3A59%3A59Z&sp=r&rscd=inline%3B%20filename%3D%22A-Government-data.
[6] Dupain, W. et al. (2021), The State of Social Enterprise in Europe – European Social Enterprise Monitor 2020-2021, Euclid Network.
[5] Elson, P., P. Hall and P. Wamucii (2016), Canadian National Social Enterprise Sector Survey Report 2016, Mount Royal University Institute for Community Prosperity & Simon Fraser University.
[19] Impact Tank (2022), Découvrez les solutions à impacts positifs, https://impact-tank.org/innovation-sociale/#les-solutions.
[10] INSEE/Flores (2021), L’économie sociale en 2018, https://www.insee.fr/fr/statistiques/5368794?sommaire=5366170#consulter.
[11] ISTAT/EURICSE (2021), L’Economia Sociale In Italia: Dimensioni, caratteristiche e settori chiave, ISTAT/EURICSE.
[14] Kehl, K. et al. (2016), Möglichkeiten, Wirkungen (in) der Freien Wohlfahrtspflege zu messen, https://www.bagfw.de/fileadmin/user_upload/Veroeffentlichungen/Publikationen/CSI_Transparenzgutachten_2016.pdf (accessed on 17 March 2020).
[13] Ministerio de Empleo y Seguridad Social (2019), Base de Datos de la Economía social: Características de los Trabajadores en situación de alta en la Seguridad Social, https://www.mites.gob.es/ficheros/ministerio/sec_trabajo/autonomos/economia-soc/EconomiaSocial/estadisticas/CaracteristicasTrabajadores/2019/1TRIMESTRE/RESUMEN_1TRIMESTRE.pdf.
[9] Observatoire Economie Sociale (2020), ÉTAT DES LIEUX de l’économie sociale 2017-18, Observatoire Economie Sociale, https://observatoire-es.be/wp-content/uploads/2020/05/EDL-2017-2018.pdf.
[2] OECD (2021), “Social impact measurement for the Social and Solidarity Economy: OECD Global Action Promoting Social & Solidarity Economy Ecosystems”, OECD Local Economic and Employment Development (LEED) Papers, No. 2021/05, OECD Publishing, Paris, https://doi.org/10.1787/d20a57ac-en.
[16] SEWF (2022), SEWF 2022: Event Report, SEWF, https://sewfonline.com/wp-content/uploads/2022/10/SEWF22-event-report.pdf.
[20] Social Innovation Evidence Portal (2020), Social Entrepreneurship Evidence Space, https://seespace.com.au/.
[17] UK Ministry of Justice (2018), Guidance: Accessing the Justice Data Lab service, https://www.gov.uk/government/publications/justice-data-lab.