SME Policy Index: Latin America and the Caribbean 2024

2. Assessment methodology and policy framework

Abstract

This chapter provides an overview of the SME Policy Index (SME PI) assessment methodology, detailing its development and application, as well as its advantages and limitations. It also introduces the 2024 assessment framework and outlines the structure of the report.

Assessment methodology

This section provides an overview of the general SME Policy Index (SME PI) assessment methodology.

For the purpose of this publication and due to varied definitions of micro, small and medium-sized enterprises across the region, the authors of this report typically use the term “SME” to refer collectively to micro, small and medium-sized enterprises, unless specified otherwise. One the same note, LA9 refers to the nine Latin American countries included in this assessment (Argentina, Brazil, Chile, Colombia, Ecuador, Mexico, Paraguay, Peru and Uruguay).

Development and application

The SME Policy Index serves as a benchmarking tool to assess and monitor progress in the design and implementation of policies for small and medium-sized enterprises (SMEs). Since its inception in 2006, it has been implemented across a growing geographical area, now covering almost 40 economies in five regions: the Eastern Partnership (EaP), the Western Balkans and Türkiye, the Middle East and North Africa, the Association of Southeast Asian Nations (ASEAN), and Latin America and the Caribbean (LAC) (OECD/EBRD, 2023[1]).

The primary objective of the Index is to collect a substantial body of information and harmonise it, thereby converting predominantly qualitative inputs into numerical information (scores) that can be compared over time, across different economies, and regions. All SME Policy Indexes share a common methodology, although they are tailored for each regional application.

In contrast to other regions, LAC lacks a formal strategic document to guide policies for SME development at the regional level. Consequently, the framework for the first 7 dimensions of the SME Policy Index was developed as a pilot version in consultation with national coordinators from the first participating countries (Argentina, Chile, Colombia, Ecuador, Mexico, Peru, and Uruguay), the Development Bank of Latin America (CAF), and the Latin American and Caribbean Economic System (SELA) in 2019. This collaborative process involved referencing the SME PI methodology and leveraging insights from OECD experts.

Drawing on the strengths of this pilot version, and to enhance its impact, the 2024 edition incorporates certain methodological changes and includes new countries (Brazil, and Paraguay). These additions capture emerging economic and policy trends and priorities while ensuring comparability at the level of dimensions and sub-dimensions with the 2019 pilot edition. Therefore, the SME PI 2024 encompasses:

• a new dimension assessing selected framework conditions for digital transformation;

• a new pilot dimension assessing green transition in the SME sector;

• a cross-cutting gender approach;

• extended and amended sub-dimensions to collect relevant information;

• re-assessment of the scores derived from the Doing Business Report for 2019 and 2024.

Advantages and limitations

The SME Policy Index methodology adds value by transcending statistics, delving into the SME policy environment across diverse areas. It offers an independent and rigorous assessment, acting as a benchmark against international best practices and offering guidance for policy reform and development based on these insights. However, like all methodologies aiming to transform a complex reality into a tangible vision, the SME Policy Index methodology has both advantages and limitations (see Table 2.1).

Table 2.1. SME Policy Index advantages and limitations

Advantages	Limitations
Embrace a multi-stakeholder approach to assessment, considering the perspectives and experiences of governments, the private sector, partner organisations, and academia.	The Index evaluate the levels of policy development across the entire policy cycle (planning and design, implementation, and monitoring and evaluation). Therefore, its design measures and compares levels of policy development rather than assessing policy performance.
Provides guidance on enhancing the SME policy framework in each country through policy recommendations and exemplifying best practices.	The weightings of the indicators are defined based on expert opinions and may therefore be open to subjectivity.
Contextualises the analysis by incorporating the country context and broader factors influencing SME development to complement the emerging scores.	The divergent definitions of SMEs pose a limitation to the comparability of data across economies.
Concentrates on a specific region where shared history, culture, and geography enable more relevant benchmarking between countries.	The full picture of government policy can be hard to capture where SME policies are implemented mainly by local governments. The current SME PI methodology focuses on policy at the national or central level.
Focuses on the LAC region as it embarks upon further economic integration, enabling policymakers to identify challenges that may lie ahead.

Source: (OECD/EBRD, 2023[1]) (OECD/CAF, 2019[2])

In essence, the SME PI framework operates under the assumption that there exists a positive correlation between effective policy practices and performance. In other words, a well-designed and thoroughly implemented policy throughout all three cycle stages is expected to be both more effective and more readily evaluated for its performance. This, in turn, enables policymakers to make decisions based on performance-oriented insights for future actions.

The 2024 assessment framework and structure of the report

2024 Assessment framework

The 2024 assessment framework for LA9 incorporates SME policy benchmarks from the pilot version's assessment framework in 2019, covering 7 dimensions and 26 sub-dimensions. Additionally, to underscore the importance of the digital economy and align with methodologies in other regions, a new dimension of digitalisation has been added, comprising three sub-dimensions. These sub-dimensions investigate i) the presence of a National Digital Strategy and its provisions for SMEs; ii) incentives and policies for improving broadband connectivity; and iii) the promotion of digital skills across the population (see Table 2.2). Furthermore, a pilot version without scores and sub-dimensions related to the green economy, along with a cross-cutting gender approach, has been incorporated at the request of the participating countries. This emphasises the need to convert these two new approaches into measurable dimensions, with enhanced data collection and analysis for future editions.

Table 2.2. 2024 SME PI assessment framework for Latin America participating countries

Dimensions and sub-dimensions
1.Institutional Framework	2. Operational Environment/Simplification of procedures
1.1 SME definition	2.1 Legislative simplification and regulatory impact analysis
1.2 Strategic planning, policy design and co-ordination	2.2 Company registration
1.3 Public-private consultations	2.3 Ease of filing taxes
1.4 Measures to address the informal economy	2.4 E-government
3. Access to finance	4. SME development services and public procurement
3.1 Legal, regulatory, and institutional framework on access to finance	4.1 Business development services
3.2 Diversified sources of enterprise finance	4.2 Entrepreneurial development services
3.3 Financial education	4.3 Public procurement
3.4 Efficient procedures for dealing with bankruptcy
5. Innovation and technology	6. Productive transformation
5.1 Institutional framework	6.1 Productivity-enhancing measures
5.2 Support services	6.2 Productive association-enhancing measures
5.3 Financing for innovation	6.3 Integration into regional and global value chains
7. Access to market and internationalisation	8. Digitalisation (new dimension)
7.1 Support programmes for internationalisation	8.1 National Digital Strategy
7.2 Trade facilitation	8.2 Broadband connectivity
7.3 Use of e-commerce	8.3 Digital Skills
7.4 Quality standards
7.5 Taking advantage of the benefits of LAC regional integration

2024 Methodological changes

Building upon its pilot version in 2019, the SME PI has undergone significant methodological adjustments, ensuring a more comprehensive and refined assessment. Firstly, to streamline the data collection process, the questionnaire was delivered through an online platform, enhancing user experience, efficiency, and accessibility. To alleviate the response burden on participants, the OECD conducted a wide-ranging overhaul of the questionnaire, clarifying ambiguous questions, removing redundant or outdated items, and introducing new questions to address emerging priorities and policy trends, particularly in response to the COVID-19 crisis.

Secondly, all indicators from the World Bank's Doing Business report were no longer considered for scoring due to the discontinuation of the exercise. However, its last report was used as a reference in some texts. In Dimension 2, Operational Environment, and Simplification of Procedures, the scores derived from this exercise for sub-dimensions 2.2 Company Registration and 2.3 Ease of Filing Taxes were replaced with new questions in the questionnaire and an analysis of the performance by the experts. In Dimension 3, Access to Finance, sub-sub dimension 3.1.3 Credit Information Bureau and thematic block 1, Creditor Rights from sub-sub dimension 3.1.2 Legal Regulatory Framework for Commercial Lending, were removed, adjusting the value percentages for the thematic blocks or sub-sub dimensions accordingly and generating an adjusted score for the 2019 version to preserve comparability across exercises. Data for sub-dimension 7.1 Trade Facilitation continue to be based on the OECD Trade Facilitation Indicators (see Table 2.3).

Table 2.3. 2024 SME Policy Index methodological changes from supplementary data

Sub-dimension or sub-sub dimension	Indicator(s)	Data source 2019	Data source 2024 or adjustment
2.1 Company registration	All indicators under Thematic Block 2: Performance	World Bank Doing Business 2019	New questions in the SME PI assessment and an analysis of the performance by the experts
2.3 Ease of filing tax	All indicators in the sub-dimension	World Bank Doing Business 2019	New questions in the SME PI assessment and an analysis of the performance by the experts
3.1.2 Legal regulatory framework for commercial lending	All indicators in Thematic Block 1: Creditor Rights	World Bank Doing Business 2019	Elimination of the indicator and readjustment of scoring
3.1.3 Credit information bureau	All indicators in the sub-sub dimension	World Bank Doing Business 2019	Elimination of the indicator and readjustment of scoring
7.2 Trade facilitation	All indicators in Thematic Block 1: OECD Trade Facilitation Indicators	OECD Trade Facilitation Indicators	OECD Trade Facilitation Indicators

Scoring and weighting

As previously indicated, the evaluation framework employed for LA9 countries comprises eight dimensions and 29 sub-dimensions (see Table 2.2) .These sub-dimensions are organised into thematic blocks, each featuring its own array of indicators. Additionally, the thematic blocks are typically segmented into three components, symbolising distinct phases of the policy cycle: planning and design, implementation, and monitoring and evaluation (M&E). In cases where this approach is not applicable to certain sub-dimensions, the thematic blocks may vary.

The evaluation framework encompasses a mix of qualitative and quantitative indicators in the following formats:

• Core indicators: These indicators define the evaluation score and can take the form of binary or multiple-choice measures in qualitative assessments.

• Open-ended questions: Open-ended questions are integrated following the core indicators for each sub-dimension in the assessment questionnaire. These questions are not scored, but help to assess the overall policy context, thus contributing to the final score.

The weighting methodology was consistent across all LA9 countries, following the approach used in the 2019 pilot version. The predominant thematic blocks - planning and design, implementation, and monitoring and evaluation (M&E) -were assigned respective weights of 35%, 45%, and 20%. This distribution underscores the significance of policy implementation, aligning with the principles of the SME PI developed for other regions.

Structure of the report

This report is divided into two parts. Part I delivers a thorough overview of the regional evaluation, where each chapter is dedicated to a specific dimension. Part II presents individual country chapters, delving deeper into the assessment outcomes for each LA9 country.

It is crucial to acknowledge that these dimensions do not operate independently; instead, they interact and support each other in various ways. The close interlinkage among all dimensions emphasises the importance of policymakers to adopt a holistic perspective. Therefore, endeavours to reform one area must be built upon advancements and a robust groundwork in all other dimensions, collaboratively contributing to the efficacy of SME support initiatives.

Assessment process

The SME PI was conducted between 2022 and 2024, comprising two assessments. Firstly, LA9 countries' governments conducted a self-assessment through an online questionnaire, providing relevant evidence. Secondly, an independent assessment was carried out in collaboration with the Fundación para el Análisis Estratégico y Desarrollo de la PyME (Foundation for the Strategic Analysis and Development of SMEs, FAEDPYME), utilising their network of academic researchers in the LA9 countries.

A cut-off date of 31 October 2023 was established for the assessment, considering only policy developments and reforms implemented by that date for the SME PI scores. Any developments after that date are reflected in the text. The final scores result from consolidating these two assessments, enhanced by additional research and consultations. The evaluation process involves these three phases:

• Review of methodology and framework (April 2022 – September 2022). The methodology and assessment framework were updated, incorporating the dimension of digitalisation in collaboration with CAF and SELA. Reference was drawn from methodologies used in other OECD regions, particularly the EaP exercise, and insights from OECD experts in relevant areas. Participating countries were consulted and invited to a launch and methodology verification meeting in Montevideo, Uruguay in June 2022 to incorporate feedback and finalise the assessment questionnaire. During this meeting, for the first time in the LAC exercise, it was decided to introduce a pilot dimension (without scoring) related to the green economy and a cross-cutting gender approach. National Coordinators for Brazil, and Paraguay were identified.

• Data collection, verification, and evaluation (October 2022 – October 2023) During the data collection phase, the LA9 participating countries carried out a self-assessment of their policy framework through an online questionnaire. This was complemented by an independent assessment. Desk research and engagement with relevant stakeholders were employed to address information gaps and resolve inconsistencies. Online workshops were conducted in all 9 countries to facilitate the data collection exercise, typically involving SME policy stakeholders. Finally, a workshop presenting preliminary results was held with national coordinators in Brasilia, Brazil, in August 2023, accompanied by an update of information provided by the countries. The preliminary results workshops of the digitalisation dimension were conducted online with each of the countries in October 2023.

• Drafting, review and publication (November 2023 – April 2024) Additional information was collected following the workshop held in Brasilia and the online workshops. A preliminary draft of the report was prepared which was reviewed by CAF and SELA, along with a group of OECD experts and the LA9 countries and comments were incorporated into the final report. The publication was formally launched at a regional meeting in Valparaíso, Chile, in July 2024, accompanied by a series of national dissemination events.

References

[2] OECD/CAF (2019), Latin America and the Caribbean 2019: Policies for Competitive SMEs in the Pacific, OECD Publishing, https://doi.org/10.1787/d9e1e5f0-en.

[1] OECD/EBRD (2023), SME Policy Index: Eastern Partner Countries 2024: Building Resilience in Challenging Times, OECD Publishing, https://doi.org/10.1787/3197420e-en.