This chapter provides guidance for developing a monitoring and reporting framework for measuring implementation progress of the skills policy actions identified in previous chapters. It provides guidance to Bulgarian authorities on designing a monitoring and reporting framework that can be used to conduct policy evaluation and to ensure that the outcomes of such evaluations are subsequently used for agenda setting and designing future evidence-based skills policy.
Technical Support for the Development of a National Skills Strategy for the Republic of Bulgaria
OECD Skills Studies
Abstract
This chapter is Output 3 of the TSI Project1. Output 1 outlined medium-term goals, or "objectives" and high-level "priority policy actions" that Bulgaria intends to implement in its Action Plan for Skills. Output 2 then expanded upon this to establish finer details of executing the action plan by defining concrete activities, and for each of these proposing responsible entities, timelines, and potential funding sources essential for the plan's implementation. Output 3 provides guidance for developing a monitoring and reporting framework for measuring implementation progress of the skills policy actions identified in Outputs 1 and 2.
Outputs 1, 2 and 3 also address all stages of the policymaking process (Figure 4.1). While the higher-level objectives and policy actions identified in Output 1 provide guidance to Bulgaria for agenda setting and policy formulation of skills policy, the more detailed proposal in Output 2 provides guidance that Bulgarian officials can use to actually implement these skills policies. The current Output 3 addresses the remaining stages of the policy cycle by providing guidance to Bulgarian authorities on designing a monitoring and reporting framework that can be used to conduct policy evaluation and to ensure that the outcomes of such evaluations are subsequently used for agenda setting and designing future evidence-based skills policy.
Source: Adapted from Exton, C. and Shinwell, M (2018[1]), “Policy use of well-being metrics: Describing countries’ experiences”, OECD Statistics Working Papers, No. 2018/07, https://doi.org/10.1787/d98eb8ed-en.
To develop an effective monitoring and reporting framework for Bulgaria’s Action Plan for Skills, this chapter first describes the existing Bulgarian and European Union (EU)-level regulations on monitoring and reporting to ensure both that the proposed framework does not conflict with existing standards and to create as many synergies as possible with existing monitoring and reporting requirements so as to lessen the administrative burden on Bulgarian officials of effectively tracking progress in skills policy. The report then presents specific milestones and indicators to monitor the implementation progress for the Action Plan for Skills in the areas of developing youth skills, improving adult skills, using skills effectively, and strengthening the governance of the skills system. Then, it outlines how these milestones and indicators should fit into a larger monitoring and reporting framework for Bulgaria’s Action Plan for Skills, including the flow of skills data and information for each of the four priority areas. Finally, the report concludes with a brief summary and description of the next steps in the project.
To develop the monitoring and reporting framework, the OECD team drew on desktop research, taking into consideration relevant examples of good policy practices from other EU Member States and across the OECD. A physical mission to Bulgaria was also conducted in order to test and refine the preliminary results of the desktop research and the initial list of indicators, which were then narrowed in line with authorities’ and stakeholders’ expertise and priorities. Afterwards, a series of bilateral meetings and regional stakeholder discussions were also held virtually in order to gather more detail about how monitoring and reporting activities are done in practice in Bulgaria, further written feedback was provided by some stakeholders, and all findings were incorporated into this chapter.
The proposed monitoring indicators and overarching monitoring and reporting framework are informed by existing regulations and indicators for monitoring and reporting both within the Bulgarian national context and at the EU-level. These two sources of regulations are fitting to inform the monitoring and reporting framework for a Bulgarian Action Plan for Skills given that the main potential funding sources for skills policy activities indicated in Output 2 were the State budget and EU-programme budgets (or more specific budgets within these). Furthermore, aligning indicators to track implementation of a Skills Action Plan with indicators already being collected for EU programmes or national strategies, to the degree possible, could help to ease the administrative burden for Bulgarian authorities of effectively adhering to the proposed monitoring and reporting framework for an Action Plan for Skills in Bulgaria.
General regulations for monitoring and reporting in Bulgaria
In Bulgaria, there is currently no established framework for strategic planning that lays out clear standards, rules, and procedures for the preparation, monitoring, and reporting across all strategic documents. Although a Strategic Planning Act was drafted in 2019, it has not yet been officially adopted. Overall, Bulgaria’s strategic planning process remains fragmented, with responsibilities split between different bodies and institutions. Numerous laws and regulations mandate strategic planning for various public authorities. Notably, excluding the National Development Programme Bulgaria 2030, all national strategies are instigated, executed and monitored by line ministries and agencies at their respective sectoral levels.
In line with Article 63 of Bulgaria’s Law on Administration, ministries, commissions and agencies, as well as regional administrations, are obligated to report to the Council of Ministers at the beginning of each year on the implementation of strategic goals from the preceding year (Bulgaria Council of Ministers, 1998[2]). In instances where institutions contribute to interinstitutional programmes, they are required to report to the leading institution. The leading institution is tasked with consolidating the provided information and presenting a report to both the Ministry of Finance and the Council of Ministers on the execution of the interinstitutional programme. However, the monitoring of national strategies and action plans implementation is currently underdeveloped, lacking defined rules and procedures for comprehensive systemic monitoring and impact evaluation (OECD, 2022[3]).
Monitoring and reporting regulations for specific strategies in Bulgaria
The National Development Programme Bulgaria 2030 (NDP Bulgaria 2030) is Bulgaria’s main framework strategic document, outlining the vision and general objectives of the policies for development of all sectors (Ministry of Finance, 2020[4]). The NDP 2030 contains 13 national priorities including a priority on education and skills and one on social inclusion, which include goals that overlap with the proposed Action Plan for Skills in Bulgaria. For each set of measures within the priorities of the NDP Bulgaria 2030, there are specified indicators, target indicator values for 2030, and an indicative financial framework. The NDP Bulgaria 2030 implementation follows three-year action plans, along with annual updates. These updates must be coordinated by the designated leading authorities and accompanied by an annual report that outlines the implementation of measures within specific priorities.
Relevant NDP Bulgaria 2030 indicators that have been incorporated or adapted in Table 4.1 include:
Pupils from age four to the starting age of compulsory education at primary level, % of the population of the corresponding age group (Indicator 1.1 in the NDP 2030, Eurostat).
Employment rate of recent graduates with a vocational upper secondary or post-secondary non-tertiary education (Indicator 1.3 in the NDP 2030, Eurostat).
The Strategic Framework for the Development of Education, Training, and Learning in the Republic of Bulgaria (2021-2030) is aligned with the vision and overarching goals of the education and skills priority of the NDP Bulgaria 2030, particularly in its sections related to preschool and school education, vocational education and training, and lifelong learning (Ministry of Education and Science, 2021[5]). Twenty-four system-level indicators are specified to monitor the achievement of goals of the strategy, as well as target values for 2030. The strategic framework will be implemented through three action plans and evaluated with two interim reports and a final analytical report. The outlined goals, measures, and indicators will be revised both in 2024 and 2027 to align with progress made or changes in identified challenges.
Relevant indicators from the Strategic Education Framework that have been incorporated or adapted in Table 4.1 and Table 4.2. include:
Net ratio of pupils from age three until entry into compulsory education at primary level, % of the population of the corresponding age group (Indicator 3 in the Strategic Education Framework, Eurostat).
Relative share of population (ages 25-64) participating in education and training (in previous four weeks) (Indicator 16 in the Strategic Education Framework, Eurostat).
The Strategy for Development of Higher Education in the Republic of Bulgaria 2021-2023 defines the main principles and priorities in the development of the higher education system until 203 0 and outlines ten specific goals, activities, and measures for their implementation, as well as 68 outcomes (Ministry of Education and Science, 2020[6]). For each measure various expected results are listed, although there are no clearly delineated indicators. Among these expected results are, for example, development and maintenance of mechanisms for tracking the progress of graduates; а greater focus on practical training, guided by clearly stated employer needs and pre-agreed job placements; an increased number of curricula and educational programmes developed with the involvement of business representatives and other external partners; and the development of dual education in higher education in professional fields with a technical focus, where at least 50% of all active students in such specialised areas receive training. The Strategy is implemented based on a multi-year operational plan, approved by the Council of Ministers, while the operational activities are conducted by the MES. The line ministries, in co‑ordination with the MES, carry out activities for the development of higher education in their respective sectors, utilising the measures outlined in the strategy. The implementation of the operational plan is reported at least once every two years.
The Employment Strategy of the Republic of Bulgaria 2021-2030 sets short-term (up to 2024) and long-term (up to 2030) priority goals and outlines specific activities for their achievement, divided into eleven groups (Ministry of Labour and Social Policy, 2022[7]). The strategy is implemented through annual national action plans for employment and monitoring of the strategy's implementation is ongoing, with annual reports prepared to assess progress. The monitoring encompasses indicators tracking progress in achieving the outlined strategy goals, as well as indicators for individual activities, according to the list of indicators attached to the Strategy. Specialised units within the Ministry of Labor and Social Policy (MLSP) and the National Employment Agency (NEA) undertake monthly and annual monitoring of employment-related activities. More specifically, NEA, through its regional divisions, conducts monthly, quarterly, and annual observation and evaluation of the implementation of active measures and programmes. Data on implemented measures are collected using methodologies ensuring comparability with data collected by Eurostat, ILO, and OECD. In addition, impact data on policy are provided by the National Statistical Institute through regular statistical surveys, primarily the Labour Force Survey conducted quarterly. The administrative statistics from the NEA serve as a source of detailed information, allowing ongoing monitoring of the implementation of active measures and programmes. For each implemented measure and programme, data on participants (enrolled, employed, those who have terminated their participation, etc.) and expenditures (financial resources provided to unemployed individuals, employers, etc.) are collected using standardised indicators. Specific data are also collected according to the objectives of the programmes and measures and are summarised quarterly and annually.
Relevant indicators from the Employment Strategy that have been incorporated or adapted to include in Table 4.2 and Table 4.3 include:
The relative share of individuals participating in formal and informal education and training (ages 25–64, Eurostat).
Registered unemployed persons involved in training to acquire key competences.
Registered unemployed persons included in vocational qualification training.
Registered unemployed persons consulted individually by case manager.
Difference in the employment rate for women and men (ages 20-64) (Eurostat).
Specific data relevant to a Skills Action Plan for Bulgaria are also collected as part of the annual National Employment Action Plan (NEAP). For example, the annexes to the National Employment Action Plan (NEAP) for 2023 include various indicators related to subsidised employment and/or training funded by active labour market policy (ALMP) funds. The reports on the implementation of the NEAP also provide data on specific ALMP indicators such as number of job fairs and informational campaigns conducted; number of youth who have participated in forms of vocational orientation; number of individual action plans created by employment mediators for newly registered youth; number of trained individuals, etc.
The Innovative Strategy for Smart Specialisation 2021-2027 aims to stimulate the transformation of society and the economy based on data and knowledge, oriented towards Industry 4.0, and also to support the green transition (Ministry of Innovation and Growth, 2022[8]). The Ministry of Innovation and Growth is tasked with implementing the Strategy and its associated programmes/plans. The monitoring and evaluation mechanism for strategy implementation involves ongoing monitoring, preparation of an interim progress report (for the period 2021-2024), and a final monitoring report with an external assessment to be developed two years after the conclusion of the Strategy’s term. As part of ongoing monitoring, information shall be collected and summarised for specific indicators for innovation performance (e.g. new doctoral graduates, population with higher education, lifelong learning participants, etc.), innovation support (e.g. supported enterprises and/or research organisations and universities; assisted researchers, etc.) and indicators for monitoring and evaluating operational goals at the operational level (e.g. personnel engaged in R&D, employment in knowledge-intensive services, innovative capacity, etc.).
Relevant indicators from the Smart Specialisation Strategy that have been incorporated or adapted to include in Table 4.3 include:
Staff/personnel engaged in R&D in enterprises, by region (NSI).
The National Strategy for Small and Medium Enterprises (SMEs) 2021-2027 outlines three strategic goals: enhanced competitiveness, specialisation in high-tech manufacturing and knowledge-intensive services, and regional equality, along with measures for their achievement (Bulgarian SMEs Promotion Agency, 2021[9]). The National Strategy for SMEs is implemented through annual action plans developed by the Bulgarian Small and Medium Enterprises Promotion Agency (BSMEPA), which is also responsible for its monitoring and evaluation. Progress reports are prepared annually, relying on official information provided by all relevant authorities. At the end of the planned period, a final report on the progress of the Strategy's implementation will be drafted. Additionally, two interim evaluations (in 2024 and 2026) of the Strategy's implementation are planned, along with a final evaluation in 2028. The strategy outlines indicators for each of the specific measures in the strategy including baseline values (2020) where relevant and target values for both 2024 and 2027.
Relevant indicators from the SME Strategy that have been incorporated or adapted to include in Table 4.1 and Table 4.2 include:
Regular training needs assessments carried out in SMEs; digital solutions in training needs assessment in SMEs; training courses conducted in SMEs (Indicator 4.3 in the SME strategy).
Number of registered enterprises, incl. SMEs, in the information database of employers who conduct dual training (work-based learning); number of students participating in dual training; support for SMEs providing dual training in order to open new classes with dual education; recognition of active SME employers in the field of vocational education and training (Indicator 4.4 in the SME strategy).
A study of the assessment and validation of informal skills in SMEs; development of national standards and procedures, as well as a certification system for key validation nationally skills and competencies, percentage of SMEs with personnel with validated skills acquired through non-formal training, percentage of SMEs that have increased the formal qualifications of their personnel (Indicator 4.5 in the SME strategy).
Development of an e-learning platform and tools and promote online-based training for SMEs, development of e-learning courses (Indicator 4.6 in the SME strategy).
However, though outlined in the SME Strategy itself, the above indicators are not, in practice, yet part of the monitoring system of the Bulgarian Small and Medium Enterprises Promotion Agency (BSMEPA) under the National Strategy for SMEs 2021-2027. Their collection both for the purposes of the SME Strategy and an Action Plan for Skills would require consultations across various institutions (e.g. NSI, MLSP, MIG, MEI) to identify the most feasible and convenient way to collect them without creating additional administrative burden for enterprises and government ministries.
The National Strategy on Migration of the Republic of Bulgaria 2021-2025 aims, among other things, to provide conditions for the reception of migrants who arrive legally in the country for purposes such as work, education, or other reasons, taking into account the needs of the labour market, especially for high-skilled workers (Ministry of Interior, 2021[10]). However, there are no clearly delineated indicators outlined in the strategy. The implementation of the strategy is based on annual action plans, adopted by the Council of Ministers upon the proposal of the National Council on Migration, Borders, Asylum, and Integration, which also monitors and reports the implementation.
EU regulations for monitoring and reporting in Bulgaria
For EU-funded programmes, Bulgaria employs both EU-wide regulations that establish standards for overseeing the implementation of such programmes across Member States, as well as its own national guidelines for monitoring and reporting on EU-level programmes. The Regulation (EU) No. 1303/2013 of the European Parliament and of the Council, also known as the Common Provisions Regulation (CPR) for the 2014-2020 period, sets a comprehensive performance framework across EU countries for tracking progress towards objectives (European Union, 2013[11]). This regulation mandates Member States to establish monitoring committees, generate reports for the European Commission, and conduct annual reviews to evaluate programme performance. Crucially, it allows Member States flexibility in applying this framework in line with their own institutional and legal structures and regional competences, ensuring effective implementation while maintaining adherence to EU goals and standards. The updated Regulation (EU) 2021/1060 for the 2021-2027 period continues this approach and aligns funds more closely with the European Union's current priorities such as sustainability, digitalisation, and social inclusion (European Union, 2021[12]).
The EU-level programmes most relevant to an Action Plan for Skills for Bulgaria are the European Social Fund Plus (ESF+) and the Recovery and Resilience Facility (RRF), both of which have been designated as potential funding sources for a number of specific skills activities detailed in Output 2 of this project. The ESF+ is a key EU tool for supporting employment policies across EU countries, focusing on full employment, quality work, and mobility, alongside enhancing education, training, and social inclusion. Its requirements are outlined in Regulations (EU) 2021/1060 and 2021/1057 (ESF+ Regulation) of the European Parliament and the Council. Specific indicators for assessing policy impacts, especially those under the lifelong learning objective such as labour market improvements and qualifications, are included in the ESF+ common indicators toolbox (European Union, 2021[12]; European Union, 2021[13]).
Relevant indicators from the ESF+ indicators toolbox that have been incorporated or adapted to include in Table 4.2 include:
“Participants in education or training upon leaving” (Indicator EECR02 in the ESF+ indicators toolbox) "Participants gaining a qualification upon leaving" (Indicator EECR03 in the ESF+ indicators toolbox)
“Participants with an improved labour market situation six months after leaving” (Indicator EECR06 in the ESF+ indicators toolbox).
Other indicators in the toolbox may either address different topics or be too broad to be directly relevant to a Skills Action Plan for Bulgaria. The regulations require Bulgaria to establish a performance framework for monitoring and evaluating ESF+ funded activities. This includes forming monitoring committees with relevant partners and organising yearly structured policy dialogues. Within this framework, system-level indicators are set to track progress and outcomes of individuals supported by ESF+ funds.
Bulgaria's National Recovery and Resilience Plan (NRRP) outlines the package of reforms and investments being carried out by the country under the European Union’s RRF from 2021 to 2026. It was launched in response to the socio-economic crisis caused by the COVID-19 pandemic and operates under Regulation (EU) 2021/241 (European Union, 2021[14]). The NRRP consists of 12 components related to innovation (including education and skills), the green and digital transitions, and socio-economic equity. Under the "Education and Skills" component, interventions aim to enhance employability by improving skills as well as by upgrading educational quality and accessibility (Bulgaria Council of Ministers, 2022[15]). The plan includes milestones and targets with related qualitative and quantitative indicators. The Ministry of Finance is the central body responsible for overall management of the NRRP implementation, with the Central Coordination Unit Directorate of the Ministry of Finance particularly responsible for monitoring the implementation of the milestones and targets of the RRP. It is also tasked with establishing a strategic framework for information and communication activities in relation to the implementation of the RRP and for administration, maintenance, and upgrade of the IT systems for programmes under shared management with the Union and under the Facility. Line ministries are tasked with implementing individual investments and reform measures in their respective area of activity. The NRRP contains a number of macroeconomic indicators that are all too broad to be of particular interest in determining indicators for an Action Plan for Skills in Bulgaria.
Relevant indicators from Bulgaria’s NRRP that have been incorporated or adapted to include in Table 4.1 and Table 4.2 include:
Number of new kindergartens and schools built and/or renovated; increased coverage of children in kindergartens as a result of renovation, reconstruction, upgrading and modernisation of the buildings.
Number of teachers who undergo training in STEM; increased competencies of teachers to work in STEM environment; percentage of schools that have trained pedagogical specialists; implemented innovative modern teaching methods in all educational institutions.
Percentage and number of students applying for university specialties in technical, engineering and natural sciences.
Developed and implemented virtual platform for e-training of adults; number of developed online training courses for adults; number of developed e-resources for informal learning for adults.
Number of adults participating in training for basic digital skills and competences.
Number of adults with validated basic skills and competences.
The Partnership Agreement on the European Structural and Investment Funds (ESIF) for Bulgaria for 2014-2020 and the Partnership Agreement between the European Commission and Bulgaria for the 2021-2027 period apply to all Bulgarian projects that are funded using ESIF (European Commission, 2014[16]; European Commission, 2022[17]). ESIF funds are used for more specific EU funds including the European Regional Development Fund (ERDF), the European Social Fund (ESF+), The Cohesion Fund (CF), the European Agricultural Fund for Rural Development (EAFRD), and the European Maritime and Fisheries Fund (EMFF) (European Commission, 2023[18]). The Bulgarian Partnership Agreement for 2021‑2027 outlines a roadmap for building national administrative capacity, focusing on three pillars: (i) strategic planning and implementation, (ii) people and organisational management, and (iii) framework conditions.
The Law on Management of Resources from the European Structural and Investment Funds (ESIF) Act is applicable to all Bulgarian programmes funded through EU Structural and Investment Funds (ESIF) (Bulgaria Council of Ministers, 2015[19]). It introduces special mechanisms to ensure the proper administration, monitoring, and evaluation of the Partnership Agreement and the EU-funded programmes by adapting the EU’s regulatory framework to the specific Bulgarian context and establishes the necessary legal grounds for the national authorities to manage the funds in accordance with both national and EU law. Among other things, the law provides the legal basis for defining the criteria and processes for monitoring, reporting, financial management, and control; setting up the necessary committees, to oversee the implementation of the operational programmes; and monitoring the progress made towards achieving the objectives and priorities in relation to the indicators defined in the joined action plan. The monitoring of the ESIF Act is overseen by a monitoring committee. This committee is responsible for tracking the progress towards achieving the strategic goals outlined in the Partnership Agreement. Additionally, it reviews information related to the implementation of various programmes and assesses their contributions to the overall objectives and target values defined in the Partnership Agreement.
The law also regulates the establishment of a Bulgarian national Information System for Management and Monitoring of ESIF Resources (UMIS). The goal of the management information system (MIS) is to collect and process all data related to the implementation, management, monitoring, evaluation, and control of all Bulgarian programmes funded by the ESIF and maintain the collected accurate and reliable information in one location, with operational standards set by the Council of Ministers. The terms, arrangement, and operational procedures of the UMIS are specified in a special regulation outlined in Resolution 243 (Bulgaria Council of Ministers, 2016[20]). Information crucial for monitoring, evaluation, financial management, and audit of each operation is electronically provided by the authorities responsible for managing and controlling funds from ESIF, as well as by the candidates and beneficiaries. This includes data about individual participants in the operations. The information, including individual data, can be utilised for statistical and analytical purposes and may be published and disseminated to ensure transparency in the management of funds, while complying with data protection requirements. The publicly accessible module of the information system provides implementation information for all programmes, data related to their indicators and evaluation results.
The Council of Ministers plays a vital role under the Law on Management of ESIF. They are responsible for approving the draft Partnership Agreement and the draft programmes before their submission to the European Commission. Furthermore, the Council of Ministers adopts strategies and operational programmes, sets priorities, and ensures alignment with the EU's objectives and Bulgaria's socio-economic needs. The Council also oversees the implementation of these programmes, designating the entities responsible for developing and implementing them and ensuring efficient fund utilisation and compliance with both national and EU regulations.
Introduction
As part of the larger monitoring and reporting framework, the OECD team has proposed a set of milestones and indicators to measure the implementation progress of Bulgaria’s Action Plan for Skills. These monitoring mechanisms aim to allow authorities and stakeholders in Bulgaria to determine whether target populations have been reached, if activities are being implemented as planned, if the governance structures put in place have been effective and efficient, if the outcomes of the Action Plan for Skills are being achieved as expected, if there are unforeseen challenges that are hindering progress, and what elements of the policy should be modified and improved (OECD, 2023[21]). A comprehensive set of milestones and indicators can allow Bulgarian authorities and stakeholders to gather information for contributing to evidence-based policymaking, informing executive decisions, enabling more effective resource allocation, and promoting accountability (OECD, 2021[22]; OECD, 2023[21]).
This chapter provides a description of the monitoring mechanisms for Bulgaria’s Action Plan for Skills, explaining the distinction between milestones and indicators, and presenting the information included for each proposed indicator, namely its type, disaggregation details, data considerations, a cross-reference to relevant activities in Output 2, and the baseline value (where relevant and available). This if followed by a list of milestones and indicators for each of the priority areas identified in earlier project Outputs.
Monitoring mechanisms
The monitoring mechanisms for Bulgaria’s Action Plan for Skills include two main elements, namely milestones and indicators (Figure 4.2). Milestones and indicators are distinct elements of the monitoring process that collect different kinds of information but are nonetheless closely related and interlinked. The milestones are specific activities in the Action Plan for Skills, such as the establishment of governance bodies or the development of more specific policy plans, that indicate key events in the Action Plan’s implementation. The achievement of these milestones is often binary (e.g. a new skills-related group has or has not been established) and they tend to be prerequisites for the implementation of other activities in the Action Plan that are better captured through the collection of indicator data. For instance, in the priority area on strengthening the governance of the skills system, Activity 13.2.1.1, which involves the creation of an Inter-institutional Data and Evidence Centre (DEC), has been identified as a milestone. Its establishment would allow for the implementation of other related activities, such as Activity 14.1.2.1, which aims to systematically evaluate public expenditure on skills development and use and is monitored through Indicators GOV.7 and GOV.8.
On the other hand, the indicators proposed for the Action Plan for Skills aim to provide accurate, timely, and reliable information on the progress of the implementation of the activities in the Action Plan, reflecting relative increases or decreases over time. They cover various aspects of policy implementation, including the amount of resources (e.g. financial expenditure, human resources) devoted to actualising the activities in the Action Plan for Skills, their scale of uptake among target groups, and the quality of their outcomes. For each of the indicators, the OECD has identified information that provide authorities and stakeholders in Bulgaria with guidance on how to design and use indicators effectively (Figure 4.2):
Indicator name and reference number. Each indicator is provided with a name that concisely describes what the indicator aims to measure, as well as a reference number to facilitate identification. The reference numbers are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator YS.1) and by letters to refer to milestones (e.g. Milestone YS.A) in order to distinguish between the two.
Indicator type. The proposed indicators are comprised of three indicator types, namely input, uptake, and outcome indicators (Figure 4.2). Inputs refer to the financial and human resources that are devoted to the implementation of the activities in the Action Plan for Skills. Uptake refers to the number and share of participants (e.g. individuals, enterprises, education and training providers) that take part in the activities, while outcomes refer to the effects of their participation in such activities, as observed in their educational and labour market conditions.2 All three indicator types are generally covered for each policy action in the four priority areas in order to explore the links between inputs, uptake, and outcomes. Taken together, input, uptake and outcome indicators can provide insight on the impact, or the higher-level and longer-term effects, of the Skills Action Plan.
Disaggregation. The OECD team has suggested that the indicators be disaggregated by various characteristics (e.g. gender, age group, municipality, sector, type of activity) in order to track differences in the inputs, uptake and outcomes for subsets of the target population or the full scope of actions detailed in the Action Plan for Skills. Some of the proposed characteristics for disaggregation may not be possible to collect at present, but should become possible to collect as the activities in the Skills Action Plan are implemented (e.g. public expenditure on activities detailed in the Action Plan). These disaggregated indicators should help Bulgarian authorities understand in a more detailed way what aspects of implementation have been more or less successful and adjust the implementation of activities accordingly.
Data considerations. The OECD team has indicated several data considerations for each of the indicators, namely if they are taken from existing indicators used in Bulgaria or not, and how feasible they would be to implement. New indicators that have been suggested in the report have been deemed as instrumental by the OECD to adequately monitor activities in the Action Plan for Skills that are not yet covered by existing indicators in Bulgaria. Feasible indicators are those which are relatively easy and straightforward to set up and track, while semi-feasible indicators would require some additional steps (e.g. development of a new methodology, administration of additional surveys, compiling data from difference sources) to adequately collect data. The column on data considerations also includes the data source, the collector, the frequency at which data should be collected, and whether metadata is present or still needs to be developed.
Output 2 activities linked to the indicator. The OECD team has provided cross-references between the indicators and the specific skills policy activities for a Skills Action Plan detailed in Output 2 in order to more clearly demonstrate the implementation progress the indicators are tracking.
Baseline value. Baseline values are the latest numerical values available for the indicators for the Action Plan for Skills. They may be taken from existing, reliable, accessible and internationally comparable sources, such as Eurostat, the OECD, and publicly available Bulgarian data registries.
Stakeholder feedback on monitoring mechanisms
During consultations, stakeholders were invited to review and provide their insights on the OECD's interim proposal for monitoring mechanisms, focusing on the number and types of indicators, their level of detail, and the allocation of responsibilities assigned to each indicator. Unless otherwise indicated, the suggestions made by stakeholders have been integrated into the content presented in this chapter.
Stakeholders agreed with the overall target of around 70 indicators for Bulgaria’s Skills Action Plan but suggested perhaps beginning with a subset of these indicators and adding the remaining indicators over time, as the Action Plan is implemented.
Stakeholders generally agreed with the OECD’s original proposal for the types of indicators (input, uptake/eligibility, output, outcome) to be included in the monitoring and reporting framework but noted that the indicators designated as output indicators were actually more suitable as milestones or pre-requisites upon which other types of indicators can build. Therefore, they suggested removing the category of output indicators and creating a distinction between milestones and indicators for each priority area.
While the stakeholders agreed with the OECD interim proposal to collect information on indicator type, disaggregation, data considerations, and baseline value (where available) for each proposed indicator, they noted that the OECD interim proposal to collect information on “responsibilities” for each indicator should be revised. Rather than identifying one responsible actor for each indicator, stakeholders suggested that the flow of data and information should be reflected for each indicator, including the data source and collection, and that responsibilities for the general aggregation, analysis and dissemination of data in the monitoring and reporting system be made clear. This approach should ensure transparency and efficiency in data management, with clear responsibilities at each stage of the data flow. Furthermore, as in Latvia (Box 4.1), the data sources should come from existing indicators and trusted primary or international data sources where possible to be most reliable. The stakeholders also suggested refining the list of responsible institutions after consulting with the National Statistical Institute and the identified entities, to ensure a comprehensive understanding of each institution’s role in the data lifecycle.
Additional recommendations by stakeholders related to input indicators, the connection between indicators and specific skills policy activities, and refining the description of indicators. The stakeholders emphasised that input indicators should be more holistic and include inputs beyond expenditures, such as infrastructure, facilities, equipment, and human capital, including teachers, policymakers, and data experts. They also expressed that the display of indicators for a monitoring and reporting framework would be enhanced by clearly connecting the indicators in the OECD interim proposal to the specific activities proposed for inclusion in the Skills Action Plan (in Output 2) for which they are relevant. Finally, stakeholders highlighted the importance of refining the description of indicators and developing metadata to create clear and consistent expectations about how to collect information on indicators, particularly for new indicators.
Latvia’s Education Development Guidelines (EDG) 2021-2027 represent a comprehensive, seven-year strategy for educational advancement, prioritising quality and inclusive education, and adaptability to societal and economic shifts. This framework encompasses the educational spectrum from preschool to adult learning and targets four key objectives: developing competent teachers and academic staff, enhancing educational quality, supporting individual growth, and ensuring effective system and resource management.
Latvia's approach to establishing metrics for monitoring the implementation of the EDG 2021-2027 combines using both existing and new indicators. Over 75% of these indicators are based on pre-existing data sources: five are derived from national databases and twelve are in alignment with international standards, including OECD's PISA and Eurostat. Newly introduced indicators focus on tracking progress in priority areas such as the value of higher education and the quality of vocational training, as well as monitoring the effectiveness of inclusive education initiatives. These targets are set with the European Union's benchmarks in mind. The EDG's tripartite action plan allows quick industry response and detailed monitoring but requires significant administrative effort and coherent policy development. Data quality is an ongoing concern, with efforts ongoing to address inaccuracies in educational data entry.
Note: These insights were shared as part of an expert presentation from Latvia during the Output 3 stakeholder consultation workshop.
Targets are quantifiable levels of indicators that a country or organisation aims to achieve within a given timeframe (World Bank, 2004[23]). Target setting is an inclusive and evidence-based process that is a key part of strategic planning. It serves multiple functions in monitoring and reporting. First, targets serve as guideposts for determining whether implementation progress (e.g. on expenditure, levels of uptake) is being made according to plan and help policymakers judge performance. Second, they help to establish clear expectations for stakeholders involved in implementation, whether among ministries or social partners outside of government. Lastly, targets promote transparency and accountability by providing information on results vis-à-vis the allocation and expenditure of resources (ILO, 2018[24]; USAID, 2010[25]). Targets for indicators must be:
Ambitious but feasible. As targets reflect a country’s objectives to improve current conditions, they must demonstrate increasing coverage (e.g. of an intervention) or increased allocation of resources (e.g. funding) through time. In some cases, targets may also be lower than the initial baseline value, such as when a smaller value reflects an improvement in conditions (e.g. student-to-staff ratio in childcare and preschools [YS.4], percentage of population who did not participate and did not want to participate in adult education and training [AS.6], percentage of population wanting to participate in education and training but not [AS.8]). Targets should therefore be backed by political will and aligned with objectives identified in country-level strategies, such as the National Development Programme BULGARIA 2030, the Strategic Framework for the Development of Education, Training and Learning in the Republic of Bulgaria (2021-2023), and the Higher Education Strategy, among others. Indicator targets for the Action Plan for Skills should reflect a balance between the local context in Bulgaria and the country’s performance in comparison to other countries in the EU. However, despite reflecting national ambitions, targets should also be feasible within the implementation period specified and be based on the resources (e.g. budgets, institutional capacity, personnel, facilities) that will likely be available during the period (World Bank, 2004[23]).
Developed in partnership with stakeholders. To determine what is feasible when setting targets, a joint discussion should be held local stakeholders and technical experts. Involving stakeholders in the target-setting process will contribute to greater transparency and will help gather valuable insights on what might be practical and possible to achieve in specific contexts, as stakeholders are more aware of local conditions and capacities on the ground. Stakeholders may be consulted through methods including surveys, interviews, and focus group discussions (USAID, 2021[26]).
Informed by research and previous indicator history. Reviewing current and historical trend data may be helpful in the target-setting process, as past data – whether on the same indicator or similar indicators – may help inform policy makers on what the future performance of an intervention may be. While current and past trends are not indicative of future performance, knowing whether an indicator is on an upward or downward trajectory or if it fluctuates throughout the year can help inform initial targets. It is also possible to draw on data, literature and research findings on similar programmes or strategies in other countries or across countries (e.g. OECD or EU average) in order to set realistic and well-informed targets (ILO, 2018[24]; USAID, 2021[26]).
Consistent with other indicators. The process of setting targets for performance indicators must take into consideration other relevant indicators that are systematically interconnected. For instance, inputs (e.g. expenditure) will determine the scale of uptake among participants (e.g. number of enterprises adopting employment support measures), which will then influence results at the outcome level (e.g. employment rate among women). When setting targets at higher levels of the results chain (e.g. outcomes), it is then useful to refer to lower levels (e.g. inputs) to determine what is realistic and feasible (OECD, 2020[27]).
Consistent with the nature of the intervention. When setting targets, it is important to consider the features of the intervention, such as the scope of the population that is eligible (e.g. all citizens in the country vs. only unemployed adults), and the modality for delivering the intervention (e.g. obligatory vs. voluntary). Such considerations will help determine who will be receiving the outputs and how much of the outputs could potentially be taken up (USAID, 2021[26]).
Stakeholder feedback on setting targets for indicators
While stakeholders largely concurred with the OECD's proposed principles above, they also suggested several enhancements. These included adapting indicators to the local context while maintaining a focus on European benchmarks and ensuring alignment with other national strategic documents in Bulgaria, like the Education Framework 2021-2030. They also stressed the importance of understanding the relationships among different indicators (e.g. between input and outcome indicators) and the crucial role of stakeholder involvement in determining ambitious yet realistic targets. They noted that mere consultations were insufficient, emphasising the need for deeper engagement with stakeholders as they are vital sources of information to develop effective targets.
In addition, stakeholders discussed the role of political will in the effective establishment of indicator targets and in incorporating these targets into the larger policy cycle. It is essential that targets are regularly reviewed at a political level to ensure they align with political priorities, are practical and realistic, are made feasible through the allocation of dedicated funding and a supportive policy environment and are consistent with targets set across government departments. Without strong political commitment, even well-designed indicators and targets could fall short of their potential impact, and therefore it is important to include discussion of political will in establishing indicator targets.
Specific milestones and indicators are displayed below for each of the four priority areas: developing youth skills, improving adult skills, using skills effectively and strengthening the governance of the skills system.
Milestones and indicators for priority area 1: Developing youth skills
The milestones and indicators for developing youth skills aim to capture the degree to which Bulgaria is increasing the skills level of the next generation by improving the development and acquisition of skills from early childhood and through post-secondary vocational and higher education. These milestones and indicators are designed to track progress in improving early childhood development, participation and outcomes, ensuring that curriculum reform and assessment practices improve students’ skills, improving equity at all stages in education, developing a highly skilled teaching workforce, and making vocational and higher education more responsive to labour market needs.
Six milestones have been identified for the priority area of developing youth skills, spanning across the short- and medium-term (Box 4.2). The milestones in the short-term focus on developing systems and institutions to help improve and track progress (e.g. by establishing indicators) over time. The two milestones for the medium term both involve creating more systemic changes within academic institutions, which will likely take more time to actualise.
The reference numbers here and in Box 4.3, Box 4.4 and Box 4.5, as well as in Table 4.1, Table 4.2, Table 4.3 and Table 4.4 are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator YS.1) and by letters to refer to milestones (e.g. Milestone YS.A) in order to distinguish between the two.
Short term (<2 years)
Milestone YS.A: Establish centres providing Early Childhood Development (ECD) services for children from vulnerable groups and their parents, including educational, linguistic, social and health services. These centres would provide general and additional support for young children beyond those provided in ECEC classrooms and groups, such as providing tailored outreach and information to parents from vulnerable backgrounds and engaging all parents of young children in ECEC more effectively, including through information, training, parent-teacher engagement, and by offering psychological and pedagogical support, and access to specialists in health and social services (Activity 1.1.2.1).
Milestone YS.B: Operationalise the National Quality Framework for early childhood education and care (ECEC) developed in 2022, including developing quality assurance indicators and benchmarks to enable periodic monitoring and analysis (Activity 1.2.2.1).
Milestone YS.C: Develop and disseminate an annual, school-based formative appraisal to generate evidence on teachers’, staff’s and principals’ training needs (Activity 4.2.1.1).
Milestone YS.D: Introduce / develop indicators to evaluate the effectiveness of continuous professional development (CPD) programmes for teachers (e.g. outcome indicators such as newly acquired knowledge and skills; improved quality of student-teacher interaction based on teachers’ surveys and teacher appraisals; process indicators such as material, equipment, and facilities; and number of training hours delivered) (Activity 4.2.2.2).
Medium term (2-4 years)
Milestone YS.E: Establish a more selective and comprehensive admission system for initial teacher education (ITE) to ensure the suitability and quality of teaching candidates. Create a working group gathering university representatives to encourage ITE providers to establish a common minimum threshold score for ITE admission based on State Matura scores and additional criteria for assessing ITE applicants (Activities 4.1.1.1 and 4.1.1.2).
Milestone YS.F: Develop a system for automatically generating user-friendly labour market information (LMI) indicators annually for higher education institutions (e.g. graduate labour market outcomes by degree programme and level of study), to inform their course offerings (Activity 5.2.1.1).
Note: All activity numbers refer to the Output 2 report.
Source: OECD (2024[28]), Guidance on the Implementation of a Whole-of-government Action Plan for Skills in Bulgaria,
In total, 18 indicators have been proposed for the priority area on developing youth skills: seven input indicators, six uptake indicators, and five outcome indicators. Seven of the indicators are taken from existing indicators used in Bulgaria or easily accessible from international data sources while eleven are new. Of the new indicators, the OECD has identified three as being semi-feasible as they would require additional methodological steps to collect, whereas the remainder were identified as feasible as they should be quite possible to collect from programme data within municipalities and MES. Because MES registers are not publicly available, the OECD proposes that the Strategic Development and Information Policy Directorate (SDIPD) within MES gather all of the relevant indicator data within MES and transmit the data to the Inter-institutional Data and Evidence Centre (DEC) (see also Figure 4.4).
Note: The reference numbers here and in Table 4.2, Table 4.3, Table 4.4, as well as in Box 4.2, Box 4.3, Box 4.4 and Box 4.5, are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator YS.1) and by letters to refer to milestones (e.g. Milestone YS.A) in order to distinguish between the two.
1. Given that government institutions (including MES) lack a mechanism to collect data from municipalities and monitor municipal funding for ECEC and provision of support for families, collecting this indicator would likely necessitate additional efforts to establish a mechanism for data collection from municipalities such as developing a methodology and instruments for data collection (e.g. questionnaires for municipal administrations).
2. MES already collects data on the construction/expansion of preschools and kindergartens under the “Programme for construction, extension, upgrade and reconstruction of nurseries, kindergartens and schools 2024 ‑2026”.
3. Though somewhat removed temporally from early childhood education and care, Indicator YS.7 could potentially also be used as a proxy outcome indicator under Objective 1, Policy Action 1.2: “Improving the quality of ECEC”.
4. This new feasible indicator is proposed as it can be collected frequently, on an annual basis. However, given that stakeholders have expressed a preference for internationally-recognised indicators, an additional measure of this indicator can be collected every five years from the OECD TALIS database to supplement the annual data from MES.
5. In order to collect data and some of the proposed levels of disaggregation (e.g. size of firm) additional steps might need to be taken to enable data collection, as there is currently no method in please to easily track this.
6. NUTS is a hierarchical system used to divide regions into three levels: NUTS 1 are major socio-economic regions, NUTS 2 are basic regions for the application of regional policies, and NUTS 3 are small regions for specific diagnoses.
Stakeholder feedback on milestones and indicators for developing youth skills
During consultations stakeholders identified three areas for improvement of the proposed indicators related to developing youth skills. Unless stated otherwise, these suggestions have been incorporated into Table 4.1. First, stakeholders noted that there is a need to remove or adjust indicators that relate to legal mandates or are incomplete and therefore would not show progress or changes over time. Second, some indicators need to be made more precise, whether in their wording, data source or collector, or their Bulgarian translation. Third, some indicators need to be added or assigned a different indicator type to accurately reflect policy goals.
Specific stakeholder feedback included:
Reassign the responsibility of the indicator on expenditure to increase participation in Early Childhood Education and Care (ECEC) (YS.1) from MES to municipalities. OECD kept MES in addition to adding municipalities, as MES has data specifically on the construction and expansion of kindergartens, which is included under YS.1.
Some indicators where municipalities have been identified as the data source (YS.1, YS.2) should be changed to be “semi-feasible” new indicators given that additional steps will have to be taken to facilitate data collection from municipalities that is not currently in place. For others (YS.5, YS.6, YS.8, YS.9, YS.11, YS.14), the source should be changed to MES and/or REDs.
Add a disaggregation by municipality or type of settlement for the indicator on children enrolled in ECEC (YS.3).
Move the indicator on staff-to-student ratio (YS.4) from an outcome indicator to an uptake indicator. The OECD ultimately decided that this acts as more of an input indicator.
Specify source of information for the indicator on school readiness among children. Alternatively, other stakeholders reflected that this is not so feasible to collect and that current assessments of children entering primary education are for the specific purpose of identifying learning difficulties. The OECD decided to remove this indicator.
Redefine or delete the indicator on the PISA mean scores (YS.7). At the moment it covers only part of the student population, making it unsuitable as an indicator. While the OECD agrees this indicator does not cover the entire student population, it has been kept as a proxy indicator of student performance.
Add an additional indicator that tracks the number of students from lower socio-economic backgrounds achieving good results using PISA indicators. The OECD has added YS.10.
Delete the indicator on hours ITE students spend in practical training, as this is mandated and would not be informative.
Ensure that teachers at all levels of education (including higher education) are included (YS.11, YS.12).
Revise the translations into Bulgarian of indicators on support for work-based learning (YS.13, YS.14). This will be noted for future implementation.
Ministry of Finance (MoF) should be added to the data source for expenditure on support for WBL (YS.13) and it should be made a semi-feasible indicator because collecting data at some of the proposed levels of disaggregation (e.g. size of firm) may require additional methodological steps.
The data source for expenditure on steering higher education students to prioritised fields (YS.16) should be the higher education directorate in MES, not the higher education institutions themselves.
Remove the following disaggregation categories from the indicator on employment rate of HE graduates (YS.18) it is not possible to disaggregate in these ways based on the Eurostat data: by industry, by area of education/study, by municipality. The OECD has applied the same to YS.15 as well.
Milestones and indicators for priority area 2: Improving adult skills
The milestones and indicators for developing adult skills are intended to track progress in increasing the frequency and quality of lifelong learning through adulthood in Bulgaria. Some of the milestones and indicators capture changes in lifelong learning over time as they relate to individual adults, others as they relate to enterprises, and others still as they relate to adult education and training providers. They include input, uptake and outcome indicators related to increasing the motivation among adults and employers to participate in adult learning, making education and training more flexible and accessible for adults and employers, and improving the quality and relevance of education and training for adults and employers.
There are four milestones within the priority area of improving adult skills, three of which are meant to be realised in the short term and one in the medium term. Two of the milestones for the short term are related to improving Bulgaria’s process for the recognition of prior learning (RPL), which can help adults in Bulgaria access jobs, trainings, and additional qualifications based on the skills they have already acquired informally. The other milestone designated for the short term aims to help Bulgarian officials understand at a more systemic level which skills and areas of adult education and training should be prioritised and incentivised through skills policy in order to meet the needs of the labour market and the demand for skills among employers. In the medium term, there is also a milestone to create greater transparency and credibility in institutions providing adult education and training by developing a process for external, objective assessment.
Short term (<2 years)
Milestone AS.A: Reform and streamline the recognition of prior learning (RPL) procedure in Bulgaria. This could be done by simplifying and shortening the administrative process for RPL both for individuals and for RPL providers (Activity 7.2.1.1).
Milestone AS.B: Alter the design of RPL certificates. The certificates should be changed to match professional qualification certificates to make them better understood and more accepted by educational and training institutions and employers (Activity 7.2.1.2).
Milestone AS.C: Develop and regularly update a list of prioritised skills and areas of adult education and training of national and/or regional importance informed by the skills assessment and anticipation, current employment forecasting projections and the list of “protected specialities” and “priority professional fields” (these lists are already used to determine financial incentives in secondary education) (Activity 8.2.2.1).
Medium term (2-4 years)
Milestone AS.D: Establish an ex post assessment process of all adult education and training providers. This assessment process should be required for all adult learning providers, including those only providing partial qualifications. The assessments should be conducted by an external evaluation team (Activity 8.1.2.1).
Note: All activity numbers refer to the Output 2 report. The reference numbers here and in Box 4.2, Box 4.4 and Box 4.5 as well as in Table 4.1, Table 4.2, Table 4.3 and Table 4.4 are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator AS.1) and by letters to refer to milestones (e.g. Milestone AS.A) in order to distinguish between the two.
Source: OECD (2024[28]), Guidance on the Implementation of a Whole-of-government Action Plan for Skills in Bulgaria,
For the priority area on improving adult skills, 16 indicators have been selected and displayed below (Table 4.2). These include five input indicators, six uptake indicators and five outcome indicators. Five of the indicators are existing indicators, while the remainder are new but feasible. Much of the data is expected to come from the National Agency for Vocational Education and Training (NAVET) and MLSP/NEA, with the rest coming from MES and international-standard surveys such as the Adult Education Survey, the Labour Force Survey and the Continuing Vocational Training Survey.
Note: The reference numbers here and in Table 4.1, Table 4.3 and Table 4.4, as well as in Box 4.2, Box 4.3, Box 4.4 and Box 4.5, are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator AS.1) and by letters to refer to milestones (e.g. Milestone AS.A) in order to distinguish between the two.
1. MLSP is involved in the development of an online adult education platform under the “Provision of Training for Digital Skills and Creation of a Platform for Adult Education” project under the ESF+ Human Resources Development Programme, and therefore will likely have relevant information to contribute to this indicator once the platform is operational (currently forecasted for 2026).
2. Alternatively or additionally, Bulgaria could consider tracking the total number of adults participating in adult education and training courses.
Stakeholder feedback on milestones and indicators for improving adult skills
For indicators related to developing adult skills in Bulgaria, stakeholders highlighted concerns about feasibility of data due to challenges in quality and/or collection. Stakeholders also requested clarity on concepts mentioned in the indicators such as “pilot sector training funds” and key stakeholders who should be mentioned like national accreditation agencies. In addition, stakeholder discussions also covered the degree to which proposed indicators capture progress in the transversality of vocational education, recognition of prior learning, quality of trainee evaluations and international co‑operation on skills development. Unless otherwise indicated, the stakeholder suggestions have been reflected in Table 4.2.
Specific stakeholder feedback included:
MES should be removed as responsible actors for the indicators on improving motivation of individuals to participate in AET (AS.1, AS.3) and an indicator related to provision of flexible and accessible AET (AS.7), but MLSP and NEA should be added as they are more involved in these areas.
MLSP and NEA should replace MEI as the data source for the indicators on improving the motivation of employers to encourage/provide AET to workers (AS.2, AS.4) as MEI does not fund or track funding/uptake in this area.
Indicators on incentives for individuals and incentives for employers need to be separated as indicators for these different target audiences had different data sources and methodologies for data collection.
Delete the following indicators: the indicator on expenditure on awareness raising activities for flexible education and training courses as it lacks reliable sources of information; the indicator on eligible individuals for adult education financial benefits as it is redundant given that all people of age 18 or over are eligible; the indicator on the percentage of adults participating in AET measures and reporting positive outcomes as it raises methodological concerns.
Expand the indicator on AET institutions and employers that use/promote more flexible offers to include universities and their recognition of courses from VET schools. In order to limit the total number of indicators, the OECD decided to include only an indicator on the number and percentage of adults participating in flexible courses (AS.7), and not another on the offer.
Credibility concerns were raised regarding the indicators on adults who assess the quality of adult learning and CVET as “good” or “very good” and the change in employees that answered ‘High’ when asked ‘Perceived value of training’ on the European Company Survey. The OECD removed these indicators for now. However, NAVET also commented that a recent update to its information system included creating a module for measuring and presenting the degree of satisfaction of trained individuals in VTCs and their employers for the purpose of tracking quality of training provided. In the future, Bulgaria may consider adding an indicator based on this module.
Remove disaggregation categories for the indicator on adults with basic education or lower participating in education and training (AS.12), as this is not possible with the data available in NSI.
For the indicator on adults participating in education and training (AS.15), remove the disaggregation by type of adult education as this is not possible if the source is the Labour Force Survey (LFS) and specify which socio-demographic characteristics are possible to track with data from LFS.
An indicator on adult participation in education and training in the past 12 months (Eurostat) can be added instead of or in addition to the present indicator (AS.15). The OECD has chosen to keep indicator AS.15 as the data for this indicator is updated more frequently in Eurostat.
In general, it is not feasible to provide disaggregated data by firm size for EU funded projects, as the assessment of the size of enterprise is not mandatory for all funded operations due to the high administrative burden. The OECD has removed disaggregation by firm size from indicators related to improving adult skills due to this feedback. However, as this can provide valuable information, the OECD would encourage Bulgaria to revisit the feasibility of collecting data according to this disaggregation in the future, where possible.
Milestones and indicators for priority area 3: Using skills effectively
The milestones and indicators for using skills effectively aim to describe how Bulgaria uses the full economic and social value from its investments in developing the skills of its youth and adults. They cover the inputs, uptake and outcomes of activities that aim to activate of skills of vulnerable groups in the labour market, foster return migration and skilled immigration to Bulgaria, and support enterprises to utilise their workers’ skills more effectively.
Two milestones have been identified for the priority area on using skills effectively, both of which are set over the medium term and involve activities under Objective 10: Fostering return migration and skilled immigration to Bulgaria (Box 4.4). Both milestones lay the foundations for the promotion of return migration and skilled immigration to the country, specifically through the development of a roadmap and an online platform. The achievements of these milestones would allow Bulgarian authorities to pursue other related activities, such as assigning clear responsibilities and adequate resources for return migration and skilled immigration, and the provision of more support services (e.g. access to information, networking events) to emigrants and skilled immigrants.
Medium term (2-4 years)
Milestone USE.A: Co-ordinate between ministries and social partners to develop a comprehensive and ambitious roadmap for return migration and skilled immigration. The roadmap should explicitly articulate the potential benefits of and a positive vision for return migration and skilled immigration; set ambitious and concrete goals and targets for return migration and skilled immigration (including for international students in Bulgaria); and align with Bulgaria’s current and anticipated skills needs (Activity 10.1.1.1)
Milestone USE.B: Develop an online platform targeting Bulgarian workers abroad and potential skilled immigrants. The platform should promote relocation to Bulgaria, provide relevant information (e.g. on Blue Cards and Startup Visas), foster communications and allow for the development of online communities (Activity 10.2.1.1).
Note: All activity numbers refer to the Output 2 report. The reference numbers here and in Box 4.2, Box 4.3 and Box 4.5, as well as in Table 4.1, Table 4.2, Table 4.3 and Table 4.4 are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator USE.1) and by letters to refer to milestones (e.g. Milestone USE.A) in order to distinguish between the two.
Source: OECD (2024[28]), Guidance on the Implementation of a Whole-of-government Action Plan for Skills in Bulgaria,
A total of 14 indicators have been identified for the priority area on using skills effectively – six of which are inputs, five of which are uptake, and three of which are outcomes (Table 4.3). Eight indicators come from existing sources, such as databases from the OECD and Eurostat, the European Company Survey, the Labour Force Survey, registries of the National Employment Agency and the National Social Security Institute, and programmes implemented by the Ministry of Innovation and Growth (MIG). While six out of 14 indicators are entirely new, a significant majority of them are feasible, as they may be directly collected from programme data (e.g. expenditure) or involve collecting data directly from the source (e.g. enterprises).
Note: The reference numbers here Table 4.1, Table 4.2 and Table 4.4 in, as well as in Box 4.2, Box 4.3, Box 4.4 and Box 4.5, are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator USE.1) and by letters to refer to milestones (e.g. Milestone USE.A) in order to distinguish between the two.
1. Currently there are only specific NEA mediators for the Roma and youth populations. However, NEA may consider adding mediators, activators and caseworkers to work with additional vulnerable population groups, as proposed in Activity 9.1.2 in Output 2 (e.g. low-skilled adults, parents unemployed after parental leave) and, if so, should monitor the number and proportion of NEA mediators, activators and caseworkers working specifically with these groups as well.
2. Active labour market policies (ALMP) refer to policies that aim to provide more individuals with access to the labour force and good jobs. ALMPs may be broadly classified into three types: policies that strengthen individuals’ motivation to work, policies that improve people’s labour-supply capabilities, and policies that expand opportunities through intermediation and improved labour demand (OECD, 2022[46]). See specific examples of relevant ALMP under Activity 9.1.3 in Output 2.
3. Employment outcomes for youth and long-term unemployed are monitored within the EU Council Recommendation of 30 October 2020 on A Bridge to Jobs – Reinforcing the Youth Guarantee and within the EU Council Recommendation of 15 February 2016 on the integration of the long-term unemployed into the labour market. Data collection is done annually by the European Commission through the Employment Committee Indicators Group on the basis of administrative data from the NEA.
4. While there is no universally agreed-upon definition of HPWPs, the OECD Centre for Skills often refers to HPWPs as practices that aim to use workers’ skills more effectively, such as by promoting flexibility in working time and autonomy in task execution, teamwork and information sharing, training and development, and the provision of benefits, career progression and performance management services (OECD, 2023[47]).
5. Until Bulgaria participates in the OECD’s Survey of Adult Skills (PIAAC), related indicators from the European Company Survey may be used.
6. Bulgarian authorities may determine whether these activities from the Action Plan for Skills may be considered as eligible entries for the expenditure and investment types included in the proxy indicator from Eurostat “Gross expenditure on R&D (GERD) by sector of performance and type of expenditure) (see here). This proxy indicator is currently disaggregated into current expenditure (labour cost, other expenditure, other expenditure - external R&D personnel, other expenditure (excluding R&D personnel) and capital expenditure (land and buildings, instruments and equipment, computer software, other intellectual property products).
Stakeholder feedback on milestones and indicators for using skills effectively
Stakeholders provided feedback that a number of indicators for using skills effectively should be deleted or revised in order to streamline the indicators and make them more precise and relevant. In addition, they suggested adaptations to improve the accuracy and applicability of the indicators. Unless otherwise stated the suggested changes have been incorporated in Table 4.3.
Specific stakeholder feedback included:
Elaborate on the indicator on active labour market policies (ALMP) (USE.2) by further explaining the meaning of ALMP, adding disaggregation by professional qualification and changing disaggregation by ‘type of skills’ to ‘key competences’.
Add vulnerable groups to the disaggregation of the indicator on NEA caseworker support (USE.1) and delete disaggregation by ‘type of support for employment’ for the indicator on expenditure on incentives for enterprises to adopt employment support measures for parents (USE.5). The OECD did not delete the latter, as it was deemed critical to the value of the indicator.
Revise or delete the following indicators due to the following concerns: it is unclear what kind of activity is measured through the indicator on expenditure on support services for emigrants/immigrants (USE.8), and NEA should be added to responsible entities; it is unclear how the indicators on registered unemployed persons receiving career counselling from the NEA (USE.3), difference in work hours between men and women (deleted by OECD) and employers who report implementing HPWP (USE.10) relate to the previous indicators in this objective and why they are relevant. The OECD has adapted and clarified the relevant indicators.
Revise indicators USE.3, and USE.11. Address translation issues in the indicator on awareness raising of HPWP in enterprises. Delete NEA as the institution responsible for the indicators on unemployed persons registered with the NEA over the total number of out of work adults (USE.3) and on the share of employers reporting awareness of HPWP, as they do not have the relevant information. Add MES as the responsible institution for the indicator on unemployed persons participating in training. The indicator on the share of employers reporting awareness of HPWP was deleted, as well as the indicator on unemployed persons participating in training. The NEA remains one of the responsible parties for the USE.3 indicator as they are responsible for providing the number of persons registered with the NEA. However, they are not responsible for providing other data required for the indicator.
Delete the indicator on raising awareness among enterprises on how to use skills more effectively, as MIG noted that activities to raise awareness and exchange good practices among enterprises regarding HPWPs are not included or envisaged under the ministry’s current programmes, including the Programme Research, Innovation and Digitalisation for Smart Transformation (PRIDST) 2021-2027 and the Programme Competitiveness and Innovation in Enterprises (PCIE) 2021-2027. The OECD has decided to delete the indicator. Expenditure for awareness-raising activities under Policy Action 11.1: Promoting effective skills use and the adoption of high-performance workplace practices (HPWPs) in Bulgarian workplaces, including in SMEs, may be considered in Output 4 of the project, which focuses on an outline for a communication campaign to raise awareness of the national skills strategy for Bulgaria.
Add an indicator on the number of employees in SMEs trained for skills in the field of Industry 4.0, which is currently being collected by the MIG under Priority 1: Innovation and Growth of PCIE 2021‑2027. The OECD has added this indicator as USE.13.
Indicate the NSI as the provider of data on the number of enterprises that have implemented an innovation (USE.14). The OECD has identified NSI as the source of data, and revised the indicator to include the number of enterprises (in addition to the share).
Merge the indicators on the roadmap for return migration and skilled immigration and for an online platform targeting Bulgarian workers abroad and potential skilled immigrants. The OECD has designated these as milestones rather than indicators.
Remove disaggregation by firm size for EU funded projects, as this is not mandatory for all funded operations due to the high administrative burden. The OECD has also removed disaggregation by firm size for Indicator USE.11, as data on gross domestic expenditure on R&D is not disaggregated by firm size on Eurostat.
Milestones and indicators for priority area 4: Strengthening the governance of the skills system
The milestones and indicators for strengthening the governance of the skills system aim to describe how effectively actors in Bulgaria can successfully implement policies to develop and use people’s skills. They cover the inputs, uptake, and outcomes of activities that aim to develop a whole-of-government and stakeholder-inclusive approach to skills policies, build and better use evidence in skills development and use, and ensure well-targeted and sustainable financing of skills policies.
Seven milestones have been identified for the priority area on strengthening the governance of the skills system (Box 4.5). The five milestones to be achieved at the short-term involve various activities, including those that enhance the co-operation of authorities and stakeholders in Bulgaria in the implementation of skills policies. These include the creation of a Skills Policy Council and the creation of the Inter-institutional Data and Evidence Centre (DEC). While the latter was originally identified as an activity to be implemented over the medium term in Output 2, the OECD suggests this be revised and considered for implementation over the short-term, given the central role that the DEC plays in collecting, quality assuring, and analysing monitoring data for the Action Plan for Skills.
The two milestones to be achieved over the medium term involve strengthening Bulgaria’s capacity to better use data for skills policymaking and promote a more stakeholder-inclusive approach to skills policies. These are through the development of a methodology for skills anticipation and the expansion of the remit of the CCVET. The achievements of these milestones would allow Bulgarian authorities to pursue related activities, such as establishing networks to improve data collection through the initiative of the DEC.
Short term (<2 years)
Milestone GOV.A: Create a Skills Policy Council for Bulgaria comprised of ministries, agencies, regional and municipal representatives, and social partners with a stake in skills policies. The council should convene regularly to oversee the skills system and ensure the achievement of Bulgaria’s skills policy objectives, for example, by monitoring and reporting on skills policy implementation and outcomes. This should include oversight of existing skills bodies (e.g. NAVET) and those that are planned (e.g. sectoral skills councils). The Skills Policy Council should also oversee and publicly report on initiatives to improve stakeholder engagement, skills need information, policy evidence, resource allocation and cost sharing, and any other measures defined in Bulgaria’s proposed action plan for skills (Activity 12.1.1.1).
Milestone GOV.B: Create an Inter-institutional Data and Evidence Centre to integrate, undertake and/or commission primary and secondary data collection, analysis, and evaluation for skills policy. The data and evidence centre should be staffed with a small team that is supported with secondments from the ministries involved in skills policy and should be tasked with improving the quality, accessibility and use of data and evaluation evidence for all key stakeholders (Activity 13.2.1.1).
Milestone GOV.C: Develop a tripartite agreement to define the division of responsibility for funding skills development. The agreement should clarify where and how government (at the national and municipal levels), employers and individuals should co-invest in education and training, in order to achieve Bulgaria’s new targets for increased expenditure across the skills system. The agreement should also seek to articulate how stable funding will be ensured over time, even with the use of European project-based funding (Activity 14.2.1.1).
Milestone GOV.D: Broaden the membership of the CCVET to include social partners, academic experts, education and training institutions, and agencies from across the whole skills system (Activity 12.2.1.1).
Milestone GOV.E: Expand the membership of SSCs to include not only MES but several ministries with responsibilities for skills, representatives from the subnational level (e.g. dedicated members or committees representing particular geographical regions), and representatives of the social partners, sectoral and branch organisations and education and training providers (Activity 12.2.2.1).
Medium term (2-4 years)
Milestone GOV.F: Develop a methodology for skills anticipation that consolidates Bulgaria’s SAA methods. This should include: defining which data and information key skills actors need from SAA initiatives; expanding existing quantitative tools to provide more sectoral, occupational, educational, demographic and regional insights; and developing qualitative insights from consultation with employers and potentially from foresight techniques (Activity 13.1.1.1).
Milestone GOV.G: Expand the remit of the CCVET to cover tertiary education and adult learning in addition to VET, and to support and advise the new Skills Policy Council (Activity 12.2.1.2).
Note: All activity numbers refer to the Output 2 report. The reference numbers here and in Box 4.2, Box 4.3 and Box 4.4, as well as in Table 4.1, Table 4.2, Table 4.3 and Table 4.4 are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator GOV.1) and by letters to refer to milestones (e.g. Milestone GOV.A) in order to distinguish between the two.
Source: OECD (2024[28]), Guidance on the Implementation of a Whole-of-government Action Plan for Skills in Bulgaria,
A total of nine indicators have been identified for the priority area on strengthening the governance of the skills system – three of which are inputs, one of which is an uptake indicator, and three of which are outcomes (Table 4.4). One of the indicators in this priority area comes from existing sources, such as data collected by the MES and stored in the information system for management and monitoring of EU funds in Bulgaria. While the rest of the indicators are new, most of them are feasible, as they may be directly collected from programme data (e.g. expenditure in Indicators GOV.1 and GOV.5, number of skills policy-related bilateral activities in GOV.2).
Note: The reference numbers here and in Table 4.1, Table 4.2 and Table 4.3 as well as in Box 4.2, Box 4.3, Box 4.4 and Box 4.5, are formed based on the initials of the priority area to which the indicator belongs: YS for developing youth skills, AS for improving adult skills, USE for using skills effectively, and GOV for strengthening the governance of the skills system. In each priority area, these initials are followed by numbers to refer to indicators (e.g. Indicator GOV.1) and by letters to refer to milestones (e.g. Milestone GOV.A) in order to distinguish between the two.
1. As stakeholders have expressed the importance of financing specifically earmarked for the monitoring and reporting framework, this expenditure may also be included in the disaggregation of this indicator.
2. The share of stakeholders may refer to the total number of sectors, stakeholder types and regions targeted as participants or beneficiaries in the Action Plan for Skills.
3. Skills assessment and anticipation (SAA) activities are tools to generate information about the current and future skills needs of the labour market, as well as the available supply of skills in the country. They are used to inform policymaking in various sectors, including employment, education and training, and migration. SAA information may also be used by enterprises to promote certain skills among their workers and inform collective bargaining processes, or by individuals to make informed educational or career choices (OECD, 2018[50]).
Stakeholder feedback on milestones and indicators for strengthening the governance of the skills system
For indicators related to strengthening the governance of the skills system, stakeholders identified the necessity for simplification and enhanced relevance of the proposed indicators. They suggested merging similar indicators for streamlined efficiency and proposed the deletion or refinement of others to ensure alignment with policy goals. Unless otherwise indicated, the stakeholder suggestions have been incorporated into Table 4.4.
Specific stakeholder feedback included:
Merge the indicators on expenditure on engaging stakeholders in skills policymaking and skills needs assessment and merge the indicators on Sectoral Skills Councils and involving stakeholders in skills needs assessments. The first two have been merged in Indicator GOV.1, the latter was simplified in Indicator GOV.3.
Indicators on creating an Inter-institutional Data and Evidence Centre (DEC), creating a regularly convened Skills Policy Council and establishing bilateral activities between ministries should all be milestones/pre-conditions rather than indicators.
Highlight the role of NSI and MES for the indicator on expenditure on DEC activities. This has been deleted or incorporated into the milestone on establishing the DEC.
Connect the indicator on users of skills assessment information to relevant skills activities. This has been done for all indicators by adding a column that cross-references specific skills activities from Output 2.
Define the indicator on a tripartite agreement for skills development funding more precisely. Specify the parties, objectives and financial incentives of/for the agreement. This has been done in the milestone section above, as this indicator was changed to a milestone.
Redefine or delete the indicator on stakeholders perceiving quality of whole-of-government collaboration on skills policy as high and the indicator on non-governmental stakeholders attending meetings of national bodies for skills policy as this is unclear in its definition and scope, and its relevance is questionable. The latter was adapted into the broader GOV.5 indicator.
Delete or move the indicator on government expenditure on education and training (GOV.9). Include the indicator in the contextual analysis of the strategy. The OECD kept this indicator as it was deemed the most useful outcome indicator for the relevant policy action.
Principles for establishing a monitoring and reporting framework
Monitoring and reporting are key steps in the policy cycle (Figure 4.2), as they provide real-time, accurate, and reliable information about the progress of implementation and the uptake of policy measures among target groups. Moreover, developing a monitoring and reporting framework can improve policy outcomes and public trust by enhancing accountability for the use of public funds, improving understanding of the link between public interventions and their results, and facilitating policy learning over time to make future policies more efficient and effective (OECD, 2023[51]).
However, despite the importance of monitoring and reporting, many countries may face a number of challenges when it comes to establishing an effective framework. These include, among others, limited technical capacity, low quality of available data, the lack of an integrated evidence management system, challenges with whole-of-government co‑operation, and ineffective use of reporting findings (OECD, 2020[27]). In Bulgaria, there is currently no established framework for strategic planning that lays out clear standards, rules and procedures for the preparation, monitoring and reporting of strategic documents. To overcome this challenge, the OECD has proposed a list of six principles and refined it with stakeholders, which could guide Bulgarian authorities in establishing an effective monitoring and reporting framework for the Action Plan for Skills. These principles include the following:
1. Employing a systematic approach for monitoring policy inputs, uptake, outcomes, and impact
To ensure that resources are used responsibly, implemented activities align with policy plans, and targeted beneficiaries are reached, a structured and streamlined monitoring and reporting framework needs to be established. The framework must be established based on indicators that capture the degree to which interventions are on track to or have resulted in desired outcomes and impact at every stage of the policy cycle (e.g. design and funding of intervention, uptake of intervention, etc.). The monitoring and reporting framework must capture the different indicator levels of the results chain (i.e. inputs, uptake, outcomes, impact), with each one having well-defined targets that would allow for a systematic and transparent approach to measuring progress. The framework also needs to determine data collection responsibilities, sources and timelines that are coherent with the policy being implemented (ILO, 2018[52]).
The current Output 3 report aims to implement this principle in practice in Bulgaria by outlining a plan for establishing the monitoring and reporting system of the Action Plan for Skills. The report, which was developed in partnership with relevant authorities and social partners in Bulgaria, aims to provide guidance on how to regularly and systematically collect high-quality data on key skills indicators, in order to help policymakers ensure the effective implementation of the Action Plan.
2. Enhancing the quality and comprehensive collection of useful indicators for assessing impact
It is crucial to ensure that indicators in the monitoring and reporting framework are comprehensive and of high quality, as they serve as measurable and tangible signs of progress or achievement (Shapiro, 2003[53]). While there are no strict rules in selecting indicators, it is commonly advised to select those – or develop new ones – that are specific, measurable, attributable, relevant and timebound. Identifying indicators early in the establishment of the monitoring and reporting framework can allow policymakers to identify a variety of data sources that may yield the highest quality data, which is key to measuring the impact of the policy (ILO, 2018[52]; OECD, 2020[27]).
Stakeholders expressed concerns regarding quality assurance of the data used for indicators for monitoring progress on skills policy in Bulgaria. They highlighted that many of the indicators that were initially proposed rely on secondary data sources for which accuracy is challenging to verify. While countries across the OECD have adopted various mechanisms to promote data quality, such as a policy or legal framework, guidelines for policy evaluation across government, competence requirements for evaluators, peer reviews (internal/external) of evaluations, and systematic and meta-evaluations, none of these mechanisms currently exist in Bulgaria (OECD, 2020[27]).
To overcome these barriers, stakeholders recommended incorporating primary data sources instead, where possible. Such primary sources have been identified in the “Data considerations” column of the indicators listed in Table 4.1,Table 4.2, Table 4.3, Table 4.4, including primary sources that come directly from programme data and information reported by partners to monitor the Action Plan for Skills. However, one possible constraint in the use of primary data sources is the lack of capacity among policy implementers and partners to report data in a complete and accurate manner, as illustrated in the case of Latvia. Some OECD countries have implemented several initiatives to overcome such technical constraints, including interactive forums (e.g. roundtables, policy dialogues) that bring together policymakers and researchers to exchange knowledge on how to collect and review evidence (OECD, 2020[54]).
In addition to primary data sources, stakeholders also noted that using internationally recognised data sources, such as PISA and TALIS, and interlinking various data registers could help to ensure higher quality of the data used for indicators. Further suggestions included building in a stage of quality assurance of the data into the larger monitoring and reporting framework and enhancing the quality and reliability of educational data in particular by developing a national framework for monitoring and evaluating educational quality in Bulgaria.
In addition to incorporating primary data sources in the framework, Bulgaria may also build on existing indicators from internationally acknowledged data sources, such as OECD surveys or Eurostat databases, as these are credibly, verified, and guided by rigorous data collection methodologies. In Bulgaria, the main agency responsible for co‑ordinating data collection and validation for these international surveys is the National Statistical Institute, which has a record of producing timely and high-quality data. However, several other ministries and regional actors that are involved in the monitoring of the Action Plan for Skills have raised issues regarding the lack of technical capacity to quality-assure and analyse data by themselves. This suggests that additional capacity building activities (including during Output 5 of this project, see Figure 4.1) and guidance will be needed to ensure that monitoring and reporting activities are implemented effectively.
3. Establishing regular and realistic monitoring and reporting requirements
When allocating responsibilities and identifying the methodology for data collection, the monitoring and reporting framework should consider the resources available across ministries and different levels of government. It is crucial to design the monitoring and reporting framework in line with existing policy frameworks that provide principles and strategic direction for monitoring and evaluation. For instance, this may include identifying and working with data that is already being regularly collected, aligning new data collection responsibilities with those that already exist across government, and making use of existing management information systems when possible (OECD, 2020[27]).
The monitoring and reporting framework also needs to determine coherent methods for data collection and the flow of data through the monitoring and reporting process (ILO, 2018[52]). For instance, in practice, this means that responsibilities for collecting data on indicators should consider the institutions or ministries responsible for implementing the activities outlined in the policy, as well as the frequency or rate at which they are expected to implement these activities. Allocating responsibilities in this manner could help ensure that monitoring and reporting data are regularly updated as activities are implemented by ministries and agencies that know best how these activities are unfolding on the ground. In keeping with these principles, proposals for the frequency with which indicator data should be collected is included in the data considerations column in the tables presented above.
In addition to regular data collection for indicators, the OECD recommends that three more comprehensive evaluation reports on the implementation of the Skills Action Plan be conducted at the end of each implementation time period proposed in Output 2 and reflected in the discussion of milestones: a report on implementation progress two years after the adoption of the Skills Action Plan (short-term), after four years (medium-term), and after six years (long-term).
4. Developing a governance framework for the flow of data and information within the monitoring and reporting framework
To ensure that data is collected on time and that the data collected meets standards, it is crucial to have a governance framework when monitoring and reporting on policies. The governance framework encompasses multiple aspects, including fostering the participation of relevant institutions and stakeholders, promoting the efficiency of the flow of data or information, and ensuring adequate financing for the system.
As stakeholders noted, co‑ordination mechanisms must be clear, with specific responsibilities for data collection, validation, analysis and dissemination assigned early on, in order to avoid the duplication of functions across various ministries and agencies. As the implementation of skills policies involves a wide variety of actors and spans multiple levels of government, ensuring the quality of data and compliance with monitoring and reporting requirements becomes more challenging. Stakeholders emphasised the need for a governance structure that creates a sustainable and self-regulating monitoring and reporting system that can continue operating through political and contextual changes, such as in Norway (Box 4.6). There was strong support for establishing centralised entities to facilitate and oversee the monitoring and reporting system, including the proposed Skills Policy Council (SPC) and Inter-institutional Data and Evidence Centre (DEC) described in Output 2. Stakeholders elaborated that dedicated monitoring and evaluation units could also be established or enhanced within certain ministries to improve effective monitoring, reporting and evaluation practices.
The flow of data must consider the mechanisms by which information will be collected from the source and submitted to the management information system (MIS) where it will eventually be stored. Stakeholders have pointed out that there is currently no centralised database that would be able to house all the indicators for the Action Plan for Skills and suggested that resources be allocated into setting up an MIS specifically for this purpose. Alternatively, Bulgaria may also invest resources into expanding the current scope of the Registry Information Exchange System (RegiX) in order to transform it into a centralised MIS that could also store the monitoring data for the Action Plan (State e-Government Agency, n.d.[55]). In both cases, the MIS should be inter-operable with other existing databases, including the country’s educational MIS (EMIS), in order to ensure the efficient exchange of data when necessary. This process of synchronisation may be overseen by various bodies, such as the NSI, the Ministry of Electronic Governance, the Ministry of Finance (Central Coordination Unit), or the Managing Authority for National and European Programmes.
In general, the stakeholders discussed that a key component of establishing an effective monitoring and reporting system is adequately resourcing (both in terms of human and financial capacity) the system itself in addition to resources for carrying out the activities that are being monitored. Such resources should be able to cover various types of expenses, such as staff and expert fees (i.e. data collection and analysis), logistics, office equipment, and operating expenses to maintain adequate digital infrastructure to house the monitoring data. Usually, M&E costs may constitute between 3% to 10% of the overall programme’s budget (Sedrakian, 2016[56]). In particular, stakeholders noted that greater human and financial resources are needed to improve the capacity for monitoring and evaluation within the MES, where a lot of the data relevant to Bulgaria’s skills system is collected and/or stored.
Norway's Skills Strategy, launched in 2017, addresses the evolving needs of its labour market in response to technological, demographic, and global changes. The strategy, developed collaboratively by the government, social partners, and various stakeholders, focuses on lifelong learning for adults, emphasising their crucial role in the nation's welfare and competitiveness.
Central to the success of this strategy are two key governance arrangements: the Skills Policy Council and the Future Skills Needs Committee, both of which play a critical role in the monitoring and evaluation of the strategy’s implementation.
The Skills Policy Council is responsible for co‑ordinating the fragmented field of skills policy and comprises representatives from government, education, and labour sectors. Its primary role is to oversee the strategy's implementation, including monitoring its progress and effectiveness. The Council also fosters stakeholder co‑operation, and if necessary, revises skills policy implementation based on the outcomes of its ongoing evaluation. The transition of the Council from a temporary to a permanent high-level group has significantly bolstered the continuity and consistency in skills policy implementation, monitoring and reporting, ensuring that the strategy remains relevant and effective even beyond its stated time frame.
The independent Future Skills Needs Committee, with members from social partners, researchers, and county councils plays a crucial role in continually assessing Norway’s future skills needs. This dynamic evaluation process enables the Committee to inform strategic decision-making more effectively, ensuring the prioritisation of skills policy implementation is responsive to evolving demands. The Committee’s reports on a variety of topics, including skills for the green transition and the impact of new technologies, are instrumental in guiding skills policy priorities.
The impact of the Skills Policy Council is particularly evident in the development of the tripartite Sector programmes for training, which target skills gaps in specific industries based on the Council’s recommendations. These programmes, while ultimately approved by the government, are shaped by the Council’s ongoing assessment of the strategy’s effectiveness and the Future Skill Needs Committee’s future-oriented insights. Thus, Norway’s system for prioritisation, implementation, monitoring and evaluation of skills policy implementation remains robust across time and changes in government.
Note: These insights were shared as part of an expert presentation from Norway during the Output 3 stakeholder consultation workshop.
5. Ensuring the credibility and wide dissemination of evaluation findings
Evaluation is closely related to – but distinct – from the monitoring process. While monitoring refers to the ongoing activity of systematically collecting evidence and data as inputs for analysis, evaluations are the periodic outputs that synthesise such analysis and use them to draw conclusions. As such, evaluations are intended to provide structured and objective assessments of the policy, helping determine if policy objectives have been effectively achieved (OECD, 2019[57]). While evaluations are outside of the scope of Output 3, monitoring and reporting lay the foundations for the conduct of evaluation studies, highlighting the need to collect timely and high-quality data throughout the implementation of the Action Plan for Skills.
The dissemination of evaluation results is a key step in the M&E phase, as it helps build support among stakeholders for the incorporation of findings in future programming or policymaking. This may be done by developing knowledge products, such as reviews, annual evaluation reports, or think pieces, among many others. Messages to communicate to stakeholders may include key evaluation findings and policy recommendations, good and innovative practices, and the relevance of the findings to the ministry or organisation’s work. These knowledge products may be targeted to different kinds of stakeholders by taking into account their information needs, their thematic expertise, and their roles in the implementation of the policy (ILO, 2020[58]).
Across the OECD, many countries develop such knowledge products that evaluate the performance of their skills systems against national goals and make these publicly available, but such initiatives in Bulgaria are limited. This may be due to challenges in data management issues, or in cases where reports are written, they are not accessible by the public or other agencies (Guthrie et al., 2022[59]). At present, Bulgaria also does not have a body responsible for research and evaluation across the skills system, an issue which the establishment of the Inter-institutional Data and Evidence Centre (DEC) will help address.
Bulgaria would thus benefit from ensuring that the work of the DEC is disseminated in various formats to relevant stakeholders not only to raise awareness for the Action Plan for Skills, but to continually strengthen buy-in for its more effective and efficient implementation. Throughout the consultations, stakeholders also highlighted the need to effectively disseminate the evaluation findings generated by the new monitoring and reporting system to relevant stakeholders in Bulgaria’s skills system, including at the regional and local levels. To address this, a dissemination strategy should be integrated into the monitoring and reporting framework from the outset, ensuring that widely and effectively sharing evaluation results is considered an integral part of the monitoring and reporting process. Additionally, stakeholders pointed out that an effective system for quality evaluation combines internal quality assessments with external audits or evaluations, which would involve partnering with researchers and NGOs.
6. Systematically utilising evaluation findings to inform the design of future policy
It is essential to systematically utilise evaluation findings to shape future policies. In the policy development phase, previous evaluations should be actively consulted and integrated. This approach ensures that new policies are informed by empirical evidence and past experiences. It also involves establishing regulations and standards that mandate the use of evaluation findings in policy design, enhancing the effectiveness and relevance of new policies. Utilising the results from evaluation activities promotes evidence-based decision-making and continuous improvement in governance practices (OECD, 2020[27]).
However, in Bulgaria, there is currently no legal requirement to provide evidence in policymaking, nor any institutionalised arrangements for exchanging information between evidence producers and policymakers (European Commission, 2017[60]). Bulgaria also ranks below the OECD average in terms of applying impact assessments and ensuring the quality of such assessments (OECD, 2023[47]). The country could therefore benefit from establishing a legislative framework for evidence-based policymaking over the longer term in order to systematically incorporate evaluation findings in future policies. However, in the more immediate future, Bulgaria can continue strengthening its efforts to include representatives from civil society (e.g. scientific groups, academic institutions, NGOs) as working groups in the development of policies (European Commission, 2017[60]).
In addition to the list of milestones and indicators specific to each priority area, this chapter also presents a monitoring and reporting framework for each priority area, describing the flow of information, processes and responsibilities for monitoring and reporting on the implementation of Bulgaria’s Action Plan for Skills in that area. As with the indicators presented in above, the methodology was refined in consultation with authorities and stakeholders and is in line with current monitoring practices in Bulgaria. In most cases, the ministries, agencies, and other actors responsible for collecting data from the primary source are those also identified as the leads or partners assigned to implement the activities in the Action Plan for Skills, as detailed in the Output 2 report.
Across all priority areas, the Inter-institutional Data and Evidence Centre (DEC) is identified as being responsible for storing and centralising the data in its management information system, as well as quality assuring, analysing, and transforming the data into useful, actionable information that could be fed back into the policy cycle (Figure 4.2). Data quality assurance and the use of evidence in policymaking are two elements that stakeholders in Bulgaria identified as a priority, as these are two elements of skills policies that currently need to be reinforced in the country.
The DEC is then tasked with submitting its analytical findings and policy implications to the Skills Policy Council, which is responsible for disseminating these findings to the main authorities and stakeholders in Bulgaria’s skills system. The findings could be used to adjust and refine the implementation of activities in the Action Plan for Skills to make them more effective and efficient, as well as to promote evidence-based policymaking and thus better inform future skills-related policies in Bulgaria.
Developing youth skills
The flow of data and information for tracking progress on the implementation of skills policy actions related to developing the skills of youth is presented in Figure 4.4. Because so much of the data being collected for this priority area is collected and overseen by various directorates within MES, stakeholders proposed strengthening the capacity of the Strategic Development and Information Policy Directorate (SDIPD) within MES to give this directorate a more active role in consolidating the relevant data to be sent to the Inter-institutional Data and Evidence Centre (DEC) for aggregation with other data in the unified management information system, quality assurance and analysis. While MES is the main collector of data, the data sources for some of the indicators come from municipalities or REDs, which are responsible for data on the ECEC institutions and schools in their jurisdiction, and from higher education institutions. A few other key indicators can be collected directly by DEC from NSI and from external OECD and Eurostat databases (e.g. PISA, Labour Force Survey).
The milestones in the priority area for developing youth skills primarily need to be initiated by the MES, and more particularly by the Preschool and School Education Content Directorate and the Higher Education Directorate. Implementing these milestones should then enhance the ability of the municipalities, REDs and higher education institutions to provide high-quality indicator data back to MES to be fed into DEC.
Once the DEC has analysed the relevant data and transmitted this data to the Skills Policy Council for further analysis and to draw policy implications, the evaluation results should be disseminated among key authorities and stakeholders involved in developing the skills of youth. Given that MES is clearly the ministry most responsible for developing the skills of youth, MES is a fitting entity to disseminate the conclusions of the SPC to other relevant stakeholders, including regional and local officials (e.g. REDs, municipalities) and directly to education providers such as higher education institutions. Furthermore, the evaluation results should be used by the various directorates in MES to inform future skills policy-making by providing an evidence base on which future skills policy is developed.
Improving adult skills
The governance of data and information as it relates to improving adult skills is presented in Figure 4.5. Many of the indicators for this priority areas come from relevant government ministries and agencies. As adult education and training (AET) overlaps with the mandate of multiple ministries and agencies, the responsibility for collecting information on implementation of skills activities related to adults is spread across MES, NAVET, MLSP and NEA. The ministry or agency tasked with collecting information also relates to whether the indicator being tracked relates to individual adults, enterprises, and/or AET providers. Because many of the indicators are sourced from multiple ministries/agencies, the OECD suggests that the DEC collect the data from each ministry/agency and aggregate these data from multiple sources within the DEC itself. As with the previous priority area, the DEC will also collect some data directly from NSI and from external Eurostat sources.
The milestones identified for the priority area on improving the skills of adults all relate to better equipping AET providers to provide relevant and quality training and services to adults in Bulgaria. Some of these milestones should be initiated by NAVET and others by MLSP.
Following the DEC’s work of consolidating, quality assuring, and analysing the data, results should be submitted to the Skills Policy Council to complete the evaluation of implementation progress, including implications for further skills policy. The evaluation results should then be disseminated to all involved ministries. The ministries in turn should be responsible for conveying the data and evaluation results to key non-governmental partners such as the social partners and AET providers.
Using skills effectively
The methodology and flow of data for the priority area on using skills effectively is presented in Figure 4.6. Multiple ministries and agencies are responsible for collecting data for the milestones and indicators, and in many cases, are also the primary sources of data as this information would come from programme data related to the activities in the Action Plan for Skills. Enterprises are also the sources of information for Indicator USE.6, which will be collected directly from the enterprises by the DEC, either through self-reporting (e.g. into a platform) by enterprises or a survey to be distributed. The DEC may co‑ordinate with the MLSP to communicate with the enterprises when gathering responses from them (e.g. communication of the link to the platform where enterprises must respond), as well as in ensuring data quality (e.g. communication with enterprises that provide low-quality answers).
Once the data has been submitted to and analysed by the DEC, the Skills Policy Council is responsible for disseminating these findings and policy implications to actors significantly involved in skills use in Bulgaria, such as the MLSP, the NEA, the Social Assistance Agency, the MIG, and other government bodies responsible for emigration and immigration policy (e.g. Ministry of Interior; State National Council on Migration, Borders, Asylum and Integration; Council for Working with Bulgarians Abroad), as well as social partners (e.g. employer and employee representatives, employers) (OECD, 2023[47]). Based on these policy findings vis-à-vis set targets, the MLSP may make decisions to adjust the implementation of the Action Plan for Skills, including expenditure on active labour market policies such as training programmes for adults (see Activities 9.1.1 and 9.1.3), as well as on other initiatives that aim to promote the full employment and use of skills of individuals in Bulgaria, including those from vulnerable groups (MLSP, 2021[61]). While the Social Assistance Agency is not directly involved in monitoring and reporting, the findings may affect its activities, especially at the local level and including the provision of provision of support and benefits to disadvantaged groups identified in the Action Plan for Skills (OECD, 2023[47]).
Strengthening the governance of the skills system
The methodology and flow of data for the priority area on strengthening the governance of the skills system is presented in Figure 4.7. The Skills Policy Council, the main body tasked to co-ordinate the implementation the Action Plan for Skills, and the MES are responsible for monitoring and reporting on three milestones each. The DEC is also tasked to report on its own activities, such as through Indicators GOV.7 and GOV.8. Two ministries have been given data collection responsibilities, namely MES, which is tasked to submit data on the number of active Sectoral Skills Councils (SSC) as it simultaneously collects this data for the Information system for management and monitoring of EU funds in Bulgaria, as well as the NSI, which regularly collects information on government expenditure on education and training for Eurostat purposes.
Once this data has been submitted to and analysed by the DEC, the Skills Policy Council is responsible for disseminating these findings and policy implications to actors involved in skills governance in Bulgaria. This includes ministries and agencies at the national level, actors at the regional level (e.g. municipalities, REDs), and non-government stakeholders (e.g. social partners, educational and training providers).
References
[15] Bulgaria Council of Ministers (2022), National Recovery and Resilience Plan of the Republic of Bulgaria: Version 1.2, https://www.minfin.bg/en/1579.
[20] Bulgaria Council of Ministers (2016), “НАРЕДБА за определяне на условията, реда и механизма за функциониране на ИСУН [ORDINANCE to determine the terms, order and mechanism for functioning of the Management Information System and monitoring of funds from the European structural and investment funds (ISUN) and for conducting proceedings before the governing bodies through ISUN]”.
[19] Bulgaria Council of Ministers (2015), Management of Resources from the European Structural and Investment Funds Act, https://aeuf.minfin.bg/upload/944/2019Management_of_Resources_from_the_European_Structural_and_Investment_Funds_Act.pdf.
[2] Bulgaria Council of Ministers (1998), ЗАКОН ЗА АДМИНИСТРАЦИЯТА [Bulgaria’s Law on Administration].
[9] Bulgarian SMEs Promotion Agency (2021), Национална стратегия за малките и средните предприятия 2021-2027 [National strategy for small and medium sized enterprises], https://www.mi.government.bg/strategy-policy/natsionalna-strategiya-za-malki-i-sredni-predpriyatiya-msp-v-balgariya-2021-2027-g/.
[43] Eurofound (2019), European Company Surveys (ECS), https://www.eurofound.europa.eu/en/surveys/european-company-surveys-ecs.
[18] European Commission (2023), European Structural and Investment Funds, https://ec.europa.eu/commission/presscorner/detail/en/ip_23_389 (accessed on 24 January 2024).
[17] European Commission (2022), Partnership Agreement with Bulgaria - 2021-2027, https://commission.europa.eu/publications/partnership-agreement-bulgaria-2021-2027_en.
[60] European Commission (2017), “Education and Training Support Mechanisms”, https://doi.org/10.2797/575942.
[16] European Commission (2014), Partnership agreement with Bulgaria - 2014-20, https://commission.europa.eu/publications/partnership-agreement-bulgaria-2014-20_en (accessed on 21 January 2024).
[13] European Union (2021), REGULATION (EU) 2021/1057 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32021R1057 (accessed on 21 January 2024).
[12] European Union (2021), REGULATION (EU) 2021/1060 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32021R1060 (accessed on 21 January 2024).
[14] European Union (2021), Regulation (EU) 2021/241 of the European Parliament and of the Council Establishing the Recovery and Resilience Facility, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32021R0241 (accessed on 21 January 2024).
[11] European Union (2013), REGULATION (EU) No 1303/2013 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32013R1303 (accessed on 21 January 2024).
[41] Eurostat (2024), Employment and activity by sex and age - annual data, https://ec.europa.eu/eurostat/databrowser/view/lfsi_emp_a__custom_9280928/default/table?lang=en.
[34] Eurostat (2024), Employment rates of young people not in education and training by sex, educational attainment level and years since completion of highest level of education, https://ec.europa.eu/eurostat/databrowser/view/EDAT_LFSE_24__custom_9446600/default/table?lang=en.
[40] Eurostat (2024), Expenditure by LMP intervention - Bulgaria, https://webgate.ec.europa.eu/empl/redisstat/databrowser/view/LMP_EXPME$BG/default/table?lang=en&category=lmp_expend.lmp_expend_me.
[49] Eurostat (2024), General government expenditure by function (COFOG), https://ec.europa.eu/eurostat/databrowser/view/gov_10a_exp__custom_9429545/default/table?lang=en.
[44] Eurostat (2024), GERD by sector of performance and type of expenditure, https://ec.europa.eu/eurostat/databrowser/view/rd_e_gerdcost__custom_9305752/default/table?lang=en.
[38] Eurostat (2024), Participation rate in education and training (last 4 weeks) by sex and age, https://ec.europa.eu/eurostat/databrowser/view/TRNG_LFSE_01/default/table?lang=en&category=educ.educ_part.trng.trng_lfs_4w0.
[35] Eurostat (2024), Population by will to participate in education and training, https://ec.europa.eu/eurostat/databrowser/view/TRNG_AES_175/default/table?lang=en&category=educ.educ_part.trng.trng_aes_12m3;.
[36] Eurostat (2024), Population wanting to participate in education and training, by reason for not participating and sex, https://ec.europa.eu/eurostat/databrowser/view/TRNG_AES_176__custom_3565161/default/table?lang=en.
[39] Eurostat (2023), Enterprises providing training by type of training and size class - % of all enterprises, https://ec.europa.eu/eurostat/databrowser/view/trng_cvt_01s/default/table?lang=en.
[45] Eurostat (2022), Community Innovation Survey 2020 - key indicators, https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Community_Innovation_Survey_2020_-_key_indicators#:~:text=innovation%20and%20novelty-,Innovation%20activity%20in%20the%20European%20Countries,the%20reference%20period%202018%E2%80%932020.
[1] Exton, C. and M. Shinwell (2018), “Policy use of well-being metrics: Describing countries’ experiences”, OECD Statistics Working Papers, No. 2018/07, OECD Publishing, Paris, https://doi.org/10.1787/d98eb8ed-en.
[59] Guthrie, C. et al. (2022), OECD Reviews of Evaluation and Assessment in Education: Bulgaria, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/57f2fb43-en.
[58] ILO (2020), “Enhancing the use and dissemination of evaluation information”, http://www.uneval.org/document/detail/1911 (accessed on 25 January 2024).
[24] ILO (2018), BASIC PRINCIPLES OF MONITORING AND EVALUATION, https://www.ilo.org/wcmsp5/groups/public/---ed_emp/documents/publication/wcms_546505.pdf (accessed on 15 November 2023).
[52] ILO (2018), Guide on Measuring Decent Jobs for Youth - Monitoring, evaluation and learning in labour market programmes., https://www.ilo.org/wcmsp5/groups/public/---ed_emp/documents/instructionalmaterial/wcms_627311.pdf (accessed on 15 November 2023).
[5] Ministry of Education and Science (2021), СТРАТЕГИЧЕСКА РАМКА ЗА РАЗВИТИЕ НА ОБРАЗОВАНИЕТО, ОБУЧЕНИЕТО И УЧЕНЕТО В РЕПУБЛИКА БЪЛГАРИЯ (2021-2030) [Strategic Framework for the Development of Education, Training and Learning in the Republic of Bulgaria 2021-2030].
[6] Ministry of Education and Science (2020), “Решение за приемане на Стратегия за развитие на висшето образование в Република България за периода 2021 - 2030 г. [Decision to adopt the Higher Education Development Strategy education in the Republic of Bulgaria for the period 2021 - 2030]”.
[4] Ministry of Finance (2020), National Development Programme BULGARIA 2030, https://www.minfin.bg/en/1394.
[8] Ministry of Innovation and Growth (2022), “Стратегията за интелигентна специализация 2014-2020 [Innovative Strategy for Smart Specialisation 2014-2020]”, https://www.mi.government.bg/files/useruploads/files/innovations/ris3_26.10.2015_en.pdf.
[10] Ministry of Interior (2021), “НАЦИОНАЛНА СТРАТЕГИЯ ПО МИГРАЦИЯ НА 2021-2025 [National Migration Strategy of the Republic of Bulgaria 2021-2025]”, https://migrant-integration.ec.europa.eu/sites/default/files/2021-05/NationalStrategyonMigrationoftheRepublicofBulgaria2021-2025.pdf.
[7] Ministry of Labour and Social Policy (2022), “СТРАТЕГИЯ ПО ЗАЕТОСТТА НА РЕПУБЛИКА БЪЛГАРИЯ: 2021 - 2030 годин [Employment Strategy of the Republic of Bulgaria: 2021-2030]”, https://www.mlsp.government.bg/uploads/26/zaetost/employment-strategy-2021-2030-2022.pdf (accessed on 22 May 2023).
[61] MLSP (2021), Bulgaria - Employment Promotion Act of 20 December 2001, as amended., https://www.ilo.org/dyn/natlex/natlex4.detail?p_lang=en&p_isn=60386 (accessed on 3 October 2022).
[28] OECD (2024), Guidance on the Implementation of a Whole-of-government Action Plan for Skills in Bulgaria, OECD, Paris, http://www.oecd.org/content/dam/oecd/en/about/programmes/dg-reform/bulgaria/Guidance-on-the-implementation-of-a-whole-of-government-action-plan-for-skills-in-Bulgaria.pdf.
[31] OECD (2024), PISA 2018 Database, OECD, Paris, https://www.oecd.org/en/data/datasets/pisa-2018-database.html.
[47] OECD (2023), OECD Skills Strategy Bulgaria: Assessment and Recommendations, OECD Skills Studies, OECD Publishing, Paris, https://doi.org/10.1787/c2eb2f34-en.
[51] OECD (2023), Policy monitoring and evaluation, https://www.oecd.org/en/topics/public-policy-monitoring-and-evaluation.html (accessed on 15 November 2023).
[21] OECD (2023), Supporting Employers in Promoting Skills Development in Latvia: Indicators and Methodology for Monitoring the Implementation of the Support Measure, OECD, Paris, https://www.oecd.org/content/dam/oecd/en/about/programmes/dg-reform/latvia/Practical-Guidelines-For-Regulatory-Framework-Latvia.pdf.
[3] OECD (2022), “Centre of government scan of Bulgaria: Strengthening strategic decision-making at the centre of government”, OECD Public Governance Policy Papers, No. 19, OECD Publishing, Paris, https://doi.org/10.1787/464a42b5-en.
[32] OECD (2022), OECD Education Equity Dashboard, OECD, Paris, https://www.oecd.org/en/data/dashboards/education-equity.html.
[46] OECD (2022), Reaching Out and Activating Inactive and Unemployed Persons in Bulgaria, Connecting People with Jobs, OECD Publishing, Paris, https://doi.org/10.1787/7b91154a-en.
[22] OECD (2021), “Incentives to Business Investment in Skills: Lessons from European Good Practices”, https://doi.org/10.1787/1eb16dc7-en (accessed on 18 January 2024).
[54] OECD (2020), Building Capacity for Evidence-Informed Policy-Making: Lessons from Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/86331250-en.
[27] OECD (2020), Improving Governance with Policy Evaluation: Lessons From Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/89b1577d-en.
[57] OECD (2019), Open Government in Biscay, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/e4e1a40c-en.
[33] OECD (2019), TALIS 2018 Results (Volume I): Teachers and School Leaders as Lifelong Learners, TALIS, OECD Publishing, Paris, https://doi.org/10.1787/1d0bc92a-en.
[50] OECD (2018), Getting Skills Right: Brazil, Getting Skills Right, OECD Publishing, Paris, https://doi.org/10.1787/9789264309838-en.
[42] Republic of Bulgaria National Statistical Insitute (2024), Employed and employment rates - national level; statistical regions; districts, https://www.nsi.bg/en/content/3996/employed-and-employment-rates-national-level-statistical-regions-districts.
[29] Republic of Bulgaria National Statistical Institute (2024), Children enrolled at kindergartens by sex and age, https://www.nsi.bg/en/content/3419/children-enrolled-kindergartens-sex-and-age.
[30] Republic of Bulgaria National Statistical Institute (2024), Kindergartens, children, pedagogical staff, places and groups in the kindergartens by statistical zones, statistical regions, districts and municipalities, https://www.nsi.bg/en/content/3430/kindergartens-children-pedagogical-staff-places-and-groups-kindergartens-statistical-zones-statistical-regions-districts-and-municipalities.
[48] Republic of Bulgaria National Statistical Institute (2024), Public and private expenditures by level of education, https://www.nsi.bg/en/content/3541/public-and-private-expenditures-level-education.
[37] Republic of Bulgaria National Statistical Institute (2023), Participants in formal or non-formal education and training, https://www.nsi.bg/en/content/3533/participants-formal-or-non-formal-education-and-training.
[56] Sedrakian, S. (2016), “Financing Monitoring & Evaluation: A Self-Study Toolkit”, http://www.ocic.on.ca (accessed on 22 January 2024).
[53] Shapiro, J. (2003), Monitoring and Evaluation, https://www.civicus.org/view/media/Monitoring%20and%20Evaluation.pdf (accessed on 15 November 2023).
[55] State e-Government Agency (n.d.), Registry Information Exchange System (RegiX), https://e-gov.bg/wps/portal/agency-en/digital-government-infrastructure/information-systems-applications/regix (accessed on 22 January 2024).
[26] USAID (2021), Performance Indicator Targets, https://usaidlearninglab.org/sites/default/files/resource/files/mt_performance_indicator_targets_final2021.pdf (accessed on 15 November 2023).
[25] USAID (2010), Performance Monitoring & Evaluation Tips, Baselines and Targets, https://pdf.usaid.gov/pdf_docs/pnadw108.pdf (accessed on 15 November 2023).
[23] World Bank (2004), A Handbook for Development Practitioners. Ten Steps to a Results-Based Monitoring and Evaluation System, https://www.oecd.org/dac/peer-reviews/World%20bank%202004%2010_Steps_to_a_Results_Based_ME_System.pdf (accessed on 15 November 2023).
← 1. Funded by the European Union via the Technical Support Instrument and implemented by the OECD, in cooperation with the European Commission.
← 2. It is important to take note how these indicators are referred to differently in European programmes and reporting processes. “Uptake” in this chapter would refer to “output”, and “output” in this chapter would refer to “results” as per EU Regulation 2021/2060.