Guidance for a Monitoring and Evaluation System for Italy’s Universal Civil Service

1. Introduction

Youth unemployment in Italy is very high compared with most other OECD countries, even though the rate has been gradually declining over the past decade. At 21.4% in the first quarter of 2024, the unemployment rate for 15‑24 year‑olds stood at double the OECD average (10.7%) and considerably above the EU‑27 average (14.7%). Even so, youth unemployment in Italy has seen a steady downward trend since the mid‑2010s when the rate peaked at more than 40%. A similar observation can be made for the share of young people who are neither in employment, education, or training (NEETs), accounting for 20.3% of all 15‑29 year‑olds in Italy in 2022 (compared to 12.8% on average across the OECD). While Italy is fourth highest in the OECD ranking, the country’s NEET rate has seen a considerable decline over the past decade (27.4% in 2014).

Italy has launched various initiatives supporting youth employment and the government also engaged in delivering the goals of the European Union Youth Strategy 2019‑27. Complementarily, Italy’s Universal Civil Service (UCS) aims to involve young people in volunteering activities, to foster practical skill development for future employment as well as civic engagement and social cohesion. The UCS represents an opportunity for personal growth and training and is an important tool to address youth unemployment and skills gaps. The successful engagement of young people in the labour market, public and political life, and society more generally have been at the forefront of the policy debate in Italy over the past years, and even more so in the context of the COVID‑19 crisis (OECD, 2020[1]; 2022[2]).

In this context, the European Commission’s Directorate‑General for Structural Reform Support (DG REFORM) and the Organisation for Economic Co‑operation and Development (OECD) are providing technical support to the Department for Youth Policies and the Universal Civil Service of Italy regarding the design and implementation of the UCS project of Italy’s Recovery and Resilience Plan (RRP). In particular, the project focuses on: (i) reducing procedural, regulatory, institutional and governance barriers for UCS effectiveness and its impact for youth outcomes; (ii) improving implementation of UCS-supported initiatives, and improved conditions for current and future UCS participants; and (iii) improving the monitoring and evaluation (M&E) framework for the UCS.

As part of Output 1 of the project (“Report on the design of indicators and the development of a Monitoring and Evaluation framework covering the UCS RRP project overall, and the compliance and outcomes of UCS-supported requests”, unpublished), the Youth and Diversity Policies Unit of the OECD Directorate for Employment, Labour and Social Affairs has prepared this technical report with guidance for an M&E system for Italy’s Universal Civil Service. This report provides an assessment of the M&E elements currently in use for the UCS, as well as suggestions for the establishment of a robust and comprehensive results-based M&E system for the UCS. The report draws on various sources of information: methodological documents on M&E, such as the OECD Recommendation on Public Policy Evaluation (OECD, 2022[3]); desk research on the UCS and good practice examples in other OECD countries; as well as interviews with UCS managers, stakeholders and researchers conducted by the OECD between July and October 2023.

The report sets the scene with a brief overview of the UCS, followed by an explanation on the purpose of an M&E system for the UCS. The report then explains the necessary steps to be considered for a comprehensive M&E system, including understanding and outlining UCS results chain through a Theory of Change; setting SMART indicators based on UCS’ Theory of Change; planning for the collection and processing of data; establishing an evaluation plan; communicating and disseminating UCS progress and results; and strengthening human, financial, and institutional capacities, mechanisms, and policy learning. A concluding section summarises the main messages of the report and provides key policy recommendations. It is important to highlight that there is not a unique way to build and maintain a results-based M&E system: this task should rather be understood as work in progress and under continuous improvement.

A short overview of Italy’s Universal Civil Service

Italy’s Civil Service was established in 1972 as part of efforts to foster social cohesion, promote civic engagement, and address societal challenges through non-military service (Gazzetta Ufficiale della Repubblica Italiana, 1972[4]). The Civil Service aimed to provide young people with opportunities for personal development, social integration, and active participation in community-building efforts.

Initially, the Civil Service primarily focused on roles related to civil protection, environmental conservation, cultural heritage preservation, and social welfare activities. Over time, its scope expanded to encompass a wider range of sectors and activities, reflecting evolving societal needs and priorities.

In 2001, Law 64/2001 established the National Civil Service, significantly expanding the scope and objectives of the Civil Service (Gazzetta Ufficiale della Repubblica Italiana, 2001[5]). Specifically, the National Civil Service was designed as a voluntary service for young men and young women aged 18 to 26 (later raised to 28 years by Legislative Decree no. 77/2002), intending to undertake a social, civic, cultural and professional training through social solidarity, national and international co‑operation activities, and the protection of national heritage. Law 64/2001 expanded the range of activities and sectors in which civil service volunteers could be engaged, while aligning more closely with broader national and European policies on youth empowerment, social cohesion, and community development. The year 2005 saw the suspension of the mandatory military service.

Another crucial development occurred with the enactment of Law Decree 40/2017 (Gazzetta Ufficiale della Repubblica Italiana, 2017[6]), which transformed the Civil Service into the Universal Civil Service (UCS). Similar to the National Civil Service, the UCS involves young people aged between 18 and 28 in social, civic, cultural, and professional training activities through social solidarity, national and international co‑operation, and the protection of national heritage. Volunteering projects, in Italy and abroad, belong to various areas – from civil protection to cultural heritage to the promotion of peace. Young volunteers are asked to commit for 25 hours per week, to be divided into at least 4 hours per day for 5/6 days a week, for a period of 8 to 12 months. Volunteers receive a monthly allowance of approximately EUR 500 (as of 2024) and a minimum of 80 hours of training, as well as other benefits including but not limited to the recognition and possible certification of the skills acquired through the UCS, university training credits if envisaged by the project, a certificate of participation in the UCS, preferential access for young people with fewer opportunities in a number of projects, and the reservation of a 15% quota in the recruitment of non-managerial personnel within public administrations (DGSCU, 2024[7]). In 2022, 50 972 of volunteer workers were initiated in the UCS (slightly above the 49 984 volunteers initiated in 2021), representing around 0.8% of the reference population (DGSCU, 2023[8]; 2019[9]; De Luca, 2024[10]).

Finally, it is important to acknowledge that the UCS benefits from support from Italy’s National Recovery and Resilience Plan (RRP) (Italian Government, 2021[11]). To address the high NEET rates and the further threats posed by the COVID‑19 crisis, Italy included a specific action (Component 1 of Mission 5) targeting inclusion and cohesion and aiming at improving employment and employability especially among young people aged between 18 and 28, in line with the Recommendation of the Council of 22 May 2018 (2018/C/189/01) (Council of the EU, 2018[12]). The RRP aimed to increase by 120 000 the number of young people undertaking a non-formal learning pathway (in other words, the UCS) for the development of transversal skills. Available financial resources dedicated to these actions between 2021 and 2023 amounted to EUR 650 million.

Purpose of a Monitoring and Evaluation system for the UCS

Governments are expected to demonstrate policy results. Systematic data collection, analysis, and interpretation are fundamental for policy makers to monitor progress, identify outcomes, evaluate the impact of public interventions in relation to established goals, and adjust where needed.

Result-oriented Monitoring and Evaluation (M&E) systems serve these purposes, by generating feedback on the progress and outcomes of public interventions. Monitoring and evaluation are distinct yet complementary practices. Monitoring is “a continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing (…) intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds” (OECD, 2002[13]). Evaluation consists of “the structured and evidence‑based assessment of the design, implementation or results of a planned, ongoing or completed public intervention” (OECD, 2022[3]).

Result-oriented M&E can support evidence informed decision-making, strategic planning, and policy design by providing timely information on an intervention; strengthen accountability and legitimise the use of public funds and resources; facilitate policy learning; ensure adaptable management; and increase the efficiency and effectiveness of spending. It can cultivate transparency, engagement, and responsibility towards internal and external stakeholders (OECD[14]; OECD, 2022[3]).

The DGSCU and UCS entities are making important efforts to monitor the implementation of the UCS and its programmes/projects, with key information on inputs, activities and outputs being regularly collected and reported. At the same time, there is room for improvement as regards the definition of the UCS’ results chain and collection of additional data allowing to better analyse its outcomes and impacts. Existing evaluations of the Civil Service prior to the UCS, for instance, find evidence of a positive correlation between active citizenship for employability (De Luca, 2023[15]; De Luca and Ferri, 2021[16]; De Luca, 2023[17]); similar analyses and data collection methods could be capitalised for the evaluation of the current version of the UCS.

The following sections focus on the key steps for building and implementing an M&E system for the UCS (Figure 1), namely:

• Understanding and outlining the UCS results chain through a Theory of Change (ToC): Identifying the UCS rationale and objectives and building its ToC – an explicit description (in the form of a graphical display, matrix and/or summary) of the intervention logic of the UCS.

• Setting SMART indicators based on UCS ToC: Defining specific, measurable, achievable, relevant, and time‑bound (SMART) indicators to measure progress towards UCS objectives, based on its ToC.

• Planning for the collection and processing of data: Identifying methods to gather information on such indicators before, during and after the implementation of UCS actions, allowing to track changes over time and establishing quality standards and mechanisms to generate robust and credible results that can be trusted and used with confidence.

• Establishing an evaluation plan: Planning different types of evaluation to analyse the intervention based on a selection of evaluation objectives, questions and criteria and ensuring that evaluations are timely and proportionate to the intended objectives.

• Communicating and disseminating UCS progress and results: Reporting and communicating results to target audiences, for instance through data dashboards containing key information on UCS outputs and outcomes, or reports summarising the M&E findings.

• Strengthening human, financial, and institutional capacities, mechanisms, and policy learning: Ensuring that i) the sufficient institutional mechanisms are in place to embed M&E in decision-making processes and ii) the UCS is endowed with the human, time, technical and financial resources needed to conduct, commission and use M&E activities effectively and in a credible manner. This should also support the use of M&E evidence to learn as well as adapt and improve the UCS.

Each of these sections illustrates the key elements to consider, analyses these elements in the framework of the UCS, and provides a set of recommendations to build a robust and regularly updated M&E system for the UCS. In this context, it is important to acknowledge that building or improving an M&E system is resource‑intensive and requires consensus building, analytical support, and technical instruments. The availability of such resources as well as strategic determination will inevitably affect whether the recommendations can be acted upon and the timeline of related actions. Moreover, an M&E system should be flexible and adaptable to changing intervention needs. As the UCS evolves, the M&E should be updated to ensure its relevance and alignment with UCS goals.

Figure 1. Key elements and steps for the UCS M&E system

Source: Authors.