In response to the COVID‑19 pandemic, Portugal implemented the ATIVAR.PT internship programme to provide rapid and comprehensive assistance to jobseekers and facilitate the integration of young people into the labour market. This chapter provides details on the internship programme and describes the methods and data used in the next chapter of this report to evaluate the impact of this programme on individuals’ labour marker outcomes.
Impact Evaluation of Active Labour Market Policies in Portugal
Connecting People with Jobs
Abstract
In the midst of the COVID‑19 pandemic, Portugal launched a comprehensive set of active labour market programmes under the ATIVAR.PT initiative to provide assistance to those affected by the crisis. A key component of this initiative is the ATIVAR.PT internship, which specifically focuses on the integration of young people into the labour market by providing practical work experience. This chapter describes the internship programme and the methods used in the next chapter to evaluate its impact on individual labour market outcomes.
The chapter begins by describing the context in which the ATIVAR.PT internship was introduced, its main features and the characteristics of the participants. The following section outlines the requirements for conducting a counterfactual impact evaluation (CIE) of the internship and the methodology for estimating its impact on employment, earnings and unemployment benefit receipt. The section also explains the timeframe and the sample chosen for the analysis. The final section describes the administrative data used for the CIE.
The programme ATIVAR.PT “Reinforced Programme of Support for Employment and Professional Training“ (Programa Reforçado de Apoios ao Emprego e à Formação Profissional) was implemented in Portugal in response to the challenges posed by the COVID‑19 pandemic. This was a crucial component of the Economic and Social Stabilization Program (PEES), designed to safeguard employment and facilitate the gradual recovery of economic activity.
The ATIVAR.PT programme includes professional training, professional internships as well as support for hiring and entrepreneurship. It uses a combination of broad active labour market programmes and tailored initiatives aimed at specific sectors and population groups, including young people and long-term unemployed. The overarching objective of the ATIVAR.PT programme is to deliver swift and comprehensive assistance to individuals seeking employment, aligning with the broader national goal of ensuring economic stability during the challenging times.
The ATIVAR.PT programme encompasses a range of measures, with the primary one being the ATIVAR.PT Internship (Estágios ATIVAR.PT),1 evaluated in this report. This measure offers internships in profit or non-profit entities for nine months or longer in specific cases. The aim of the internship is to facilitate the integration of the unemployed, in particular young people, into the job market and support their professional retraining through the development of practical experience in the work context. Interns receive a monthly internship grant (Bolsa de Estágio), which corresponds to their qualification level, along with other benefits, such as meal allowances, transport allowances and work accident insurance co-funded by the Portuguese Public Employment Service (PES), the Instituto do Emprego e Formação Profissional (IEFP). The internship grant is treated as earnings and is subject to taxes and social security contributions (see Box 3.1).
In addition to the internship, the ATIVAR.PT initiative also offers other measures, such as providing employment incentives to employers. The ATIVAR.PT Employment Award (Prémio ao Emprego ATIVAR.PT) provides employment incentives to the entity that signs an open-ended employment contract with a former participant in the ATIVAR.PT internship. The amount of the award depends on the employee’s salary and is paid to the entity in two instalments. This measure incentivises the continuity of employment for individuals transitioning from internships to permanent jobs. This measure is not evaluated in this report. The ATIVAR.PT Incentive (Incentivo ATIVAR.PT), on the other hand, provides financial support to employers for hiring individuals who did not benefit from PES-funded internships.2 Employers sign open-ended or fixed-term employment contracts lasting for at least 12 months with an obligation to provide professional training to the hired workers. Participation is open to people of all ages, including young people. The primary goal of this measure is to foster sustainable employment and enhance the employability of the groups that face significant barriers in the labour market, for example, recipients of social assistance, disability benefits, persons recovering from addiction, victims of gender-based violence, etc. These measures are not evaluated in this report due to data availability.
Among these three ATIVAR.PT measures, the internship had the largest number of recipients and recorded the highest expenditure in 2021‑22 (Figure 3.1). In 2022, the expenditure dedicated to the ATIVAR.PT internships nearly reached EUR 130 million, constituting 17% of the total expenditure on active labour market policies (ALMPs).
Expenditure on ATIVAR.PT measures and as a share of total PES expenditure on ALMPs, 2021‑22
Note: The horizontal axis shows expenditure on different ATIVAR.PT measures. Percentages next to the bars express expenditure as a share of total PES expenditure on ALMPs in a given year.
Source: Monthly reports of the Public Employment Service (IEFP) on physical and financial execution of active labour market measures (“Relatório de Execução Física e Financeira”), available at www.iefp.pt/estatisticas.
Portugal’s Public Employment Service (IEFP) is responsible for the management and supervision of the ATIVAR.PT internships. To initiate a new internship, an entity must submit an application to the IEFP, providing a comprehensive description of the desired candidate profile based on the proposed professional activities, together with details of the training supervisors. In addition, the entity must ensure that the internship is not intended to replace an already filled position. The IEFP evaluates applications according to pre‑defined criteria that take into account the entity’s location, size, and employability of interns. Potential internship candidates may be proposed by the entities (subject to their registration with and approval by the IEFP) or selected directly by the IEFP from its database of registered unemployed, based on the requirements specified by the entity. Candidates must fulfil the eligibility conditions set out in the internship legislation (Box 3.1) and have the appropriate academic or professional qualifications for the internship area. The final selection decision rests with the promoting entity.
The ATIVAR.PT internship (Estágios ATIVAR.PT) is regulated by the Ordinance 206/2020 of 27 August 2020, amended by Ordinances 122‑A/2021 of 14 June 2021, 331‑A/2021 of 31 December 2021, and 293/2022 of 12 December 2022. The latter of the amendments is not considered in this report since the analysed data cover only the internships which started between November 2020 and April 2022.
A new internship can be initiated by a natural or legal person, profit or non-profit, by submitting an application to the Public Employment Service of Portugal (Instituto do Emprego e Formação Profissional, IEFP). To qualify for the internship, individuals must be registered as unemployed in the IEFP. The internship programme is open to:
individuals aged 18‑35 (18‑30 from 2022) with at least upper secondary education (qualification Level 3 to 8);
individuals aged over 35 (30 from 2022) and up to and including 45 who have been registered as unemployed for more than six months and who have obtained at least upper secondary education or higher level of education (qualification Level 3 to 8) less than three years ago or are registered at the centres for the qualification of adults “Centro Qualifica”;3
individuals over 45 who have been registered as unemployed for more than six months (12 months from 2022), have completed primary or upper secondary education (qualification Level 2 or 3) and are registered in the “Centro Qualifica” or have at least non-higher post-secondary education (qualification Level 4 to 8).
In addition, special groups such as people with disability, victims of domestic violence, refugees, ex‑prisoners, homeless, etc. are eligible for the internship without any specific age or qualification requirements.
Interns are entitled to a monthly internship grant (Bolsa de Estágio), a meal allowance, work accident insurance and, in some cases, a transport allowance. The grant is subsidised by the IEFP at a rate of 75‑95% (65‑95% from 2022) depending on the characteristics of the promoting entity and the intern. The amount of the internship is linked to the Social Support Index (Indexante dos Apoios Sociais, IAS), which is used in Portugal for calculations of various social benefits. The internship grant varies from 1.0*IAS to 2.4*IAS (1.3 to 2.5 from 2022) according to the qualification level of the intern. In 2020, IAS was EUR 438.81 per month. For comparison, the social assistance benefit (Rendimento Social de Inserção) was 43.525% of IAS for a single person, the unemployment assistance benefit (Subsídio Social de Desemprego) was 80% of IAS and the minimum wage was EUR 635 per month, which is approximately 145% of IAS. Thus, the grant offers a wide range of earnings above typical assistance benefits, allowing employers to hire individuals with lower qualification levels at wages below the legal minimum, while paying higher wages to those with higher qualifications.
The internship grant is treated as earnings and is subject to social security contributions and income tax payments, although amounts at or below the minimum wage are not taxed. If an intern pays social security contributions for at least 180 calendar days (which is less than the typical length of most internships), they are entitled to at least five months of unemployment assistance benefit.
During the internship, interns cannot carry out any type of other professional activity. Recipients who have completed a professional internship financed, in whole or in part, by the Portuguese State may attend a new internship under the ATIVAR.PT regulation only if after the start of the previous internship they have obtained a new level of qualification or qualification in a different area. The second internship may occur only 12 months after the completion of the previous internship.
Source: The Ordinance 206/2020 of 27 August 2020 (https://diariodarepublica.pt/dr/detalhe/portaria/206-2020-141259624) and its amendments; the OECD Tax-Benefit database for Portugal 2020 (https://www.oecd.org/social/benefits-and-wages/benefits-and-wages-country-specific-information.htm).
The ATIVAR.PT internship complemented and partially replaced the Professional Internship (“Estágios Profissionais”), which was introduced in 2017 and shared a similar design. The new rules revised the amount of internship grant and introduced new categories of possible recipients, such as homeless people and informal caregivers. In addition, the new measure incorporated a set of transitional rules to address the exceptional circumstances that the country faced during the pandemic, including a rapid inflow of new unemployed. For example, the new transitional rules included an extension of the eligibility criteria for the main target group of young people up to the age of 35, as opposed to the previous limit of 30. For people over 35 years, the prerequisite of being registered as unemployed for 12 months was reduced to six months. The transitional rules also increased the IEFP contribution to the internship grant, raised the proportion of the support paid upfront, and made a number of other minor adjustments, which were reversed in 2022.
The ATIVAR.PT internship was launched in August 2020 and the first participants, who applied for the programme in October, were able to start their internships in November 2020. The enrolment quickly climbed to an average of around 2 000 new entrants per month. By April 2022, the end of the observation period of the evaluation, the number of individuals actively engaged in internships in a given month nearly reached 20 000. While this constitutes only about 5‑6% of the stock of all registered unemployed, the programme covered up to 30‑40% of the young unemployed under the age of 25 (Figure 3.2).
Share of internship participants out of all registered unemployed at the end of the month, November 2020‑April 2022
Note: The data cover internships which started between November 2020 and April 2022. Calculations refer to the continental Portugal.
Source: OECD calculations based on Public Employment Service (IEFP) administrative microdata on internships and the IEFP reports on registered unemployed “Informação Mensal do Mercado de Emprego” available at www.iefp.pt/estatisticas.
Most internship participants are young, with an average age of around 25 years. While the programme is open to individuals over the age of 35, they can only access it after being registered as unemployed for at least six months and meeting some other requirements with respect to their qualification levels (see Box 3.1). Only about 4% of participants in the data are aged 35 or above (Figure 3.3, Panel A).
The standard duration of internships as defined by the legislation is nine months. Only 8% of internships in the sample last longer than nine months (Figure 3.3, Panel B). These longer internships are available to special groups of unemployed, mainly individuals with disability, those recovering from addiction, homeless, and informal caregivers, have an opportunity to pursue internships for 12 months. Also, the internships promoted by entities operating under the special regime of strategic interest vary in duration from 6 to 12 months.4 The evaluation in this report focuses on participants engaged in internships for nine months or less, which constitutes more than 90% of all internships. This focus enables observing outcomes for a longer period of time (for more details see Section 3.3.4. and the technical report (OECD, 2024[1])
More than 80% of the internships start in the first six months after registration in PES (Figure 3.3, Panel C). The report focuses on evaluating this segment of internships to optimise the observation period for post-treatment outcomes, as explained in more detail in Section 3.3.4.
Characteristics of internship participants and internship duration from November 2020 to April 2022
Note: The data cover internships which started between November 2020 and April 2022. Calculations refer to the continental Portugal.
Source: OECD calculations based on the Public Employment Service (IEFP) administrative microdata on internships.
Although the eligibility rules for internships do not explicitly require tertiary education, 70% of interns had higher education and performed jobs in professional and technical occupations requiring high skill levels. For example, 61% of all participants worked as “Professionals” and an additional 16% as “Technicians and associate professionals” during their internships (Figure 3.4, Panel A). In contrast, only around 22% of registered unemployed under 35 not involved in internships had higher education and about 17% sought professional occupations. Internship opportunities spanned across various economic sectors (Figure 3.4, Panel B), but almost a quarter of internships (22%) were in entities specialising in professional, scientific and technical activities. This suggests that internships tend to develop in new and more innovative employment areas that require higher qualification levels.
After the end of the internship, the promoting entity may receive an additional recruitment incentive, the ATIVAR.PT Employment Award (Prémio ao Emprego ATIVAR.PT), if it signs an open-ended employment contract with a former intern who participated in the ATIVAR.PT internship. This measure aims to facilitate the transition of individuals from internships to permanent employment. In 2020, the subsidy was equal to three basic5 monthly salaries of the new employee, up to a ceiling of 7*IAS (Social Support Index). The entity is obliged to maintain the employment contract and employment level for 12 months. The grant is paid in two equal instalments: the first within 30 working days of the grant approval and the second in the 13th month of employment, provided that the employment obligations have been met.
The aggregate data provided by the IEFP show that about 55.2% of the participants, who completed their internships between 2021 and 2023, remained employed with the same employer, while only about 18.6% of the interns benefited from the subsidy. This implies that the subsidy is not necessarily a decisive factor in employers’ hiring decision. The available microdata do not allow identifying receipt of this employment subsidy for former internship participants and, therefore, the analysis in this report cannot distinguish between contracts that benefit from this subsidy and unsubsidised contracts (see Section 3.4 and technical report (OECD, 2024[1]) for more details).
Internships in Portugal play an important role in enhancing the employability of young people, facilitating their smooth transition from education to the labour market by providing valuable practical work experience. Previous studies have consistently demonstrated the positive and lasting impact of internship participation on individuals’ labour market outcomes in Portugal. For example, Duarte et al. (2020[2]) assessed the impact of internships within the Youth Employment Initiative (2013‑18) on young people not in employment, education, or training (NEETs), while Capucha and Godinho (2022[3]) studied the effect of employment and professional internships (2014‑20) on unemployed adults over 30 years of age. The magnitude of the estimated effects varies depending on the type of intervention and the specific population groups involved. Notably, these studies have not yet assessed the impact of the ATIVAR.PT internship in the context of the COVID‑19 pandemic, which is precisely the focus of this report.
Note: The data cover internships which started between November 2020 and April 2022. Calculations refer to the continental Portugal. Occupations are classified according to the International Standard Classification of Occupations (ISCO). Sectors of economic activity are coded according to the International Standard Industrial Classification of All Economic Activities (ISIC).
Source: OECD calculations based on Public Employment Service (IEFP) administrative microdata on internships.
This section presents the methodology and requirements for a thorough impact evaluation of the internship programme. It outlines the reasons for employing quasi‑experimental techniques through the use of linked administrative data for the impact evaluation. The section also details the outcomes assessed, the timeframe of the analysis, and describes the final sample chosen for the evaluation.
3.3.1. Quasi‑experimental techniques are used to estimate programme impact
The evaluation of the ATIVAR.PT internship programme’s causal impact presents several methodological challenges. Simply comparing the outcomes of participants to those of non-participants is not sufficient. Participants are “selected”, meaning that they do not have the same characteristics as non-participants and the two groups are therefore not comparable (see Chapter 4). To overcome this challenge, counterfactual impact evaluations (CIE) seek to compare participant outcomes (the treatment group) with those of a set of individuals as similar as possible to them (the control group) (OECD, 2020[4]; OECD, 2020[5]). The only difference between the treatment group and the control group is that the latter did not participate in the programme. The control group therefore provides information on “what would have happened to participants in the absence of the programme”, that is the counterfactual case.
CIEs methodologies are broadly divided into two categories: experimental evaluations, also known as randomised controlled trials (RCTs), and non-experimental (quasi‑experimental or observational) evaluations. In RCTs, participants are randomly assigned to either the treatment or control group within a target population. If the selection process is truly random, the characteristics of the individuals in the two groups do not differ on average: the groups are therefore statistically equivalent. Thus, the direct comparison of outcomes of these two groups allows to determine the programme’s causal effect.
The ATIVAR.PT internship programme is open to all target population members (as described in Section 3.2). Its voluntary nature implies that participants may differ from non-participants in non-trivial dimensions. Since participation is not determined randomly, it is necessary to use non-experimental methods to analyse the programme’s impact. These methods aim to simulate the randomisation process described above by constructing a control group as similar as possible to the treatment group so that they are statistically equivalent prior to participation in the internship programme. Various methods for estimating the causal impact of programmes using non-experimental data are available, depending on the type of information accessible to researchers (see (OECD, 2020[5]) for a discussion). The technical report (OECD, 2024[1]) provides additional insights into the different methods considered and underpins the method retained for the analysis in this report.
The analysis in this report relies on comprehensive administrative data to compare participants to non‑participants, employing propensity score matching to select a control group of non-participants for comparison. Through this methodology, all observable characteristics influencing programme participation are summarised into a “propensity score”. This score captures an individual’s likelihood of participating in the programme, reflecting all known (and measurable) factors that affect this likelihood. To ensure that estimated programme effects are unbiased and reflect its true impact, it is crucial that the propensity score is calculated taking into account characteristics impacting both programme participation and outcomes. For instance, assume that possessing a higher education level (e.g. a degree) correlates with higher earnings, even in the absence of the internship programme. Suppose it also increases the likelihood of participating in the internship programme. Neglecting to include education level in the propensity score calculation could result in an over-estimation of the effects of the internship on earnings. The estimation would attribute increased earnings to the internship, when in fact it simply reflects the fact that more people with higher levels of education participated in it. By incorporating the education variable into the propensity score, comparisons between treatment and control groups are restricted to individuals with comparable education levels. Section 3.4 illustrates the wide array of variables incorporated in the analysis, providing assurance that such oversights are unlikely to occur. A second critical assumption for the validity of matching methods is achieving a balanced distribution of the propensity score across treatment and control groups. Similar individuals need to be found to compare to one another. Statistical tests are employed to verify this balance and the technical details are provided in the technical report (OECD, 2024[1]).
3.3.2. Data requirements for the impact evaluation of the internship programme
The choice of methodology for evaluating the internship programme significantly influences the data requirements. When employing an RCT to evaluate the impact of a programme, in principle, the only difference between individuals is their enrolment in the programme itself. The design ensures the groups being compared are statistically equivalent, thereby minimising the need for extensive data and control variables for adjustments. Such data are primarily used to verify the balance between the two groups and ensure that the random assignment was properly conducted. However, exhaustive data on socio-demographic characteristics or employment history are not as vital in an RCT. Thus, the primary focus is to collect data on the primary outcomes of interest. For example, to study impacts on employment, then data on employment status are required for participants and non-participants.
It is common for ALMPs like the ATIVAR.PT internship to be implemented first and then evaluated without adhering to a specific evaluation design. In the context of propensity score matching or other quasi experimental or observational evaluations, the data requirements are more extensive than those in experimental evaluations. These methodologies attempt to approximate an RCT setting under specific assumptions. To test these assumptions and ensure the validity of the methodology, it is crucial to gather a significant amount of data.
The primary hypothesis of propensity score matching for causal identification is that, given the measured control variables, outcomes are independent of treatment assignment (the “conditional independence assumption”), thereby allowing to mimic an RCT. Consequently, having a rich set of covariates (or control variables) is essential. Typically, in studies of ALMP, this includes detailed socio‑economic data and previous labour market histories of unemployed people. Without these data, there is a risk that any impact evaluation conducted might merely reflect inherent differences between individuals, rather than any differences driven directly by the programme participation itself. Portugal has provided good quality detailed data for this study which are presented in Section 3.4.
Besides data requirements concerning individual characteristics, data on the outcomes the study intends to examine are equally important. In the case of the ATIVAR.PT internship, the primary objective is to assess whether the internships contribute to improving individuals’ prospects in the labour market. Such prospects could be measured as higher earnings, enhanced job stability, or access to positions offering greater flexibility in working arrangements. However, data on these outcomes are not necessarily available for research purposes. Data related to job flexibility or subjective job satisfaction may only be obtained through surveys designed to collect such information. The data available for Portugal and their use in the analysis are presented in Section 3.4 and in the technical report (OECD, 2024[1]).
For the impact evaluation of the internship programme, Portugal has provided a rich set of administrative data linked across the registers of the IEFP and Social Security. Such data are indispensable for conducting causal impact analyses on policies where randomisation has not been feasible (OECD, 2020[4]).
3.3.3. The impact evaluation examines a range of outcomes
The primary objective of the internship programme is to enhance participants’ employability. Accordingly, the evaluation focuses on assessing its impact on labour market integration, captured by the following outcome variables:
Employment: This variable indicates whether individuals are employed (i.e. have an ongoing employment contract) at a specific point in time. It measures whether participants in the internship programme have a higher probability of finding employment due to their participation (and are thus more likely to be employed) than if they had not participated.
Quality of employment: Beyond determining whether individuals have increased access to employment, it is essential to assess the quality of the jobs found. Due to data constraints, this evaluation predominantly relies on earnings data obtained from social security records. Other dimensions such as contract stability (e.g. permanent, fixed-term, temporary) were not accessible due to data limitations or high levels of missing values, as detailed in Section 3.4 and the technical report (OECD, 2024[1]). Earnings are measured both monthly and cumulatively over time. It is noteworthy that cumulative earnings offer a comprehensive view of participants’ financial progress over time. By tracking their total earnings throughout and after the programme, it is possible to gauge economic well-being. Ideally though, the evaluation should include further outcomes on job quality, such as contract characteristics and prospects in terms of occupation and career mobility.
Unemployment Benefits: An additional dimension crucial for evaluating the internship programme’s impact is the assessment of participants’ reliance on unemployment benefits. Participation in internships and subsequent employment can reduce benefit dependency, while social security contributions paid during internships (see Box 3.1) can make individuals eligible for unemployment benefits if they remain unemployed at the end of the programme. This measure tracks whether individuals receive unemployment benefits. Similar to earnings, unemployment benefits are measured both on a monthly basis and cumulatively over the study period. Examining unemployment benefits alongside employment and earnings provides a comprehensive understanding of participants’ economic circumstances. Moreover, the evaluation of unemployment benefits offers insights into the programme’s potential to alleviate financial strain on individuals and government resources, contributing to a more robust assessment of its overall impact.
Sub-groups of participants are also examined to see whether impacts vary
The effects of the internship programme may differ among different individuals. Existing academic literature on ALMP evaluation has highlighted varying impacts across sub-groups concerning other categories of ALMPs, such as training, subsidies, or job search assistance. This observation is underscored in the meta‑analysis conducted by Card, Kluve, and Weber (2018[6]). To investigate whether similar trends apply to participants of internships in Portugal, the impact analysis in this report is conducted separately for several sub-groups, defined by gender, age, rural and urban residence, previous work experience and by level of education.
3.3.4. Analysis is conducted on data from 2020 onwards to 2023
The analysis focuses on unemployed individuals with an open unemployment spell in July 2020, extending to registrations as of May 2023. Data on outcomes are observed up to August 2023. Given this window of observation, a critical consideration for defining the sample and the definition of “treatment” is ensuring a sufficient post-treatment observation period for analysing outcomes.
To optimise the observation period for post-treatment outcomes, treatment is restricted to individuals enrolling in the internship programme within six months from their registration as unemployed and for internships lasting nine months. This choice ensures a minimal nine‑month window post-treatment for observing outcomes, aligning with the programme’s timeframe and evaluation objectives.
As a result, the final sample comprises the first unemployment spell6 of individuals registering as unemployed between May 2020 (as those registered before would not be able to enrol within six months due to the programme’s starting date) and August 2021 (as registrations after this date would not allow for the observation of outcomes for 24 months: six months to start treatment, nine months of treatment duration, and nine months post-treatment). This approach ensures a logical and consistent framework for analysing the impact of the internship programme on labour market integration.
The final sample used for propensity score matching consists of 495 420 individuals, of which 13 148 were treated (participated in the internship) according to this definition.
Figure 3.5 illustrates the timeline of participants’ registration in the PES, their enrolment in the internships and their subsequent outcomes after the end of their participation. Within the final sample, around 22% of the treated participants started internships within less than one months after their registration in PES, another 52% within 1‑3 months of registration and the remaining 26% within 4‑6 months.
The timeline of participants’ registration in the PES, their enrolment in internships and subsequent outcomes
The evaluation in this report uses data from two main sources: (i) the registry of the Public Employment Service (Instituto do Emprego e Formação Profissional, IEFP), and (ii) the administrative records of the Social Security Institute (Instituto da Segurança Social, ISS).
The IEFP data comprise several datasets containing information on registered jobseekers, including details on their unemployment spells, demographic characteristics, and the types of jobs they are seeking or have previously held. In addition, this data source provides information on the participation of jobseekers in the ATIVAR.PT internships, their duration and the type of activity undertaken. The data cover all jobseekers who had at least one day of registration between 1 August 2020 and 31 May 2023 and all internships started between 13 November 2020 and 30 April 2022.
However, the IEFP data have two important limitations. First, they lack information on whether individuals received the ATIVAR.PT employment subsidy after participating in an internship, should the promoting entity hire a former intern on a permanent contract. As a result, it is not possible to distinguish between subsidised and unsubsidised employment when assessing employment outcomes. Second, the data do not provide sufficient information on participation in other ALMPs to include it in the analysis. Such data could have been valuable in controlling for unobserved differences in motivation between participants and non-participants prior to their participation in the internship, or as an additional outcome variable. A more detailed discussion of the content of the IEFP data and their limitations could be found in the technical report (OECD, 2024[1]).
The ISS data comprise three datasets focusing on employment, earnings, and unemployment benefit receipt. These datasets cover the same individuals as identified in the IEFP data. They provide details on employment spells, employment status, certain job characteristics, monthly remuneration declared by employers and monthly unemployment benefits, if applicable. The data cover the period from 1 August 2017 to 31 August 2023. Due to this extended time frame, the data allow not only to define outcomes for participants and non-participants in terms of employment, wages and benefit receipt, but also to control for pre‑participation outcomes along the same dimensions. It should be noted, however, that information on contract type is missing for almost 70% of employees and information on occupation in employment is not available. Consequently, these characteristics cannot be used to assess the quality of jobs obtained after the internship.
The data provide good quality information necessary to construct a sample of unemployed individuals, who could potentially participate in the programme, and their unemployment spells, identify treatment (i.e. participation in the internship), and define control variables to mitigate potential bias in the impact evaluation. These control variables include socio‑economic factors, unemployment-related characteristics, pre‑treatment outcomes, and geographical information (see Table 3.1 for the list of controls). Including detailed controls for these factors in impact evaluations based on administrative data can effectively mitigate selection bias and account for factors that might otherwise remain unobservable (Lechner and Wunsch, 2013[7]).
Variables used to select non-participants that are alike to internship participants
The technical report (OECD, 2024[1]), which accompanies this report, provides more detailed information on data access, content, quality and limitations. Chapter 4 presents the results of the analysis based on the data and methodology outlined in this chapter.
This chapter has outlined the details of the ATIVAR.PT internship programme introduced in Portugal amidst the COVID‑19 pandemic. Although the programme was open to a wide range of jobseekers, its main focus was on facilitating the integration of young people into the labour market. Between 2021 and 2022, the number of participants increased significantly, establishing the internship programme as one of the most important measures in the ATIVAR.PT package, both in terms of the number of beneficiaries and expenditure. Although not explicitly designed as such, the internship predominantly targets young people with higher education who are seeking positions in professional and technical occupations, thus contributing to the development of new and innovative sectors of the economy.
The report uses quasi‑experimental evaluation techniques to ensure that internship participants are compared to similar non-participants. Using the good-quality administrative data on personal characteristics of jobseekers provided by the IEFP and ISS, this comparison accounts for past labour market outcomes, socio‑economic and personal characteristics of the matched individuals. While the available data allow the assessment of standard labour market outcomes, such as employment, earnings and receipt of unemployment benefits, they unfortunately lack or provide insufficient information on such aspects of employment, as contract type, occupations and subsidised employment.
References
[3] Capucha, L. and R. Godinho (2022), Avaliação intercalar do programa operacional inclusão social e emprego, Ministério do Trabalho, Solidariedade e Segurança Social, https://portugal2020.pt/wp-content/uploads/1.-Av.-Intercalar-POISE-Relatorio-Final.pdf (accessed on 8 March 2024).
[6] Card, D., J. Kluve and A. Weber (2018), “What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations”, Journal of the European Economic Association, Vol. 16/3, pp. 894–931, https://doi.org/10.1093/jeea/jvx028.
[2] Duarte, N. et al. (2020), The evaluation of the Youth Employment Initiative in Portugal using Counterfactual Impact Evaluation methods, UR 30318 EN, Publications Office of the European Union, Luxembourg, https://op.europa.eu/en/publication-detail/-/publication/fa41a92f-dd0a-11ea-adf7-01aa75ed71a1/language-en (accessed on 8 March 2024).
[7] Lechner, M. and C. Wunsch (2013), “Sensitivity of matching-based program evaluations to the availability of control variables”, Labour Economics, Vol. 21, pp. 111-121, https://doi.org/10.1016/j.labeco.2013.01.004.
[1] OECD (2024), Technical Report: Evaluation of active labour market policies in Portugal, OECD, Paris, https://www.oecd.org./content/dam/oecd/en/about/projects/technical-reports-and-presentations-dg-reform/portugal/Technical-Report-Evaluation-of-active-labour-market-policies-in-Portugal.pdf.
[4] OECD (2020), “Impact evaluation of labour market policies through the use of linked administrative data”, OECD, Paris, https://www.oecd.org/els/emp/Impact_evaluation_of_LMP.pdf.
[5] OECD (2020), “Impact Evaluations Framework for the Spanish Ministry of Labour and Social Economy and Ministry of Inclusion, Social Security and Migrations”, OECD, Paris, https://www.oecd.org/els/emp/Impact_Evaluations_Framework.pdf.
← 1. Alternatively, Estágios ATIVAR.PT can be translated as the ATIVAR.PT traineeship.
← 2. The ATIVAR.PT incentive cannot be used by a person who has participated in an internship financed by the Public Employment Service or who has been employed in the same entity or entity belonging to the same group in the previous 24 months, except in some special cases.
← 3. The “Qualifica Centers” form a national network of specialised centres for adult qualification, which are essential structures in implementing the adult qualification strategy in Portugal. The centres are in operation since 2017. The network includes more than 300 centres across the country, thus ensuring greater proximity to the Portuguese population.
← 4. Individuals pursuing internships for more than nine months do not differ substantially by their demographic and labour characteristics from the participants engaged in internships up to nine months.
← 5. Basic salary is the salary that a worker gets for normal working hours. It does not include allowances, bonuses, overtime pay, etc.
← 6. Focusing on the first unemployment spell is essential to eliminate potential biases in the analysis that could arise from multiple registrations or programme participation by the same individual (for more details, see the technical report (OECD, 2024[1])).