The broad question when considering training is whether and how it influences skill development and acquisition to connect people better to jobs. Labour market training (LMT) and self-motivated training (SMT) are designed to achieve this objective with very different underlying training offers. With the introduction of SMT in 2010, better financial support became available to acquire degree level education. This raises the question of how its introduction affected jobseekers’ outcomes as a whole. Does SMT replace LMT for some individuals? Does it complement the training that LMT provides? This chapter addresses these questions by evaluating the impact that SMT has had on the mix and availability of training to jobseekers.
Evaluation of Active Labour Market Policies in Finland
Connecting People with Jobs
7. Interactions between training programmes
Abstract
7.1. Introduction
The analysis in the preceding chapters has evaluated whether self-motivated training (SMT) and labour market training (LMT) are effective in improving jobseekers’ labour market outcomes when considered separately. However, a supplementary question is how the introduction of SMT financing has altered the training landscape in Finland and interacted with LMT. It is now easier for jobseekers to undertake two distinctly different types of training provision, as SMT provides greater financial assistance for degree education for jobseekers than was previously the case. SMT tends to be longer and more academically focussed. LMT has more of an emphasis on vocational courses, which are shorter in duration. There is a small difference in their eligibility criteria, with SMT restricted to individuals aged 25 or older.
What does this mean for the totality of the training provision made available by the Ministry of Economic Affairs and Employment (TEM)? Has the introduction of this additional funding for degree education for jobseekers caused overall outcomes to improve, or has it simply displaced individuals from one type of training to another without leading to any change in outcomes? Has the expansion of training funding provision caused different individuals to participate than might have otherwise been the case? Is training complementary, so that the participation in one programme (LMT, SMT or even the study subsidy (SS) from the Social Insurance Institution (KELA)) improves the outcomes experienced in another?
These are important questions for TEM to consider when considering the optimal mix of training provision for its customers. At present, TEM is only able to influence the amount and content of training provided through LMT. Even for LMT, the planning and procurement for training is delivered by the economic and employment (TE) Offices and the ELY centres and part of LMT funding is provided by the Ministry of Education and Culture via the system of central government transfers to local government. SMT was introduced as a means to complete further education and requires no direction from TEM, except via job counsellor agreement that education would fill an educational gap that would help the individual progress in the labour market. This means that currently TEM has no ability to directly influence the use of this training pathway. Consideration of the extent of autonomy that TEM has to influence training for jobseekers, which formulating policy and legislation, is important in ensuring that jobseekers get the right training.
To address some of the questions above, the report considers the following distinct analytical questions.
1. How has the introduction of SMT financing affected the aggregate outcomes for jobseekers at TEM?
2. Has the increased financial contribution of SMT improved educational attainment and therefore outcomes relative to education financed using the SS from KELA?
The chapter proceeds to address these questions in turn, using administrative data on participation in training and on labour market outcomes to determine what effect the introduction of SMT funding has had. In the first section, the first cohort of individuals that had both SMT, SS and LMT available to them are compared to their previous cohort for whom only LMT and SS were available. Section 7.3 compares SMT participants to similar SS participants, to determine whether SMT offers better outcomes in the labour market.
7.2. Have aggregate outcomes for training participants changed after the introduction of self-motivated training?
SMT introduced two mechanisms by which outcomes for training participants might change. Outcomes for the same individual may be better or worse with different training provision. Compositional differences might also occur, with different types of individuals undertaking training or different types of training. In this case one might expect the introduction of additional financial support for training to expand the number of participants in training, rather than reduce it, but both outcomes are possible.
To attempt an answer to this question, this section looks at outcomes for two different cohorts of jobseekers. These are individuals which were unemployed at the end of either 2008 and 2009 and who participated in either LMT or SMT (via some interaction with TEM) or SS in the following year. Those unemployed in 2008 undertook LMT or SS in 2009, the last year for which only the two were available. Those unemployed in 2009 undertook either LMT, SS or SMT in 2010, when the three different programmes were available. Outcomes for these two cohorts are compared, to determine whether for jobseekers on average, outcomes changed when the suite of available training provision changed.
The timing of the introduction of SMT, shortly after the global financial crisis, means that comparisons are made during a period of labour market flux. Unemployment in 2009 had risen to 8.3%, a 1.9 percentage point increase from 2008. Whilst unemployment remained relatively constant into 2010 (at 8.5%), this period of elevated unemployment and labour market flux in the shorter term means that analysis is potentially complicated by these changes. For example, whilst aggregate unemployment rates remained similar in those two years, the composition of unemployed people changed significantly, with rates of long-term unemployment (12 months plus) increasing significantly in 2010, relative to 2009. Whilst training programmes have been shown to be effective during labour market contractions (Card, Kluve and Weber, 2018[1]), it does mean that comparisons between 2009 and 2010 are being made on across potentially different cohorts of individuals.
The investigation in this chapter relies on comparing cohorts across time periods, which limits the ability to scrutinise different years of training cohorts. Ideally there would be cross-sectional variation in training provision, so that at the same point in time, individuals could be compared under the two different regimes. If this were the case, then the impact of training could be estimated, absent of any effects solely related to time (for example, if the economic cycle impacts upon rates of job finding). However, because SMT was rolled out nationally in 2010, there is no cross-sectional variation in this manner. This means that cohorts must be compared across time, on the assumption that differences related to the time periods themselves do not cause differences in outcomes. To make this assumption as plausible as possible, cohorts that are as close to one another as possible (cohorts of jobseekers who undertook training in 2009 and 2010) are chosen, rather than cohorts further apart in time.
As per the preceding analysis, samples are chosen from the pool of unemployed people at 31 December in the previous year. To ensure the estimates present a true representation of programme impacts, outcomes (such as earnings and employment) are analysed relative to their levels prior to when the person was observed being unemployed. For example, if earnings in year two were EUR 10 000 and previous earnings were EUR 8 000, the estimate would be EUR 2000 for year two. In this way, only relative changes are compared. This removes all differences between individuals that are stable over time. This additional step is taken, relative to the analysis in Chapters 5 and 6, to try to ensure as much as possible that differences between cohorts, which are not observable using administrative data, are accounted for. The introduction of SMT financing for education potentially widen the pool of jobseekers who might take up education or training. The use of differencing outcomes means that all of the time‑invariant differences between these groups of individuals are removed, without any reliance on administrative data to account for observed characteristics.
In order to control for differences that might also change over time, only individuals that look similar to each other are compared to one another (more detail on this is given in Chapter 4, Section 4.4). To determine how successful this method of analysis has been in removing potential differences that would cause estimates to be incorrect, outcomes are reviewed in the period prior to unemployment and training participation. It should be the case that no differences are found between the groups being compared in this period. In the periods before any training programme has taken place, differences must only be driven by individuals having different innate characteristics.
The analysis includes the following steps. First, unemployed individuals are identified at the end of the years 2008 and 2009 and are matched to training spells in the following calendar year (2008 cohort with LMT or SS in 2009, 2009 cohort with LMT, SS or SMT training in 2010). Second, the analysis keeps only individuals with training in either year and defines an indicator which is set to one for individuals unemployed in 2009 and zero for individuals unemployed in 2008. Next, the likelihood of an individual belonging to the 2009 unemployment cohort is estimated using detailed socio‑economic and labour market data (see Chapter 4 for more details). Fourth, these estimated likelihoods are used to construct a group of LMT/SS participants from the 2008 cohort who look similar to the 2009 LMT/SS/SMT cohort. Finally, it is estimated whether the 2009 LMT/SS/SMT cohort experiences higher/lower/the same changes to their employment and earnings, relative to their previous employment and earnings, compared to the 2008 LMT/SS cohort in the years following their training participation.
7.2.1. The national roll-out of self-motivated training in 2010 makes it difficult to compare across training cohorts in different years
Despite being able to successfully control for pre‑programme earnings and employment differences between a cohort of jobseekers in 2008 that undertook LMT or SS training in 2009 and a cohort of jobseekers in 2009 that undertook either LMT, SS or SMT in 2010, comparison of post-programme outcomes is biased by effects due to differences in the economic cycle. Figure 7.1 shows that the pattern of effects on earnings for the training cohorts is very similar to the pattern of effects on earnings when comparing jobseekers in the same two years who do not undergo any training (labelled “Robustness” in the chart). For the analysis to be robust, there should be no difference in earnings patterns for the unemployed who receive no training. This means it is not possible to interpret this trajectory as a result of participation in LMT, SS or SMT relative to LMT and SS only. This analysis is similar for effects on employment and is detailed in the technical report (OECD, 2023[2]).
One additional analytical check can be performed to try to remove the effect of the economic cycle, by using the changes to the unemployed without training as a proxy for year specific impacts. This is done by subtracting the changes to jobseekers without any training from the changes to LMT+SS+SMT cohort. When this is done, no clear pattern emerges on the impact of LMT+SS+SMT across earnings, though when repeated for employment there is a positive impact across the shorter term (years one to three). This relies on the assumption that changes to outcomes for the non-training participant unemployed provide a good proxy for the changes that would be experienced by the training participants in the same period (more detail is provided in the technical report (OECD, 2023[2])).
In order to provide compelling evidence on how the introduction of SMT affected the uptake of and outcomes from training, relative to a world where only LMT or SS was available to jobseekers, it would be beneficial to undertake a trial which altered eligibility for some individuals. The nature of SMT means it is the individuals who participate in training that determine this participation, to a greater extent than counsellors in TE Offices. Therefore, it is not viable to use potential differences in regional intensity of training to determine its impact. This is because differences are driven by individuals and not due to differences in policy interpretation by offices or staff. There is currently no means by which to robustly compare differences in eligibility between individuals as the same point in time. A randomised trial would resolve this and remove the challenges to analysis that are currently evident from comparing cohorts across different time periods.
Figure 7.1. Changes to earnings over time are driven by underlying differences in the economic cycle
LMT: Labour market training. SMT: Self-motivated training. SS: Study-subsidy.
Note: The main series in blue compares the additional earnings that a cohort of jobseekers in 2009 with either LMT, SS, or SMT participation in 2010 experience relative to a cohort of jobseekers in 2008 with LMT or SS only in 2009. The “robustness” series presents the additional earnings a set of unemployed jobseekers in 2009 relative to a set of unemployed jobseekers in 2008, neither of whom participated in any training.
All outcomes are expressed as differences compared to their level in year ‑4. Year 0 is the year in which an individual is identified as being unemployed at the end of the year. For the cohorts with training, this training starts in year 1. The analysis presents nearest-neighbour propensity score matching results which matches individuals based on several characteristics: duration in unemployment until the point of observation, history in unemployment over the past two years (spells and days), age, gender, marital status, education level and field, Finnish nationality, the type of municipality, the quarter of registration into unemployment (time trend), as well as previous year employment status, income. The confidence intervals are shown at the 5% level of significance and represented by the whiskers delimiting the dotted lines on the charts.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
7.3. Does self-motivated training differ to the study subsidy?
It is important to understand how SMT interacts with or replaces other benefits which provide support for studying. One such support is SS, paid by KELA to adults to continue education. Its financial contribution is small, averaging around EUR 1 100 per annum for the unemployed cohorts observed between 2012 and 2014. As discussed in Chapter 4, SMT supports individuals with a higher payment than they could receive under SS, so it is possible this additional payment helps them to continue in their studies and complete their education. In both SMT and SS the underlying education can be the same, the only difference is in the amount of financial support that an individual receives. If it is the case that SMT improves educational attainment, then this additional cost to the taxpayer may be worthwhile if it helps to improve labour market outcomes This chapter asks the question of what outcomes look like for participants that begin SMT relative to those for individuals that start using the SS, to discern whether any differential patterns emerge as a result.
To address the question, the SMT participants are compared against a group of SMT non-participants, all of whom participate in education and receive the SS instead. Participants are compared across the years 2012‑14 to utilise the same timeframe as the individual analysis on SMT and LMT in the preceding chapters. To make the groups more comparable, individuals are classed as participating in SS if it is the first instance of them having done so in the data, to avoid individuals who may have begun receiving this subsidy previously. This reduces the sample sizes somewhat. There are an annual average of 16 000 unemployed individuals in this period receiving SS, for whom around 7% are in receipt for the first time (around 900 per annum). This contrasts to around 7 500 individuals starting SMT per year. Furthermore, analysis is restricted to individuals aged 24 and above, to preclude the inclusion of SS participants that would be ineligible to participate in SMT. This reduces the sample size further, so that only 1 250 participants remain across the sample period. Participation for SS is defined as having a positive annual payment of SS.
The small sample sizes involved and the large degree of difference between SMT participants and SS participants mean that it is challenging to use propensity scores to generate similar groups and the results should be interpreted with a degree of caution. The matching manages to create a pool of SS participants that are more similar SMT participants, though some differences remain between the two groups. Table 7.1 demonstrates that whilst across most dimensions, matching is successful at reducing differences between SMT and SS participants, for some characteristics matching reduces the similarities of the two groups. For example, the proportion of car ownership is more similar between the SMT participants (44%) and the unmatched SS participants (42%) than it is between SMT and matched SS participants (30%). This can be a problem for algorithms in any matching analysis but is exacerbated when the sample sizes are small, as in this case, when it becomes more difficult to find an individual which matches another closely across all dimensions (also see OECD (2023[2]) for more details on standardised difference in means pre‑ and post-matching). When evaluating earnings and employment prior to unemployment, this matching does manage to remove the differences between the SMT and SS for employment outcomes. It is similarly successful in balancing annual earnings in the year before the first observation, but in year zero, earnings for the SMT cohort are above those for the SS.
Table 7.1. Participants in self-motivated training and study subsidy differ considerably
Comparison of observable characteristics by treatment status
Note: Marital status, geographic location, Level and Field of Education, Female, Finnish National and Car ownership all express proportions of the population. Other variables are mean values. Earnings in Euros. Field of Education – Unknown/other includes Natural Sciences and Education, to avoid statistical disclosure of small groups. For employment and unemployed history, years are relative to the year in which the individual was unemployed at the end of the year. So that ‘year‑1’ represents annual values of the variable in the year preceding the unemployment (if an individual was unemployed in December 2012, year‑1 would represent variable values for 2011). SMT participants represent those participants that have been matched with a SS participant. This does not include all SMT participants, as nearest-neighbour matching is used with a 0.05 caliper, meaning that individuals without a match within +/- 0.05 of their propensity score are disregarded. 92% of all SMT participants have a matched SS participant. Unmatched SS participants represent all SS participants, aged 24 and above, matched SS participants include only those matched to a similar SMT participant and is weighted so that participants that are matched to more than one SMT participant contribute more to the overall variable value.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
When looking at future outcomes over time, there is tentative evidence to suggest that in the longer run, there may be additional benefits of SMT, relative to those individuals who use only SS grants (Figure 7.2). Similar to the main analysis in Chapter 5, a larger lock-in period is observed for SMT participants relative to those receiving SS. However, a statistically significant effect is found on employment (0.77 months of additional annual employment) is observed by year four and the point estimate for annual earnings is positive, if insignificant (EUR 2 200). Given that SMT is designed to help individuals augment skills that may give rise to lifelong benefits in the labour market, such early evidence on the emergence of positive trends is welcome. It is at least suggestive that the introduction of SMT may have allowed individuals who would have otherwise utilised the SS route towards meaningful accumulation of further skills, that enable them to progress their careers as is discussed in Chapter 5. The larger lock-in period observed, for educational programmes that should be similar between SS and SMT, suggests that the additional financial contribution that SMT provides does help individuals to continue in education. In this respect it seems that SMT is well-targeted towards enabling jobseekers to increase their educational attainment and there are no obvious displacement effects whereby the payment of SS at a lower rate could generate the same outcomes at a lower cost to the taxpayer. There is no evidence to suggest that increasing the lower age limit of SMT, in the expectation that individuals could instead utilise SS, would be beneficial to individuals and may decrease their likelihood of completing studies. On the contrary, neither is there evidence that lowering the age limit for SMT below 25 would be beneficial either. The current lower age limit of 25 for SMT seems an appropriate compromise between facilitating participation in SMT (and thereby increasing the likelihood of continuing in education) and restricting substitution at lower ages, for whom studying without unemployment benefit is a routine occurrence and represents the continuation from lower levels of education.
Figure 7.2. Self-motivated training has a larger lock-in period but better longer-term outcomes relative to the study subsidy
Note: The analysis presents nearest-neighbour propensity score matching results which matches individuals based on several characteristics: duration in unemployment until the point of observation, history in unemployment over the past two years (spells and days), age, gender, marital status, education level and field, Finnish nationality, the type of municipality, the quarter of registration into unemployment (time trend), as well as previous year employment status, income. The confidence intervals are shown at the 5% level of significance and represented by the whiskers delimiting the dotted lines on the charts.
Source: OECD calculations based on Statistics Finland’s repository: FOLK and TEM datasets.
7.4. Conclusion
To summarise, it is challenging to determine precisely how the introduction of additional financing for education via SMT affected outcomes from training participation in aggregate for jobseekers. This is because it was introduced close to the global financial crisis and rolled out nationally rather than piloted first. However, it is important to recall that the evaluation of each programme separately in the preceding chapters has shown that both SMT and LMT have positive impacts on longer term labour market outcomes. Qualitatively the introduction of SMT has also introduced a mechanism by which jobseekers can access different types of education for skills accumulation with greater financial support. The fact that both programmes are independently able to provide benefits to their participants (Chapters 5 and 6) provides reassurance that the system of training for jobseekers in Finland is helping individuals to connect with jobs in the labour market.
To provide more insight into how the suite of provision is viewed by both jobseekers and case workers at TE offices, qualitative analysis may be beneficial to further investigate the mechanisms underlying the choices between training and the subsequent outcomes. A detailed survey of caseworkers and training participants (and non-participants) motivations, thoughts and feelings regarding the suite of training packages on offer would allow a thorough process evaluation to be conducted, enabling more scrutiny of potential causal mechanisms, outside of a narrower quantitative evaluation. Similarly, considering how administrative data are collected and reviewed in the future may aid operational decision-making. At present, there are few data within TEM to guide policy makers and operational staff on how SMT are selected, progressed and completed by individuals, because it is unnecessary for its practical implementation (education is run by the Ministry for Education and Culture and ongoing participation is confirmed to KELA and the unemployment insurance funds). However, this makes it more challenging to scrutinise and review how the policy actually operates. This is of increasing importance now as Finland embarks on labour market reforms, which introduce more contact between jobseekers and TE Office counsellors. This is centred around conditionality and jobseeking requirements, however, it also bears more thought into precisely whether and how caseworkers might keep contact with individuals undergoing training, particularly those undergoing SMT. Keeping jobseekers abreast of labour market developments, to minimise as far as possible the lock-in effects that are present from delaying job search to participate in training, may enable better outcomes for both individuals and government.
An analysis of the relationship between SMT and SS has provided some tentative evidence that they are complementary. When reviewing SMT alongside SS, the results suggest that in the longer-term SMT provides for greater growth in earnings and employment. Despite a larger initial lock-in period, individuals appear less likely to drop out of education and therefore improve their potential labour market outcomes. The ability for participants to retain unemployment benefits whilst in the education system, is important as it allows older participants, who are more likely to have families, greater financial support to continue their education (see Table 4.2 in Chapter 4 for descriptive statistics on SS and SMT). The differences in the cohorts undertaking SS and SMT mean that there is not a large degree of overlap between the two cohorts to begin with. Furthermore, the age restriction on SMT, so that individuals under the age of 25 cannot access it, helps to keep the SS and SMT cohorts relatively distinct. For those individuals that utilise SMT rather than SS, they have relatively better long-term outcomes, so that any displacement from SS to SMT produces better results for participants. The lower age cut-off at 25 for SMT seems to be an appropriate compromise between allowing participants to benefit from SMT’s financial support without crossing into the routine educational system where university courses are a natural successor to prior education and are relatively separate to the labour market.
References
[1] Card, D., J. Kluve and A. Weber (2018), “What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations”, Journal of the European Economic Association, Vol. 16/3, pp. 894-931, https://doi.org/10.1093/JEEA/JVX028.
[2] OECD (2023), Technical Report: Evaluation of active labour market policies in Finland, OECD, Paris, https://www.oecd.org/els/emp/FITechnicalReport2023.pdf.