What sets high-quality career guidance apart? After reviewing survey evidence on users’ satisfaction and outcomes, this chapter discusses policy measures that countries have put in place to improve quality. Measures include: certifying organisations against quality standards; defining staff qualifications and competencies; measuring outcomes; producing and using high-quality labour market information; and developing tools to promote tailored career guidance services.
Career Guidance for Adults in a Changing World of Work
Getting Skills Right
3. Quality and impact
Abstract
In Brief
Career guidance for adults has the potential to improve employment, education and training outcomes, while mitigating skills shortages and smoothing the business cycle. To achieve these desired outcomes, services must be of high quality. High-quality career guidance is personalised, based on current labour market information, and delivered by well-qualified advisors. Governments can support quality service delivery by defining quality standards and encouraging the monitoring of outcomes. The findings of this chapter can be summarised as follows:
Three-quarters (75%) of adults who received career guidance reported being satisfied or very satisfied with the guidance they received. Most adults (70%) experienced an improvement to their employment, education or training status within six months of receiving guidance. However, only 22% said that guidance was useful in achieving that outcome.
Receiving a personalised career development roadmap is associated with a 25% higher likelihood of improved employment prospects within six months of receiving career guidance. It is also positively associated with enrolling in education and training. Face-to-face provision of career guidance makes a positive difference to employment outcomes. There is also some evidence that the type of provider influences outcomes, underscoring the importance of professionalised services: provision by employers or employer associations are found to be linked to a positive employment outcome and provision by education and training providers is positively associated with education outcomes.
Quality standards were used in 7 out of 21 career guidance programmes reported in the OECD policy questionnaire, ‘Career Guidance for Adults’. In some cases, providers must meet prescribed quality standards in order to receive public funds. Voluntary standards also exist, and are a useful tool for motivating quality improvements and attracting users.
While the job of a ‘career guidance advisor’ is not a regulated profession in most countries, employers generally require minimum training and qualifications for employment. Requirements vary by context, but a tertiary degree is usually required. Advisors working in public employment services have lower qualification requirements, but are often required to complete in-service training. A growing number of countries use competency frameworks to design training and qualifications.
Monitoring outcomes of career guidance holds providers accountable. Providers can conduct self-evaluations or have external bodies conduct evaluations. Typical outcomes measured are economic (e.g. employment, wages) and social/psychological (e.g. job satisfaction, user satisfaction, overall well-being).
High quality labour market information is objective, up to date, fit for purpose and sufficiently granular. Adults have different information needs from young people. For instance, they require advice on flexible career pathways that specify how to transition from one occupation to another while focusing training efficiently on their skill gaps.
Tailored career guidance starts with a thorough assessment of an adult’s skills. Skills profiling tools are still uncommon across OECD countries, while interviews and self-assessment tools are more common. Half of all surveyed adults report receiving a personalised career development roadmap as an output of guidance.
Introduction
Career guidance for adults has the potential to improve employment, education and training outcomes. For the economy as a whole, it can mitigate skills shortages, smooth the business cycle by facilitating structural adjustment, and boost productivity by connecting adults with education and training opportunities. To have these desired positive outcomes, however, services must be of high quality.
Assessing and assuring the quality of career guidance services is made challenging by the variable nature of service delivery, which is ideally adapted to different contexts and to different user’s needs. As outlined in Chapter 2, the provision of career guidance spans multiple settings. Adults needing guidance do not fit one mould: they may be unemployed; employed but at risk of displacement; employed but looking for a career change; or returning to work after years out of the labour force. Each of these users has different guidance needs, requiring different resources and tools. The variable nature of career guidance services poses challenges for defining what constitutes high-quality service.
This chapter first presents survey evidence of the perceived impact of career guidance services, focusing on adults’ overall satisfaction with the services they received and their employment and training outcomes. The chapter then discusses policy measures that countries could put in place to improve quality provision. It elaborates three components of high-quality provision: producing and using high-quality labour market information, tailoring career guidance to individual needs, and standardising the training and qualifications of career guidance advisors. It then looks at two ways to ensure quality: certifying providers against quality standards, and monitoring outcomes.
3.1. How satisfied are adults with career guidance services?
The OECD Survey of Career Guidance for Adults (SCGA) provides insights into the perceived impact of career guidance services. Across the countries surveyed, overall satisfaction with career guidance was high, with 75% of adults who had received career guidance services in the last five years reporting that they were satisfied or very satisfied with the guidance they received (Figure 3.1). General satisfaction was highest in Chile (83%) and the United States (79%) and lowest in Italy (67%). Satisfaction levels tend to be higher the more users report that guidance was well informed by labour market information and tailored to their needs (Figure 3.1).
Overall satisfaction with career guidance varies by provider. While the public employment service (PES) tends to be the largest provider of adult career guidance, dissatisfaction with its services is high (Chapter 2). Dissatisfaction with counselling offered by the PES could point to a failure to meet the more specialised needs of employed adults who seek guidance to progress in their current job or to change jobs. Improving the quality of services for employed adults means better tailoring services to their needs, by adapting the training and qualification requirements of staff, providing more relevant labour market information, and using appropriate tools to assess their skills and to define personalised career and training pathways. These policy approaches are discussed later in this chapter.
Figure 3.1. User satisfaction with guidance, and perception of guidance as well-informed and well-targeted
Note: The average includes the six countries covered by the SCGA: Chile, France, Germany, Italy, New Zealand and the United States. Data refer to the last time the respondent spoke to a career guidance advisor.
Source: OECD 2020 Survey of Career Guidance for Adults (SCGA).
3.2. What are the outcomes of guidance?
Box 3.1 summarises the evaluation evidence on the impact of career guidance on three types of outcomes: learning and skills, participation in training, and employment. The literature suggests that career guidance is highly effective at improving learning and skills and training participation among adults. It is also effective at helping unemployed workers to find jobs, though evidence is less robust as to its impact on career progression and job satisfaction.
According to the SCGA, the majority (70%) of users report some change to their employment or training status in the six months after receiving career guidance (Figure 3.2). A quarter (25%) of users made progress in their job (e.g. obtained a promotion), while 19% enrolled in an education or training programme. The next most common change was moving to a new job in the same industry (17%), followed by moving to a new job in a different industry (16%). In Italy and Germany, users were least likely to report any change to their employment and training status. Adults in Chile were more likely to report making progress in their job or enrolling in an education or training programme. Adults in the United States were more likely to move to a different job in either the same industry or a different one.
Table 3.1 summarises results from a regression of the likelihood of achieving employment or training outcomes after receiving career guidance, while controlling for a set of individual, job and firm characteristics. Two factors stand out as being highly associated with positive employment outcomes: receiving a personalised career development roadmap (increases the likelihood by 25%), and using services delivered by an employer or employer group (both increase the likelihood by 10%, relative to services delivered by the PES). Having face-to-face interaction with a career guidance advisor is also associated with a higher likelihood of achieving positive employment outcomes (4% higher than remote alternatives). When it comes to education and training participation, an adult is most likely to enrol in a programme after receiving career guidance from an education or training provider (17% higher than when provided by the PES), followed by an employer group (12% higher), or a dedicated public career guidance service (8%). Receiving a personalised career development roadmap also raises the likelihood of enrolling in an education or training programme by 7%.
Figure 3.2. Employment, education and training outcomes of career guidance
Note: Average for the six countries covered by the SCGA: Chile, France, Germany, Italy, New Zealand and the United States. Respondents could choose more than one answer. Data refer to the last time the respondent spoke to a career guidance advisor. Respondents were asked whether any of these outcomes occurred within six months of receiving career guidance.
Source: OECD 2020 Survey of Career Guidance for Adults (SCGA).
While most users reported a change to their employment and training status, few attributed the change to having received career guidance services. Only 22% of users say that career guidance was useful in achieving that outcome (Figure 3.2). Perhaps adults do not fully appreciate the impact of career guidance, given the conflating influence of other factors, including family and friends. Adults who seek out career guidance may have particular characteristics, like strong motivation, that make them more likely to network, apply for jobs and enrol in training. Relative to their own efforts, users may not view their meeting with a career guidance advisor as having an important impact on their employment and training outcomes. Self-perceptions of impact are subjective in nature. Without a counterfactual of what might have happened had users not received career guidance, it is impossible to attribute outcomes reliably.
Nevertheless, it is important to note that adults value career guidance services beyond their capacity to bring about employment and training outcomes. High satisfaction levels despite an absence of perceived employment and training outcomes provide evidence for this. Users may value the psychological benefits of career guidance, such as higher self-esteem, sense of well-being, self-confidence or insight, awareness of opportunities, and future direction (Kidd, Jackson and Hirsh, 2003[1]). They may also value the proven opportunity to learn new skills, like decision-making and information-seeking skills (Maguire, 2004[2]).
Table 3.1. Employment, education and training outcomes of career guidance
Marginal effects from a probit regression
Note: The dependent variable “Employment outcome” takes value 1 if a respondent reported at least one of the following outcomes: “Found a new job in the same industry”, “Found a new job in a different industry”, “Progressed in my current job (e.g. got a promotion)”, or “Moved from informal employment (without a contract) to formal employment,” and 0 otherwise. The dependent variable “Education outcome” takes value 1 if the respondent reported that they “Enrolled in a new education or training programme “and 0 otherwise. The regression includes additional controls for country, age, place of residence, education, gender, employment status, migration, firm size and contract type. The table reports marginal effects, i.e. percentage change in the outcome variable following a change in the relevant explanatory variable. Marginal effects for categorical variables refer to a discrete change from the base level. *,**,***: statistically significant at the 1%, 0.1%, and 0.01% level, respectively.
Source: OECD 2020 Survey of Career Guidance for Adults (SCGA).
Box 3.1. Evidence on the outcomes of career guidance for adults
Learning and skills
Empirical evidence shows that career guidance can have a positive impact on short-term learning outcomes like decision-making skills, information-seeking skills, self-awareness, and job search skills (Killeen and Kidd, 1991[3]; Bimrose, Barnes and Hughes, 2009[4]; Maguire, 2004[2]; Kidd, Jackson and Hirsh, 2003[1]). It can increase confidence and motivation (Bimrose, Barnes and Hughes, 2009[4]), and improve adults’ attitudes towards learning (European Commission, 2015[5]).
Training participation
There is evidence that voluntary exposure to guidance increases the likelihood of adults participating in education and training relative to similar adults not exposed to guidance. Several impact evaluations of publicly funded career guidance for adults in the United Kingdom found a significant impact of guidance on participation in education or training (Lane et al., 2017[6]; Killeen and White, 2000[7]). In experiments with Individual Training Accounts (ITAs) in the United States, evaluations showed that take-up of ITAs was highest where counselling was offered on a voluntary basis without being too directive, while worse results were obtained when counselling was mandatory (Gautié and Perez, 2012[8]).
Employment
Evidence on longer-term impacts – like the quantity and quality of employment matches, or career progression – is less robust. The UK impact evaluations mentioned above found no positive impacts on employment, benefit receipt, career progression or job satisfaction. In a meta-analysis of evaluations of active labour market programmes, Card, Kluve and Weber (2015[9]) found that job search assistance (which included guidance as a component) increases the probability of employment in the short-run and is more cost-effective than other active labour market programmes (i.e. training, private sector incentives, public employment). The impact on the long-term probability of employment is small. In another review of the literature, Brown (2006[10]) concluded that interventions designed narrowly to help people find a job (with a focus on imparting job-search and interview skills, but also including a guidance component) are very effective in helping people find jobs, but may not be as successful in ensuring satisfaction in those jobs.
3.3. Policies to promote high-quality career guidance services
Countries can influence the quality of career guidance services in several ways. This section first elaborates three components of high-quality provision: producing and using high-quality labour market information, tailoring career guidance to adults’ needs, and standardising the training and qualifications of career guidance advisors. It then looks at two ways to ensure quality: by certifying providers against quality standards, and monitoring outcomes.
3.3.1. Producing and using high-quality labour market information
Providing effective career guidance depends on producing and using high-quality information about the current and future labour market. Career guidance professionals rely on such information to provide clients with accurate advice about their labour market prospects. The availability of high-quality labour market information is also crucial for the many adults who search online to learn about their career, education and training options (e.g. through online portals, see Chapter 2).
Producing high-quality labour market information
Labour market information (LMI) is systematically collected and disseminated in all OECD countries (OECD, 2016[11]), though the type of information and the approaches and tools used to collect and disseminate this information vary. They include surveys of employers, adults or graduates; administrative data; online vacancy data; forecasts or foresight exercises; and sectoral or occupational studies. High-quality labour market information is objective, timely, sufficiently granular, fit for purpose, and well-coordinated (Box 3.2).
Box 3.2. What constitutes high-quality labour market information?
High-quality labour market information (LMI) is:
Objective. LMI should be free of bias, and grounded in research evidence.
Timely. Given how quickly the labour market evolves, LMI should be regularly updated.
Sufficiently granular. Local, regional or sectoral-level data are often more scarce than national-level data due to higher costs associated with achieving sufficient sample sizes. But granular LMI enables advisors to provide tailored advice.
Fit for purpose. To inform quality career guidance for adults, advisors need information about current and future labour market needs, as well as about flexible pathways that facilitate transitions from one occupation to another.
Well-coordinated. Building a national LMI system requires coordination from many different stakeholders. Information about employment and education pathways is most useful when it is integrated.
Source: Summarised from OECD (2016[11]), Getting Skills Right: Assessing and Anticipating Changing Skill Needs, https://dx.doi.org/10.1787/9789264252073-en.
Adults and young people have different LMI needs. Compared with young people, adults may be more interested in learning about shorter education and training programmes that can be carried out close to home and in flexible formats, e.g. part-time, during evenings, weekends, or in modules. Information on flexible pathways from one occupation to another, based on an analysis of skills gaps, will also be of particular interest to adults.
Advancements in scraping technologies, big and open data, the use of artificial intelligence, and online surveys have enabled diverse players to produce LMI, and this has both advantages and disadvantages. It enables the production of more data at finer levels of granularity (e.g. local, sectoral). It also means that data are updated more quickly. But such data have disadvantages as well, including the underrepresentation of certain groups. For instance, a UK study compared occupational demand using scraped online vacancy data versus labour force survey data. Low-skilled occupations were under-represented in the scraped online vacancy data relative to the more traditional labour force survey data (Souto-Otero and Brown, 2016[12]). Another challenge with having so many players producing LMI is quality assurance.
Some countries have taken steps to assure the quality of LMI. The United Kingdom set up LMI for All; an online repository of data that collects, vets and standardises existing labour market data. Career development practitioners work with software developers to design online platforms that showcase selected data from LMI for All in a way that suits their clients’ needs. Another approach is to set quality standards for LMI and its use. One outcome of the 2003 Danish Act on Guidance was generating conditions for tailored and high-quality LMI in guidance. The Austrian PES (Arbeitsmarktservice, AMS) sets central minimum standards for service delivery, including access to up-to-date and gender-sensitive career information. These standards apply nation-wide and each AMS decides autonomously how to implement them.
Using labour market information in career guidance
Countries could make better use of LMI in career guidance. Possible uses of LMI in career guidance include training advisors in the most up-to-date LMI available, promoting development of skills in high-demand, and updating online platforms. In the OECD 2020 Policy Questionnaire, ‘Career guidance for adults’, only 12 out of 21 countries reported that they use LMI information to inform career guidance. Furthermore, while most adults say that career guidance is well informed (Figure 3.1), less than half received information about education and training opportunities, job vacancies, or sectors currently in high or low demand (Figure 3.3). Adults are even less likely to receive information about sectors forecasted to be in high or low demand in the future, the quality of training providers or about financial support for training.
Figure 3.3. Type of information that adults receive from career guidance advisors
Note: Average for the six countries covered by the SCGA: Chile, France, Germany, Italy, New Zealand and the United States. Respondents could choose more than one answer. Data refer to the last time the respondent spoke to a career guidance advisor.
Source: OECD 2020 Survey of Career Guidance for Adults (SCGA).
Training career guidance advisors in the most up-to-date LMI is one way to make better use of LMI in career guidance. Without support, advisors struggle to locate the information users need, and to interpret it correctly. Canada’s Labour Market Information Council conducted a survey of career development practitioners and found that only 60% think that LMI is easy to understand, and less than half (43%) say that they received training to help them access or make sense of the data (LMIC, 2019[13]). Advisors are often expected to keep themselves informed about labour market developments, though training is provided in some countries. In the Cités des Métiers centres in Belgium, advisors participate in weekly information sessions delivered by a specialist. In Sweden, the PES (Arbetsförmedlingen) and firm representatives work in close collaboration to offer labour market information training sessions to teachers and career counsellors in schools. The career guidance advisors in charge of France’s CEP receive training sessions to stay up to date about government reforms, economic changes, labour market cycles, innovations and digital transformations affecting the labour market. As part of Mexico’s Employment Support Programme, advisors are offered training on the behaviour of local and regional labour markets.
Promoting the development of skills in high demand is another way to make better use of LMI in career guidance. Only 7 out of 21 countries reported promoting the development of skills in high demand as a specific aim of career guidance programmes. In Belgium’s Dispositif d’orientation tout au long de la vie (OTLAV), guidance advisors promote skills in high demand in both group information sessions and individual guidance interviews. Users of career guidance in Belgium’s VDAB (PES) are encouraged towards high-demand occupations based on sectoral development plans, while taking into account their capacity, interest and competences. To benefit from free training, low-skilled adults in Estonia must first undertake career counselling (Karjäärinõustamine) and they are encouraged towards training in skills in demand. In Spain, guidance for the long-term unemployed supports upskilling and reskilling in strategic sectors.
Finally, updating online portals is another way to make better use of LMI. As noted in Chapter 1, 69% of surveyed adults searched online for employment, education and training information in the last five years. The majority (72%) of adults who looked online for information about employment, education and training opportunities said that they found this information to be user-friendly or very user-friendly (Figure 3.4).
Figure 3.4. User-friendliness of online information
Note: The average includes the six countries covered by the SCGA: Chile, France, Germany, Italy, New Zealand and the United States. Data refer to the last time the respondent spoke to a career guidance advisor. Online information users looked online for information on employment, education or training opportunities online in the last five years.
Source: OECD 2020 Survey of Career Guidance for Adults (SCGA).
3.3.2. Tailoring career guidance services to adults’ needs
To be effective, career guidance should be tailored to an adult’s particular needs. This requires taking the time to understand the client’s objectives and to assess their unique skill set. According to the SCGA, 70% of career guidance users felt that the advice they received was targeted to their specific needs (Figure 3.1). However, only half (51%) confirmed that they received a personalised career development roadmap (Figure 3.6).
Organisational pressures can create disincentives to tailored service. Public employment services often reward counsellors for quickly matching jobseekers with jobs. This approach contributes to less personalised services, by prioritising quick entry into employment over high-quality job matches. A “revolving door” phenomenon can result, whereby adults rotate back and forth between periods of employment in poorly fitting jobs and periods of unemployment. A more personalised approach takes the time to explore and address underlying obstacles to employment and to support the adult in finding work that is a good fit. It may entail first helping them to identify and complete training to address skills gaps.
This section looks at two aspects of tailored career guidance: first, assessing an adults’ unique skill set, and then developing a personalised career development roadmap that plots out a sequence of activities to achieve his or her objectives. It also considers how to tailor the information and advice presented on online career guidance portals.
Assessing an individual’s unique set of skills
Assessing an adult’s skills is a necessary first step in advising them about possible occupations to consider and building a personalised career development roadmap. For adults who are in need of retraining but are not aware of their options, carrying out a thorough assessment of their skills constitutes an essential starting point to design individualised reskilling pathways.
The most common approach is to interview clients and ask them questions about their work experience, qualifications and current skills (Figure 3.5). Two-thirds (67%) of users were asked in interviews about their skills and experience, and 24% were asked about their qualifications and certificates. Interviews provide highly useful information that helps counsellors assemble a well-rounded perspective on the user’s potential and needs. It also helps to build trust and rapport with the individual. But interviews are subjective, and tend to rely heavily on job history or educational qualifications as a proxy for skills. Some individuals may possess skills not fully used or not used at all at work, making their job history a less than perfect proxy for what they can actually do (Quintini, 2011[14]). Moreover, individual interviews are potentially costly in terms of the staff time required to conduct them.
Figure 3.5. Methods for assessing skills
Note: Average for the six countries covered by the SCGA: Chile, France, Germany, Italy, New Zealand and the United States. Respondents could choose more than one answer. Data refer to the last time the respondent spoke to a career guidance advisor.
Source: OECD 2020 Survey of Career Guidance for Adults (SCGA).
Career guidance advisors sometimes complement interviews with self-assessment tests. These tests generally ask the user to rate their comfort using particular skills. Common self-assessment tests employed by advisors include interest and personality tests, psychometric tests, and vocational aptitude tests (Table 3.2). An advantage with a self-assessment is that it may prompt users to take stock of skills they acquired outside of formal employment or education. For instance, the European Commission developed a skills assessment tool designed to be used by organisations providing services to third-country nationals. The tool prompts the interviewer to collect information about skills the interviewee acquired while working but also those acquired outside of formal employment, e.g. childcare, volunteering. Ireland’s skills measurement tool (My Journey) prompts users to self-assess their capacity in five soft skills: literacy and numeracy; confidence, goal setting and self-efficacy; communication skills; connection with others; and general work readiness.
Though still rare, skills profiling tools provide a more objective measure of a person’s abilities by having them complete a test that can be graded against an answer key. Benchmarking performance against other test takers provides an objective metric. By assessing abilities beyond those documented through work history and certificates, skills profiling tools support flexible pathways and redeployment of adults from declining to growing jobs and sectors.1 They have been used for migrants and refugees, as well as for specific skill domains such as literacy, numeracy and digital skills, but have not yet been developed for a broad range of users or skills. The PES in Italy and Spain piloted an online version of the OECD’s Survey of Adult Skills to test the literacy, numeracy and digital skills of jobseekers (Education and Skills Online). In France, the government made available an online tool for testing, developing and certifying digital skills (https://pix.fr/).
There is a growing role for advisors to help individuals obtain formal recognition of prior learning (RPL), i.e. the skills they have acquired informally (Cedefop, 2009[15]). Recognition of prior learning is a more involved process than skills profiling and can lead to formal certification for skills acquired outside of formal training. The process involves demonstrating achievement of competencies, often by preparing a portfolio of relevant work or demonstrating one’s ability to carry out tasks in practice. RPL can shorten retraining pathways by giving adults credit for skills they already have, and thus accelerating their transition to new jobs or sectors. Career guidance advisors can help adults to navigate RPL processes. Portugal’s Qualifica Centres and Finland’s competency-based VET programmes combine career guidance with support in recognition of prior learning processes (Box 3.3).
Box 3.3. Support for recognition of prior learning processes within career guidance
Finland
Adults who want to complete a competence-based qualification (CBQ) or a preparatory training for a competence-based qualification, can have their skills validated. In order to complete a CBQ, candidates must demonstrate certain skills and competences required in the profession, outlined in the Requirements of Competence-based Qualifications defined by the Finnish National Board of Education. Education providers are responsible for providing personalised guidance and support to students as they carry out the validation process. Adults receive a personalised learning plan that charts and recognises the skills they already have, those they need, and in which learning environments they can be acquired. Certificates are awarded by Qualification Committees (Näyttötutkintotoimikunta), which are sector-specific tripartite bodies that oversee the quality of the provision of CBQs.
Portugal
The Portuguese Qualifica programme has as an objective to increase qualification levels and improve the employability of low-skilled adults, providing them with skills needed in the labour market. It also aims at reducing illiteracy rates and adapting the offer and the training network to the needs of the labour market. In addition to a first diagnosis and provision of information and guidance, individuals can also take part in a recognition of prior learning procedure. The RPL can lead to total or partial certification. In case of partial certification, an individual receives a personal qualification plan and is encouraged to take part in further training or education activities. The RPL takes place both by self-evaluation and as an evaluation by the Qualifica team.
Source: OECD 2020 Policy Questionnaire ‘Career Guidance for Adults’.
Personalised career development roadmap
A personalised career development roadmap – also called an individual or personal action plan or training plan – spells out a sequence of activities that should be taken to achieve an individual’s training or employment objectives. It starts from an assessment of an individual’s skills, aspirations and background. Only half (51%) of career guidance users confirmed that they received a personalised career development roadmap as an output from their career guidance service (Figure 3.6). If adults are involved in the process of developing their own career development roadmap, this can be a powerful tool to motivate them to take informed action towards their goals. According to the SCGA, receiving a personalised career development roadmap increases an adult’s probability of achieving employment outcomes by 25% (Table 3.1). It also raises the probability that they will enrol in an education or training programme by 7%.
Figure 3.6. Personalised career development roadmap
Note: The average includes the six countries covered by the SCGA: Chile, France, Germany, Italy, New Zealand and the United States. A personalised career development roadmap (also called an individual action plan) is a resource prepared by the advisor based on the user’s background and skills. It sets out a planned sequence of activities to help the user towards achieving their education and employment objectives.
Source: OECD 2020 Survey of Career Guidance for Adults (SCGA).
Personalised career development roadmaps are a required output in many career guidance programmes (Table 3.2). Australia’s Career Transition Assistance programme gives mature job seekers a skills assessment which informs the development of a personalised Career Pathway Plan that provides information on retraining opportunities in line with local labour market needs. In Flanders (Belgium), the VDAB provides users with an individualised action plan immediately upon registering. After carrying out a self-assessment, the user receives an online account with personalised tips on how and where to search for a job, as well as suggestions for jobs to apply to, based on their profile and preferences. The VDAB is also experimenting with an ‘Amazon model’, using big data to make job suggestions based on an individual’s personal and work experience. As mentioned above, a personalised learning plan is a required output from Finland’s education guidance towards obtaining a competency-based qualification (Table 3.2).
Tailoring information and advice in online portals
Online career guidance portals are most useful when they provide tailored information and advice. The best ones start from an assessment of the user’s skills, and then provide tailored information and advice based on the results of that skills assessment. For instance, New Zealand’s online career guidance portal, CareersQuest, invites users to complete a self-assessment. Then it suggests occupations that align with the user’s skills and interests based on the self-assessment. Spain is using artificial intelligence to develop a digital profiling tool that tailors the information provided by SEPE’s online guidance platform (www.sepe.es). The Czech Republic’s JOBHUB helps users appraise their skills and interests using a self-evaluation tool, and then suggests occupations that might be a good fit. Greece’s Internet Portal for Adults operated by EOPPEP (http://e-stadiodromia.eoppep.gr/) suggests activities that would help users to develop their career based on three psychometric online tests (a job interests test, a values test, and a vocational decisions test) as well as career management skills exercises. Since March 2020, Japanese O-Net enables workers to appraise their current skill set based on their job history. Users are then shown the skills gap between the job they want and their current skillset, and Japanese O-Net describes the training pathways that would lead them there. On Japan’s Hello Work Internet Service Site, users can create their own “my-page” to see their job search history, and receive tailored job advertisements from PES offices. England’s National Career Service website (https://nationalcareers.service.gov.uk/skills-assessment) includes a self-assessment tool asking a battery of questions about what users like to do and which skills they like to use. With this information, it suggests jobs they might be interested in pursuing.
Features that enable speaking or chatting with someone are another way to tailor the information and advice provided in online career guidance portals. Some users require opportunities to talk through the online information with someone to grasp what it means for them personally. Online portals sometimes include features that enable interaction with someone who can help them interpret the LMI. For instance, Poland’s Talent Development Center (Centrum Rozwoju Talentów) website has a live chat option that allows for direct contact with professional advisors from the Centre. See Chapter 2 for more country examples.
Table 3.2. Tools used to tailor career guidance service provision
Source: OECD 2020 Policy Questionnaire, ‘Career Guidance for Adults’.
3.3.3. Standardising training and qualifications of career guidance advisors
Standardising training and qualifications for career guidance advisors can promote high quality service delivery. Career guidance advisors should have an understanding of the specific theories and methods central to career guidance. How advisors are expected to acquire such specialised knowledge varies across and sometimes within countries. This section summarises the minimum training and qualification requirements for employing advisors in specific career guidance programmes. It also describes the role of continuing professional development and professional certifications in standardising the training and qualifications of career guidance advisors.
Minimum training and qualification requirements
In most countries, ‘career guidance advisor’ (or its national equivalent) is not a regulated profession, meaning that there is no legislation specifying which certificate, license or registration must be attained to use the occupation title. Nevertheless, even if not written in legislation, many countries do define minimum training and qualifications requirements for employing advisors in specific programmes. Minimum requirements for training and qualifications often also form the basis of professional certifications.
The OECD 2020 Policy Questionnaire, ‘Career Guidance for Adults’, polled countries about whether advisors working with adults must have any minimum training or qualifications to practice in their country (Figure 3.7). A tertiary degree is generally a minimum requirement for most adult career guidance programmes: in 18 out of 29 career guidance programmes, a first-cycle or second-cycle programme at a university or other higher education institution was required. Only four programmes had no minimum training or qualification requirement, while five required a short uncertified course or in-service training.
Figure 3.7. Minimum required training or qualification of career guidance advisors
Note: A total of 18 OECD countries responded to this question about 29 programmes. Countries sometimes reported multiple responses when qualification requirements varied for different programmes within the same country.
Source: OECD 2020 Policy Questionnaire, ‘Career Guidance for Adults’.
While a tertiary degree is a common minimum requirement for most publicly subsidised programmes, that qualification may not provide any specialised training in career guidance. OECD (2004[16]) identified five main training and qualification models for advisors, as outlined in Table 3.3. Each of these models vary in terms of how advisors are expected to gain skills and knowledge to provide career guidance.
Table 3.3. Training and qualification models
Source: Framework developed in OECD (2004[16]), Career Guidance and Public Policy: Bridging the Gap, https://dx.doi.org/10.1787/9789264105669-en.
A growing number of countries require that advisors have tertiary qualifications with specialised training in career guidance. The German Federal Employment Agency trains career guidance professionals at the University of Applied Labour Science in a dedicated bachelor’s study course (Career Guidance for Education, Career and Employment) (Cedefop, 2020[17]). Modules include intensive training in counselling techniques for different target groups, as well as training on the labour market and education system, recent trends, and sociology. The University of Applied Labour Science also offers a part-time master’s course in Labour Market Oriented Guidance, which most employed career guidance specialists have obtained. To obtain a permit to work as a career counsellor in Quebec (Canada), both a bachelor’s and a masters’ degree in career counselling must be completed, which include modules on the production and dissemination of labour market information, online sources of labour market information, and how to incorporate labour market information in career counselling. The same permit allows the counsellor to work in different settings, including schools and universities (60% of graduates), employment and rehabilitation services (20%), in private practice, human resources or skill development departments of large companies, and in the mental health sector (Cedefop, 2016[18]). In Alberta (Canada), employers are increasingly seeking applicants who have a certificate, diploma, or degree in career development. The Career Development Association of Alberta grants the Certified Career Development Professional (CCDP) designation to applicants who meet educational, experiential, and ethical requirements.
A common model is to require a tertiary qualification in any one of a broad range of fields related to career guidance – including psychology, education, economics, and social sciences – but without any specific training in guidance or counselling. Alternatively, advisors may be expected to have a general qualification in guidance or counselling, but without any specialised training in career guidance. In Finland, vocational guidance psychologists in the PES must have a master’s degree in psychology. In France, guidance professionals working in schools or universities must hold a master’s degree in psychology and, after passing a selective competition, they undertake a one‑year university training in psychology, sociology, economics and educational sciences (Cedefop, 2020[19]).
While specialised qualifications in career guidance are becoming more common, many countries still require only general tertiary qualifications with no specific focus on career guidance. This general model is often used when no academic programmes providing specialised training in career guidance are available in the country. A risk with this approach is that career guidance advisors may lack the specialised skills, attitudes and knowledge to provide high-quality career guidance. Lack of specialised training may also result in a varying standard of practice across providers.
OECD (2004[16]) called on countries to develop competency frameworks as a first step in addressing this issue. Competency frameworks spell out what career guidance advisors should know and do, and form a foundation for designing training and qualifications. They are useful in recruitment and allow advisors to self-assess and benchmark their competencies. Many countries have developed competency frameworks (Table 3.4). In Greece, the competency framework for career guidance advisors developed by EOPPEP forms the basis for a system of accreditation, as well as a national professional register. In Germany, both the bachelor’s and the master’s programmes discussed above are based upon the Federal Employment Agency’s “guidance concepts” (Beratungskonzepte), which is a competency framework that is part of the country’s quality assurance system (Cedefop, 2020[17]).
Table 3.4. Competence frameworks for career guidance advisors
Note: “/”: no response provided by country.
Source: OECD 2020 Policy Questionnaire, ‘Career Guidance for Adults’ plus author elaborations based on desk research.
In some PES programmes, the only requirement is a relatively brief course offered by a tertiary institution. Case workers in Ireland’s PES complete training through the Department of Employment Affairs and Social Protection, as well as a certificate in employability services provided through the National College of Ireland. Also common, is for the PES to hire people with no specific qualifications in career guidance, and then to provide them with in-service training. Generally this in-service training leads to no formal qualifications and covers a range of administrative and procedural aspects of their work as well as client-interaction skills (Cedefop, 2009[15]). The length of training varies from a few months to a full year. The Austrian PES, for instance, puts new recruits through a year-long in-house training programme.
Only 4 of 29 programmes reported in the policy questionnaire require no minimum training or qualification. In such cases, relevant professional experience may be prioritised over qualifications or training. Managers of VDAB centres in Flanders (Belgium) must have at least three years of professional experience in the sector of career guidance, career coaching, outplacement or job seeker guidance. Counsellors in VDAB centres must have either a bachelor’s degree or two years of relevant field experience. However, candidates who lack relevant qualifications or work experience can obtain a validated attestation of “otherwise obtained competences” through a recognition of prior learning procedure. In Korea, career guidance advisors working in Employment Centres or Workplus Centres must first pass a civil servant recruitment test.
Box 3.4. Competency frameworks
Austria
In Austria, career guidance advisors use the European Career Guidance Certificate (ECGC), which is based on the MEVOC standards (Quality Manual for Educational and Vocational Counselling). The MEVOC standards, a Leonardo da Vinci project of the European Union, were developed under the leadership of Austria (ibw – Institut für Bildungsforschung der Wirtschaft) in cooperation with 19 partner institutions from nine countries. They describe quality standards for educational and vocational counsellors. Based on the MEVOC standards, a competence grid was developed with 35 required competences in the following four areas: education, career, counselling practice, personality and ICT competences. To achieve the ECGC, counsellors can complete an online test and an assessment centre.
Canada
The Canadian Standards and Guidelines for Career Development Practitioners (S&G) outline the competencies needed to provide effective and people-centred guidance across the lifespan. Canada’s S&Gs were the first competency framework developed internationally in the field, and have served as a model for other countries. The S&Gs were funded by Employment and Social Development Canada (ESDC) and matched by contributions from career development partners. The objectives in developing these national standards were: to define career development as a legitimate specialisation; to provide a foundation for designing training; to provide quality assurance to the public; to recognise and validate the diverse skill sets of practitioners working in the field; and to create a common voice and vocabulary for career development. The competencies are organised in three areas: core competencies, specialisation competencies and ethical principles. Five provinces in Canada have developed professional certifications linked to the S&Gs (British Colombia, Alberta, Ontario, New Brunswick, and Nova Scotia).
European Commission
The European Commission published the European Reference Competence Profile for PES and EURES counsellors. The profile is intended to serve as a reference tool for European countries in their recruitment and training of PES counsellors, while recognising that each country will adapt it based on their particular business model and labour supply. Three competence areas make up the profile: foundational competences (general practitioners’ values and skills), client interaction competences (working with jobseekers and employers) as well as supporting competences (systems and technical). The development of the profile was initiated in response to the PES to PES Dialogue programme, which found room for improvement in the professionalisation of PES counsellors.
Source: OECD 2020 Policy Questionnaire ‘Career Guidance for Adults’.
Continuing professional development
The skills and knowledge needed to offer high-quality guidance services change regularly with developments in technology and the labour market, making continuing professional development an essential element of quality service delivery. Across the 21 OECD countries that responded to a question about refresher training in the policy questionnaire, only 9 cited that career guidance advisors are required to participate in ongoing refresher training, ranging in frequency from several times per year to once every five years. For instance, the Employment Service of Slovenia offers an annual catalogue of internal professional courses and trainings (in person or e-learning) and there is budget available to refer counsellors to external professional courses, trainings, conferences, study visits, and seminars. In Japan, refresher training for Career Consultants in the PES is mandatory and ongoing self-development is expected. Under Japan’s new national qualification for career counselling, counsellors must renew their certification once every five years with a minimum of 38 hours of training (OECD, 2021[20]). In Italy, career guidance advisors working as Eurodesk Mobility Advisors must adhere to an EU level Competence Framework which requires regular refresher courses. In Estonia, Eesti Töötukassa organises training sessions and provides guidelines and information materials for advisors. It also pays for its employees’ professional qualification standard exams and encourages its career counsellors to take the exam.
Continuing professional development helps advisors develop and maintain digital skills and knowledge of the labour market. The need for advisors to keep abreast of labour market information was discussed earlier. Career guidance advisors also need to develop digital literacy skills amid rapid technological developments which have revolutionised career products and services, like online provision or supporting adults in their search for labour market information (National Careers Council, 2013[21]). The need for digital skills became even more urgent during the COVID‑19 health pandemic as career guidance provision shifted online (see Chapter 2). To hone digital skills, some countries have experimented with online or blended learning approaches to continuing professional development (Bimrose and Brown, 2019[22]).
The European Public Employment Service Network also identified the increasing need for career guidance advisors to develop teaching skills, as their role in the PES shifts from job broker to facilitator or coach (European Public Employment Services, n.d.[23]). This means being prepared to teach career management skills, like self-awareness and labour market research, in order to help adults navigate complex career transitions.
Professional certifications
Professional certifications signal that a career practitioner has the qualifications, experience, skills and knowledge to provide high-quality career guidance. In some cases, professional certifications may be required for employment. They are particularly useful as a way for private practitioners to advertise their credentials to potential clients. In addition to minimum qualifications and work experience, applicants must usually demonstrate participation in continuing professional development.
For instance, the United Kingdom Career Development Institute developed the Register of Career Development Professionals. To qualify for the register, applicants must demonstrate that they are qualified in a career development subject to a minimum of QCF Level 6/SCQF Level 11,2 adhere to the CDI’s Code of Ethics, and undertake and record a minimum of 25 hours of continuing professional development each year. Those without formal qualifications may gain entrance to the register via a competency route, based on the National Occupational Standards. Similarly, in Ireland, the Institute of Guidance Counsellors maintains a register of accredited counsellors.
3.3.4. Certifying providers against quality standards in service delivery
Quality standards in service delivery establish basic requirements for how career guidance is provided. They are set either by a public authority or by the provider, and can govern all aspects of service delivery including professional standards, partnerships, labour market information, client satisfaction, evaluation and leadership (Dodd et al., 2019[24]). Only 7 of the 21 countries that responded to the OECD 2020 Policy Questionnaire, ‘Career guidance for Adults,’ reported employing quality standards in service delivery.3
Certification against a quality standard can be a mandatory condition for providers to receive public funds. But voluntary standards also exist, and can be a useful tool for quality improvement. Obtaining certification against voluntary standards is a way for private providers to signal the quality of their service to potential users. Some voluntary standards do not offer certification, but instead provide a framework for providers to use towards quality improvement.
Mandatory quality standards
With mandatory quality standards, guidance providers must demonstrate that they meet the standards in order to receive public funds. The Flemish Government introduced a national quality framework to assure quality under the PES’ career voucher system. All service providers – public or private – who offer career guidance under the voucher programme must abide by the national quality framework. France has put in place quality specifications (cahier de charges) to which career guidance practitioners who participate in the national career guidance programme (Conseil en evolution professionnelle, CEP) must conform. Providers must meet eligibility requirements to be certified against the quality label (orientation pour tous). In England (United Kingdom), all organisations that receive public funds have to meet national quality standards. All providers who deliver the National Careers Service must achieve the criteria set in the Matrix Standard – a quality assurance system set up specifically for career guidance providers. For other guidance services, these standards are voluntary. The quality standard used by Austria’s free educational guidance programme and Korea’s Employment Success Package Program provide other examples of mandatory country-specific quality standards (Box 3.5).
Standards are often set at national level, while allowing for regional adaptation. The Austrian PES (Arbeitsmarktservice, AMS) sets central minimum standards for service delivery, including access to up-to-date and gender-specific career information, as well as minimum duration of client interviews. Standards are developed at national level, but each AMS can decide how to put them into practice (Cedefop, 2020[25]). In the United States, American Job Centres must pass a certification process that establishes a minimum level of quality and consistency of services across the state. The Workforce Innovation Opportunity Act requires the State Workforce Development Board (WDB), in consultation with firms and local WDBs, to set objective criteria to use when certifying American Job Centres.
Countries do not always develop country-specific quality standards, or even quality standards specific to career guidance services. They sometimes use generic standards, which are intended to be used by any organisation, not limited to career guidance providers. To become a jobactive provider in Australia, for example, potential providers must adhere to the Department of Employment’s Quality Principles, as well as obtain certification against the generic ISO 9001: 2015 quality standard.4
Box 3.5. Mandatory quality standards in career guidance
Austria
In Austria, providers of the country’s free adult educational guidance programme must be certified by the IBOBB (Information, Counselling and Orientation for Education and Career) certification. The certification was introduced in 2009 to support the country’s National Lifelong Guidance Strategy and 53 certification procedures have been carried out since then. The Federal Ministry of Education, Science and Research awards the certification. The review procedure includes an on-site assessment of the impartiality of the guidance being provided and how tailored it is to customers’ needs. The quality criteria are available in a handbook that describes the procedure for certification.
Belgium
Under the Cité des Métiers network, which started in France in 1991 and now operates in seven countries, a provider must obtain the Cité des Métiers quality label in order to open. The label is managed by Universcience – Cité de Sciences et de l’Industrie. Acquiring the label involves demonstrating that certain service conditions are met: cost-free services for all individuals without a prior appointment; a welcoming space that is well sign-posted; partnerships with a wide variety of industries and services; and neutral information. Career guidance counsellors must also undergo regular training about services and labour market opportunities.
In Belgium, the Cités appoint a quality referent from their professional team, who finds a balance between implementing nation-wide quality standards and adapting their implementation to local conditions. Being part of a large network of Cités is meant to facilitate the exchange of best practices and the continuous improvement of services.
France
All providers of France’s professional development consulting programme (conseil en évolution professionnelle, CEP) must abide by the cahier des charges. The cahier specifies objectives, beneficiary groups, service provision, methods, and the skill requirements of advisors. It also specifies how providers go about promoting, coordinating and monitoring the CEP. The guidelines established by the cahier are designed to be broad enough that providers can experiment with innovative practices and adapt the service to each adult’s needs.
Korea
The Employment Success Package Program (ESPP) is carried out by the Job Centres and PES for low-income workers and by contracted private employment agencies for youth and middle-aged workers. Every year, the government evaluates the comprehensive performance of all private agencies to assure the quality improvement of employment services. To participate in the programme the following year, private agencies must achieve a minimum score evaluated based on quantitative and qualitative performance criteria.
Source: OECD 2020 Policy Questionnaire ‘Career Guidance for Adults’.
Voluntary quality standards
Voluntary quality standards provide a metric for quality improvement, signal quality to potential users when certification is obtained, and promote consistency of service across jurisdictions. The UK’s Matrix Standard is a requirement for some public career guidance programmes, but it also serves as a voluntary benchmarking tool that allows providers to improve their service and receive accreditation. Thirty percent of accredited providers seek the Matrix Standard for reasons other than obtaining publicly-funded contracts (BIS, 2015[26]). The majority of providers who sought the standard voluntarily vouch for it improving the quality of their service, the reputation of their organisation, and the competency of their staff. Similarly, obtaining Germany’s BeQu quality label is intended to put providers through a voluntary quality improvement process (Box 3.6).
In Greece, voluntary quality standards signal quality to potential end users, and also offer competitive advantages to career guidance providers. Greece’s EOPPEP (National Organization for the Certification of Qualifications and Vocational Guidance) developed quality standards based on the European Lifelong Guidance Policy Network (ELGPN) quality assurance framework. EOPPEP’s register of approved private providers informs the public about available and high-quality providers. The Ministry of Labour also uses the register as a selection device for giving preferential support to providers when applying for European programmes.
Voluntary quality standards also promote consistent service quality across diverse regions. The Blueprint for Lifework framework has been developed in Canada, the United States and Australia (Hooley et al., 2013[27]). In Canada, the Blueprint framework is implemented on a voluntary basis by government agencies, professional associations, community agencies and corporations in most provinces and territories (Box 3.6).
Box 3.6. Voluntary quality standards in career guidance
Canada
Canada’s Blueprint framework articulates the concept of career management skills for a range of audiences (careers workers, policy makers, teachers and end users). It outlines the career management skills necessary to support an individual over the life course in their education, training and employment decisions, and it specifies different expected outcomes for different age groups (e.g. early years, primary/elementary school, high school, post-secondary and adult populations). It specifies ten broad competencies in three areas: personal management; learning; and career. The Blueprint is a companion piece to the Canadian Standards and Guidelines for Career Practitioners. Career, curriculum and human resources specialists in public and private agencies from every region of Canada collaborated to produce the Blueprint. The Blueprint is not accredited, meaning that providers cannot obtain formal certification.
Germany
Germany’s National Guidance Forum developed a quality concept in guidance (BeKo) in cooperation with many actors in the guidance landscape. This concept has three pillars. First, a set of quality standards, called BeQu, for guidance in education, occupation and employment. These support both counsellors and guidance institutions in further developing the quality and professionalism of their services. Second, a quality development framework, and third, a competence profile that outlines the competencies required of a career guidance advisor. Any provider of career guidance that would like to obtain the standard can receive support from the National Guidance Forum. To use the BeQu quality label, providers must formally apply to the National Guidance Forum, commit to using the quality standards, and have participated in a mandatory workshop.
Source: OECD 2020 Policy Questionnaire ‘Career Guidance for Adults’.
3.3.5. Monitoring outcomes
Monitoring outcomes of career guidance is carried out for a number of purposes: to help providers evaluate and improve their performance, to hold the system to account, and to measure the economic and social value of activities relative to their cost. This section focuses on the first two as important components of a quality assurance system.
In the OECD 2020 Policy Questionnaire, ‘Career Guidance for Adults,’ only 10 out of 19 country respondents said that they assessed the effectiveness of their career guidance programmes against pre-set objectives.
Quality of career guidance is generally measured by looking at employment, wages, training participation, unemployment benefit receipt, and user satisfaction. Outcomes can be measured using a variety of monitoring and evaluation methods, implemented either by external quality assurance bodies, research groups and academics, or internally by self-evaluations.
Type of outcomes measured
Employment outcomes are a common way to assess the effectiveness of career guidance services or re-employment programmes that include career guidance services as a component. In a systematic review of re-employment programmes across the United States (which included career guidance services), evaluators tracked short-term and long-term employment, earnings, and the receipt of Unemployment Insurance benefits. Austria’s PES (BIS) monitors changes to employment status 12 months after the counselling service. In the Job Centres run by the UK’s Department of Work and Pensions, outcome measures focus on the number of people who transition into employment. In Australia’s PES (jobactive), providers are selected on the basis of their performance placing jobseekers into jobs, taking into account differences in caseload and regional labour market characteristics. Providers are also partially funded by outcome-based fees, which reward employment matches if a jobseeker remains employed for four weeks, 12 weeks and 26 weeks.
Training participation is another commonly measured outcome. In Slovenia, national network coordinators of lifelong guidance centres monitor whether clients participate in a training programme (either formal or non-formal) after they receive guidance. In evaluating the Guidance and Orientation for Adult Learners (GOAL) project, follow-up surveys asked users whether or not they had attained or made progress towards their educational goals, as well as whether they had enrolled in a course (Carpentieri et al., 2018[28]). Other training-related outcomes include greater educational and training attainment, improved retention rates in education and training programmes, and higher levels of skills.
Changes to employment or training status may not occur in the months immediately following the career guidance service, and may therefore escape observation during monitoring and evaluation exercises. Some evaluations consider shorter-term outcomes, like whether the individual has acquired new skills or knowledge as a result of counselling. In Canada, with help from the PRIME data management system, career development practitioners collect data on employment and training outcomes, but also the quality of job matches and incremental progress towards employability. By measuring progress along a continuum, rather than only focusing on whether or not someone became employed or entered a training programme, the tool is able to capture a richer picture of the impact of career guidance services.
Asking users about their satisfaction with the service is another way to assess quality. The French PES, Pôle Emploi, regularly surveys clients about their satisfaction with the services offered and France Compétence collects user satisfaction data about the public career guidance programme (Conseil en Evolution Professionnel, CEP) through quality and perceived usefulness questionnaires. The National Careers Service, which offers careers advice to adults and young people in England (United Kingdom), measures outcomes from face-to-face support through user satisfaction, i.e. whether the adult accepted the action plan produced and confirmed they have had high-quality career guidance. It also tracks education and training outcomes, as well as whether the adult took steps to manage their career (e.g. uploading a CV online).
Often, a combination of outcomes are measured. The vocational guidance department of the German PES tracks an index composed of several indicators, including successful integration into apprenticeship training, sustainable integration after 6 months, duration of employment placement process, and a user satisfaction rating. Using this index, the quality of local services is benchmarked against clusters of regions with comparable labour market situations ( (Plant, 2012[29]).
In addition to the above economic outcomes, evaluations also look at psychological or social impacts, including sense of well-being, self-confidence or insight, awareness of opportunities, and future direction. An evaluation of a group-based counselling programme in Ontario (Canada) conducted surveys of participants before and after the counselling. Participants were vulnerable adults facing significant employment challenges. While employment and training outcomes were measured, the focus was on whether participants had experienced changes to their overall well-being, motivation, and optimism about the future (OCWI, 2018[30]).
Methods used to assess outcomes
By collecting data on pre-set outcomes and making quality improvements based on the data, providers can improve service delivery. This involves putting in place processes to stimulate regular monitoring. Data may be collected by external bodies or as part of self-evaluations.
External bodies are often contracted to monitor career guidance services. When external audits find performance lacking, career guidance providers may be required to improve performance in order to maintain their right to continue delivering services. In Sweden, the Schools Inspectorate (Skolinspektionen) conducts regular inspections of education institutions, from pre-school to adult education, and one of their focus areas is guidance. The Inspectorate advises schools about what they would need to change to meet legislative requirements. In Wallonia (Belgium), career guidance centres must pass regular quality audits to continue receiving public funding to provide guidance for persons with disabilities (Phase de determination de projet socio-professionel). The audits, conducted every three years, assess objectives set by the Walloon Code of Social Action and Health (Code Wallon de l’action sociale et de la santé). France and Scotland both contract monitoring of outcomes to arms-length government agencies (Box 3.7).
Self-evaluations are another means for monitoring the outcomes of career guidance services. They can be a powerful tool for motivating service improvement, though risk of bias may be greater than with external audits. Greece produced self-evaluation guidelines for career guidance advisors that align with its National Quality Assurance System of Guidance Services. Any gaps in service provision identified by the self-evaluations are discussed with stakeholders at national, regional and local levels. Portugal’s Qualifica Centres must submit data on user enrolment, referral to education and training pathways, and recognition of prior learning activities to the National Agency for Qualification and Vocational Education (ANQEP) on a monthly basis. ANQEP analyses the data and sends it back to the Qualifica Centres in order to encourage self-evaluation and quality improvement.
Box 3.7. Monitoring carried out by external bodies
France
The French Government entrusts France Compétence with monitoring the career guidance programme (Conseil d’Évolution Professionelle) at the national level. At the regional level, monitoring is carried out by regional inter-professional joint committees that send an annual report to France Compétence. France Compétence regularly collects user satisfaction data through “quality and perceived usefulness” questionnaires. It also collects data directly from providers about beneficiaries, their reasons for seeking guidance, and the support they received.
Scotland
Skills Development Scotland works with Education Scotland to collect data to monitor the quality of its all-ages career guidance service. In 2013, Education Scotland started a six‑year cycle of external assessment, with an aim to inspect delivery in each of Scotland’s 32 local authority areas by March 2020. During external reviews, the review teams observe group activities, carry out one‑to-one coaching sessions and hold discussions with users, staff and stakeholders. They consider information on the quality of career information, advice and guidance. The agreed quality framework supports Skills Development Scotland’s internal self-assessment processes as well as the external review.
Source: OECD 2020 Policy Questionnaire, ‘Career Guidance for Adults’; Skills Development Scotland website, https://www.skillsdevelopmentscotland.co.uk/.
An impact evaluation provides a more rigorous measurement of outcomes than either external audits or self-evaluations. The main difference between monitoring outcomes and a real impact evaluation is that the latter uses a counterfactual to estimate what part of the observed outcome can be attributed to the guidance intervention. An impact evaluation of a career guidance programme would compare the outcomes of participants to the outcomes of similar adults who for non-systematic reasons did not participate in the guidance programme.
However, impact evaluations in the field of career guidance are rare due to the many challenges involved (Plant, 2012[29]). Career guidance entails bundles of activities, and it can be difficult to isolate which activities are most effective. It is also challenging to distinguish the impact of career guidance from other influences (e.g. advice from non-professionals, training, job search effort, or networking). Recipients of career guidance are also very different, and their needs are different. As noted in OECD (2004[16]), obtaining clear answers about impacts under these circumstances requires large-scale research with complex experimental designs.
Given the challenges, impact evaluations are rarely carried out by individual providers as part of routine monitoring exercises, but are instead conducted by public institutions or academic researchers to build research evidence that will improve the quality of career guidance services on a wider scale. Several countries invest in public programmes to better understand what works in employment and training services more generally, with career guidance as one component of these services. Australia’s Try, Test and Learn Fund tests innovative approaches to moving workers at risk of long-term welfare dependence onto a pathway towards employment. Through the Workforce Innovation Fund, the United States’ Department of Labor evaluates innovative approaches to employment and training services and supports grantees in meeting rigorous evaluation requirements.
References
[4] Bimrose, J., S. Barnes and D. Hughes (2009), Adult career progression and advancement: a five year study of the effectiveness of guidance, University of Warwick. Institute for Employment Research (IER), Coventry, England, http://www2.warwick.ac.uk/fac/soc/ier/publications/2008/eg_report_4_years_on_final.pdf (accessed on 19 November 2020).
[22] Bimrose, J. and A. Brown (2019), “Professional identity transformation: supporting career and employment practitioners at a distance”, British Journal of Guidance & Counselling, Vol. 47/6, pp. 757-769, http://dx.doi.org/10.1080/03069885.2019.1698008.
[26] BIS (2015), Evaluation of the effectiveness of the matrix Standard: BIS RESEARCH PAPER NUMBER 218, Department for Business Innovation & Skills, London, http://www.gov.uk/bis (accessed on 1 August 2020).
[10] Brown, S. (2006), “Encyclopedia of Career Development”, in Greenhaus, J., G. Callanan and D. Gibson (eds.), Business Faculty Book Gallery.
[9] Card, D., J. Kluve and A. Weber (2015), What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations, Forschungsinstitut zur Zukunft der Arbeit Institute for the Study of Labor.
[28] Carpentieri, J. et al. (2018), Guidance and Orientation for Adult Learners: Final cross-country evaluation report, UCL Institute of Education, https://adultguidance.eu/images/Reports/GOAL_final_cross-country_evaluation_report.pdf.
[25] Cedefop (2020), Inventory of lifelong guidance systems and practices - Austria, CareersNet national records, https://www.cedefop.europa.eu/fr/publications-and-resources/country-reports/inventory-lifelong-guidance-systems-and-practices-austria (accessed on 14 August 2020).
[19] Cedefop (2020), Inventory of lifelong guidance systems and practices - France, CareersNet national records, https://www.cedefop.europa.eu/fr/publications-and-resources/country-reports/inventory-lifelong-guidance-systems-and-practices-france (accessed on 14 August 2020).
[17] Cedefop (2020), Inventory of lifelong guidance systems and practices - Germany, CareersNet national records, https://www.cedefop.europa.eu/en/publications-and-resources/country-reports/inventory-lifelong-guidance-systems-and-practices-germany (accessed on 14 August 2020).
[18] Cedefop (2016), Labour market information in lifelong guidance: Case study visit focusing on IMT en ligne, Placement en ligne, REPÈRES, in Quebec, Canada, European Centre for the Development of Vocational Training, https://www.cedefop.europa.eu/files/5555_en_case_study_canada.pdf (accessed on 14 August 2020).
[15] Cedefop (2009), Professionalising career guidance: Practitioner competences and qualification routes in Europe, European Centre for the Development of Vocational Training, Luxembourg, https://www.cedefop.europa.eu/files/5193_en.pdf (accessed on 7 July 2020).
[24] Dodd, V. et al. (2019), Quality Assurance Standards: A synthesis of quality standards across partner countries, Erasmus, http://guidancequality.eu/quality-assurance-standards-a-synthesis-of-quality-standards-across-partner-countries/.
[5] European Commission (2015), An in-depth analysis of adult learning policies and their effectiveness in Europe, Directorate General for Employment, Social Affairs and Inclusion, https://op.europa.eu/en/publication-detail/-/publication/c8c38dc9-89d0-11e5-b8b7-01aa75ed71a1/language-en (accessed on 6 November 2020).
[23] European Public Employment Services (n.d.), EU Network of Public Employment Services Strategy to 2020 and beyond, PES Network, https://ec.europa.eu/social/BlobServlet?docId=9690&langId=en.
[8] Gautié, J. and C. Perez (2012), Promoting Life Long Learning through Individual Accounts : Asset-Based versus Capability-Based Policies, Maison des Sciences Économiques, Paris, http://centredeconomiesorbonne.univ-paris1.fr/bandeau-haut/documents-de-travail (accessed on 11 September 2020).
[27] Hooley, T. et al. (2013), “The ’Blueprint’ framework for career management skills: A critical exploration”, British Journal of Guidance and Counselling, Vol. 41/2, pp. 117-131, http://dx.doi.org/10.1080/03069885.2012.713908.
[1] Kidd, J., C. Jackson and W. Hirsh (2003), “The outcomes of effective career discussion at work”, Journal of Vocational Behavior, Vol. 62/1, pp. 119-133, http://dx.doi.org/10.1016/S0001-8791(02)00027-1.
[3] Killeen, J. and J. Kidd (1991), Learning outcomes of guidance: a review of recent research, Department of Employment; National Institute of Careers Education and Counselling (Great Britain), http://hdl.voced.edu.au/10707/78443 (accessed on 19 November 2020).
[7] Killeen, J. and M. White (2000), The Impact of Careers Guidance on Adult Employed People, Department for Education and Employment, London.
[6] Lane, M. et al. (2017), An economic evaluation of the National Careers Service, Department for Education, London, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/603929/National_Careers_Service_economic_evaluation.pdf (accessed on 6 August 2020).
[13] LMIC (2019), LMI INSIGHTS- Issue No. 20, Labour Market Information Council, Ottawa, https://lmic-cimt.ca/wp-content/uploads/2020/01/Insights-20_ENG_v6.pdf (accessed on 11 August 2020).
[2] Maguire, M. (2004), “Measuring the Outcomes of Career Guidance”, International Journal for Educational and Vocational Guidance, Vol. 4/2-3, pp. 179-192, http://dx.doi.org/10.1007/s10775-005-1022-1.
[21] National Careers Council (2013), An Aspirational Nation: Creating a culture change in careers provision, National Careers Council - England, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/354644/bis-13-919-national-careers-council-report-an-aspirational-nation-creating-a-culture-change-in-careers-provison.pdf (accessed on 7 July 2020).
[30] OCWI (2018), In Motion & Momentum, Ontario Centre for Workforce innovation, https://ocwi-coie.ca/project/in-motion-and-momentum/.
[20] OECD (2021), Creating Responsive Adult Learning Opportunities in Japan, Getting Skills Right, OECD Publishing, Paris, https://dx.doi.org/10.1787/cfe1ccd2-en.
[11] OECD (2016), Getting Skills Right: Assessing and Anticipating Changing Skill Needs, Getting Skills Right, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264252073-en.
[16] OECD (2004), Career Guidance and Public Policy: Bridging the Gap, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264105669-en.
[29] Plant, P. (2012), “Quality assurance and evidence in career guidance in Europe: counting what is measured or measuring what counts?”, International Journal of Educational and Vocational Guidance, Vol. 12, pp. 91-104, http://dx.doi.org/10.1007/s10775-011-9195-2.
[14] Quintini, G. (2011), “Over-Qualified or Under-Skilled: A Review of Existing Literature”, OECD Social, Employment and Migration Working Papers, No. 121, OECD Publishing, Paris, https://dx.doi.org/10.1787/5kg58j9d7b6d-en.
[12] Souto-Otero, M. and P. Brown (2016), Changing Conceptions of Merit: The Use of Labour Market Analytics in the Study of Educational Attainment and Social Mobility.
Notes
← 1. Skills profiling tools differ from the more generic “profiling tools” widely employed by public employment services to assess the job-finding prospects of jobseekers. With profiling tools, data are collected to estimate the jobseeker’s risk of long-term unemployment. Those with a higher risk of long-term unemployment receive more intensive counselling and support services. Skills profiling tools test the skills of the individual for the purpose of identifying current skills and any skills gaps that would need to be addressed to pursue desired employment or education pathways.
← 2. The Qualifications and Credit Framework (QCF) is the classification framework for further education qualifications in the United Kingdom. It has recently been replaced by the Regulatory Qualifications Framework (RQF). The Scottish Qualifications and Credit Framework (SQCF) is a similar system that applies in Scotland.
← 3. Most quality standards apply either to services for both adults and young people, or adults only. A handful of quality standards only apply to young people in schools (e.g. the Quality in Careers Standards in England).
← 4. When relevant, they can alternatively be certified by the National Disability Standards for Disability Services.