Quality assurance faces new requirements given the introduction of the apprenticeship levy, the replacement of frameworks by standards and other connected reforms. This chapter argues that apprentice standards need to be coherent with the wider system of vocational qualifications and manageable in number. They should be designed so as to ensure the transferability of skills and allow for the recognition of prior learning. England proposes to achieve consistency in assessment standards through multiple bodies conducting the assessment. The chapter argues that given the key role of consistent assessment standards in the credibility and reputation of apprentice qualifications these plans should be reviewed.
Apprenticeship in England, United Kingdom
Report
OECD Reviews of Vocational Education and Training
Table of contents
Share
Facebook
Twitter
LinkedIn
Chapter 5. Quality in apprenticeship qualifications and assessment in England
Abstract
Introduction: The role of quality
Quality in apprenticeship policy in England and around the world
The shift from frameworks to standards is intended to enhance quality
In England, plans to expand the number of apprenticeships have been matched by reforms which aim to improve quality, through a new form of apprenticeship embodied in apprenticeship ‘standards’, and new quality assurance institutions. These reforms are taking place against a background of concern that, previously, too many apprenticeships were undemanding and low level, and were therefore not providing the substantial contribution to workforce skills that might be needed by the economy, or expected of the apprenticeship brand. Chapter 4 argued that quality should be the priority of apprenticeship reform because the levy is likely to encourage the restructuring of other forms of skills development as apprenticeship.
‘Quality’ as an explicit goal, emerged first in management theories
The notion of quality has in recent decades become more prominent in public policy debate, including in relation to vocational education and training (see for example European Training Foundation, 2014). The language used can be fluid, so that quality ‘management’, quality ‘control’, quality ‘improvement’ and quality ‘assurance’ are all used somewhat interchangeably, but also to impart different nuances. In the context of vocational education and training, concerns about low quality led to the establishment of the European Quality Assurance Reference Framework for Vocational Education and Training (EU, 2009).
At its widest, good ‘quality’ means anything desirable
Quality potentially covers all desirable features of apprenticeship. Quality may be attributed to the highest ‘system’ level covering the entire apprenticeship system, at a more intermediate ‘design’ level, covering individual apprenticeship programmes and qualifications, and at the ‘delivery level, covering the learning experience of the individual apprentice. Thus a strong apprenticeship system will support the design of individual apprentice qualifications – for example through a qualifications framework that provides clarity about the relationship between apprenticeship and wider educational qualifications. Good design, in the form of the right qualification will then underpin effective delivery of individual apprenticeships. Quality and quality assurance in vocational education and training are commonly associated with some combination of inputs, processes and outputs (EU, 2009; European Training Foundation, 2014; Broek and Buiskool, 2013; Sweet, 2014). These may be taken to apply at the system, design or delivery stage, implying a matrix as in Table 5.1.
Table 5.1. Types of quality and quality assurance in apprenticeship and the levels at which they may be applied
|
|
Inputs |
Processes |
Outputs |
|---|---|---|---|
|
System features such as wide coverage across different industrial sectors, high status and a good brand, extensive employer buy-in. Strong and effective support institutions in terms of a legal and institutional framework that engages key actors in supporting the system. |
e.g. Strong policy-making bodies, co-ordinating institutions |
e. g. Effective consultation procedures |
e. g. Well-respected apprenticeship system |
|
Design features such as well constructed qualifications, apprentice standards and approaches to assessment relevant to labour market needs. Apprentice wages that work for both apprentice and employer. |
e. g. Well-qualified people, preparing qualifications and standards, adequate input from employers |
e. g. Time and resources to construct qualifications, standards etc. Arrangements for updating. |
e.g. Well-respected qualifications and standards, meeting employer and student needs |
|
Delivery, especially training quality and effectiveness. |
e. g. Accredited training providers, well-trained trainers |
e. g. Pedagogical technique |
e. g. High completion rates and good labour market outcomes |
Source: Adapted from Broek, S. and B.-J. Buiskool (2013), Developing the Adult Learning Sector: Quality in the Adult Learning Sector (Lot 1), Final Report (Open Call for tender EAC/26/2011), Panteia, Zoetermeer, http://arhiv.acs.si/porocila/Quality_in_the_Adult_Learning_Sector-final_report.pdf; Ulicna, D. and A. Curth (2013), Study on quality assurance in continuous VET and on future development of EQAVET, Final Report, ICF GHK, Brussels, www.eqavet.eu/Libraries/2014_Publications/Study_on_QA_in_CVET.sflb.ashx; Sweet, R. (2014), Work-based Learning: A Handbook for Policy Makers and Social Partners in ETF Partner Countries, prepared for the ETF (European Training Foundation), http://ec.europa.eu/education/library/publications/etf-wbl-handbook_en.pdf; European Training Foundation (2014), Quality assurance in vocational education and training a collection of articles, European Training Foundation; EU (2009). Recommendation of the European Parliament and of the Council of 18 June 2009 on the Establishment of a European Quality Assurance Reference Framework for Vocational Education and Training, http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32009H0708(01)&from=EN.
Reform and a new approach to quality in England
Some serious quality problems were identified in the past
Under the apprenticeship frameworks – now being phased out in England - serious quality weaknesses in some sectors have coexisted with strands of high-quality provision. Ofsted used a survey of providers, apprentices and employers to show that in around one-third of the providers, apprentices were not receiving sufficient high-quality training, and were often just having their existing low-level skills accredited; this was particularly likely among apprentices aged over 25, especially at employer-providers in the retail and care sectors. Often providers lacked a clear rationale for their apprenticeship programmes in terms of local labour market requirements. Conversely, the report noted the high quality of many apprenticeships in some more technical areas (Ofsted, 2015). The regular apprenticeship evaluation exercise (BIS, 2014) also revealed that a significant minority of apprentices were not receiving the minimum required level of training, and as noted in Chapter 2, nearly one in five apprentices was paid less than the minimum required wage. The largest private provider of apprentice programmes in England received an 'inadequate' rating in an Ofsted report, and will lose government funding as a result (FE Week, 2017). Poor quality, in one form or another, is one of the major challenges facing English apprenticeships.
The key new quality institution is the Institute for Apprenticeships (IfA)
The IfA formally came into existence in April 2017, simultaneously with the introduction of the apprentice levy. It is envisaged as an employer-led body, with some independence from government, and will have as a very central part of its mission the quality of apprenticeships. Most directly, the IfA will develop quality criteria, and use them to approve apprenticeship standards and assessment plans, and ensure the quality assurance of end-point assessments (IfA, 2017).
Developing quality assurance in the context of rapid change
Wholesale reform in the apprenticeship system implies new requirements for quality assurance
The shift from apprenticeship frameworks to standards, to be achieved by 2020, alongside the introduction of the apprenticeship levy and associated funding mechanisms, mean that quality assurance will face new requirements. Under the old apprenticeship framework system, apprenticeship was and is quality assured by the DfE in setting policy, Ofqual in agreeing to the qualification and an associated assessment developed by the awarding body; the Skills Funding Agency in ‘accrediting’ the training provider by placing it in a register of approved providers; and Ofsted in examining individual training providers and employers other than employer-providers (if only on a sample basis). Under the new standards, while there is some continuity in the role of the Department and Ofsted, key new issues are that:
The IfA will regulate the quality of apprenticeship standards and assessments, advise government on the funding of training and assessment against apprenticeship standards, and from 2018 will have wider responsibilities to include the technical education qualifications (see IfA, 2017).
The Education and Skills Funding Agency ESFA (successor to the SFA and EFA) will be responsible for registers of apprenticeship training providers and assessment organisations, as well as running the National Apprenticeship Service (NAS).
Quality assurance will be challenging given change and growth
Inevitably, the new standards will have teething problems – so that some features of standards that looked sensible initially will turn out to have problems. Similarly, in implementing the standards in training, some providers will pursue teaching approaches which turn out not to work well. Quality assurance capacity in the IfA and in Ofsted will need to be developed over time, gradually learning about the potential pitfalls of standards and their implementation, and how they might be overcome in practice. Stakeholders presenting evidence to the parliamentary committee enquiry made this very clear (House of Commons, 2017). Few countries have attempted apprenticeship reforms of the scale and speed of England. This is because slow evolution is a common characteristic of apprenticeship systems, often reflecting the need to obtain consensus between government, employers and often trade unions on the shape and direction of change. Quality assurance systems have, for the same reasons, also developed slowly, allowing time for the necessary expertise to be developed, issues to be identified and problems ironed out.
But the quality of apprenticeship must be assured
One of the greatest challenges in apprenticeship policy will therefore be to ensure high quality at a time of rapid change and reform. As argued in Chapter 1, careful prioritisation will be necessary to ensure that key challenges are addressed, while other longer-term requirements are addressed more gradually. But as Chapter 1 also argues, regardless of the pressures, some quality issues cannot be postponed. The whole rationale for the range of current reforms is to replace weak training options with good quality apprenticeships. If quality in the new apprenticeship system cannot be effectively ensured, that rationale collapses. The policy issues addressed below fall into the category of those which cannot be postponed.
One very important dimension of quality is equity
In England, a successful high-quality system will necessarily be inclusive, and offer routes to good jobs for vulnerable and disadvantaged groups. For example, the European EQAVET indicators of quality in vocational education and training rightly include equity measures, including completion rates and the inclusion of disadvantaged groups (European Commission, 2014). These challenges are addressed in Chapter 6.
Policy issue 5.1: Delivering a coherent apprenticeship qualifications system
Challenge: Managing the changing landscape of apprentice qualifications
Reform is changing both apprenticeship and other vocational qualifications
One key element of quality is not simply the content of individual apprentice qualifications, but also their ability to play a sensible role within a wider education and training system, complementing other qualifications, and helping to provide a coherent offer to potential students. The introduction of apprentice standards, alongside a wider qualification reform following the Sainsbury review, imply a reorganisation of the apprentice qualifications system, and their relationship to other qualifications.
Reform offers a promising opportunity to clarify the offer
By common consent England has suffered a difficult history of proliferating, overlapping and over-numerous vocational qualifications, with the then Skills Minister, Robert Halfon, reporting to parliament that current reforms seek to simplify a “spaghetti junction of qualifications” (Hansard, 2017). Following the Sainsbury and Richard reviews, the government is implementing, both for school-based and apprentice qualifications in the form of standards, the welcome principle that there should be just one qualification for each target occupation, rather than competing qualifications offered by different awarding bodies. This reflects the recommendation of a previous OECD review (Musset and Field, 2013).
But challenges are emerging in implementation
While the overarching logic of reform is clear, challenges are emerging in implementation. These challenges relate both to the number of apprentice qualifications and their relationship to wider qualifications.
Policy pointer 5.1: Delivering a coherent apprenticeship qualifications system
A credible and robust system of apprentice qualifications needs to be coherent with the wider system of vocational qualifications and manageable in number. International experience offers some guidance:
Apprentice standards represent the requirements for the target occupation, and should therefore be closely articulated with any related technical qualification. One option would be to require all graduates of associated technical qualifications to take the apprenticeship exam to certify their occupational competence. A second option would be to establish a technical qualification as a preparatory programme for a linked apprenticeship.
To ensure the transferability of skills, the IfA needs to ensure that each proposed standard represents a wide occupational field and therefore reject proposals that do not do so, aiming to keep the eventual total number of standards well under one thousand.
In the context of upskilling adult learners, a more effective framework for recognising prior learning needs to be developed within the frame of apprenticeship standards and levy funding. This will need to support the top-up training and assessments for those who are able to pass the end-point assessment, but have not pursued regular apprenticeships.
Analysis: Alignment between apprenticeship and other vocational qualifications
There are three ways in which apprentice qualifications may relate to other vocational qualifications
Apprenticeship qualifications are normally linked to wider education and training qualifications. This allows both students and employers to see where apprenticeship fits within an educational progression, and what jobs and careers might result from a given apprenticeship. For any given target occupation, there are three possible ways of articulating apprentice qualifications with other vocational and educational qualifications.
First, the target occupation, and the competences it requires, may determine whether these competences should be acquired through an apprenticeship or through some other more appropriate form of training, recognising a division of labour between apprenticeship and other forms of training.
Second, there may be alternative routes to the same target occupation, allowing apprenticeship to offer one way of acquiring the competences, while other routes are also possible, including school or college-based training. This model is often associated with a competence-based final assessment, permitting different means of preparing for that assessment. For example, in the Netherlands there is both a school-based and an apprenticeship route to every upper secondary vocational qualification.
Third, there may be sequential programmes. For example, in Norway apprentices spend the first two years of their programme in school-based upper secondary education with a relatively broad curriculum, followed normally by two years with an employer, gaining the work-based experience that will allow them to qualify as an apprentice. (Some students also continue in a school-based path after their first two years and graduate in that way).
The logic of current reforms in England implies the ‘alternative routes’ model, but uncertainty remains
The Sainsbury review refers to two alternative modes of vocational study at the same level, alongside a new emphasis on having only one qualification for each occupation. This seems to imply an adherence to the alternative routes approach (Independent Panel on Technical Education, 2016). The government's post-16 skills plan states there will be “only one approved tech level qualification for each occupation or cluster of occupations within a route. These tech levels could play a role within the relevant apprenticeships, but only if employers decide that should be the case, and the Institute will need to consider the implications of this single tech level approach.” (BIS, 2016). This wording leaves some uncertainty. At the level of degree apprenticeships, there would, of course, still be two routes to the degree – a traditional university degree programme, and a degree apprenticeship.
Apprenticeships need to be very closely aligned with related vocational qualifications
A close alignment between apprenticeship standards and any associated technical qualifications reflects the logic and the spirit of the Sainsbury review, and best international practice. The responsibility of the Institute for Apprenticeships in ensuring that close alignment is therefore very much to be welcomed. Once an industry sector has defined the required competences for an occupation in the form of an apprenticeship standard, it would make no sense to reinvent that standard in different occupational requirements for a T-level qualification, (although such a qualification could reasonably include some additional cross-curricular competences). There are real risks of fragmentation, for example if there emerge slightly different qualifications, alongside apprenticeship, offering a confusing landscape of competing possibilities for the student, and indeed the employer. It is precisely this landscape of confusion, which, by common consent has been one of the weakest points in the English vocational training system, which current reforms are designed to tackle. The models for alignment in the Netherlands and Estonia offer possible models.
The Estonian system offers one option which could be applied in England
Pursuing the Estonian model, for each occupation there would be an examination – in effect the end-point assessment linked to the apprenticeship standard. Passing this examination might then offer the industry-recognised credential for the occupation, since it corresponds to a standard developed by employers in the sector. This same examination might then be pursued not only by apprentices, but also those that have been prepared for the examination – by taking a related T-level, and perhaps in other ways. Those completing T-level qualifications might be encouraged or indeed (as in Estonia) required to take the assessment examination. The principle would be that there is a single standard, defined in the apprenticeship standard and associated assessment plan by employers in the sector, and this should underpin the content of both apprenticeship and technical school-based programmes in the relevant field.
Alternatively, there could be a sequential approach
Chapter 2 showed that there is very limited general education, including basic skills, in English youth apprenticeships relative to many other countries. As suggested in that Chapter, one potential way of addressing that challenge, as well as articulating apprenticeship relative to T-levels, would be to define relevant T-levels as programmes to be pursued prior to entering apprenticeship, allowing general education to be pursued in the T-level followed by a narrower approach on occupational skills during the apprenticeship. This approach also has several attractions.
A clearer place for recognition of prior learning needs to be found
It has been widely recognised, in the Richard review and many other contexts, that in the past some so-called apprenticeships had degenerated into a simple exercise in recognition of existing skills, and current arrangements are designed to preclude that possibility. But the Richard review also argued that for those who already have the required skills, it should be possible to pursue the end-point assessment without having to go through an unnecessary apprenticeship programme (Richard, 2012). The OECD endorses that view, as this possibility of direct access to the end-point assessment, and the associated qualification, potentially represents an important route for certifying and making transparent those skills which have been acquired informally, recognising that in England most apprentices have work experience, and many will therefore already have some or even all of the competences necessary for the end-point assessment. Box 5.2 sets out how, in Australia and Denmark among many countries, apprenticeships may be completed on accelerated timetables, and how in Germany, an apprentice qualification can be acquired without pursuing an actual apprenticeship. These are two distinct options, depending on whether they do or do not involve an apprenticeship programme.
Box 5.1. Apprenticeship and school-based routes to the same qualification in the Netherlands and Estonia
In the Netherlands, the upper secondary vocational system (MBO) includes two parallel structures: an apprenticeship track (Beroepsbegeleidende Leerweg or BBL) and a school-based track (Beroepsopleidende Leerweg or BOL). Both tracks combine learning and working. In the apprenticeship track, at least 60% of the learning takes place in the workplace, often in the form of around one day of formal schooling and four days in the workplace. The school-based track includes at least 20% of workplace training and typically around 30%. Both tracks lead to the same qualifications. Participants in the school-based pathway are mainly youngsters, while almost 50% of those following a dual pathway are 24 or over. A contract (an employment contract in most cases) with a firm is mandatory to enrol in the dual/apprenticeship track. There is no such obligation for the school-based track.
In Estonia, just over 500 occupational standards, setting out the competences required for each occupation and developed by industry sectors, lightly regulated but with the co-operation of government, provide the foundation of the vocational system. Each standard is reviewed and updated every five years, and more often when necessary. Examinations to test realisation of the standard are developed and administered by 100 ‘awarding bodies’ (typically industry professional groups), granted the authority to run the examinations for a five-year period by government in consultation with the relevant industry sector and with government. Fees are regulated. These examinations form a required end- point assessment for all vocational programmes, (except those where there is no relevant examination). In upper secondary vocational programmes for example, there is a school-based test leading to the award of an upper secondary diploma, but all students are also required to take the occupational examination, and only two-thirds of them pass this typically more demanding test. These occupational examinations can be the end-point of quite different types of programme - an apprenticeship, or a school-based full-time programme for a young person, or a part-time programme for an adult – or the examination may be pursued without formal preparation by someone who has acquired the relevant competences working in the occupation.
Source: Fazekas, M. and I. Litjens (2014), A Skills beyond School Review of the Netherlands, OECD Reviews of Vocational Education and Training, OECD Publishing. http://dx.doi.org/10.1787/9789264221840-en.
Box 5.2. Recognition of prior learning in apprenticeships: Accelerated completion and qualification without taking the apprenticeship programme
In Australia, apprentices may receive course credits for skills they already have and their prior work experience, reducing training duration. The apprenticeship contract defines a nominal term, which can be reduced (or extended), as all apprenticeships are “competency based”. If an apprentice can demonstrate that they have acquired the required skill level, they may progress to the next stage of their training or complete the apprenticeship. Competency is assessed first by the “registered training organisation” (training provider). Then employers need to confirm that the apprentice is able to apply the same skills in the workplace.
Source: Australian Apprenticeships (2017), “For employers”, www.australianapprenticeships.gov.au/employers (accessed 30 August 2017); State Government of Victoria, Australia (2017), “TAFE and training”, www.education.vic.gov.au/training/Pages/default.aspx (accessed 30 August 2017).
In Denmark, Adults aged 25 and above may complete an apprenticeship through two alternative pathways, in addition to the regular apprenticeship programme. They first undergo individual competence assessment (which lasts between half a day and ten days), which determines which pathway they may follow. Those with at least two years of relevant experience are exempted from the basic course (which includes school-based vocational training), and follow a shortened main course (which normally combines school-based and company-based training) and do not need on-the-job training. Those with some relevant work experience or prior education and training may follow a shortened basic course (up to 20 weeks), a shortened main course and up to two years of on-the-job training.
Source: Danish Ministry of Higher Education and Science (2017), “Admission to vocational education and training (VET)”, https://ufm.dk/en/education-and-institutions/recognition-and-transparency/recognition-guide/admission-vet (accessed 30 August 2017).
In Germany, individuals may take an “external examination” (Externenprüfung), involving the final assessment of regular apprenticeship programmes without completing the programme itself. Access to this exam is limited to those who have worked in the target occupation at least for one and a half times as long as the duration of the apprenticeship, and they have been performing skilled tasks in their job. Relevant school qualifications may reduce or replace the required minimum work experience. Candidates may prepare for the assessment by themselves (e.g. taking tests from past years, reading the technical literature as proposed by websites that advise potential candidates) or following preparatory courses. In 2009 candidates who took the external examination accounted for 6.4% of successful apprenticeship final assessment candidates.
Source: BIBB (2017), “Externenprüfung: Voraussetzungen, Beratung und Vorbereitung” www.perspektive-berufsabschluss.de/de/501.php (accessed 30 August 2017).
While SFA funding rules allow for recognition of prior learning, their practical application will be challenging
SFA funding rules require the cost, content and duration of an individual apprenticeship to take full account of prior learning, so that it should not be possible to offer funding to learn skills which the apprentices already have (SFA, 2017). For example, the apprenticeship standard for equine groom includes many competences which someone familiar with horses may be expected to have already, but as a Level 2 apprenticeship occupying 12-18 months, with a minimum apprenticeship length of 12 months, it will be very difficult to address the needs of an experienced person, who needs no more than a top-up of skills and knowledge to realise the standard, or indeed those capable of succeeding immediately in an end-point assessment (see GOV.UK, 2017). If such individuals are shoehorned into full 12 month minimum apprenticeships, this will be wasteful. Equally, employers may not wish to offer apprenticeships, with 20% off-the-job training, to their employees if those employees are already close to fully skilled. The needs of individuals in these circumstances need to be addressed in some other way, leading to an end-point assessment similar to that offered in Germany. At present in England, there is no framework for supporting top-up training, or for the assessments, for those in this position. Nor is there the nomenclature in place to describe the qualification that would be obtained by such a person – although this could easily be rectified, for example by granting all those who successfully pass the end-point assessment a 'diploma' in that occupation, that might not always arise from an apprenticeship. Such a framework, allowing apprenticeship qualifications to be used to recognise prior learning, needs to be developed, recognising that those concerned will not be apprentices, and could not therefore be funded under the standard rules for apprentices.
Analysis: Defining the number of apprenticeship standards
Other countries have limited number of apprenticeship standards
Other countries with strong apprenticeship systems typically manage with a relatively limited number of apprentice training occupations. In Austria, Switzerland and Germany, although half or more of the youth cohort enter apprenticeship, the number of apprentice standards is between two and three hundred in each country (Bliem et al., 2016). In Ontario in Canada, apprenticeships are available in 150 skilled trades (Government of Ontario, n.d.). In Estonia, there are just over 500 occupational standards, and all apprenticeships need to be directed at one or other of these standards. In Denmark apprenticeship may be pursued in 111 different fields, at various levels in each field (European Commission, n.d.). Sometimes a specialisation may be permitted within an apprenticeship, without removing the main title of the qualification.
Pressures from employers to create overly narrow apprentice standards are to be expected
As set out in OECD (2010), employers in a specific occupational and industrial niche sector naturally want to develop the skills of those working in the sector, but do not want to lose those skills through mobility to other sectors, even in related sectors where skills requirements are similar. Apprentice qualifications, drawn narrowly to fit the niche sector alone, serve to protect the interests of employers in the niche sector, but by the same token can be an artificial barrier to labour market mobility, because they disguise the extent to which the skillset is transferable to other similar occupations. In the interests of apprentices, and the labour market as a whole, the IfA therefore needs to insist on broad apprentice standards, even if this slows down the replacement of frameworks by standards. In some countries – for example Austria - the involvement of trade unions in agreeing new training occupations balances employer interests, since trade unions, with an eye on the careers of their members, are typically interested in ensuring that apprentice graduates can work in a broad industrial sector. In Austria, the outcome of social partnership negotiation of individual apprenticeship profiles results in a fruitful compromise between meeting very specific employer needs and the longer-term career interests of the graduate apprentices (see Hoeckel, 2010).
Narrow apprentice standards will multiply the burdens on the IfA and employers
Every apprentice standard needs to be updated periodically in consultation with employers. Engaging employers in this task may be much harder than creating a standard, particularly in cases where employer consultation is necessary to pursue the unexciting but essential work of winding up redundant standards, or merging them into other standards. Each standard also requires bodies not only to provide training, but also to provide assessments – and this latter task is already a challenge. This needs to be pursued carefully and with adequate resources, both in the IfA, and on the employer side, to maintain quality in the apprenticeship system. The greater the number of standards the harder this task will be.
It is worrying that 500 apprentice standards are already under development
The IfA is already reporting concerns that some standards may be overlapping or amount to micro-qualifications (IfA, 2017). Some of those giving evidence to the House of Commons select committee suggested that as many as 1 500 apprenticeship standards may emerge, many times more than in comparable countries (as indicated above), and increasing the confusion for employers and learners (House of Commons, 2017). The narrowness of standards suggested by these numbers implies difficulty for the graduate apprentice in career development, challenges because of the burden of updating such a large number of standards, and greater difficulty in recruiting appropriate assessment bodies. It will also make apprenticeships less attractive by narrowing career opportunities for apprentice graduates. Taken together, these problems could seriously damage the apprenticeship system in England.
Policy issue 5.2: Ensuring reliable end-point assessments
Challenge: Thinking through the effects of markets in the provision of assessment
End-point assessments represent one of the key elements of quality assurance
In England, the new apprenticeship standards are attached to an assessment plan setting out the elements of the final assessment – paper and pencil examinations, interviews and practical tests. End-point assessments take place once the individual employer of the apprentices is satisfied that the apprentice has acquired all the relevant competences. The function of the end-point assessment is therefore to ensure that the confidence of the individual employer in the skills of the apprentice can be backed by an external and objective quality check. Such a check is in effect quality assurance, on behalf of employers and apprentice graduates more generally, that the skills of this individual apprentice graduate are up to the standard expected in the industry. It is envisaged that there will be a market in how assessments are conducted. So, while the post-Sainsbury reforms have eliminated competition in the market offering qualifications in favour of regulated monopolies subject to a franchise competition, an open market in the delivery of assessments will remain.
Policy pointer 5.2: Ensuring reliable end-point assessments
Few, if any, other countries seek to achieve consistency in assessment standards through multiple bodies conducting the assessment and consistency in standards will be impossible to achieve with current plans for multiple assessment bodies for individual standards. Given the key role of consistent assessment standards in the credibility and reputation of apprentice qualifications these plans should be reviewed.
Analysis: The impact of competition
Most other countries maintain one assessment body per assessment
Looked at across countries, there are few or no directly comparable systems, in which there is the possibility of choosing between competing assessment bodies to deliver the same final assessment. In the English context, it may not seem unusual to have such competition, given the history of regulated but quite open markets in the development of qualifications and associated assessments by awarding bodies. But one of the effects of the Sainsbury review is to move away from this model, at least in respect of qualifications, given a history of qualification proliferation, quality problems and potentially a race to the bottom in standards.
Several countries balance flexibility in how skills are acquired with single final assessments
The well-regarded Swiss post-secondary vocational education and training (VET) system allows a very open and competitive market in training providers, preparing students for the professional examinations, but there is a single examination, organised nationally under the aegis of employers, but subject to light touch federal regulation (see Fazekas and Field, 2013). A different example lies in the Dutch school system, which, by PISA standards at least achieves outstanding results with no national curriculum, and with schools that have extensive autonomy over how they teach, but subject to single national examinations. The principle in all these cases is that the single end-point assessment, administered by a single body, provides a counterbalancing form of quality assurance and accountability to an open market and choice in the forms of learning that lead to that end-point assessment (OECD, 2016).
Sometimes there will be little competition in the provision of assessments
For some, possibly many apprenticeship standards, it is already clear that there may be few bodies willing to offer assessments (see House of Commons, 2017). In these cases, the task of quality assurance will become that of ensuring that there is at least one provider of assessments, and regulating that provider. This task would be easier if single provision were built into the design of the system.
More common apprenticeships will involve competition between assessment providers
There will be little price competition in the market for assessment providers, given that employers that have not exhausted their levy pots will be relatively indifferent to price, while those that cannot draw on a levy pot will only have to pay 10% of the cost. Assessment providers will instead compete very narrowly on different approaches to exam questions and the training of assessors. At the same time, assessment bodies and the way in which they go about their work will inevitably have some influence on the pass threshold. This is worrying on two grounds. First, the incentives falling on an employer in choosing an assessment body will be to choose a body which is unlikely to challenge the judgement of that employer that an apprentice is ready to graduate. Providers, who may advise on the choice of assessor, will also not wish to see an apprentice that they have trained fail the assessment, not least as providers are judged on successful completions. Second, while there is no single objectively ‘right’ pass threshold, it is undesirable for such thresholds to vary between different assessment bodies, and such variability is inevitable given multiple assessment bodies.
Multiple assessment providers will lead to inconsistent assessment standards
A practical example highlights the problem. The assessment plan for the adult care worker apprenticeship standard includes a ‘judgement test’ and a ‘professional discussion’. The judgement test poses 60 multiple choice questions on real life scenarios. Achieving the 40 correct answers required for a pass clearly depends on the choice of questions and scenarios. Equally, the standardised interview template for the more open-ended professional discussion cannot avoid an element of subjective judgement on the part of the assessor. This is no criticism of the assessment plan, but it underlines the fact that consistent application of a pass standard is demanding, and even if just one assessment body is responsible, consistency between different tests at different points in time and different assessors presents many challenges – a challenge discussed in the context of educational testing in terms of test ‘reliability’ (Wells and Wollack, 2003). But a single assessment body could take clear responsibility for consistency/reliability, and address it by using common question banks and sets of scenarios for multiple choice questions, and by ensuring that assessors are similarly trained, and meet regularly to discuss the formal and informal criteria applied in the professional discussion, and converge their approaches. Multiple competing assessment bodies will make such convergence impossible. Any supposed overarching quality assurance will remain beside the point in the absence of a credible methodology for delivering consistency/reliability. In fact the terminology of ‘quality assurance’ misses the target: two good quality assessments can have inconsistent pass thresholds.
Many informed stakeholders are concerned about consistency in end-point assessments
The recent report by the parliamentary select committee, and evidence given to that committee by a wide variety of well-informed stakeholders including Ofsted, the Association of Colleges, and the organisations of awarding bodies, has underlined the serious challenges of ensuring consistency/validity in assessment standards, challenges which in the view of the select committee could not be adequately addressed by the proposed quality assurance arrangements (House of Commons, 2017). The OECD shares that view. The logical conclusion is that the challenge of consistency would be much easier to handle if England followed the well-established model of other countries, and franchised a single assessment body for each apprenticeship standard, or group of standards. As with vocational qualifications, some contestability should be preserved by franchising the body responsible for assessment in relation to a particular occupational standard for a limited period – say five years.
References
BIS (Department of Business, Innovation and Skills) (2016), Post‑16 Skills Plan, BIS, www.gov.uk/government/uploads/system/uploads/attachment_data/file/536068/56259_Cm_9280_print.pdf.
BIS (2014), “Apprenticeships evaluation: Learner survey”, BIS Research Paper, No. 205, www.gov.uk/government/publications/apprenticeships-evaluation-learner-survey-2014.
Bliem, W., A. Petanovitsch and K. Schmid (2016), Dual Vocational Education and Training in Austria, Germany, Liechtenstein and Switzerland, Institut für Bildungsforschung der Wirtschaft, www.dcdualvet.org/wp-content/uploads/Dual-VET-Comparative-Study_Full-Study.pdf.
Broek, S. and B.-J. Buiskool (2013), Developing the Adult Learning Sector: Quality in the Adult Learning Sector (Lot 1), Final Report (Open Call for tender EAC/26/2011), Panteia, Zoetermeer, http://arhiv.acs.si/porocila/Quality_in_the_Adult_Learning_Sector-final_report.pdf.
DfE (Department for Education) (2017), “Register of apprenticeship training providers”, www.gov.uk/government/uploads/system/uploads/attachment_data/file/562465/Provider_register_policy_doc.pdf.
European Commission (2014), Report from The Commission To The European Parliament And The Council, European Commission, Brussels, http://ec.europa.eu/dgs/education_culture/repository/education/policy/vocational-policy/doc/eqavet_en.pdf.
European Commission (2009), Recommendation of the European Parliament and of the Council of 18 June 2009 on the establishment of a European Quality Assurance Reference Framework for Vocational Education and Training.
European Commission (n.d.) , Apprenticeship Toolbox, www.apprenticeship-toolbox.eu/.
European Training Foundation (2014), Quality Assurance in Vocational Education and Training a Collection of Articles, European Training Foundation, www.etf.europa.eu/webatt.nsf/0/270970490A6E9327C1257CA800407038/$file/Quality%20assurance%20in%20VET.pdf.
Fazekas, M. and S. Field (2013), A Skills beyond School Review of Switzerland, OECD Reviews of Vocational Education and Training, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264062665-en.
GOV.UK (2017), “Guidance. Search for apprenticeship standards. New apprenticeship standards are developed by employer groups known as ‘trailblazers’, published 3 November 2017, www.gov.uk/guidancesearch-for-apprenticeship-standards.
Government of Ontario (n.d.), "Start and apprenticeship", www.ontario.ca/page/start-apprenticeship#section-3.
Hansard, House of Commons (2017), 27 March 2017. Evidence given by Robert Halfon Minister for Apprenticeships and Skills, https://hansard.parliament.uk/commons/2017-03-27/debates/8baab83a-aa21-4419-9313-efe97dcbb520/DraftPublicSectorApprenticeshipTargetsRegulations2017.
Hoeckel, K. (2010), OECD Reviews of Vocational Education and Training: A Learning for Jobs Review of Austria 2010, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264113695-en.
House of Commons (2017), Apprenticeships, Business, Energy and Industrial Strategy and Education Committees Sub Committee on Education, Skills and the Economy, www.publications.parliament.uk/pa/cm201617/cmselect/cmese/206/206.pdf.
Independent Panel on Technical Education (2016), Report of the Independent Panel on Technical Education, www.gatsby.org.uk/uploads/education/reports/pdf/report-of-the-independent-panel-on-technical-education1.pdf.
Institute for Apprenticeships (2017a), Driving the Quality of Apprenticeships in England, https://media.wix.com/ugd/4971f0_66f9a66ab82f456b97ce5fc656b923d1.pdf.
Institute for Apprenticeships (2017b), Driving the Quality of Apprenticeships in England (Institute for Apprenticeships operational plan), www.gov.uk/government/uploads/system/uploads/attachment_data/file/609008/IfA_Driving_Quality_.pdf.
Institute for Apprenticeships (2016), Adult Care Worker Assessment Strategy, www.gov.uk/government/uploads/system/uploads/attachment_data/file/539997/ACW_Assessment_plan.pdf.
Kis V. (forthcoming), “Making skills transparent: Recognising vocational skills acquired through work-based learning”, OECD Education Working Papers, OECD Publishing, Paris.
Kis, V. (2010), OECD Reviews of Vocational Education and Training: A Learning for Jobs Review of Ireland 2010, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264113848-en.
Musset, P. and S. Field (2013), A Skills beyond School Review of England, OECD Reviews of Vocational Education and Training, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264203594-en.
OECD (2016), Netherlands 2016: Foundations for the Future, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264257658-en.
OECD (2010), Learning for Jobs, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264087460-en.
Ofsted (2017), Further Education and Skills Inspection Handbook, www.gov.uk/government/uploads/system/uploads/attachment_data/file/608481/Further_education_and_skills_inspection_handbook_for_use_from_April_2017.pdf.
Ofsted (2015), Apprenticeships: Developing Skills for Future Prosperity: How Well Do Apprenticeships Meet The Needs of Young People, Their Employers And The Economy?
Pilz, M. (2012), “Modularisation of vocational training in Germany, Austria and Switzerland: Parallels and disparities in a modernisation process”, Journal of Vocational Education & Training, Vol. 64/2, pp.69-183, http://dx.doi.org/10.1080/13636820.2011.628757.
Richard, D. (2012), The Richard Review of Apprenticeships, School for Startups, London, United Kingdom, www.gov.uk/government/uploads/system/uploads/attachment_data/file/34708/richard-review-full.pdf.
Sweet, R. (2014), Work-based Learning: A Handbook for Policy Makers and Social Partners in ETF Partner Countries, http://ec.europa.eu/education/library/publications/etf-wbl-handbook_en.pdf.
U.S. Congress, Office of Technology Assessment (1994), Testing and Assessment in Vocational Education, OTA‑BP‑SET‑123, Government Printing Office, Washington, DC, United States, http://ota.fas.org/reports/9437.pdf.
Ulicna, D. and A. Curth (2013), Study on Quality Assurance in Continuous VET and on Future Development of EQAVET, Final Report, ICF GHK, Brussels, www.eqavet.eu/Libraries/2014_Publications/Study_on_QA_in_CVET.sflb.ashx.
Wells, C. and J. Wollack (2003), An Instructor’s Guide to Understanding Test Reliability, Testing & Evaluation Services, University of Wisconsin, https://testing.wisc.edu/Reliability.pdf.