This chapter defines the occupational area of “automotive technician” as assessed by PISA-VET and elaborates the framework for automotive technician that has been designed for the Development Phase of the project, including descriptions of the competencies and constructs to be assessed. It presents and explains the processes, content knowledge, and contexts and provides several sample items with descriptions of task characteristics. The chapter also discusses how performance in the automotive technician occupational area is measured and reported against proficiency levels and scales.
PISA Vocational Education and Training (VET)
2. Automotive technician
Copy link to 2. Automotive technicianAbstract
Introduction
Copy link to IntroductionThis chapter of PISA-VET assessment framework defines the occupational area of “automotive technician.” It has been prepared by a group of experts from a range of countries that are well versed in the training and assessment of automotive technicians. The chapter draws on previous work in assessing automotive technicians, such as the work undertaken as part of the ASCOT (+) initiative and WorldSkills competitions.
There are several reasons for selecting the occupational area of automotive technician for the Development Phase of PISA-VET, including the significance of vehicles for the economy, the international comparability of the occupation and key learning objectives of the associated VET programmes, as well as its role in the green and digital transformations:
Economic importance of automotive technology: In most OECD countries, individual motorised transport is the dominant form of transportation. In the long term (and despite the green transition, see below), it is still a growing market, especially in newly industrialised economies. The automotive industry is in many ways a hub that sets technological and especially digital standards worldwide. It prompts suppliers and vehicle workshops to adapt accordingly. Furthermore, it is a world leader and a major driver of innovation (e.g. future mobility), and accounts for a large number of jobs around the world.
International comparability and importance of the occupational area in VET: In all countries, technicians service, overhaul, and diagnose vehicles. They deal with internationally comparable technologies and so perform similar work tasks in each country. Automotive technicians are typically trained through VET programmes. The complexity of vehicles increases rapidly, changing capability profiles, requiring workers to participate in elaborated VET programmes, and setting new challenges to VET teachers and trainers. Baethge and Arends (2009, p. 33[1]) examined several countries and concluded that the number of students and trainees of automotive technicians in VET is large enough internationally to draw suitable samples.
Green transition: The transport sector is the second largest emitter of carbon dioxide, accounting for about 20 % of carbon dioxide emissions worldwide. About 75% of these emissions come from road transport (Ritchie, 2020[2]). If vehicles are not regularly and properly maintained and adjusted, their CO2 emissions can increase significantly. Consequently, the work of automotive technicians can contribute to environmental and climate protection. Moreover, automotive technology is developing rapidly due to the energy transition that vehicles are currently undergoing. It is important that automotive workshops follow this development to make their contribution.
Digital transition: The occupational area is highly relevant in terms of the digital transformation. Due to the change in thinking towards electric vehicles, the work and service profile of workshops will become more complex than in the past and therefore their impact on employment will continue to be relevant. The electrification and complexity of vehicles require automotive technicians to use different software, computer-based expert systems, and other digital information and communication systems. The transformation to e-mobility creates new environmental challenges due to the materials, infrastructure, and energy needed. For example, future assessments of automotive technicians will need to include high-voltage technology in connection with e-mobility/hybrid vehicle technology as well as the complex area of pollutant reduction in internal combustion engines (especially diesel engines),
Defining the automotive technician occupational area
Copy link to Defining the automotive technician occupational areaOccupational definition and associated VET programmes
Copy link to Occupational definition and associated VET programmesAutomotive technicians are defined as professionals who work in workshops and service, overhaul, and troubleshoot light vehicles (passenger cars, light commercial vehicles). Within the International Standard Classification of Occupations (ISCO-08), automotive technicians are part of the “Motor Vehicle Mechanics and Repairers” occupation (i.e. ISCO-08 code 7231). It is defined as: “Motor vehicle mechanics and repairers fit, install, maintain, service and repair engines and the mechanical and related equipment of passenger cars, delivery trucks, motorcycles, and other motor vehicles. Their tasks include: a) detecting and diagnosing faults in engines and parts; b) fitting, examining, testing and servicing motor vehicle and motorcycle engines; c) replacing engine components or complete engines; d) fitting, examining, adjusting, dismantling, rebuilding and replacing defective parts of motor vehicles; e) installing or adjusting motors and brakes, and adjusting steering or other parts of motor vehicles; f) installing, adjusting, servicing and replacing mechatronics components of motor vehicles; g) performing scheduled maintenance services, such as oil changes, lubrications and engine tune-ups, to achieve smoother running of vehicles and ensure compliance with pollution regulations; and h) reassembling engines and parts after being repaired”. The European Skills, Competences, Qualifications and Occupations (ESCO, 2020[3]) classification, further breaks this down into ten more narrow occupations, of which that of “vehicle technician” is mostly closely related to the occupation under consideration in this chapter. It is defined as: “Vehicle technicians inspect, test and maintain vehicles, motorcycles, engine tune-ups and tyre replacements. They repair engine failures, lube replacements. Vehicle technicians replace vehicle components and evaluate warranty options.”
The capabilities to perform these tasks are acquired in VET programmes corresponding to ISCED level 3 and EQF level 4 or equivalent. PISA-VET covers learners in the last six months of their vocational training.
Table 2.1. Examples of VET programmes for automotive technicians
Copy link to Table 2.1. Examples of VET programmes for automotive techniciansSelected programmes in data collection countries of PISA-VET
Australia |
Germany |
Portugal |
United Arab Emirates |
|
---|---|---|---|---|
Title |
AUR30620 - Certificate III in Light Vehicle Mechanical Technology |
Kfz-Mechatroniker/-in |
Tecnico de mecatronica automóvel |
Certificate 4 in Light Vehicle Service |
ISCED level |
4 |
3B |
3 |
3 |
EQF level |
3 |
4 |
4 |
3 |
Duration (years) |
4 |
3.5 |
3 |
2-3 |
Note: EQF: European Qualifications Framework; ISCED: International Standard Classification of Education; Kfz-Mechatroniker/-in: Motor vehicle mechatronics technician
International comparability of the occupational area
Copy link to International comparability of the occupational areaThe occupational area of automotive technician is comparable at the international level for the following reasons:
It has been found that VET programmes for automotive technicians in different countries aim to teach comparable capabilities, although the VET programmes differ internationally (e.g. in terms of the amount of school-based and vocational learning; (Baethge and Arends, 2009, pp. 3-4[1]).
As vehicles from different manufacturers are present all over the world, the requirements in this area are similar internationally.
The tasks and the tools used to fulfil these requirements are highly similar worldwide, as they are closely linked to vehicle technologies. An increasing number of vehicle technologies are based on shared platforms as this helps vehicle manufacturers to reduce costs.
Due to requirements established by automobile manufacturers, automotive technicians must often use standardised procedures.
Standardisation increasingly leads to comparable safety, environmental, and climate protection requirements and thus to comparable requirements overall.
Organising the domain of automotive technicians
Copy link to Organising the domain of automotive techniciansIn this section of the chapter, the domain of automotive technicians is first organised and described by specifying the context, contents, and processes of the domain. While the context refers to both work tasks and the environment in which automotive technicians work, the contents cover the major subjects that automotive technicians deal with in performing their core tasks. The processes refer to activities that are required to fulfil the core tasks of automotive technicians. These specifications are then used to define the key capabilities of automotive technicians, which form the basis for the development of the assessment described in the further sections of this chapter.
Context
Copy link to ContextContext category: workplace and work orders of automotive technicians
Copy link to Context category: workplace and work orders of automotive techniciansAutomotive technicians work in two kinds of workplaces and deal with different types of vehicles.
Regarding the workplace, two context categories can be identified: work in the production of vehicles or in workshops. Following the VET-LSA feasibility study (Baethge and Arends, 2009[1]) and the 2022 WorldSkills Occupational Standards, automotive technicians working in workshops are selected as the focus for PISA-VET. Sub-categories of workshops can be identified, such as brand workshops, independent workshops, fleet workshops). As automotive technicians in these workshops deal with comparable content (e.g. systems, technologies) and perform comparable tasks, no distinction is made between these subcategories in the remainder of the chapter. In terms of vehicles, the focus is on light vehicles, i.e. passenger cars and light commercial vehicles from various manufacturers, in line with the 2022 WorldSkills Occupational Standards. Other vehicles such as agricultural vehicles, commercial vehicles, motorbikes, or construction vehicles are disregarded.
There are three types of work orders in workshops, each of which is formulated in the form of specific, often combined work orders: service orders, overhaul orders, and troubleshoot orders. Autobody repairers are not included in this framework, as they carry out other work orders.
Desired distribution of items
Copy link to Desired distribution of itemsThe items used in the assessment relate to the above context category, i.e. light vehicles in workshops and orders for service, overhaul, and troubleshooting in workshops.
Table 2.2. Desired distribution of items by context
Copy link to Table 2.2. Desired distribution of items by context
Context category |
Percentage of items in the assessment |
---|---|
Workshop |
100 |
Processes and the underlying automotive technician capabilities
Copy link to Processes and the underlying automotive technician capabilitiesAs described above, automotive technicians engage in the work orders of servicing, overhauling, and troubleshooting light vehicles. There can be considerable overlap between these: Service includes inspection and may require overhaul. Overhaul includes replacing, restoring, and adjusting parts and components. Troubleshooting is about diagnosing the cause(s) of malfunctions and taking action to correct them by replacing, restoring, or adjusting parts and components.
In PISA-VET however, the processes must be distinct, as each assessment item should be clearly attributable to a single process. Otherwise, the assessment results are either difficult to interpret and do not provide a sound basis for policy recommendations, or they require complex statistical models whose results are difficult to communicate or may lead to misinterpretation. Therefore, the following five separate processes are included in the assessment of automotive technicians based on the three overlapping work orders above: inspect, diagnose, replace, restore, and adjust.
To carry out these processes, automotive technicians need certain fundamental capabilities. Possessing capabilities means that someone can perform a given task. Capabilities are mentally represented in the form of procedural knowledge. Procedural knowledge can represent heuristics and internalised systematic approaches to inspect, diagnose, replace, restore, and adjust parts, components, and subsystems of light vehicle systems. It can also refer to algorithms and internalised rules that clearly define each step to be performed in these processes. Procedural knowledge includes both the capabilities to perform the process and the knowledge needed to describe how to perform the process. Automotive technicians need procedural knowledge for:
Obtaining information from standardised sources (e.g. access mounting positions or wiring diagrams using repair manuals or digital expert systems).
Operating measuring instruments (e.g. multimeter, brake tester, wheel aligning system).
Performing measurements (e.g. measuring voltage on the pins of a connector, measuring the freezing point of engine coolant).
Adjusting (e.g. adjusting headlights, adjusting the amount of engine oil, tire balancing).
Restoring or replacing vehicle parts, components, or subsystems (e.g. carrying out a cable repair, replace the cabin filter), etc.
When the automotive technician applies the procedural knowledge, content knowledge can be required (e.g. to justify the steps, sequences, and results of a certain course of action). Content knowledge categories are described in the next section.
For substantive and pragmatic reasons, the five processes are grouped into the following two process categories: investigate and rectify. The investigate category comprises inspect and diagnose, and the rectify category includes restore, replace, and adjust. The substantive reason for this grouping is that “inspect and diagnose” are about investigating vehicles and drawing conclusions from the information obtained and that “restore,” “replace,” and “adjust” aim to rectify light vehicles, i.e. to establish or re-establish desired states of light vehicles. This grouping also allows to report proficiency on two scales instead of five, which makes it easier to communicate assessment results.
Investigate
Copy link to Investigate“Investigate” includes two sub-processes: inspect and diagnose light vehicles. While "inspect” aims to examine whether vehicles have undesired states, “diagnose” aims to identify the cause(s) of a certain kind of undesired states, namely vehicle malfunctions.
Inspect
Copy link to Inspect“Inspect” includes activities to determine if there are any undesired states of vehicles. It comprises (1) gathering information about the vehicle systems, and (2) drawing conclusions from this information.
1. A standard procedure is used to collect information about the vehicles. In the standard procedure, each step of information collection is clearly defined. The steps typically represent sensory tests of the actual sates of operating fluids, parts, components, or subsystems of the vehicle.
2. The information obtained about the vehicle states is compared with target values and used to determine whether undesired states exist and, if so, their specification. Undesired vehicle states can be malfunctions (e.g. light does not work) or parts, components, or subsystems of a vehicle that do not comply with standards (e.g. brake pads that are too thin). Standards are legal requirements or requirements set by automobile manufacturers.1 Based on the specification of undesired states, the next actions are derived. In case of deviations from the standard, the action following the inspection is "rectify” (see below), in case of malfunctions, it is "diagnose.”
Diagnose
Copy link to Diagnose“Diagnose” aims to identify the cause(s) of a vehicle malfunction and suggest actions to eliminate it. Actions to eliminate malfunctions are “(dis)assemble and replace,” “rebuild and restore,” and “adjust” (see below). “Diagnose” refers to a coordination between evidence and hypotheses in which automotive technicians generate hypotheses based on the available evidence and then generate and interpret further evidence to test the respective hypotheses (Abele, 2018[4]; Klahr and Dunbar, 1988[5]). “Diagnose” comprises various steps, which in reality do not usually run in a linear fashion: (1) generating evidence to formulate hypotheses about causes of malfunctions and to evaluate these hypotheses, (2) generating hypotheses about causes of malfunctions, and (3) drawing conclusions from this evidence about the cause(s) of the malfunction and the actions to eliminate it.
1. Evidence is information relevant (a) to formulate hypotheses about the causes of undesired states and (b) to evaluate those hypotheses. (a) The evidence needed to formulate hypotheses comes from activities such as reading work orders, collecting information on the vehicle, and investigating its undesired state. (b) To evaluate hypotheses, tests are required. Tests require strategies and can be carried out with equipment (e.g. measurement tools) or without equipment through sensory testing (e.g. visual checks). Measurement strategies can either be self-developed or retrieved from long-term memory or external sources (e.g. computer-based expert system). In addition to measured values, target values for parts, components, and subsystems are needed to evaluate hypotheses. Target values can be obtained from long-term memory or external sources (e.g. technical documents).
2. Hypotheses provide potential explanations (i.e. causes) for undesired states of the vehicle. They are formulated based on the knowledge gained and hypotheses that have already been rejected.
3. To infer the cause(s) of the undesired states, all the evidence and hypotheses obtained must be integrated. The cause(s) can then be specified and described precisely. Causes of undesired states can be, for example, defective parts, components, or sub-subsystems of vehicles. From the description of the cause(s) it can be concluded how to eliminate the undesired state (replacing, rebuilding, or adjusting).
"Diagnose” is a special type of problem solving (Jonassen, 2000[6]) and meets the main characteristics of problem solving as described in the PIAAC framework: It refers to dynamic situations, as it includes manipulations of vehicles that change their states (e.g. switching off the ignition can change the measured values). In diagnostic situations, methods to identify the cause(s) of undesired states are not immediately available, especially at the third level (see section 3.3.4). Finally, diagnosis involves defining the diagnostic situation and searching for information (e.g. to obtain evidence) and applying the solution (i.e. the conclusion on the actions to be taken to eliminate the undesired state).
Rectify
Copy link to RectifyThis process2 includes actions in which automotive technicians actively alter the condition of a vehicle to establish or re-establish a specified condition or desired state. It includes but is not limited to the actions of “replacing and (dis)assembling,” “rebuilding and restoring” and “adjusting.”
Typical reasons for such actions include: restoring functionality after damage or malfunction (i.e. repairing), compensating for wear, maintaining a vehicle's functionality and safety (i.e. preventing future damage, malfunction, or excessive wear), meeting the manufacturer's specifications, upgrading a system, or fulfilling a customer's request.
Replace and (Dis)assemble
Copy link to Replace and (Dis)assembleThe first sub-process “Replace and (Dis)assemble” includes all actions in which some part, component, subsystem, or operating fluid (e.g. engine oil, coolant, etc.) is removed from a vehicle and either reattached or replaced. There are various reasons for such actions. Parts or components are often removed to gain access to other parts or components; in this case, they are later re-attached without replacement. Parts or components are also removed if they need replacement. Such replacement might be technically necessary, e.g. to restore functionality after damage, to maintain functionality in case of ageing or wear, or because of product recalls. A replacement might also be in order at a customer's request without being technically necessary, e.g. if a vehicle owner wishes to upgrade an existing system, or to change the vehicle's appearance according to individual taste.
The replacement parts, components, subsystems, or operating fluids can be of the exact same type as the removed one, or of a different type (e.g. an improved version).
Replacing a part, component, subsystem, or an operating fluid can be an easy task (e.g. replacing a wheel or replacing engine oil) or a complex operation (e.g. replacing the timing belt of an engine or replacing the radial seal of a crankshaft). It might require disassembling subsystems, components and/or parts to gain access to the subsystem(s), component(s), or part(s) to be replaced. Sometimes, existing structures must be torn down or destroyed to remove a subsystem, component, or part and/or to install a replacement subsystem, component, or part, e.g. when an exhaust pipe is cut off to replace a damaged silencer.
Automotive technicians usually begin the (dis)assembling or replacement process by retrieving information about the target or goal of the operation – usually by reading a work order. The next step involves using their expertise to determine the precise location of the part, component, system, or operational fluid, as well as the necessary procedure for (dis)assembling or replacing it. They also consider the required condition of the vehicle, the specific tools and their settings, and any necessary equipment and materials. In case of any missing materials, they ensure that the required materials are ordered or provided promptly. If the technicians' knowledge is not sufficient, they will have to retrieve this information from external sources such as maintenance and repair instructions, exploded view drawings, schematics, or diagrams, which requires literacy and – in case of digital workshop information systems – ICT skills.
Technicians use different tools and equipment to (dis)assemble and replace fluids, parts, and components. Usually, they follow a pre-defined sequence of steps, but under certain circumstances it might also be necessary for them to develop an individual professional solution to the problem. During the procedure, the technicians should monitor their progress frequently. After they completed the operation, they usually conduct a final assessment to ensure success. Sometimes, such assessments involve a systematic examination or comparison of an actual state to a specified state; such actions fall under the core process investigate.
Examples include:
replacing engine oil
replacing air filter
replacing brake pads
replacing spark plugs
replacing a silencer
replacing a ball joint
replacing a timing belt
replacing a windshield.
Restore and Rebuild
Copy link to <strong>Restore and Rebuild</strong>The second sub-process “Restore and Rebuild” refers to corrective actions that reinstate the function and/or specified state of a broken, damaged, malfunctioning, or excessively worn part, component, or subsystem without replacing it.
Rebuilding or restoring a part, component, or subsystem can be an easy task (e.g. fixing a punctured tyre with a tyre repair kit) or a complex operation (e.g. fixing a damaged threaded hole in an oil sump using a threaded insert repair kit). It might require disassembling parts, components, or subsystems to gain access to the part that needs repair. Sometimes, existing structures must be torn down or destroyed to prepare or conduct the rebuilding, e.g. when a stripped bolt head or nut needs to be cut off or drilled out to remove a part. In these cases, (dis)assembly and/or replacement skills are necessary before and after the actual repair task is performed.
Automotive technicians usually begin the rebuilding/restoring process by retrieving information about the target or goal of the operation—usually by reading a work order. In the next step, they access their knowledge about the procedure, required tools, equipment, and material(s). If the technicians' knowledge is not sufficient, they will have to retrieve information from external sources such as service manuals, exploded view drawings, schematics, or diagrams, which requires literacy and – in case of digital media – ICT skills.
Technicians use different tools, equipment, and materials to rebuild or restore fluids, parts, and components. They might follow a pre-defined sequence of steps, but it might also be necessary to develop an individual professional solution to the problem. During the procedure, the technicians should monitor their progress frequently. After they completed the operation, they usually conduct a final assessment to ensure success. Sometimes, such assessments involve comparing an actual state to a specified state; such actions fall under the core process investigate. If a part, component, or subsystem is repaired by simply replacing one of its parts or components, such actions fall under the sub-process “Replace and (Dis)assemble.”
Examples include:
Fixing a punctured tyre with a tyre repair kit.
Soldering a cut wire with a soldering connector.
Restoring a damaged threaded hole in a cylinder head using a threaded insert repair kit.
Adjust
Copy link to AdjustThe third sub-process “Adjust” refers to actions in which automotive technicians change the setting of a car's part, component, or subsystem to specified values. Adjusting is only possible if the part, component, or subsystem in question is adjustable, i.e. the manufacturer provided it with a mechanism, interface, or other option to tamper its settings or calibration.
Adjusting often involves the use of specialised equipment such as wheel alignment systems or headlight adjustment tools. Reasons for adjustments can be:
Restoring correct settings after replacement, rebuilding or incorrect adjustment.
Compensating for engineering tolerance.
Compensating for wear.
The process of adjusting often involves the comparison of an actual state to a specified state; in this regard it is like investigation tasks. It also includes acts of changing a setting to a specified state; in that regard it is like rebuild and replace. However, in contrast to diagnosis, adjusting involves an active change of state or condition. In contrast to the other rectifications processes – which eliminate damage or malfunction – adjustments are taken out on intact parts, components, or subsystems.
Adjustments can be easy tasks (e.g. inflating a tyre to a specified pressure) or complex operations (e.g. adjusting the camber, caster, and toe on a car with a multi-link suspension). Automotive technicians usually begin the adjustment process by retrieving information about the target or goal of the operation – usually by reading a work order. In the next step, they access their knowledge about the procedure, required tools, and equipment. If the technicians' knowledge is not sufficient, they will have to retrieve information from external sources such as service manuals, exploded view drawings, schematics, or diagrams, which requires literacy and – in case of digital media – ICT skills.
Depending on what part, component, or subsystem is adjusted, technicians use different tools and equipment. They usually follow a pre-defined sequence of steps and rarely improvise. During the procedure, the technicians should monitor their progress frequently. After they completed the operation, they usually conduct a final assessment to ensure success. Sometimes, such assessments involve comparing an actual state to a specified state; such actions fall under the core process investigate.
Examples include:
Adjusting the headlights using a headlight adjustment tool.
Adjusting the camber, caster, and toe of the wheels using a wheel alignment system.
Adjusting the ground clearance of a car after equipping it with height adjustable springs.
Calibrating the sensors of a driver assistance system.
Desired distribution of items by automotive technicians’ processes
Copy link to Desired distribution of items by automotive technicians’ processesThe item distribution by automotive technicians’ processes is shown in the following Table. It was extensively discussed and agreed upon by a group of experts from different countries. The item distribution reflects the frequency of processes in the daily work of automotive technicians. "Investigate” (diagnose, inspect) is overall more frequent, because rectification measures (replace and [dis]assemble, restore, and rebuild, adjust) usually involve inspection steps as well (e.g. to monitor progress and to ensure success of the rectification task). The higher percentage of "investigate" compared to "rectify" is also supported by the VET-LSA feasibility study (Baethge and Arends, 2009[1]) and by the WorldSkills Occupational Standards.
Table 2.3. Desired distribution of items by automotive technicians’ processes
Copy link to Table 2.3. Desired distribution of items by automotive technicians’ processes
Process category |
Percentage of items in the assessment |
|
---|---|---|
Investigate |
Diagnose |
50 |
Inspect |
10 |
|
Rectify |
Replace and (Dis)assemble |
30 |
Rebuild and Restore |
5 |
|
Adjust |
5 |
|
Total |
100 |
Within the first process “Investigate,” the sub-processes "Diagnose" and “Inspect” were given a different weight. Even though both sub-processes occur frequently in daily work, inspection tasks are usually much less demanding than diagnostic tasks. As a result, items to assess inspection skills will only cover the lowest part of the ability scale, whereas the “Diagnose” process covers the whole range of item difficulties. Therefore, “Inspect" is represented by much fewer items compared to "Diagnose".
Within the second process “Rectify,” the sub-processes “Replace and (Dis)assemble” were given a higher weight because in modern workshops it is much more common to replace damaged or defective parts and components instead of rebuilding and restoring them. Also, most rectification tasks include at least some steps of (dis)assembly.
Fundamental capabilities underlying the automotive technician processes
Copy link to Fundamental capabilities underlying the automotive technician processesTwo broad capabilities and associated sub-capabilities are fundamental and underlying the requirements for the assessment of automotive technician learners: investigation and rectification capability. The learners can achieve different levels of these capabilities.
Investigation capability is defined as an automotive technician’s ability to gather information about light vehicles in workshops, to draw conclusions from the information obtained and to justify the actions taken and results achieved. The investigation capability includes capabilities to (1) inspect vehicles and to (2) diagnose vehicles:
1. To inspect vehicles requires procedural knowledge to investigate the states of the vehicles systems and, in case of undesired states, to conclude whether diagnosis or what type of rectification is needed. This includes justifying the conclusions about the undesired states and the inspection steps taken.
2. To diagnose vehicles requires procedural knowledge to investigate undesired states of vehicle systems, formulate failure hypotheses and to infer the cause(s) of the undesired states and the actions to rectify them. This includes justifying the steps taken to explore the undesired states and the conclusions about the cause(s) of the undesired states as well as the rectification actions proposed.
Rectification capability is defined as the ability to establish or re-establish desired states of light vehicles to remove damage, correct malfunctions, maintain functionality, meet manufacturers' specifications, or fulfil customers' wishes. Rectification capability includes the abilities (1) to (dis)assemble and/or replace, (2) to rebuild and/or restore, and (3) to adjust light vehicles’ parts, components, or subsystems. It also includes the associated content and procedural knowledge.
1. To (dis)assemble and/or replace, automotive technicians require procedural knowledge about single steps, required tools, and equipment involved in the process. They should be familiar with sources of technical information such as repair manuals, wiring diagrams and digital expert systems, and should be able to use these sources for information retrieval.
2. To rebuild and restore, automotive technicians require procedural knowledge about single steps, required tools, and equipment involved in the process. They should be familiar with sources of technical information such as repair manuals, wiring diagrams and digital expert systems, and should be able to use these sources for information retrieval.
3. To adjust, automotive technicians require procedural knowledge about single steps, required tools, and equipment involved in the process. They should be familiar with sources of technical information such as repair manuals, wiring diagrams and digital expert systems, and should be able to use these sources for information retrieval.
Content knowledge for automotive technicians
Copy link to Content knowledge for automotive techniciansContent categories
Copy link to Content categoriesAutomotive technicians deal with light vehicles, and these provide the structures of knowledge content for this occupational area. Light vehicles consist of several systems:
drive unit
powertrain
undercarriage
assistance
chassis.
Apart from the chassis, these systems are relevant content categories and topics for automotive technicians. The chassis is excluded because in many countries this content and the related activities do not fall within the scope of automotive technicians or their training curriculum, but rather into the occupational area of autobody repairer. Each system has central functions (e.g. to convert other forms of energy into kinetic energy). The systems consist of subsystems (e.g. fuel injection) that are interconnected and made up of components (e.g. common rail injector) which can be also interconnected. Components, in turn, are composed of parts. Parts are elements that cannot be further disassembled (e.g. coil) without destroying them. The functions of the systems are realised through the connections and interactions of the subsystems, components, and parts. Further details about the four systems can be found in the Annex of this chapter.
In modern light vehicles, the four systems can be distinguished according to their functions. They cannot be distinguished by the technical principles (e.g. electrotechnical, mechanical, hydraulic, etc. principles) they use to provide these functions, as they all use a mixture of technical principles. From this principle-based perspective, service, overhaul, and troubleshooting of modern light vehicle systems require some of the same content knowledge. Because of this overlap in knowledge, the four content categories are not considered as empirically distinct dimensions of knowledge. Rather, these categories are used to ensure that the relevant knowledge of automotive technicians is broadly covered in PISA-VET. All principles and related knowledge required to service, overhaul, and troubleshooting vehicles should be included in the assessment. Assessment of abstract knowledge that is not relevant to understanding and performing these activities and to justifying the steps, sequences, and results of these activities should not be assessed.
Content knowledge
Copy link to Content knowledgeTo service, overhaul, and troubleshoot light vehicles and their systems, automotive technicians need content knowledge. In this chapter, content knowledge is defined as disciplinary knowledge (see Appendix A for definitions), i.e. the knowledge of concepts of vehicle systems. Following de Jong and Ferguson-Hessler (1996[7]), concepts represent facts, technical terms, and principles (e.g. function of systems). Content knowledge functions as additional information on the work order at hand, that automotive technicians use to understand and perform the order (De Jong and Ferguson-Hessler, 1996, p. 107[7]). Content knowledge is also needed to justify the steps, sequences, and results of a certain course of action. The most relevant content knowledge includes:
Knowledge of technical terms and facts, such as for example legal requirements, requirements of vehicle manufacturers, types and signs of malfunctions, damage, and wear (e.g. visual appearance of corroded cables), most common designs etc. for the respective system.
Knowledge of scientific principles (e.g. principles of electronics, fluid mechanics, material science) relevant to the system in question.
Knowledge of the structure of the subsystems, components, and parts including their interdependences and connection types (e.g. electrical wire, hydraulic lines) for the respective system.
Knowledge of the functions of each system, the system-related subsystems, and components (e.g. knowing the function of a fuel temperature sensor).
Desired distribution of items by content knowledge
Copy link to Desired distribution of items by content knowledgeThe table below lists the four categories of content knowledge. The items should be evenly distributed among the categories, as they can be considered equally important. It is proposed to develop the content knowledge items by selecting a concrete work order and determining associated process(es). Afterwards, the content knowledge (i.e. the knowledge of the technical terms and facts, the principles, the structure, and the functions) necessary to carry out this process should be determined. Finally, the corresponding items can be designed.
Table 2.4. Desired distribution of items by content knowledge
Copy link to Table 2.4. Desired distribution of items by content knowledge
Content knowledge |
Percentage of items in assessment |
---|---|
Content knowledge of the drive unit |
25 |
Content knowledge of the power train |
25 |
Content knowledge of the undercarriage |
25 |
Content knowledge of the assistance system |
25 |
Total |
100 |
Assessing automotive technician learners
Copy link to Assessing automotive technician learnersIn PISA-VET, the capabilities of automotive technician learners to carry out investigation and rectification processes will be assessed. The assessment should provide objective, reliable, and valid results about the learners’ capabilities. It should accurately measure the capabilities learners need for the occupational area, and predict their future job performance, while minimising the impact of confounding factors on the results (e.g. local standards and contexts).
To obtain reliable and valid results, the items should authentically and broadly represent the constructs. The number of items and their coverage of the constructs should be as high as possible within the constraints imposed by student fatigue and survey implementation considerations.
Technology-based assessments allow for a cost-effective assessment that is also feasible in international large-scale settings. There is evidence for the domain of automotive technicians that technology-based assessments can provide objective and reliable results in reasonable testing times (Abele et al., 2014), valid interpretations of these results (e.g. Gschwendtner et al., 2009), and objective process data which can be used to increase the informative value of assessments (Abele and von Davier, 2019).
To assess the investigation and rectification capabilities, technology-based authentic scenarios can be used. The scenarios include authentic work orders, authentic work environments, and where possible and appropriate allow for authentic interactions. They are presented in a computer environment and can use digital simulations and/or videos to imitate relevant features of the automotive technicians’ workplace as authentically as possible. An example of such a scenario would be a situation where learners must diagnose the malfunction of a vehicle in a computer simulation that represents a typical work order, (parts of) the light vehicle as well as the work environment and allows typical diagnostic activities.
Authentic scenarios are used for technology-based assessment. Each scenario starts with a work order which is a formal request to carry out a single task (e.g. “Replace the timing belt of the engine”) or several tasks (e.g. “Diagnose why the engine does not start, and eliminate the cause of the malfunction”). A task requires test takers to perform actions that fall under one of the five processes (inspect, diagnose, etc.). Within each task of the scenario, test items assess specific aspects of the task. Tasks usually contain more than one item.
For example, a scenario may involve tasks of diagnosing and rectifying a specific vehicle malfunction. The test items may assess the test takers knowledge to:
identify the cause(s) of the malfunction (e.g. defective component)
justify their conclusion
explain how the diagnosed defect causes the symptoms of the malfunction at hand, and
replace the defective component.
The capabilities are assessed in a two-step procedure: First, the sub-capabilities of each capability are measured. For reporting purposes, the results are then used to build scores for the investigation capability and rectification capability.
To assess the sub-capabilities reliably and validly, at least three tasks per process are required. Consequently, at least fifteen tasks are used in the assessment, as there are five processes. The number of scenarios could be less than 15 as they include work orders and work orders can integrate more than one task (process). The number of items per task depends on the testing time available and the percentages given above. Assuming a total of 150 items and a testing time of 2 minutes per item, this results in a total testing time of 5 hours and, considering test fatigue, a booklet design is needed. Using 150 items as an example, “diagnose” and the content topic “electric and electronic” would each be represented with seventy-five items, “adjust” with eight items (rounded up).
Item formats and scoring
Copy link to Item formats and scoringTechnology-based assessments can contain simulation items. Simulation items have the potential to offer a more engaging experience for test takers as they are capable of emulating real-world (and thus more relevant) settings through complex interactions and the use of multiple media, such as animations, and sounds. Albeit complex to develop, simulations have the potential to validly assess the investigation and rectification capability as they can authentically represent parts of the work environment.
Another challenge with simulations is grading – subject matter and assessment experts will be involved in the development of (automatic) marking rubrics and how to collect data on test takers’ strategies to potentialise the reporting of the outcomes. The next sections provide examples of simulation items and references that show that such items are feasible and can lead to reliable and valid results.
Technology-based items can be open-ended constructed-response items, closed constructed-response items, and selected-response items. They are expected to be combined in the assessment and are selected based on content and difficulty considerations.
The number of open-ended constructed-response items will be kept to a minimum as scoring such items is reliant on dedicated systems and coder training, thus time-consuming and less practical in international large-scale assessments. Additional challenges notwithstanding, open-ended items can be especially useful to assess certain, more complex components of the framework and to infer whether learners can generate responses based on more complex cognition and integrated knowledge. They are beneficial, for example, to assess how and to what quality learners justify the results they have achieved or the actions they have taken. Where such items are needed, algorithms will be explored for automatic scoring to decrease the workload on national counterparties as much as possible, but useful algorithms are difficult to develop and to validate, meaning no definite implementation can be decided now. Closed constructed-response items have a finite number of responses. In these, users manipulate the state by their input but do not have unlimited choices (not free typing or drawing), as they can only pick from a certain number of actions, while they have more choices than in a typical multiple-choice item. Examples are selection and dragging and dropping of objects, drawing graphs with fixed start and endpoints, etc. Such items are also useful to tap more complex knowledge and cognition but are limited in this respect compared to open constructed-response items.
Selected-response items (simple and complex multiple-choice items) are employed, for example, to assess content knowledge such as the knowledge of facts (e.g. components of a system, legal requirements). The development of such items considers measures to reduce rate biases (e.g. a high number of choices) and the risk of assessing superficial thinking and knowledge (e.g. the use of many appropriate and complex distractors).
The items will be scored dichotomously or with partial credits based on expert generated scoring guides to encode test takers’ responses. Two response types are used for the item scoring: learners’ written responses and process data. The process data results from the learners’ interactions with the technology-based interactive task environments and is recorded by the computer, or by data related to student response reported by expert judges. Such data is used to score for example, whether the learners have performed a relevant diagnostic activity (e.g. measuring the resistance of a particular sensor in a computer simulated vehicle). In this case, the natural processes do not have to be interrupted for item scoring purposes. The item scoring can be done afterwards with the help of the process data.
Demonstration tasks could also be used in the assessment. In this case, expert judges, and relevant material (e.g. vehicles) must be present and prepared. Demonstration tasks have the potential to assess learner capabilities with a considerable amount of flexibility, given the possibility of interactions between test takers and judges and the nuanced assessment of learner performance. Demonstrations can be capital for the validity and reception of results as results are directly connected to trained professionals in the field of occupation being assessed. Benefits notwithstanding, demonstrations are challenging to scale in large-scale settings, due to the human and material resources required (e.g. judges, vehicles, measurement devices, computer-based-expert systems) and the preparation effort, internationally comparable training protocols to be developed and statistical models to be employed. Demonstration tasks pose a particular challenge for standardisation as they require identical vehicles, measuring equipment, computerised systems, etc., which is not easy to achieve in a globally conducted assessment.
Reporting proficiency of automotive technician learners
Copy link to Reporting proficiency of automotive technician learnersThe proficiency of the automotive technician learners may be reported for the investigation and rectification capability or using a single scale. Two scales are preferable but testing time must be considered. The scales cover different levels of capability (see table 2.3.). The lower score limits will be determined with data in the second stage of the Development Phase according to the standard PISA scaling procedure.
The levels of investigation capability are described based on an evidence-based proficiency scale (Nickolaus et al., 2012[8]). The levels depend on how the information is obtained during the investigation of vehicles: At the first level, it is obtained with sensory checks and without technical aids such as measurement devices. At the second level, it is generated by measurements with equipment and measurement strategies that are retrieved rather than developed independently. At the third level, the information is obtained by measurements with equipment and the independent development of measurement strategies. The third level refers only to diagnosis, as inspection follows standardised procedures that are either given in external sources or recalled from memory.
Table 2.5. Summary description of the levels of investigation proficiency in PISA-VET
Copy link to Table 2.5. Summary description of the levels of investigation proficiency in PISA-VET
Level |
Lower score limit |
Descriptor |
---|---|---|
3 |
To be confirmed with data |
At level 3, learners can diagnose different systems and types of light vehicles in situations where they need to independently develop strategies for obtaining information and use measuring equipment. |
2 |
To be confirmed with data |
At level 2, learners can inspect and diagnose various systems and types of light vehicles in situations where information can be obtained by applying standard procedures from memory or external sources and using measurement devices. |
1 |
To be confirmed with data |
At level 1, learners can inspect and diagnose various systems and types of light vehicles in situations where information about the vehicle can be obtained by applying standard procedures from memory or external sources and using sensory testing. |
The task difficulty of rectification activities is determined by various parameters. One of these parameters is experience: Certain tasks are quite common and are conducted frequently during the training, while other tasks are rare, so that learners lack opportunities to become familiar with them and to develop routines. Experience builds on a continuum starting with very frequent service activities (e.g. oil change, change of brake pads), less frequent service activities (e.g. changing the timing belt of an engine), up to novel, problem-based activities (e.g. overhauling a gearbox). Another parameter that affects task difficulty is the scope of work: The more work steps that either must be retrieved from memory and/or from external information sources, the more complex, difficult, and error-prone the work. Task difficulty is also determined by whether individual action steps must first be organized by content knowledge, e.g. if an adequate solution to a problem must be developed by choosing from possible options, which is often the case with more complex repairs. Finally, difficulty is affected by the tools and equipment involved in the task: Using a screwdriver is easy while using a laser-based wheel alignment system is more difficult.
Against this background, we distinguish three rectification capability levels. At the first level, the learners solve routine tasks (such as frequent service tasks) with a varying amount of pre-defined sequence of steps. At the second level, the learners solve tasks of less routine character (less frequent service tasks), which often involve a larger number of work steps and require content knowledge and specialised tools to be applied successfully. At the third level, the learners solve novel problem-based tasks, resp. fewer routine tasks in a novel context, which is accompanied by a larger number of work steps using specialised tools and more sophisticated content knowledge. Despite their high degree of difficulty, the requirements of the third level are in principle covered by the training contents and can be solved.
Table 2.6. Summary description of the levels of rectification proficiency in PISA-VET
Copy link to Table 2.6. Summary description of the levels of rectification proficiency in PISA-VET
Level |
Lower score limit |
Descriptor |
---|---|---|
3 |
To be confirmed with data |
At Level 3, learners can solve problem-based tasks that are rich in complexity and require them to independently develop problem solving and information retrieval strategies and use specialised tools. Following a step-by-step repair manual is not sufficient to solve such tasks. |
2 |
To be confirmed with data |
At Level 2, learners can solve tasks of medium complexity that require them to use specialised tools as well as to apply content knowledge. It is possible to successfully perform such tasks by following a step-by-step repair manual. |
1 |
To be confirmed with data |
At Level 1, learners can solve routine tasks (such as frequent rectification tasks) that consist of a varying number of standardised steps and require the use of standard tools. It is possible to successfully perform such tasks by simply following a step-by-step repair manual. |
Examples of technology-based assessment
Copy link to Examples of technology-based assessmentTechnology-based assessment utilising computer-simulated scenarios
Copy link to Technology-based assessment utilising computer-simulated scenariosThe following technology-based scenario relates to investigation and focuses on diagnosis. The scenario was developed and evaluated as part of the ASCOT+3 initiative of the German Federal Ministry of Education and Research. It involves an authentic work order to diagnose the malfunction of a car. In this case, the scenario represents only one process, namely, “diagnose.” In the scenario, a customer complained about a constantly burning engine control lamp. The malfunction is diagnosed in a computer simulation (Meier et al., 2022[9]). The simulation is made up of authentic drawings, photos, and screenshots of the following parts of the work environment of automotive technicians: (1) a selection of four relevant car systems, (2) a toolbox with various tools, and (3) relevant segments of the computer-based expert system (see Figure 2.1 below).
The car systems are the drive unit (subsystem: engine control unit, access via button “Motorraum”) and the assistance system (subsystems: radio, air conditioning, power window, access via button “Innenraum;” subsystem: lighting unit, access via button “Beleuchtungsanlage”; subsystem: anti-lock-brake unit, access via button “Fahrwerk”). The focus is on malfunctions in the engine control, which are diagnosed in the engine compartment (top-right in Figure 2.1). Numerous components (e.g. sensors, cables, and fuses) are available for electrotechnical measurements. These components have different numbers of measuring points (e.g. different numbers of plugs or terminals on these plugs). The car can be in four different operating states (ignition off & engine off, ignition on & engine off, ignition on & engine starting, ignition on & engine running). The components and operating states allow for 3840 different measurements of voltages, resistances, and signals.
For these measurements, the test takers have access to various tools. Moreover, the simulation covers relevant segments of the computer-based expert system. It offers a great variety of information such as electrical circuit diagrams, installation plans, descriptions of components, diagnostic instructions, or reference values for electrotechnical measurements. A total of fifty-four malfunction scenarios are available in the computer simulation as realistic work orders on which initial symptoms are listed. When starting a diagnosis in the simulation, a test taker would usually first read the work order and then scan the fault memory of the engine control. Depending on the entry in the fault memory, the learners would then start looking for the cause of the malfunction. Every mouse click made in the simulation is recorded in log-files with a timestamp (Abele and von Davier, 2019[10]).
Example 1: Simulation items to measure the content and procedural knowledge of malfunction diagnosis.
Copy link to Example 1: Simulation items to measure the content and procedural knowledge of malfunction diagnosis.In the following example the test takers' content and procedural knowledge of malfunction diagnosis are measured. The example refers to an authentic scenario and a situation in which the test takers must identify the cause(s) of a malfunction in the computer simulation and justify their conclusion. To this end, they must conduct a complete diagnostic process (as described in the chapter "Diagnosis"; see p. 9). All necessary information, tools, and corresponding measuring points on the vehicle (such as connector plugs, control units, sensors etc.) are available in the computer simulation. While performing the diagnosis, every action is recorded by the program in a log file (see Figure 2.2 below). Test takers are given both constructed-response items and selected-response items. Their responses are scored according to relevant criteria.
The scenario starts with a work order that refers to a car that will not start (see Figure 2.3 below). The test takers must interact with the computer simulation to identify the cause(s) of this malfunction. They measure the resistance of the coil of the speed sensor and find that the measurement value is different from the reference value according to manufacturer’s specifications, for example. They conclude that the sensor is defective (item 1, result of the diagnosis), and justify their conclusion by pointing out the difference between measurement value and reference (item 2, justification). Moreover, they are asked to explain how the defective sensor would cause the engine not to start (item 3, explanation). Test takers’ responses are recorded via free text entry (open-ended constructed-response items) or drop-down menus (selected-response items). The identification of the defective sensor (item 1), the justification of the conclusion by giving the relevant measurement value (item 2), and the explanation how this defect causes the malfunction (item 3) are scored. In addition, relevant diagnostic actions (e.g. measurements) can be scored. The log data can be used for this purpose (e.g. by rewarding a correct measurement in the log file with a point). Given that the universe of actions that are logged by the simulation is exceptionally large yet still limited, scoring by log entries refers to closed constructed-response items. To increase reliability and still have an acceptable test time, items could be added to the tasks that do not cover a complete diagnosis but critical diagnostic requirements and require a comparatively short test time (Abele, 2014[11]). The tasks cover all diagnostic requirements.
Example 2: Simulation items to measure the procedural knowledge of tool selection.
Copy link to Example 2: Simulation items to measure the procedural knowledge of tool selection.The example below (Figure 2.4) is also from the computer simulation and measures procedural knowledge. In several stages of the diagnostic process, it is crucial to select adequate tools. This is realised with a virtual toolbox from which test takers can select a small range of measuring equipment.
For example, at a certain point, the diagnose might require checking an electric signal. To perform this measurement action correctly, test takers must pick the oscilloscope and adapter cable from the list of available tools. Which tools they select is logged by the simulation. Again, the corresponding action from the log file represents a selected-response item. The selection of the correct tool(s) is used for scoring.
Such virtual toolboxes can be integrated in a variety of digital assessment tools (video-based, simulation-based, text-and-picture based etc.).
The same item principle can also be used to measure if automotive technicians would select the correct parts for a replacement (e.g. from an electronic parts catalogue or a virtual storage shelf) or the correct materials (such as oil, lube, thermal grease, or anti-squeal paste from a virtual tool cart or trolley).
Technology-based assessment utilising scenarios
Copy link to Technology-based assessment utilising scenariosA second type of technology-based assessment involves scenarios rather than simulations. In such scenarios, an authentic work order and necessary information about the initial situation are provided as text. Furthermore, the test takers are granted access to necessary sources of information (e.g. circuit diagrams, testing instructions).
Example 3: Testing protocol to measure the content and procedural knowledge of generating and testing malfunction hypotheses.
Copy link to Example 3: Testing protocol to measure the content and procedural knowledge of generating and testing malfunction hypotheses.The following example (see Figure 2.5 below) measures the test takers’ content knowledge and procedural knowledge of diagnosis and more specifically, of generating fault hypotheses and planning how to test them. Based on the information provided in the scenario, the task of the test takers is to formulate reasonable hypotheses about the cause(s) of the described malfunction (the assumed defect) and specify how to evaluate these hypotheses. The test takers type their responses to a standardised testing protocol.
In each row of the protocol (representing one hypothesis and associated diagnostic strategy), the test takers name the appropriate testing instruments, measurement ranges, measuring points, and target values. In some cases, all information can be obtained from a manual; in other cases, test takers must develop their own diagnostic strategy.
Correct answers are determined in advance based on pilot studies and expert solutions. To fill out the cells of the testing protocol, both open-ended constructed-response items (free text entry; see Figure 2.5: measuring points) and selected-response items (single choice, e.g. via drop-down menus; see Figure 2.5: measuring device) are feasible. To score the items, the number of appropriate hypotheses and correct tests could be evaluated, for example.
Figure 2.5 shows a work order in which a customer's vehicle has engine trouble such as rough idle and reduced performance. A reasonable hypothesis about the cause of the malfunction that could be derived from the available information is an interruption of the voltage supply between the engine control unit and a fuel injector, caused by a broken cable. The test takers could further conclude that a reasonable measurement to test this hypothesis is a voltage test on the plug at the injector side of the cable. For this, they would select the multimeter as the testing instrument and identify pin one and pin two on the injector as the measurement points. They would read the target value of 12 volts from the available testing instructions. They enter all this information into the testing protocol. Afterwards, they can enter further hypotheses and corresponding tests until they cannot think of any more or until the test time is over.
To arrive at such results, the test takers must a) read and understand the description of the malfunction and the context, b) use available electrical circuit diagrams and testing instructions to obtain necessary information, c) combine a, b, and their content knowledge to derive reasonable hypotheses about the assumed defect, and d) derive and plan the necessary tests for the hypotheses using available information. The hypotheses about the cause of the malfunction and relevant measurements must be stated in the testing protocol. To score the items, the number of appropriate hypotheses and correct tests could be evaluated, for example.
Technology-based assessment utilising video vignettes
Copy link to Technology-based assessment utilising video vignettesAnother potential format to assess automotive technicians’ work-related capabilities are video-based assessments (Gschwendtner et al., 2017[12]). Hartmann, Güzel, and Gschwendtner (2023[13]) measured rectification knowledge with an instrument that uses scripted video vignettes. The instrument was developed and evaluated as part of the ASCOT+ initiative of the German Federal Ministry of Education and Research. The instrument consists of a series of short video clips showing a technician (played by an actor) who performs rectification tasks on a car. The videos and related test items cover the entire replacement process, starting with the presentation of a work order and ending with the final checks after the work is completed.
Some of the actions in these clips are correct whereas in others the actor intentionally makes mistakes which are known to be common for automotive technician learners. Each video clip is followed by one or more selected-response items which refer to the actions shown in the videos. The items are presented on the computer screen.
The distractors are based on empirical studies of learners' pre-concepts regarding the action addressed by the items. It is assumed that learners can only differentiate between correct and incorrect rectification actions in the videos if they know how these actions are adequately performed. Therefore, their responses can be interpreted as an indicator of their rectification-related knowledge. Evidence from validation studies suggests that even though there are limitations, this format is capable of measuring rectification-related capabilities. The instrument is currently used in the practical exams of automotive technician learners in the Stuttgart Region (Germany) (Hartmann and Gschwendtner, 2022[14]). A comparison of exam scores indicates that the video-based assessment generates results like those of ‘classical’ hands-on exams.
Example 4: Video-based item to measure the content and procedural knowledge of replacing the timing belt.
Copy link to Example 4: Video-based item to measure the content and procedural knowledge of replacing the timing belt.The item example below (Figure 2.6) is from the video-based instrument and measures content and procedural knowledge. The instrument utilises a series of video clips in which an automotive technician replaces the timing belt of an engine. The task in the shown item is the replacement of the timing belt itself. It consists of a series of actions. Among those actions, the technician removes and later reattaches a special tool that holds the camshaft wheels in position, which is the correct procedure for this replacement task. He should also replace a tension pulley but skips the replacement. After the video has stopped, the test takers are presented with two statements (Option A and Option B) that refer to these two aspects of the replacement task. If they judge the statements correctly, they must conclude that Option A is incorrect, and Option B is correct. Therefore, the correct response is answer 2: “Only option B is correct”.
The selected-response item is complemented with several documents including a repair manual. It is expected that to solve the item correctly, technicians must either be familiar with the task or be able to extract the essential information from the repair manual.
Example 5: Text- and image-based item to measure the content knowledge about the structure and function of the timing belt
Copy link to Example 5: Text- and image-based item to measure the content knowledge about the structure and function of the timing beltThe item example below (Figure 2.7) is from the same video-based instrument and measures content knowledge. Again, the item is part of a series of items that represent the complete task of replacing a timing belt.
To measure knowledge of the structure and function of the engine timing subsystem, test takers are presented with a damage of a pulley. They are then asked what consequences this defect would have on the related (sub)system. To solve the selected-response item, test takers must know how the damaged component is integrated in the subsystem and in which ways it contributes to the subsystem’s function. The pulley in question regulates the tension of the timing belt. It is not connected to the drive belt. Therefore, Option A is incorrect, and Option B is correct. The correct response would be answer 2 (“Only option B is correct”).
In order to make the test format even more authentic with even more varied realistic decision processes, it would be conceivable to rework the instrument in such a way that it allows greater freedom with regard to, for example, the selection and setting of tools (e.g. supplemented by menu windows which - similar to a toolbox - allow a selection of the most varied tools) and the like.
References
[4] Abele, S. (2018), “Diagnostic Problem-Solving Process in Professional Contexts: Theory and Empirical Investigation in the Context of Car Mechatronics Using Computer-Generated Log-Files”, Vocations and Learning, Vol. 11/1, https://doi.org/10.1007/s12186-017-9183-x.
[11] Abele, S. (2014), Modellierung und Entwicklung berufsfachlicher Kompetenz in der gewerblich-technischen Ausbildung, https://doi.org/10.25162/9783515107556.
[10] Abele, S. and M. von Davier (2019), “CDMs in Vocational Education: Assessment and Usage of Diagnostic Problem-Solving Strategies in Car Mechatronics”, in Methodology of Educational Measurement and Assessment, https://doi.org/10.1007/978-3-030-05584-4_22.
[1] Baethge, M. and L. Arends (2009), “Feasibility study VET-LSA: A comparative analysis of occupational profiles and VET programmes in 8 European countries - international report”.
[7] De Jong, T. and M. Ferguson-Hessler (1996), “Types and qualities of knowledge”, Educational Psychologist, Vol. 31/2, https://doi.org/10.1207/s15326985ep3102_2.
[3] ESCO (2020), European classification of Skills/Competences, Qualifications and Occupations, https://esco.ec.europa.eu (accessed on 16 December 2022).
[15] Gschwendtner, T., S. Abele and R. Nickolaus (2009), “Computersimulierte Arbeitsproben: Eine Validierungsstudie am Beispiel der Fehlerdiagnoseleistungen von Kfz-Mechatronikern”, Zeitschrift für Berufs- und Wirtschaftspädagogik, Vol. 105/4, https://doi.org/10.25162/zbw-2009-0038.
[12] Gschwendtner, T. et al. (2017), “Multidimensional Competency Assessments and Structures in VET”, in Methodology of Educational Measurement and Assessment, https://doi.org/10.1007/978-3-319-50030-0_12.
[14] Hartmann, S. and T. Gschwendtner (2022), “Premiere für digitale Prüfungsstationen in der praktischen Kfz-Gesellenprüfung”.
[13] Hartmann, S., E. Güzel and T. Gschwendtner (2023), “Digital measurement of hands-on performance? Ecological validation of a computer-based assessment of automotive repair skills”, Empirical Research in Vocational Education and Training, Vol. 1.
[6] Jonassen, D. (2000), “Toward a design theory of problem solving”, Educational Technology Research and Development, Vol. 48/4, https://doi.org/10.1007/BF02300500.
[5] Klahr, D. and K. Dunbar (1988), “Dual space search during scientific reasoning”, Cognitive Science, Vol. 12/1, https://doi.org/10.1016/0364-0213(88)90007-9.
[9] Meier, J. et al. (2022), “Promoting car mechatronics apprentices’ diagnostic strategy with modeling examples: Development and evaluation of a simulation-based learning environment”, Studies in Educational Evaluation, Vol. 72, https://doi.org/10.1016/j.stueduc.2021.101117.
[8] Nickolaus, R. et al. (2012), “Fachspezifische Problemlösefähigkeit in gewerblich- technischen Ausbildungsberufen – Modellierung, erreichte Niveaus und relevante Einflussfaktoren”, Zeitschrift für Berufs- und Wirtschaftspädagogik, Vol. 108/2, https://doi.org/10.25162/zbw-2012-0016.
[2] Ritchie, H. (2020), “Cars, planes, trains: where do CO2 emissions from transport come from?”, Biofuels, Bioproducts and Biorefining, Vol. 9/5.
Further reading
Copy link to Further readingAbele, S., Walker, F. and Nickolaus, R. (2014), Zeitökonomische und reliable Diagnostik beruflicher Problemlösekompetenzen bei Auszubildenden zum Kfz-Mechatroniker [Time-Saving and Reliable Diagnostics in Measuring Professional Problem-Solving Competence in the Domain of Car Mechatronics], Zeitschrift für Pädagogische Psychologie, Vol. 28/4, pp. 167-179, https://doi.org/10.1024/1010-0652/a000138.
Annex 2.A. Systems, subsystems, components, and parts
Copy link to Annex 2.A. Systems, subsystems, components, and partsEach content category for the automotive technician occupational area is described by specifying the topics, i.e. the basic functions of the respective system and naming subsystems, components, and parts. Given the complexity of vehicle systems, it is not the intention to describe the systems exhaustively, but rather to use typical and representative examples to define them.
Drive unit
Copy link to Drive unitThe central functions of the drive unit are to convert other forms of energy into kinetic energy, to provide the energy to the other systems of the motor vehicle and to store the energy (e.g. fuel tanks or batteries). In the case of the charging subsystem, the direction of energy conversion is reversed. The (still) most widespread drive units are internal combustion engines. These generate motion through controlled, combustion processes of fuels (e.g. diesel, gasoline, or gas). Other engine types such as electric motor drives are becoming increasingly important. There are also mixed forms of these different engine types, so-called hybrids, which are being used more frequently.
It is composed of the following subsystems, components, and parts:
Subsystems: air subsystem (air intake and exhaust, e.g. exhaust gas recirculation systems, exhaust gas aftertreatment systems), fuel injection subsystem (e.g. pumps, fuel metering systems, fuel tank), ignition subsystem (e.g. spark plugs, rotating high voltage distribution, ignition coils), engine management subsystem (e.g. engine control unit, emission control, timing belt), starting and charging subsystem (e.g. starter, generator, battery), etc.
Components: engine control unit, sensors (e.g. temperature sensors, speed sensors, mass airflow sensor), actuators (e.g. starter motor, fuel pump, injection valves), transmission components (e.g. cable harnesses, battery), etc.
Parts: High-pressure rail, coils, intake and exhaust valves, pistons, camshaft, crankshaft, permanent magnets, etc.
Powertrain
Copy link to PowertrainA central function of the powertrain is to transmit the engine power (rotation and torque) of the drive unit to the vehicle's drive wheels with as little energy loss, friction, vibration, and noise as possible. In principle, engine power can be transmitted to one (front-wheel or rear-wheel drive) or to all vehicle axles (all-wheel drive). This is the transmission function of the powertrain. At the same time, the powertrain also has a transformation function, i.e. speeds and torques of the driveline can be – if necessary and technically possible – specifically transformed into different peripheral speed and torque at the vehicle wheels. Associated with the transformation function is the compensatory function in that different rotational speed and torque at the vehicle´s axles and/or individual drive wheels can be permitted (by means of a differential gear) or purposefully changed (by means of a transfer gearbox, or torque vectoring through a clutch system).
It is composed of the following subsystems, components, and parts:
Subsystems: powertrain management systems, non-shiftable and shiftable mechanical, automated and automatic transmissions (including planetary gear), non-shiftable and shiftable clutch systems (e.g. cam clutches, multiplate clutches, viscous clutches, hydrodynamic torque converters), shaft systems, etc.
Components: powertrain control unit (which can be either a separate unit or integrated in the engine control unit), sensors (e.g. RPM or temperature sensors), actuators (e.g. electro-hydraulic actuators, gear oil pump), complex dynamic seals (e.g. simmering), rolling bearings (e.g. needle bearing), synchronisation device, flange couplings, multiple-disc clutches, park locks, drive/axle shafts, housings, etc.
Parts: shafts, plain bearings, static gaskets, gearwheels, shift sleeves, synchronous body, synchronizer ring, springs, etc.
Undercarriage
Copy link to UndercarriageThe central function of the undercarriage is to provide the vehicle with a load-bearing framework that allows relative motion between the vehicle and the underground, a controlled change of direction, and a reduction of speed up to a full stop in a controlled manner.
Modern undercarriage systems are combined of mechanical, hydraulic, and electric/electronic parts and components:
Subsystems: suspension, steering, brake system, electronic stability control (ESC), etc.
Components: wheel, MacPherson strut, brake calliper, angular rate sensors (ARS), steering mechanism, tie rod, etc.
Parts: tyre, spring, stem (of a shock absorber), brake piston, tone wheel, ball and socket (of a ball joint), etc.
Assistance
Copy link to AssistanceThe central functions of the assistance system are to support the passengers’ well-being and to facilitate the vehicle operation. It protects against hazards (e.g. accident risk and consequences) and enables a pleasant use of the vehicle. They are designed to intervene (semi-)autonomously in other systems and/or warn the driver shortly before and/or in critical situations through suitable instruments. The functions of the assistance system are provided by the following subsystems, components, and parts:
Subsystems: lighting systems, airbag (e.g. driver airbag, head airbag, pedestrian airbag), driver assistance systems (e.g. park assistance system, brake assistant system, driving dynamic systems), ventilation and air conditioning (HVAC; e.g. parking heater, circulation air system), multi-media systems (e.g. radio, head-up display, headrest monitors), interior comfort systems (e.g. electrically adjustable steering wheel, massage seats, noise insulation), data bus systems (e.g. CAN, LIN, FLEXRAY, MOST), etc.
Components: sensors (e.g. temperature sensors, air quality sensors), actuator (e.g. servomotors, electrical valves), control units (e.g. air conditioner control unit, anti-lock-brake control unit), transmission components (e.g. cable harnesses, plug connectors), interior climate compressor, seatbelts, airbags, etc.
Parts: cabin filter, coils, rotors, capacitor plates, wires, cable sheathing, permanent magnets, speaker, etc.
Notes
Copy link to Notes← 1. Besides such global standards, there are also local standards. Compared to the global standards, the local standards are much less relevant and must not be included in the assessment for reasons of comparability.
← 2. This process is often referred to as “repair” in common speech. However, vehicles are often repaired by simply replacing a broken part; the broken part itself is not repaired. Other actions that in common speech fall under the term “repair” also often do not meet the definition of a repair in a strict sense, e.g. when undamaged parts are replaced at customer's request or in order to maintain functionality before a part has worn beyond the limits specified by the manufacturer. Therefore, we named the process “rectify”, which by definition involves different kinds of corrective actions such as repairs, replacements, adjustments, etc.
← 3. ASCOT stands for “Technology-based Assessment of Skills and Competences in VET” and the "+" for the transfer of results from the predecessor initiative ASCOT (2011-2015) into training and examination practice.