Quentin Vidal
OECD
OECD Digital Education Outlook 2023
Towards an Effective Digital Education Ecosystem
OECD Digital Education Outlook
Abstract
This chapter provides an overview of the digitalisation of national and central assessments across OECD countries and jurisdictions. It explores the opportunities that new technology and smart data offer to transform assessment practices and compares it with what countries and jurisdictions effectively leverage while digitalising their assessment. Balancing those tensions, the chapter offers insights as to why most system-wide student evaluations are now digitised while high-stakes examinations remain predominantly paper-based. Finally, it makes suggestions that policy makers may consider to move towards an effective digital assessment ecosystem.
In the ever-evolving landscape of education, the effective assessment of student learning outcomes, and the continuous evaluation of education systems’ capacity to provide quality education and ensure equitable opportunities for all, play a pivotal role in shaping educational policies and practices.
At the class and individual levels, two main types of student assessment coexist: formative assessments and summative assessments. Conducting formative assessments is the ongoing process of assessing and providing feedback on student learning throughout their learning journey. Formative assessment aims to monitor students’ progress to provide opportunities for improvement and guide instructional decisions to enhance learning outcomes. In countries and jurisdictions that have incorporated formative assessment practices into the classrooms, teachers typically use quizzes, assignments, projects, or class discussions to gauge student progress. Administering summative assessments, on the other hand, is measuring students’ overall learning achievements to determine the level of mastery of specific learning objectives at the end of a specific period of learning. Summative assessments can take the form of final exams or end-of-year assessments, but include all sorts of assessments, including teacher-given assessments, that are administered primarily to assess student learning. Summative assessments are the focus of this chapter.
At the level of an education system as a whole, countries and jurisdictions typically resort to summative assessments to assess key student learning outcomes and the quality and equity of education systems. At the central or jurisdictional level, assessments may take the form of student examinations and evaluations. Examinations are often administered at the end of primary or secondary education. Exams carry high stakes for students as they determine their progression to higher levels of education or provide them with a qualification before they enter the labour market. But system-wide assessment may also consist in standardised student evaluations, which are assessments designed to measure entire cohorts of students’ skills in specific key subjects, such as reading, mathematics or science. Countries and jurisdictions may administer those evaluations in a census-based approach, where all students of a given cohorts are assessed on the same subjects; or using probability sampling, where only a representative sample of students are evaluated in each subject.
Digitalisation presents both opportunities and challenges for the administration of exams and evaluation. Digitised (or computer-based) assessments offer adaptivity, enabling the tailoring of assessment questions’ difficulty based on responses for more accurate skill measurement (see Box 4.1). They may assess new or traditional skills through varied means. Immediate feedback empowers students to recognise content areas that they master from those where they struggle, enhancing and paving the way for more granular, personalised learning. Learning analytics, generated from the analysis of log data, offer insights that extend beyond the final outcomes of the assessment, allowing to understand how students reflect on their assessment and not only whether they get the right answer (OECD, 2021[1]). Digital assessments also provide flexibility and accessibility, accommodating diverse learning situations and notably helped ensure assessment continuity during the COVID-19 crisis and worldwide school closures (OECD, 2020[2]). Finally, in some cases they can prove to be cost-efficient, with reduced expenses due to automation (of parts of the grading process for instance), replication, and lowered printing and distribution costs – although experience suggests that building valid, reliable and fair digital assessments is not necessarily cost-effective to measure simple constructs (Buckley et al., 2021[3]).
However, transitioning to digital assessments presents challenges for countries and jurisdictions. Reliable technology infrastructure, data privacy, and security measures, along with disparities in access in face of digital divides, create obstacles (see (Fragoso, 2023[4])). Furthermore, designing valid and reliable digital assessment items aligned with educational goals is a complex process, let alone when featured in high-stakes examinations – which further demands innovative solutions to address concerns over cheating. Transitioning to digital assessments thus requires a solid digital infrastructure, inclusive of the humans who comprise it. Teacher training and professional development are essential for good design and effective implementation (see (Foster, 2023[5])).
Digital assessments have advantages and disadvantages. Some of the most obvious advantages include the possibility of rapidly marking them and providing feedback, reducing human biases and errors, and the fact that they, at least partially, address teachers’ workload (Leaton Gray and Kucirkova, 2021[6]). Other more technical advantages include the possibility of using adaptive testing or using sophisticated trackers of academic progress across an extended period of time (Veldkamp and Sluijter, 2019[7]). On the other hand, there are challenges to consider, such as how data should be collected, used and stored (Timmis et al., 2015[8]). Ethical questions may also emerge in relation to inequalities, as it cannot be assumed that all teachers and students have the necessary skills to use technological devices/software. This may impact on students’ assessment results (Wyatt-Smith, Lingard and Heck, 2021[9]). Other challenges arise with specific technology-enhanced assessments. For example, assessments marked by algorithms may be less transparent and hard to verify, particularly with machine learning (e.g. essay marking) (Veldkamp and Sluijter, 2019[7]).
In its review of the challenges of computer-based testing in England (United Kingdom), Ofqual studied its introduction in Finland, Israel, and New Zealand (Ofqual, 2020[10]). The authors found that successful implementation required a strong “risk appetite” – recognition and acceptance by stakeholders that some things would go wrong and an understanding there would be some “learning by doing”. Each of the countries and jurisdictions took a different approach to implementation. For example, New Zealand took a gradual, voluntary approach. This resulted in a longer timeline for introduction but enabled public perception to change as the adoption grew to ensure positive user perception prior to use. Technical glitches were easily overcome by the ability to revert to paper-based testing. The downside of this approach was the dual running of paper and computer-based assessments, which risks to give an unfair advantage or disadvantage to groups of students. Biases related to the method of assessment delivery could ultimately hinder performance comparisons (McClelland and Cuevas, 2020[11]) ; (Kolen and Brennan, 2014[12]); (Sandene et al., 2005[13]). Another example is provided by Australia, which underwent a broad research and development programme before and during the transition of its NAPLAN to a computer-based delivery.
Adaptive testing is efficient, as students do not waste time and effort on questions that are too hard or too easy for them. As candidates only answer items that are paired to their ability, the assessment length can be reduced in comparison to linear assessment forms, and the assessment administration can, if desired, be more flexible in terms of time as a result of individualised testing (Veldkamp and Sluijter, 2019[7]). At the same time, some international evidence shows that test-takers report feeling discouraged after taking a computer-based adaptive test, and this feeling can reduce learning self-efficacy and motivation (Kimura, 2017[14]). In addition, adaptive assessments require a large bank of test questions, which increases development costs (OECD, 2013[15]). Recent breakthrough in generative text AI may alleviate some of these costs in the future. Furthermore, it may be challenging to explain to students and their parents why test experiences differ between students, as well as how the test results should be interpreted (Somers, 2021[16]).
Source: Extracted from (OECD, 2023[17]), “Strengthening the design and implementation of the standardised student assessment reform of the Flemish Community of Belgium”, OECD Education Policy Perspectives, No. 72, OECD Publishing, Paris, https://doi.org/10.1787/85250f4c-en.
This chapter provides an overview of the increasing use of digital tools to administer various types of assessments. The first section looks at the digitalisation of standardised student evaluations, examining the motivations that drive countries and jurisdictions in the transition from paper-based to computer-based administration while addressing its challenges. The next section focuses on digital (high stakes) student examinations, suggesting why they are less often taken digitally than student evaluations. Finally, the chapter explores the landscape of digital support systems that countries and jurisdictions have developed to support the administration, grading and processing of various assessments, whether digital or not. The chapter concludes with a reflection on the trends in the digitalisation of assessments, taking stock of the (so far limited) transformation that digitalisation has brought about, and envisioning what countries may do next to reap the benefits of the innovative affordances allowed by digital assessments.
Our comparative analysis covers 29 countries and jurisdictions. All OECD countries and jurisdictions were asked to answer a survey on digital education and infrastructure. The answers to the questionnaires were validated through meetings with the responding team and expanded by desk research to describe countries’ digital education infrastructure and governance (see the related publication (OECD, 2023[18])).
Most of the countries and jurisdictions that have taken part in our data collection have developed digital tools to assess students and/or to support their assessment (Figure 4.1). Out of 29 countries and jurisdictions, 18 (or about two-third) administer system-level standardised student evaluations that are fully or partially taken digitally, among which two are federal countries where all or most of their states or regional authorities conduct student evaluations at their level. Seven have fully or partly digitised some of their high-stakes examinations; and 16 have developed digital tools to support the administration of assessments, digital or not.
Note: N=29.
Digitalisation of standardised student evaluation
Standardised evaluations of education systems are typically the first type of assessment that countries and jurisdictions have digitised, and the most common type of (publicly administered) digitised assessment as of 2024. Student evaluations are usually relatively short and generally made of closed-answer questions to facilitate their systemic completion and grading at the jurisdictional level. Moreover, they are cyclical and fully or partially repeated over years to assess trends in a short range of key student achievements: literacy (in the national language(s)), numeracy, and, in some cases (e.g., Italy, Norway or Sweden), in other subjects such as English as a foreign language. Finally, standardised evaluations carry low (or no) stakes for individual students (although they may have one for schools and their staff): results are only analysed at the larger unit of the class, the school, or the jurisdiction as a whole. For those reasons, standardised evaluations appear easier to digitise than traditional exams.
Country coverage and main characteristics
Over the last two decades, more and more education systems have deployed their own standardised student evaluation. Across the OECD, most countries, including countries where the education governance is largely decentralised to regional (e.g., federal countries) or local authorities (e.g., Nordic countries), administer such standardised student evaluation nationally.
Data from OECD Education at a Glance 2023 (EAG) show that national/central evaluation are more prevalent at primary and lower secondary levels than at upper secondary level (OECD, 2023[19]). About four-fifths of the 39 countries and jurisdictions participating in EAG with data available conduct at least one national/central evaluation on students at primary level (33 countries and jurisdictions). The share is similar for lower secondary level (32 countries and jurisdictions). In upper secondary education, less than two-fifths (14 countries and jurisdictions) conduct such evaluations. National/central evaluations take place in different grades throughout primary and secondary levels. They are commonly conducted six and nine years after the start of primary education (in 19 countries and other participants for each grade). Before 2023, information on the national/central evaluations put in place by countries were last collected by EAG in 2015. Both rounds of the survey collected data on the use of digital technology in national/central evaluations in three subject areas: reading, writing and literature; mathematics; and natural sciences. Between 2015 and 2023, the numbers of countries and jurisdictions using digital technology in their national/central evaluations in at least one level of education almost tripled, from 8 to 21. The shift to digital testing was notable particularly in the last five years (e.g. Canada and France since 2019, and Hungary and Korea since 2022). Several systems are planning to also have digitised evaluations in the coming years (e.g. the French Community of Belgium, Germany).
Unless otherwise stated, the rest of this chapter will focus on the 29 countries and jurisdictions that participated in the specific data collection conducted for the Digital Education Outlook 2023.
Note: N=29.
Out of 29 education systems for which we have comparative information, 24 have national/central student evaluations. Among them, 18 are partly or fully digitised (about two-thirds of all surveyed countries and jurisdictions, and three-quarters of those with standardised evaluations), as shown on Figure 4.2. The left-hand part of Table 4.1 below synthetises information on national and central student evaluations.
Notes: ✓ indicates that some parts of the assessments (evaluations or examinations) are digitised. ✓+ indicates that some parts of the digital assessment (not necessarily the main one though) integrate innovative feature, such as adaptive testing. All other digital assessments use uniform testing. ✗ indicates that a national/central assessment exists but that it is not digitised.
A grey cell indicates that no national/central assessment exists.
A star (*) indicates that the assessment is administered at a lower level of government (e.g. province, state, region).
1: In Denmark, only the non-compulsory part of the Folkeskolen's National Skills Test still integrates adaptive features.
2: In England (United Kingdom), the main standardised student evaluations are the End of Key Stages 1 & 2 national curriculum assessments, and they are not digitised.
For more details on OECD countries and jurisdictions’ assessments, see OECD Education at a Glance 2023 (https://doi.org/10.1787/dfccf5a6-en).
Out of the 29 countries and jurisdictions for which we have comparative information, two federal countries have digitised the standardised evaluations that they administer or coordinate from the national level. In Canada, all provinces conduct the Pan-Canadian Assessment Programme (PCAP) that assesses students in grade 8 (secondary education) in reading, mathematics, and science. Across all 10 provinces, in partnership with the Council of Ministers of Education, Canada (CMEC), samples of students are drawn to participate in this evaluation so that their results are representative of their provinces. In addition to PCAP, most provinces and territories also conduct their own evaluation on a census basis, in some cases through the use of digital means. In the United States, the National Assessment of Educational Progress (NAEP) is also completely computer-based: while all subjects provide a range of selected-response questions, the digital medium is also used for items that would be difficult to implement non-digitally, such as interactive scenario-based tasks (in the 2018 technology and engineering literacy assessment). Most (if not all) US states and districts carry out digital standardised evaluations. A third federal country, Mexico, used to conduct standardised student evaluations at the federal level, whose results were then stored and transferred for reporting in their Sistema Integral de Resultados de las Evaluaciones (SIRE), a 2016 system connected to SIGE, the federal student information system (see (Vincent-Lancrin and González-Sancho, 2023[20])). However, the standardised evaluations have been discontinued.
Nordic countries have also long embraced the digitisation of their standardised student evaluation. In Finland, the Finnish Education Evaluation Centre (FINEEC) evaluates randomly selected samples of students in years 3 and 9 (primary and secondary education) to monitor and evaluate the education system. FINEEC started experimenting with digital parts of the national evaluations in 2014, and by 2017 all were fully digitised and taken on Koda, FINEEC’s own digital evaluation platform. As per its 2020-2023 National Plan for Education Evaluation, Finland envisions to develop new evaluation systems and methods to diversify tasks and allow for automated scoring. In Sweden, the National Agency for Education also undertook the digitalisation of the national student evaluation for students in grades 6 and 9 (primary and secondary education). Since 2018, the answers to essay questions for the Swedish and English language assessments are taken on computers. By 2026, the objective is to have most national assessments entirely digitised and taken from a single digital environment that will allow for direct student identification, innovative assessment design, automated grading, and provide visualisation tools for the results.
A few countries and jurisdictions actively encourage the development of innovative assessment design. In the United States, the federal department of education encourages the implementation of innovative assessment by encouraging state education agencies (SEAs) to apply for the Innovative Assessment Demonstration Authority (IADA) programme and improve state-wide assessments. Since 2018, Louisiana, New Hampshire, North Carolina, and Georgia have joined the IADA programme, which has provided them with the authority to establish and operate an innovative assessment system in their public schools.1
Yet, uniform assessments, where all students complete a fixed set of assessment items, are still far more common than adaptive assessments, where the difficulty of the tasks is adapted to students’ abilities. Out of 39 countries/jurisdictions, computer-based uniform testing is used in 18, whereas computer-based adaptive testing is used only in Australia (National Assessment Program – Literacy and Numeracy), Denmark (only for non-compulsory assessment in natural sciences as well as in English and in Danish as a second language), France (in the upper secondary entry test) and Scotland (United Kingdom) (for National Standardised Assessments) (OECD, 2023[21]).
While one of the advantages of digitalising assessment is the possibility to implement innovative assessment features, most countries and jurisdictions do not take advantage of these digital affordances yet. On the contrary, some countries explored them before reverting to more traditional (yet digital) forms of assessment. In Denmark, since 2022 and the implementation of the new Folkeskolen’s National Skills Test in reading and mathematics, the compulsory part of the national student evaluations is no longer adaptive. Following complaints from education researchers, politicians on both sides of the political spectrum and parts of the public opinion, they decided to return to linear assessments where all students take the same questions in the same order. Students reported that they faced higher stress in answering the first questions of the adaptive assessments, as teachers had warned them that those were the questions that would matter the most for the rest of the assessment and their final outcome. Moreover, teachers faced challenges in explaining why students may experience different assessments, and how parents should interpret their results. The former, adaptive national evaluation will remain available on a voluntary basis in certain disciplines (biology, geography, chemistry, physics, English and Danish as a second language) until the 2025/26 school year.
As of 2024, most countries and jurisdictions that have paper-based standardised student evaluation plan to digitise them to a certain degree. For instance, Japan aims to digitally administer (parts of) its National Assessment of Academic Ability (Zenkoku Gakuryoku Gakushuu Joukyou Chousa) by 2025. This low-stake annual national standardised evaluation of Japanese language and mathematics is taken by all students in the final years of primary and lower secondary education. Similarly, Brazil plans to transition to computer-based administration for its Basic Education Assessment System (SAEB) by 2024, starting with students from the last year of primary education to year 3 of upper secondary education (OECD, 2021[22]).
Finally, countries and jurisdictions with no national or central standardised student evaluation planning to develop one typically consider digital administration from the beginning. For instance, by 2024 the Flemish Community of Belgium aims to develop and implement a digital standardised evaluation to support the monitoring of learning outcomes, generate data for research, and help improve the quality and accountability of the education system. It will initially focus on the Dutch language and mathematics. The computer-based administration of the evaluation will allow one to randomly select three out of the ten competences assigned to a school for testing (for example, “number theory”, “geometry” and “statistics” in the case of mathematics). Each student in that school will be assessed in two randomly assigned competences out of the three (for example, “number theory” and “statistics”). As a result, the digitalisation of evaluation will allow to cover all competences while limiting the testing burden on schools and students.2
Turning assessment results into formative information
Digitising student evaluation simplifies the transfer, storage of and access to student results at the system level. Linking those results to longitudinal and unique student identifiers provides policymakers, administrators, school leaders and teachers with granular information on students’ results throughout their learning pathways. This is only possible for assessments based on comprehensive cohorts of students. When based on a probability sampling, the assessments still provide a national or jurisdictional benchmark to schools and teachers (assuming they can test their students with comparable tests).
Student information systems (SIS) sometimes integrate, or allow easy linkage to, student standardised assessment results (see (Vincent-Lancrin and González-Sancho, 2023[20])).
Out of 29 education systems for which we have comparative information, 15 can link student unique identifiers to the results of their jurisdictional standardised assessment (see Table 4.1).
Some student information systems are inherently built on the management of standardised evaluation results. In England (United Kingdom), student results to national standardised evaluations are transferred to Analyse School Performance, a student information system co-developed (though no longer linked) with the Office for Standards in Education (Ofsted).3 It helps education stakeholders review and compare national and local authority performances of students, and thereby plan school improvement. Analyse School Performance is primarily intended to help schools identify strengths and areas of performance (based on their students’ results). Other data available in Analyse School Performance come from a range of sources, including census, exam and performance data collected by the English department for education, or from other sources of data that are sent to the department for education. Analyse School Performance uses tables, graphs, and charts to show the attainment and progress of schools, broken down by different contextual factors and student characteristics (ethnic group, poverty status, etc.). Schools can in turn select relevant data from the platform to personalise their own report. The type of information end-users can access on this system varies depending on their status (for instance, administrators and trustees will only be entitled to view non-student-specific information).
In the United States, the results to standardised student evaluation are typically included as a data element in states’ (or jurisdictions’) longitudinal data systems. By contrast, European countries often keep information about students’ results separate from their student information systems, making them only possible to use for research. This may relate to privacy or institutional reasons: in Italy, a specific public agency (Invalsi) is responsible for the national assessment, and it is separate from the ministry of education that administers the national information system; in Luxembourg, the University of Luxembourg (LUCET) is responsible for the administration of the national standardised assessment, and it is separate from the ministry of education (or higher education). In countries and jurisdictions where the information is not directly accessible within the student information system (e.g. Hungary, Italy, Luxembourg or New Zealand), standardised learning outcomes are typically connected to students’ unique identifier so that this information can be retrieved through data linkages. Results can then be made available to teachers, schools, and parents as well as to researchers – while being adequately aggregated to preserve anonymity.
Hungary is a good example of a country where student evaluation is an integral part of the student information system used at the national level – although results are no longer directly available in the system. Hungary started collecting information from the National Assessment of Basic Competencies in 2001, but it was only after introducing individual student identifiers in 2007 that the system allowed longitudinal tracking across grades. Since 2010, Hungary has aimed to turn these data into actionable information to school practitioners. Individual reports are available for each student (and the school aggregated reports are open to the public). School principals can connect to the database through the FIT analysis software (FIT elemző szoftver) to make customised comparative reports about their school and students and make comparisons with similar schools. Moreover, the Education Authority, an agency of the Ministry of Interior, provides an expert consultant service through its regional pedagogical centres to help schools and teachers understand and utilise the results. In 2022, literacy, mathematics, natural sciences, and languages were assessed and expanding the scope of the information is in progress (history and digital culture). The covered grades were also broadened to students from 4th to 11th grade. This expansion generates results under a new data structure, which will require updating the FIT analysis software. The reporting of results is also currently under revision.
Finally, in some countries and jurisdictions standardised assessment results are accessible through the student information systems used by schools – referred to as learning management systems in this report. In Denmark, upper secondary education students take digital assessments on Netprøver, a digital platform that automatically checks students’ submissions for plagiarism, assigns them grading by appointed examiners, and transfers students’ grades to their schools’ learning management system. In New Zealand, tabular reports of students’ scores to the Assessment Tool for Teaching and Learning (e-asTTle) assessment can be easily integrated with schools’ learning management systems, thus combining information from tailored assessment with school-level administrative and operations data. A feature of e-asTTle, having been initially designed as a formative assessment, is the ability to transform assessment results into prompt and interpretative feedback for teachers and school leaders through a range of graphical reports. These show student progress and areas of weakness and strength, as well as comparing performance at the individual, class or school levels to curriculum requirements, national averages, or normed gender, ethnicity, language, or socio-economic groups. An example is the Individual Learning Pathway report, which gives information on a student’s strengths and gaps. Another application links assessment results to the What Next website, an indexed library of resources to help teachers and learners identify appropriately targeted learning materials. Providing this individualised feedback in real time would be impossible without a digital tool.
Digitalisation of student examinations
In the context of this report, student examinations refer to summative assessments that students typically take at critical points in their learning course, such as at the end of a study year or a study cycle. Those exams carry high stakes for students as they can impact their academic trajectory, access to higher education, career opportunities, or eligibility for scholarships or certifications. Moreover, they are usually taken at the same time by all students. For these reasons, student examinations tend to be administered in a more controlled setting than system evaluations. They require stricter security measures to prevent cheating and maintain equality of exam-taking conditions across examinees. They may also cover a broader range of content and skills as they aim to assess students’ comprehensive knowledge and skills in various subject areas. Compared to national evaluations, digitising exams appears as more politically and technically complex.
The digitalisation of high-stakes student examinations also offers potential benefits. Digitising exams enables innovative assessment design, which may ultimately enable the assessment of new types of skills, including digital literacy. As for digital national evaluations, digitised exams may lead to faster results and could perhaps even be an end-of-cycle national evaluations if governments can ensure the fairness, validity, and reliability of the assessment process.
Country coverage
Out of 29 education systems for which we have comparative information, 7 (24%) countries (Denmark, Estonia, Finland, Italy, Latvia, Lithuania, and New Zealand) have partly digitised some of their high-stakes student examinations. In five of them, high school graduation exams are concerned (see Table 4.1, right-hand columns). In other countries and jurisdictions, only subjects that would typically be taken on a computer, such as coding, may be digitised.
In Denmark, students take their end-of-cycle exams on the Testogprøver in primary and lower secondary education and Netprøver in upper secondary education. In Finland, the high school graduation exam is entirely digitalised (Box 4.2). In Estonia, secondary education students take their exams in presence on an online platform, the Eksamite Infosüsteem (EIS). The system also allows schools to register students for exams, to create exam schedules, and to manage other aspects of the examination process: it provides tools for generating exam papers (for paper and pencil exams), for conducting exams digitally, and for processing exam results. In Italy, some parts of the high-stakes end-of-year examinations for upper secondary and VET education are conducted online. To facilitate the management of the examinations process, the ministry has developed the Commissione Web application for secondary education and the Gestione Esami ITS application for state examinations in VET.
The fruit of a public-private collaboration
In 2012, Finland decided to digitise the matriculation exams taken by students at the end of secondary education to enter higher education. The ministry publicly procured the services of a private firm specialised in digital products to build a platform operating on the free and open-access Linux operating system. After four years of research and development, in 2016 the first digital exams were held in subjects with a smaller number of candidates: geography, philosophy, and German language. By 2019, all matriculation exams were digitised.
Candidates can take the matriculation exams on their own laptop or borrow one from their schools; whichever they opt for, all students have to access the Linux operating system from a thumb drive distributed by the matriculation board at the start of the exams. In this restrained, self-contained environment, candidates cannot access their local files or the Internet but only the pre-installed software and materials. There is no cloud-based architecture to minimise the risk of technical difficulties: both the assessments and students’ answers are saved on offline local severs.
Support from the ministry: a digital tool, capacity building, and a training programme
To support schools in this transition, the Matriculation Examination Board has developed Abitti, a digital system for exam administration. Abitti provides instructions and guidance on the administration of the digital exams and allows teachers and students to get accustomed to the digital environment before they take the exams. The exam digitalisation was accompanied by an extensive training programme, provided by the ministry in a waterfall approach: across the country, 50 teachers were trained on the platform during a series of workshops so that in turn they could champion this transition and disseminate their knowledge among their peers.
Note: Digital Matriculation Exams: https://www.ylioppilastutkinto.fi/en/matriculation-examination/digital-matriculation-examination
Other countries and jurisdictions plan to digitise their student examinations in the near future. As of 2024, Austria was piloting a digital matriculation exam (Digitale Reifeprüfung) in federal schools at the upper secondary level (including VET). The digitalisation of the matriculation exam will accompany the imminent launch of a digital credential system. In England (United Kingdom), all main exam boards were piloting digital exams for the GCSE and A-levels.
Compared to their central or national evaluations, few countries and jurisdictions had digitised their student examinations as of 2024. This is partly because not all countries organise national/jurisdictional graduation or end-of-cycle exams. But other explanations can also be put forward.
First, it may be less advantageous to digitise exams than national evaluations. End-of-course exams may cover a broader range of subjects as they assess students’ knowledge and mastery of specific learning cycles, which reduces the gains from replication over time and automation.
Second, exams carry high stakes for students. Historically, attempting to innovate such exams in one way or another has led to stronger resistance from education stakeholders and the general public. Cultural norms and societal attitudes may favour traditional pen-and-paper exams as a wide range of stakeholders, including students, parents, educators, and employers, may have reservations about the validity and reliability of digital exams. Security, equity, and technical issues are also often brought forward. Digitising exams requires that countries and jurisdictions develop a solid digital infrastructure that allows for strict security measures (against cheating, hacking, etc.), equitable access for all students everywhere, equivalent student proficiency in using computers, and proven robustness against technical glitches or device malfunction . They also require sufficient physical spaces with the electrical and network facilities suitable for numbers of students to take the assessments on devices concurrently. In addition, it often requires changing the format of the exams and to negotiate this change socially.
However, while the process tends to be slower than for mere system evaluations, it is likely that an increasing number of countries and jurisdictions will digitise their national exams given the emphasis on student “digital competencies” of most curricula. Transitioning from paper-based to digital exams demands long-term planning but also investments in a sufficient, secure hardware infrastructure, inclusive of connectivity and digital equipment in schools (or exam centres).
Digital tools to support assessment
Countries and jurisdictions have developed a host of digital systems to support the administration of their assessment, whether (low-stakes) system evaluations or (high-stakes) student examinations. Those systems typically provide a digital environment for the administration of digital assessments; but they can also assist with the administration of paper-based assessments (e.g., student registration, distribution, completion), or facilitate the grading and transfer of grades to other digital systems or registers. Finally, some support systems were developed to enable the remote proctoring of students during assessments.
Country coverage
Out of 29 countries and jurisdictions for which we have comparative information, 16 (more than half) have developed digital tools to support the administration of assessments, whether the assessments are digital or not, and whether they consist in evaluations, formative assessments, or high-stakes exams (see Table 4.2).
Note: N=29
To provide an environment for digital assessment
By definition, digital assessments are taken on a digital testing platform. Countries and jurisdictions that have entirely digitised their national evaluations (or are in the process of doing so) provide a digital environment to support their administration and their completion. In Luxembourg, for the national student evaluation (Épstan), all students from specific grades take their computer-based test on the Oasys platform, which was developed by the University of Luxembourg (LUCET) in close collaboration with the ministry. In Sweden, which aims to digitise most of its national evaluations by 2026, the national agency for education has developed Skolverkets provtjänst (“Test service”), a digital system for assessment administration that provides a single environment for student’s identification and assessments’ design, delivery, grading (automatic and manual) and results.
In Hungary, multiple digital tools support the administration, management and analysis of the standardised evaluation. Schools have to use a data collection and management system for national measurements (Országos mérések adatbegyűjtő és -kezelő rendszere) to manage and organise the assessment (e.g. ensuring students take part in the right assessment on the right day) and to anonymise student results via identifiers. With another software (Tehetsegkapu), students can assess their preliminary results, before final results are available on a dedicated website (FIT-jelentések) in the form of automated reports. Schools, at their end, have access to preliminary results on their own platform (intézményi gyorsvisszajelző). Eventually, stakeholders can use the FIT analysis software (FIT elemző szoftver) to analyse the results of the assessment.
Such support tools also exist in countries and jurisdictions that have digitised (parts of) their central examinations. This is for instance the case of Lithuania’s NECIS system, an open-source online system provided by TAO.4 NECIS is used both for administering several parts of the national student assessments, which are carried out online in mathematics and in Lithuanian, as well as for administering some parts of the final exams in upper secondary education (including VET). After a pilot phase, the objective is to fully digitise both types of assessments by 2026, so that exams and evaluations are entirely administered, taken, and graded online through digital systems such as NECIS. Finland’s Abitti system (see Box 4.2) is also used to support and monitor the administration of matriculation exams. In Italy, the ministry has developed the Commissione Web application for secondary education examinations and the Gestione Esami ITS application for state examinations in VET. In Denmark, the ministry manages the administration of the student evaluations and exams, which students take on Testogprøver and Netprøver, with the XPRS tool.
To support the design of teacher-given assessments
Alongside the digitalisation of assessments conducted at the system level, countries and jurisdictions have developed digital assessment tools for the benefits of teachers. This category ranges from public banks of assessment items to tools that are made modulable so that teachers can design and carry out their own formative or summative assessments with their students.
In the Czechia, the InspIS SET platform was primarily developed to administer the country’s digital standardised evaluation. But schools and teachers can also use the InspIS SET platform to carry out their own assessments online, using their own items, or picking from a public bank of test items – which will soon be supplemented by items released from international assessments. In Brazil, digital tools support the administration of the Basic Education Assessment System (SAEB), the paper-based national standardised assessment in literacy and mathematics. While insufficient digital infrastructure and logistic constraints has posed challenges to the digitalisation of this evaluation, the federal government provides states and schools with digital tools that facilitate its administration on paper, in particular as regards logistics and financing. Similarly to the Czech platform, teachers can leverage some of those tools for their own assessments, using banks of items (e.g. BNI) a grading essays. Some of those tools are maintained by the federal or local governments whereas others are procured from private contractors for use during the assessment.5
Some countries and jurisdictions centrally provide digital tools for teachers’ formative assessment. In Chile, the DIA is available online, on a voluntary basis, for schools willing to assess their students in core subjects as well as their socio-emotional skills. DIA is an example of a digital formative assessment, designed to be delivered by teachers at three distinct moments of the school year: a diagnostic assessment to begin with, a follow-up halfway, and a closing assessment at the end. Albeit available for all grades, DIA’s uptake is larger in primary schools. In New Zealand, the New Zealand Qualification Authority (NZQA) has provided an Assessment Tool for Teaching and Learning (e-asTTle) since 2002. E-asTTle is an educational platform that enables educators to design and generate standardised and curriculum-aligned tests in reading, writing and mathematics in either English or Māori languages, in primary and secondary levels of education. A large bank of calibrated items allows teachers to customise assessments to the specific needs of their classrooms. Contrary to other computer-assisted assessment systems, e-asTTle was also designed with a strong emphasis on formative assessment. Those two examples of digital assessments differ from the self-assessment resources discussed in Chapter 5 (Yu, Vidal and Vincent-Lancrin, 2023[23]) in the sense that they are meant to be administered by teachers but benefit from a large‑scale administration that can allow for comparisons across schools and school boards. Self-assessment resources can be accessed and used by individual people.
To support the administration of paper-based assessments
Countries and jurisdictions have also developed digital tools to support the administration of non-digital assessments. In Chile, the government publicly provides digital tools to support the administration of SIMCE (Sistema de Medición de la Calidad de la Educación), the paper-and-pencil national evaluations. They include online item grading platforms and digital tools for data reading and processing. In Luxembourg, the ministry requires schools to manage the administration of the two end-of-cycle exams with two separate digital tools: PFS supports the management of the national exam at the end of primary education, and BAC, the national end-of-secondary-school exam. Those two tools embark features that facilitate student registrations, grading, and tracking of their exam sessions.
Finally, in education systems where standardised evaluations are run on probabilistic samples of students (as is often the case), educational authorities may provide regional administrators and schools staff with digital tools to assist them with the administration of the assessment. In France for instance, several standardised student evaluations coexist: Cedre (mathematics and literacy), LSE (digital reading literacy), Socle CE1 (mathematics and literacy in grade 2), Socle Sixième (mathematics and literacy in grade 6), as well as TIMMS and PISA. All tests are administered on randomised samples of students. To support sampled schools in the administration of the right assessments to the right students, the ministry has developed a digital tool, called Application de Suivi des Passations (ASP).
To support the grading of exams and the transfer of grades
Digital tools are also used to support the grading of paper-based exams. Those tools may enable collaborative grading and automated grade sharing with the educational authorities in charge of the examination.
In France, the ministry of education has developed a platform called Cyclades to support the grading of its paper-based high school graduation examination. Since June 2021, schools scan and digitise the end-of-high-school and other high-stake exams’ response sheets, filled in by students with paper and pencils; teacher-graders grade the copies online, annotate the document and leave comments that students can access almost instantly after the validation process. This in principle avoids the possible loss of response sheets, but could also, over time, give rise to the machine analysis of the response sheets.
In Ireland, the high school graduation exams are also administered as paper-based assessments. The State Examinations Commission (in charge of secondary education high-stakes assessments) uses a digital platform to support their grading. In a low-stakes context, Ireland’s Educational Research Centre (ERC) also offers machine coding services for schools that administer the (optional) Drumcondra standardised assessments on paper.
To enable proctoring
A less common type of digital systems provided by educational authorities to support the administration of assessments are proctoring systems. Proctoring systems are digital tools used to monitor students during online (or offline, digital) assessments, preventing them from cheating and from unauthorised activities during a remote exam (and sometimes a computerised exam). When the exam or test is taken remotely, they may ensure the integrity of the assessment by employing methods such as video monitoring as well as screen, keystrokes, or mouse tracking. In many countries and jurisdictions, the use of proctoring systems in education is perceived as intrusive and raising privacy concerns (Nigam et al., 2021[24]).
Latvia uses an AI-based proctoring system (ProctorEdu) to monitor secondary education students participating in school subject Olympiads, a range of competitive but low-stakes assessments for students throughout the country. The monitoring tool features desktop, audio and video recording, biometric verification of users’ identity as well as real-time behavioural monitoring of the test sessions (among other features). Candidates log in and sit the assessment at any time, in any place, and on any device, in asynchronous and synchronous ways. As of 2024, the system was no allowed for high-stakes state examinations.
The Finnish and Danish examples presented above illustrate a more limited form of digital proctoring for in-presence exams: digital proctoring system merely limit the computer environment while humans continue to proctor the exam as would be the case for a paper-and-pencil test. One can imagine that this form of proctoring may become more common to limit the uses of generative AI until new forms of assessment are developed and tested.
More and more countries and jurisdictions digitise the student assessments that they administer at the central or jurisdictional levels. Digital assessments can be adaptive and more personalised; they may assess skills in ways that paper-and-pencil do not allow; they may power learning analytics; they can provide flexibility to allow for remote exams in certain situations; and they can improve the cost-efficiency of assessments through automation (e.g., grading), replication over time, and lowered long-term administration costs. However, more innovative forms of assessment can be costly to develop and not be appropriate for high stakes exams. Concerns relate to the validity, reliability, and security of the assessments. They can also raise an equity issue, given digital divides in terms of hardware, software and human competences. And the digitisation process may infer transitions costs and a path dependency that can be more difficult to disrupt than in the case of paper-and-pencil assessments or exams.
Digitising the simpler things first
Low-stakes evaluations are twice as often digitised as exams. Out of 29 education systems for which we have comparative information, 18 have digitised (part of) their standardised student evaluation. Among them, 7 have also digitised (part of) their student examination. No country has embarked in the digitalisation of high-stakes exams without experimenting with the digitalisation of low-stakes evaluations in parallel.
This chapter suggests four main reasons to explain why exams are less often digitised than evaluations. First, digitising low-stakes standardised evaluations is less complex than digitising exams. Second, because student exams carry higher stakes for students and families, attempting to innovate them leads to stronger political resistance from education stakeholders. Third, because exams are high stakes and must be fair, it is difficult to implement them in an educational environment that is not digitalised, which may become the case as countries and jurisdictions develop students’ digital skills. Transitioning to digital exams for entire cohorts of students often require stronger investments in digital infrastructure, including investments in the training and digital literacy of teachers and school staff.
Our comparative data collection further suggests that countries and jurisdictions tend to use the digitalisation of their student evaluation as a first step towards the possible digitalisation of their exams, if any. The transition from paper-based to digital evaluations has required countries to strengthen their digital infrastructure in schools, in assessment centres, or directly at students’ place of residence.
Whether they have digitalised their national or not, countries have developed digital tools that support the administration of assessments. 16 out of 29 education systems use such support tools. Most tools are used to provide a digital environment for the administration of digital evaluations or, to a lesser extent, digital exams; but some support the administration of paper-based assessments (e.g., student registration, distribution of exam tasks, completion status), or facilitate the grading and transfer of grades to other digital systems or registers. Only one country (Latvia) uses an AI-based digital proctoring system in a low-stake context.
A digitalisation without transformation?
As of 2024, countries and jurisdictions that had digitised (part of) their student assessment had usually not used many innovative affordances allowed by digital assessments. In terms of content and design, most digitised assessments replicate the paper-based assessments they replaced. Almost no assessment uses adaptive assessment techniques, not to mention games, simulations or scenario-based formats. Denmark tried to embed adaptive features to its low-stake standardised evaluation before policy makers decided to revert to a traditional design given the resistance of the public opinion.
Even the most innovative digital features, such as the use of digital assessment environments or proctoring systems, are usually deployed to replicate the conditions of traditional, paper-based assessments taken in class. As of 2024, the advantages of digitising assessments have mainly lied in the simplification of the administration, data management and sometimes grading processes. Most common data linkages are done with jurisdictions’ student information systems (or central student registers).
Looking forward: where to go next?
Addressing technical challenges. Digital assessments have technical prerequisites, which include access to enough compatible devices; sufficiently reliable Internet and/or local network capabilities; sufficient staff with the expertise to support the introduction and ongoing use of computer-based assessment; physical spaces with the electrical and network facilities suitable for numbers of students to take the assessments on devices concurrently; and strong IT infrastructure to manage security risks. An offline modality of taking the assessment may provide a means for minimising some of these risks, but this requires some additional time and guidance to schools for installing the software beforehand and limits the use of digital tools and the adaptability of the assessment. Countries willing to meet these requirements will have to plan for an initial financial investment and ongoing maintenance costs.
More innovative assessment features. The transition to digital assessments opens up opportunities for incorporating more innovative features into the assessment process. However, our data collection shows that we are not there yet. Perhaps the new assessments that will be designed, from their infancy, to be taken digitally, may embark more innovative assessment features than those that were mainly designed to replace their paper-based counterparts. The next generation of digital assessments could foster the deployment of personalised learning experiences, adaptive features that adjust based on student performance, automated grading that allows for quicker feedback and immediate access to results, among other digital affordances. Countries will seek to transform their assessment practices to reap the benefits of the digital transformation in education.
Broader content coverage. As of 2024, most of the digital assessments developed and administered by countries and jurisdictions are digitised versions of their standardised evaluation, hence they mostly assess students’ literacy and numeracy. Yet, digital assessments provide the means to evaluate a broader range of subjects and competences, including other traditional curricular subjects than language and mathematics, but also a wider set of knowledge, skills, attitudes, and values. Countries that seek to integrate digital competences into teachers pre-service and in-service training, or into curriculum requirements, should also look to measure those skills so as to monitor all education actors’ digital literacy. Digitising assessment serves both as a means and as an end to that.
Better integration within countries’ assessment ecosystem. Countries and jurisdictions may explore how to better integrate their digital assessments into their broader assessment ecosystem, where different education stakeholders have access to different assessment resources, such as past exams or sample of items that teachers can use to create their own assessment; or that students can use for self-assessment. They can also be readily integrated within countries’ digital credential systems.
Better linkages with other education and student information systems. Sitting a digital assessment typically requires students to self-identify online or on a digital platform, using unique identifiers – except in the context of sample-based assessments where student results are aggregated. This allows to store and transfer student results and, if those identifiers are longitudinal, to monitor students’ progress and trajectory over time. This linkage is typically done through student information systems; and allow schools and teachers to compare their students’ results with other (similar) schools and teachers. Additionally, student results to digital assessments can be immediately and automatically mobilised as key signals by early warning systems, that can help make visible students who are at risk of dropping out, for instance (see (Vidal, 2023[25])).
Easier and deeper access to data for research. Digital assessment offers advantages over paper-based assessment in terms of data accessibility and analysis. With digital assessments, or with digital tools that support the administration of paper-based assessments, student data may be scored automatically and stored electronically, allowing for easy and quick access. Building on appropriate data governance to mitigate the risks posed to data protection and privacy (see (Vincent-Lancrin and González-Sancho, 2023[26])), these digital capabilities streamline the research process, making it more efficient and reliable compared to handling similar amounts of paper‑based data. Furthermore, if those data are tagged with unique, sometime longitudinal, student identifiers, then they may provide even richer insights for research when integrated with other student and school information stored in other databases.
References
[3] Buckley, J.; L. Colosimo; R. Kantar; M. McCall and E. Snow (2021), “Game-based assessment for education”, in OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, OECD Publishing, Paris, https://doi.org/10.1787/9289cbfd-en.
[5] Foster, N. (2023), “Teacher digital competences: formal approaches to their development”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[4] Fragoso, T. (2023), “Hardware: the provision of connectivity and digital devices”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[14] Kimura, T. (2017), “The impacts of computer adaptive testing from a variety of perspectives”, Journal of Educational Evaluation for Health Professions, Vol. 14, p. 12, https://doi.org/10.3352/jeehp.2017.14.12.
[12] Kolen, M. and R. Brennan (2014), Test Equating, Scaling, and Linking, Springer New York, New York, NY, https://doi.org/10.1007/978-1-4939-0317-7.
[6] Leaton Gray, S. and N. Kucirkova (2021), “AI and the human in education: Editorial”, London Review of Education, Vol. 19/1, https://doi.org/10.14324/lre.19.1.10.
[11] McClelland, T. and J. Cuevas (2020), A comparison of computer based testing and paper and pencil testing in mathematics assessment.
[24] Nigam, A.; R. Pasricha; T. Singh and P. Churi (2021), “A Systematic Review on AI-based Proctoring Systems: Past, Present and Future”, Education and Information Technologies, Vol. 26/5, pp. 6421-6445, https://doi.org/10.1007/s10639-021-10597-x.
[18] OECD (2023), Country Digital Education Ecosystems and Governance. A companion to Digital Education Outlook 2023, OECD Publishing, https://doi.org/10.1787/906134d4-en.
[19] OECD (2023), Education at a Glance 2023: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/e13bef63-en.
[17] OECD (2023), “Strengthening the design and implementation of the standardised student assessment reform of the Flemish Community of Belgium”, OECD Education Policy Perspectives, No. 72, OECD Publishing, Paris, https://doi.org/10.1787/85250f4c-en.
[21] OECD (2023), “What assessments and examinations of students are in place?”, in Education at a Glance 2023: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/dfccf5a6-en.
[22] OECD (2021), “National assessment reform: Core considerations for Brazil”, OECD Education Policy Perspectives, No. 34, OECD Publishing, Paris, https://doi.org/10.1787/333a6e20-en.
[1] OECD (2021), OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, OECD Publishing, Paris, https://doi.org/10.1787/589b283f-en.
[2] OECD (2020), “Remote online exams in higher education during the COVID-19 crisis”, OECD Education Policy Perspectives, No. 6, OECD Publishing, Paris, https://doi.org/10.1787/f53e2177-en.
[15] OECD (2013), Synergies for Better Learning: An International Perspective on Evaluation and Assessment, OECD Reviews of Evaluation and Assessment in Education, OECD Publishing, Paris, https://doi.org/10.1787/9789264190658-en.
[10] Ofqual (2020), Online and On-screen Assessment in High Stakes, Sessional Qualifications: A Review of the Barriers to Greater Adoption and How These Might Be Overcome, https://assets.publishing.service.gov.uk/government/uploads/system/upl.
[13] Sandene, B.; N. Horkay; R.E. Bennett; J. Braswell; B. Kaplan and A. Oranje (2005), Online Assessment in Mathematics and Writing: Reports from the NAEP Technology-Based Assessment Project, https://files.eric.ed.gov/fulltext/ED485780.pdf.
[16] Somers, G. (2021), Feasibility Study into the Introduction of Central Assessments in Flanders: Legal-technical Aspects (Lot 3).
[8] Timmis, S.; P. Broadfoot; R. Sutherland and A. Oldfield (2015), “Rethinking assessment in a digital age: opportunities, challenges and risks”, British Educational Research Journal, Vol. 42/3, pp. 454-476, https://doi.org/10.1002/berj.3215.
[7] Veldkamp, B. and C. Sluijter (eds.) (2019), Theoretical and Practical Advances in Computer-based Educational Measurement, Springer International Publishing, Cham, https://doi.org/10.1007/978-3-030-18480-3.
[25] Vidal, Q. (2023), “Digital assessment”, in OECD Digital Education Outlook. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[26] Vincent-Lancrin, S. and C. González-Sancho (2023), “Data and technology governance: fostering trust in the use of data”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[20] Vincent-Lancrin, S. and C. González-Sancho (2023), “Education and student information systems”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[9] Wyatt-Smith, C., B. Lingard and E. Heck (2021), Digital Disruption In Teaching And Testing, Routledge, New York, https://doi.org/10.4324/9781003045793.
[23] Yu, J., Q. Vidal and S. Vincent-Lancrin (2023), “Digital teaching and learning resources”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
← 1. United States: Innovative Assessment Demonstration Authority: https://www2.ed.gov/admins/lead/account/iada/index.html
← 2. The curriculum competences assessed will vary among students in the Flemish system and even within schools. In this example, two-thirds of the students in that particular school will get assessed on the competence “number theory”, two-thirds of the students will get assessed on the competence “geometry” and two-thirds of the students will get assessed on the competence in “statistics”. Other schools could be assigned three different mathematics competences out of the ten available (OECD, 2023[17]).
← 3. In the England (United Kingdom), the Standards and Testing Agency administers Multiplication Tables Check, a statutory digital assessment for all Year 4 students (8-9 years old, primary education). As of 2023, another statutory assessment for measuring English and maths skills of new primary school students, Reception Baseline Assessment, is administered partially digitally, but the department is considering digitalising it further. In addition, a new Digital Test for Literacy and Numeracy is in the pipeline, which will involve a series of short digital activities and will be taken by Year 9 students (13-14 years old, secondary education).
← 5. Brazil’s National Bank of Assessment Items (BNI): https://www.gov.br/inep/pt-br/areas-de-atuacao/avaliacao-e-exames-educacionais/bni