Stéphan Vincent-Lancrin
OECD
OECD Digital Education Outlook 2023
Towards an Effective Digital Education Ecosystem
OECD Digital Education Outlook
Abstract
This chapter presents a vision of what a digital transformation of education could look like and what some of its benefits and challenges are. It argues that digital technology, including AI, could improve the effectiveness and quality of education by personalising education, be it teaching and learning or other education services, by making it more inclusive and possibly equitable, and by improving the cost-efficiency of the sector. A digital transformation of education also comes with risks that must be mitigated. The findings of the report about where countries stand suggest a few areas where they should focus their efforts to catalyse their ongoing journey of digitising the education operations towards a proper digital transformation. Taking advantage of the possibilities of widespread data collections and the use of advanced digital tools and resources to solve their educational problems will require further effort.
Digitalisation opens new possibilities for education. While education has always been a sector rich in data, such as grades and administrative information, the use of data to help students learn better, help teachers teach better, and inform everyone’s decision making in education systems – from parents and students through to administrators – is still nascent. Could digital technology, and, notably, smart technology based on artificial intelligence (AI), learning analytics, robotics, etc., transform education in the same ways it is transforming the rest of society? If so, what might this digital transformation look like and how can countries harness it?
This chapter starts by restating what current digital technology can do for education based on previous work on the trends and frontiers of educational technology (OECD, 2021[1]). The OECD Digital Education Outlook 2021 highlighted different uses of digital tools and resources that had the potential to improve teaching and learning as well as the management of education institutions and systems. The COVID-19 pandemic accelerated the uptake and lived experience of digital technology for teaching and learning, but it also exposed the relative scarcity and basic nature of most digital resources and tools used in education (OECD, 2022[2]; Vincent-Lancrin, 2022[3]).
The OECD Digital Education Outlook 2023 shows a similar picture, with a slow penetration of digital tools in education, used mainly as a way to digitise existing educational processes. The visible emergence of generative AI has been a wake-up call for education policy makers: it has raised their awareness of the possibly disruptive nature of advanced technology but also of its imminent impact on our societies. While the frontiers outlined in the OECD Digital Education Outlook 2021 may have appeared as distant then, their time horizon is now perceived as very close. However, despite a chapter dedicated to generative AI (Vidal, Vincent-Lancrin and Yun, 2023[4]), most educational AI is embedded in systems that provide a diagnosis based on a large amount of data, that suggest decisions, or that capture learners’ or teachers’ information to provide them with feedback or suggestions. One of the big challenges for a digital transformation of education is for policy makers and teachers to have a better grasp of existing AI tools and resources specifically designed for education and to better use the data collected by a variety of digital systems to make education more effective and personalised for every student. A digital transformation of education could be about supporting students and teachers in their decisions based on observations that are not immediately accessible to them, and about designing policy reforms based on an unprecedented amount of (reusable and analysable) information.
The findings presented in this book are based on mixed method research: the OECD carried out a survey about countries’ digital education infrastructure and governance, which was supplemented by a series of interviews and desk research. The comparative analysis in this book is based on the descriptive analysis of each country’s digital education ecosystem and governance, which is presented in detail in a companion report (OECD, 2023[5]). All the comparative analyses and data presented in these books, and notably its tables and examples, were double checked by countries in December 2023.
After recalling some of the opportunities and challenges of a digital transformation, the chapter will provide a quick overview of where countries are on this journey, arguing that most countries still face the challenge of shaping a digital education ecosystem that provides teachers and students with the appropriate tools to improve their teaching and learning, that can make the data collected by a variety of digital tools reusable to address important educational objectives, and that empower teachers and students in their educational career. Many building blocks are still missing for such an ecosystem to be effective: a stronger emphasis on teachers’ professional learning, the availability and interoperability of some key digital tools, investment in hardware and connectivity where not of sufficient quality, and the establishment of new types of institutions that help to implement digital strategies – from “support organisations” to “innovation labs” that can create useful resources for education systems and negotiate their responsible use with all stakeholders.
Personalising learning and education
The personalisation of education is one of the major potentials of digitalisation. Personalisation does not imply or assume that education is no longer social and collective; it simply refers to the delivery of education that helps learners individually in their educational journey. While the contexts can be different, the personalisation of education and learning is based on the same principles: capturing and detecting information that is specific to a student or that can be inferred from detections made on “similar” students; using the detected information to make a diagnosis, for example a recommendation; and in some cases, having an intervention based on this diagnosis, usually under the supervision of a human being (OECD, 2021[1]). These principles can be used for instructional decisions when giving study and careers advice, for designing specific educational interventions, etc. What the diagnosis phase requires is usually a large amount of data or observations that allows comparisons to be made between a specific person and others who share some of the same relevant characteristics.
Here are three examples of how digital tools (and notably AI-based tools) can support personalised learning or education.
In the classroom, AI applications that directly support student learning show early promise with the development of adaptive learning systems, including intelligent tutoring systems. Personalised learning aims to provide all students with the appropriate curriculum or task and scaffold them to solve specific problems based on a diagnosis of their knowledge and knowledge gaps. Increasingly, this personalisation of learning can rely on digital tools, which not only focus on “what” students should practice next, but also take into account how students learn and consider factors such as self-regulation, motivation, and effort (OECD, 2021[1]). These digital learning resources can be used and remain helpful outside the classroom too, for homework, as automated private tutoring or practice solutions, and for lifelong learning. While still too expensive to be present in education, social robots may perform similar tasks in different ways in the future: they can use adaptive learning to tutor students with natural language, but they can also teach or motivate them to learn by playing the role of a peer student (Belpaeme and Tanaka, 2021[6]).
While adaptive learning data are typically collected when learners interact with a specific software, AI in education can provide diagnosis information to teachers and school leaders about their students based on data collected for administrative purposes. Where countries collect standardised assessment data for each student over time, or just teacher-given grades, AI models can gradually infer a development or growth model for students’ learning based on their “past trajectory” and a comparison with students sharing similar characteristics. This can give rise to a variety of recommendation tools. In many cases, the collected data is provided back to schools through dashboards so they can interpret it themselves and take action to improve students’ performance (if needed). In a few cases, predictive models about individual students’ “growth” can be designed, alerting teachers or educators when specific students do not follow the expected path. This may lead to different types of interventions. Early warning systems based on AI algorithms are based on the same model (Bowers, 2021[7]). Although they may use different types of data (e.g. absence patterns), they usually provide schools with an indication that a specific student is “at risk” of dropping out, notably identifying students that school staff do not necessarily suspect to be at risk. Here again, once the diagnosis is made, human beings have to intervene (or ignore the recommendation).
A third example relates to student’s career planning and educational guidance. Given the complexity and variety of study paths (and possible careers), countries offer careers and study guidance services. They support students to navigate their education system and its different tracks (if any) but also help them to shape their expectations for a transition into the labour market. Some of these services are based on digital platforms with interactive services: they typically propose a personality test to identify students’ tastes and preferences to propose a few possible related occupations and services. While this provides some level of customisation, one could imagine that some of this guidance could be personalised further using not only students’ preferences but also their observed strengths and interests within the education system – thus providing more individualised advice.
These three cases present different modes of personalisation (or individualisation) of education but highlight how a digital transformation could make it possible. In all cases, this requires the collection of data not only about the individual about whom the advice is given, but a number of other subjects. This also requires being able to link data and build digital systems that can reuse relevant information for the mentioned purposes.
Inclusion and equity
The digitalisation of learning tools and resources can expand access to learning and teaching materials, and thus learning opportunities. Educational platforms proposing open educational resources or massive open online course (MOOC) platforms are good examples. At least in some parts of the world, they allow learners to access learning materials that may be superior to what they can access locally. When provided universally, closed-access resources limited to students enrolled in an education system can also provide students with more learning opportunities. Contrary to textbooks, digital resources can be made accessible at scale on a mere use basis. When provided by the national or central government, all students within the education system can access them and learn under the supervision of their teachers (but also possibly on their own). In the analogic world, this would be equivalent to providing students with all available textbooks and allowing them to choose the ones that work best for them, something that is not feasible under public resource constraints.
Some of the personalisation tools mentioned above can also have a positive impact on equality. Few studies show that adaptive technology (or personalised learning) reduces the achievement gap between students with more and less prior academic knowledge. For intelligent tutoring systems to reduce achievement gaps, they have to be more effective with students with more initial difficulties. Evaluated through a randomised control trial, an intervention in the US state of Maine showed that this may become the case (Murphy et al., 2020[8]; Roschelle et al., 2016[9]). Teachers in the intervention schools used an adaptive learning software to provide students with mathematics homework. The system provides feedback to students as they solve mathematics homework problems and automatically prepares reports for teachers about student performance on daily assignments. Teachers received training and coaching on formative assessment. The study found that students in the schools using the software learned more compared with their peers in the control schools, with large effect sizes, and that the impact was greater for students with lower prior mathematics achievement. A reduction of the achievement gap between different group of students is thus possible.
Just as important, digital technologies can reduce inequity by facilitating the inclusion of students with special needs and by adapting learning to different learning styles. Technology has, for example, made it much easier to support the diagnosis of learning difficulties such as dysgraphia, and remedial digital responses have also been developed. A variety of smart technologies applied to learning solutions also makes it easier for blind or visually impaired students as well as deaf or hard-of-hearing students to access learning materials and easily perform the educational tasks required from other students. AI-enabled speech to text (and vice versa) or automatic subtitles are the most obvious examples. Learning technologies also help address more difficult inclusion issues, for example by supporting the socio-emotional learning of autistic children (OECD, 2021[1]).
There are many other ways in which technology can support equity as well as the implementation of countries’ policy efforts towards equity. While early warning systems give an example of AI-based recommendations to provide individualised educational services for those at risk of dropping out, many other individualised interventions can contribute to alleviating inequalities. Digitalisation makes it easier for countries and jurisdictions to individualise their services and target students with locally identified characteristics. In some countries, it has enabled shifts from school- or neighbourhood-based equity policies to individualised ones.
Enhancing the quality of teaching
As teaching is key to students’ success, and as human educators are key to the wellbeing and holistic education of children in school, digital technology that supports and provides feedback to teachers and other educators offers another opportunity to improve the quality of education. The examples of personalised education presented above provide teachers with suggestions, recommendations and food for thought about specific students, unless the information is trivial. Perhaps this can make teachers realise that specific students needed more attention, or that they would have been expected to perform better as they do, or that they might be at risk of dropping out. This information derived from past data points that are usually not accessible to them, or from a comparison with other students within a system, enables teachers to reflect on their instruction practices and on how to customise them for a given student or class. In some cases, these digital tools not only provide information to teachers, but they also make suggestions on teaching and learning resources, etc. While teachers can ignore them, as is the case for medical doctors who receive information from their “expert systems”, this can hopefully provide them with ideas to improve their teaching for a given context.
In the same way as digitalisation makes a wider array of digital learning resources available to students, it does so for teachers. Not only can teachers access open educational resources as well as multiple platforms of digital learning resources, they can also have dedicated platforms with digital teaching resources. The variety of resources can help them design their lesson plans, integrating digital elements into them, but also connect with their peer teachers teaching similar classes or subjects. Here again, the non-rival character and near-zero cost of reproduction of digital resources make it easier to provide teachers with more options to find their relevant teaching resources, made available by their government, their local authority, their school, or cultural agencies nationally and internationally.
Finally, while still work in progress and largely absent from OECD schools, classroom analytics may also support teachers to teach more effectively. Instead of taking students as the unit of analysis, classroom analytics focus on the entire classroom and provide teachers with real-time or post-hoc feedback on how to improve or “orchestrate” their teaching. Many applications already show how a variety of solutions could support teachers in better using their time in class, for example, by suggesting when it is a good time to shift to the next teaching or learning activity after students were given individual activities, identifying who would require their attention the most, and recommending how they could engage the whole class in collaborative learning activities. While some classroom orchestration solutions are designed to help teachers in real time, they also provide feedback on teachers’ professional practice, measuring, for example, how much they talk (compared to their students) and to whom, or how they divide their time between different types of activities (Dillenbourg, 2021[10]). Both real-time and post-hoc feedback are akin to personal professional learning opportunities for individual teachers in question, and they furthermore contribute to the personalisation agenda as their recommendations target the specific teacher who was (digitally) observed rather than a theoretical or general teaching practice. By providing individual teachers with reflective opportunities on their teaching practices and thus professional learning opportunities, digital technology could subsequently contribute to the wellbeing and learning outcomes of students.
Improving efficiency
In many business and government sectors, beyond effectiveness, a major rationale for digitalisation lies in efficiency. Many countries have embarked on digital government strategies to this effect, notably to make processes more efficient and easier for their users. The OECD has developed a Recommendation and principles highlighting these different objectives (OECD, 2020[11]).
There are different ways in which digital technology can increase cost efficiency in education. One example lies in student application (and admission) processes for educational institutions. Applications are sometimes undertaken through digital platforms, especially for the transition towards higher education, where a “matching” (or selection) process is often necessary. In open-admission institutions, when no selection is required beyond rule-based criteria, implementing seamless automated admission processes is even possible. The implementation of the National Education Information System (NEIS) in Korea, an e-government system that allows, among other things, for the digital transfer of students’ academic records from one school to the other (as well as from school to university) was estimated to have saved USD 237 million a year when a cost-benefit analysis was undertaken in 2010.
A second area where digitalisation could lead to cost efficiency is the provision of verifiable degrees and other credentials, for example using blockchain technology (Smolenski, 2021[12]). The gradual development of an infrastructure for digital credentials and the adoption of open standards may lead to a different way of certifying and holding degrees, with individuals being able to manage their qualifications themselves.
A third area where cost efficiency is underway is the collection of system-level statistical information. While in the past statistical information often relied on the establishment of statistical panels (of representative samples of individuals or institutions) and often involved multiple handlings of the same data, the use of administrative data (when combined with the interoperability of diverse systems) has made it much easier to get statistical information from operational services in almost real time. Essentially, the latter avoids that administrators re-enter the same information several times.
But efficiency is also about how teachers use their time. Digital technology could help free some of teachers’ time, allowing them to focus on the most stimulating aspects of their work. An obvious example is formative assessment, or developments in the automated grading of open-ended essays, because grading and designing assessments are time-intensive tasks when done manually. Another example lies in some of the administrative tasks that teachers have to perform that could be supported by computers. By freeing up time for teachers, smart technologies can allow them to dedicate more time to learners who most need their attention, and to focus on their own continuous professional development or on supporting complex aspects of students’ learning, including the acquisition of higher-order or of socio-emotional skills.
Enhancing research and innovation
Digitalisation helps to promote another aspect of efficiency and effectiveness: improving policy design and reform based on evidence, research and quick innovation (OECD, 2019[13]). In a digitalised education sector, the unprecedented amount of collected data allows researchers and governments to undertake research on their education systems in order to reform it and achieve their goals.
While digital tools have a practical utility, their development also helps to uncover educational patterns that were not previously visible. They help to better understand education systems, their actors’ behaviours, and thus to design better policies and better practical interventions. For example, the research on early warning systems has not only led to predictive tools, it has also enabled researchers to recognise that different profiles of students were at risk of dropping out and that the types of interventions they required were thus different. Bowers and Sprott (2012[14]) showed that the majority of high school dropouts did not match common wisdom about dropout and these students were thus likely “invisible” to many education stakeholders. This is one example among many showing the value of collecting and analysing robust data and having a strong data infrastructure for better policy design.
Making education more relevant to modern times
Regardless of the benefits of personalisation and cost-efficiency, a strong argument for the more intensive use of digital tools and resources in education lies in the development of learners’ digital skills. This is one of countries’ main educational objectives, recognising that education should reflect and prepare students for modern societies. While in the past, most evaluations of digital technology in education focused almost exclusively on their effects on students’ learning outcomes, usually in mathematics or language, both the COVID-19 pandemic and the ongoing digital transformation of our societies have shown that this may not be the only rationale for digital education. Even if the use of digital technology did not improve the effectiveness of education compared to its non-digital equivalent, it might still be important to use digital tools to develop students’ digital competences: to ensure a better mastery of digital technology, to familiarise them with it, and to help them understand broadly how it works. Many countries have made “digital competences”, defined in different ways, a transversal competence (and made “computer science and/or computational thinking” a more important part of their curriculum).
Generative AI is an interesting point in case. While its emergence was considered as disruptive by many and framed in terms of “cheating”, it may be an opportunity to prepare students for the modern age. Assuming that in the near future generative AI becomes more prevalent in the labour market and our lives, getting students used to working with it, getting skilled at preparing prompts, knowing what to (and what not to) expect from it, are all just another dimension of developing their digital competences. Moreover, as a productivity tool, it can allow teachers and students to do much more than what would have been possible before: produce more text, create and refine images that would have been very time consuming to produce, get help in producing music and songs, etc.
Opportunities usually come with challenges and unknowns, especially when digital technology is new and evolving at a fast pace. Harnessing the promises of a digital transformation requires both awareness and mitigation of those risks, and a careful cost-benefit analysis. While some of the risks are new and specific to digitalisation, many are not; digital risks need to be compared with the risks of a non-digital education.
Digital divides
Despite the possible benefits of digital tools for equity, the COVID-19 pandemic has exposed inequalities of access to connectivity and digital tools within education systems, and notably the inequalities of access to digital devices and connectivity (Thorn and Vincent-Lancrin, 2021[15]; Vincent-Lancrin, 2022[3]). As long as access to high quality connectivity and to sufficiently recent digital devices is not universal, digitalisation will present challenges to equality of opportunities and equity. As noted by Fragoso (2023[16]), the availability of appropriate hardware is a necessary condition for a digital transformation of education, one that most countries are aware of and upon which they focus their investments and digital strategies. Remote learning during the pandemic highlighted that school education did not stop at the school doors but continued at home. A lack of available devices and connectivity at home is a problem for digital education. Providing quick and affordable broadband or mobile data connectivity across a country is usually not in the ministry of education’s portfolio. Many countries had interesting initiatives during the COVID-19 crisis to create a more even playing field: most of those initiatives, such as making education platforms free of charge to end users, have since been discontinued. Interesting initiatives to alleviate discrepancies in access to digital devices and connectivity continue though, as is the case in Japan, Luxembourg or the Flemish Community of Belgium for devices, and the United States for connectivity (OECD, 2023[5]).
A second challenge of digitalisation lies in the availability of advanced technology within countries. Even though they can be considered as more or less centralised, all countries have decentralised education systems. The devolution of responsibility takes different shapes and forms across countries, but regional governments, local governments and schools themselves play a role in choosing digital tools and resources. In addition, depending on the school funding formula, public schools located in different parts of a country may have budgets that are significantly different and allow for a very different provision of digital tools and resources to students and teachers. It is, for example, possible that local education authorities in richer neighbourhoods provide their schools and students with more and better AI-based tools (e.g. intelligent tutoring systems) and that this increases the achievement gap and inequality of opportunities with students (and teachers) in poorer neighbourhoods and schools. The variation in the cost of (and budget spent on) textbooks and other paper learning resources across schools is likely to be smaller. Depending on the effectiveness of digital learning tools compared to textbooks and static learning resources, digitalisation may lead to more inequalities unless governments address the issue and ensure that there is at least a minimum basis of digital tools and resources available to all schools in their territory (as is for example the case in France, see (OECD, 2023[5])).
A third challenge that may correlate with the previous one comes from the inequality in the digital competences of teachers within countries. Even though the COVID-19 pandemic led to a forced use of digital tools in education and has made teachers more familiar with digital teaching and learning resources and tools, there are still widespread variations in teachers’ confidence and interest in integrating digital resources in their teaching. Where those resources are not available, lower competences can come from the lack of opportunities to develop them.
The jury is still out on whether digitalisation is likely to widen or help close the gap between educational outcomes in high- and low-income countries (or high- and low-income regions within countries). On the one hand, digitalisation requires continuous investments in hardware (connectivity and devices), for which access is still a limitation in many countries. It also requires digital teaching and learning tools and resources that are adapted to local contexts and thus a certain level of available expertise within countries. On the other hand, digitalisation makes knowledge available in countries where people struggled to access recent knowledge, and light models of generative AI that run on a mobile phone with low bandwidth requirements may support teachers and learners around the world, regardless of their country’s income level. Some middle-income countries/jurisdictions have shown that digitalisation could be used to improve system performance without introducing advanced technology products or services in the classroom. Digitalisation is an incremental process, and all countries can reap some of its benefits by clearly identifying the purpose and means of using digitalisation to solve a problem. In Gujarat (India), for example, where absenteeism of both students and teachers was a problem, the digital monitoring of school attendance coupled with the provision of dedicated human resources and services have led to a significant reduction of the problem (Vincent-Lancrin and González-Sancho, 2023[17]).
Performance of digital tools
While digital tools hold many promises for more effective education, they do not have perfect performance yet – contrary to calculators, for example. It is possible that some of the most advanced tools will always have their shortcomings, as is the case for human individual and collective intelligence. As they may make mistakes in the advice or recommendations they provide to students, teachers, parents, etc., it is important to understand their limitations and that they are used under the supervision of competent human beings.
For example, while some early warning systems now approach very good predictive power, Bowers (2021[7]) shows that a significant number of them rely on predictors that are no better than a random guess. In the areas of student engagement, D’Mello (2021[18]) points to new approaches that are developed to better measure students’ engagement in learning using facial image analysis and other ways but also notes the inaccuracy of many of the measures used in the field of learning engagement. In the area of classroom analytics, Dillenbourg (2021[10]) notes that some solutions manage to identify whether learners are working individually or in groups with a very high level of accuracy (90%) but identifying the type of teaching and learning activity remains more challenging (67% of accuracy). Those are just three examples, showing that accuracy levels can be very high, but are not guaranteed for any AI-powered education application. Despite their impressive natural language generative power, AI text generators also have “hallucinations” and provide erroneous information with perfect syntax.
Most of the time, these shortcomings do not matter because the stakes are low: AI systems make recommendations that may be more or less correct but are checked and can be discarded by a human being. Human beings also make mistakes and give advice that is not always worth following. While AI-based digital tools should be able to demonstrate a certain level of performance to be put on the market, some level of mistake is not necessarily a serious problem as long as those have no serious consequences for the users. We are used to all sorts of errors, made by machines and humans.
For example, an early warning system that makes good predictions 7 or 8 times out of 10 would actually be very useful – assuming it makes visible those signs or patterns of dropout that are not so visible to teachers and school leaders. In the 2 to 3 cases when such a system is wrong, human educators may realise this is a false alert not to be followed and hopefully the interventions put into place will not be harmful to students that are not really at risk. The cost of those mistakes (in terms of inefficiency and annoyance for “false positive” cases) should be compared to the benefits of the system (compared to a uniquely human detection of potential dropout cases).
However, when a system has high stakes for individuals, our tolerance for errors should be minimal, and the systems we use should have perfect or very high levels of performance. For example, if early warning systems were not meant to provide support to students to prevent a bad outcome to happen, but led to an intervention that would be extremely costly and risky for students, it would be unethical to rely on a digital tool with imperfect performance (even if humans were also making imperfect decisions).
New or amplified biases
Some AI-based digital tools have been shown to perform better for some population groups than for others. For example, an intelligent tutoring system that is used universally could work better for, say, girls than boys. Depending on the initial situation, it may actually increase or decrease achievement gaps between girls and boys. A speech to text software may work much better for white than for black English speakers, making only one share of the population able to reap its benefits. While these issues are also a “performance” problem, they relate to equity and are not easily identifiable based on the overall performance of a digital tool or resource: an AI tool may have a good performance for the entire population, but work badly for some minority groups and put them at a serious disadvantage.
Some digital tools are designed to work better for certain groups of the population, as is for example the case with assistive technologies for students with disability or with special needs. The idea is thus not that all digital tools should always perform the same for everyone. The real problem arises when they unintentionally advantage some groups compared to others and amplify rather than reduce societal biases such as racism, sexism, anti-migrant biases, etc. While human beings have prejudices and are the origin of societal biases, machines built on these biases will replicate them in a systematic, automated way that could amplify their effects compared to human bias.
Some cases of algorithmic bias with extreme consequences were highlighted in (mainly) other sectors than education (e.g. finance, justice) (O’Neil, 2016[19]). While education does not use much automated advice to make final decisions, Baker (2023[20]) shows that educational tools have also been shown to have unintentional differing performance for diverse groups. Should it happen for decisions that matter for eligibility to certain support services, admissions to schools or universities, or disciplinary sanctions, this would be extremely problematic. This is certainly a new challenge and presents upcoming risks that countries will have to address.
Inefficiencies of a digital ecosystem
The past introduction of digital tools in education, notably computers, has led to cost-inefficiencies – simply because the computers were not used. Instances of a lack of use and a lack of usefulness of education technology have given rise to repeated critiques of education technology (Cuban, 1986[21]; Reich, 2020[22]). Many education officials perceive the COVID-19 pandemic as a wakeup call for education staff that were not always using or aware of what governments provided. At the same time, the increased use of technology in classroom instruction represents one of the biggest changes in classrooms of the 2010s (Vincent-Lancrin et al., 2019[23]), which was accelerated by the remote learning and alternate modes of schooling during the pandemic. Sometimes the lack of use can be ascribed to the quality of the digital education tools that are proposed. Education technology is sometimes designed and proposed because it is technologically possible rather than because it is useful and provides clear benefits to end users in education. Most education technology products are mere educational derivatives of solutions designed for other sectors. Even when technology applications are useful and beneficial, some teachers, learners and users may have no interest in using them. There is thus a risk that digital teaching and learning resources are available and publicly provided, but that they are not used by education stakeholders, therefore increasing costs without changing outcomes.
Another possible source of cost-inefficiencies lies in the fragmentation of the digital education ecosystem, which is always comprised of a variety of digital tools and resources. If not properly managed, this diversity may lead to an increased workload for teachers and administrators, with multiple data entries of similar information in different software. As we will see below, efforts towards interoperability can help address such sources of cost-inefficiencies.
Privacy and data protection
Digitalisation raises new issues (and costs) related to privacy and data protection. It also raises new possibilities that expose children to access inappropriate interactions or content. New privacy challenges emerge as an increasing amount of data are collected, especially when they can be linked. The challenge is exacerbated as most people post personal information about themselves on the Internet, making it easier to reidentify them from a pseudonymised dataset. As technology and service providers collect and manage increasing amounts of information on behalf of schools and education agencies, more and more data shift outside the direct stewardship of education agencies, feeding concerns that personal information about students or teachers could be used inappropriately or lead to privacy breaches. Harm arising from a privacy breach can affect individuals or communities, may be objective or subjective, and can involve economic, legal, psycho-emotional or reputational injuries. Privacy and data protection has become a major focus of digital education governance, as discussed in the next section of this chapter.
Ethics of AI
The ethics of AI in education (and elsewhere) has become a major policy concern. Ethics only really matters when there is no regulation. For any serious issue, regulation should trump the ethical behaviour of stakeholders. Over-regulating is a risk, especially for evolving and not well-known technology, but leaving decisions that could lead to serious harm to the ethics of individuals would be unreasonable.
There are two types of ethical problems raised by digitalisation. One type is about what algorithms are allowed to do. For example, where people feel discomfort in the monitoring of students’ emotional states, directly or indirectly, even if it would help to identify and address cyberbullying or support their learning, regulation is the right option. Regulation should not inhibit finding weaker ways to balance the costs and benefits, for example by mandating data deletion immediately after processing, which would avoid keeping records of emotional states while reaping the benefits of monitoring (assuming it is accurate and contributes to protecting children or improving their learning performance). Recently published guidelines and forthcoming regulation (in the European Union) address this issue by recommending or planning to limit the uses of AI technology.
A second type (enabled by the first) is about the use of AI by human beings. For example, if the identification of students at risk of dropping out from high school leads to their stigmatisation or to their expulsion from school, for example as a way for school leaders to preserve their school’s graduation rates, this would be an unethical use of digital tools as it would harm the students that the algorithm identified as requiring support. If classroom analytics designed to support teachers to improve their teaching practice could be used against them as a “performance assessment” tool, this would also be problematic ethically. The ethical challenge in these cases does not stem from the technology affordances, but from how human beings use them. Guardrails about how AI and other advanced technology should be used by humans is thus crucial to enable its beneficial uses.
Last but not least, many observers and stakeholders worry about digitalisation implying an overhaul of non-digital forms of learning. Concerns used to be expressed as excessive “screen time”, but as digital technology also involves so many other non-screen activities, the question is more about digital technology time. As education is and will remain a portfolio of very diverse educational activities, it is difficult to imagine an education that would be only digital and that would not allow students to develop perception and knowledge through the direct experience and use of their five human senses. A digital transformation of education does not imply that all educational activities would become digital, as the opponents of digitalisation sometimes claim. It is difficult to pre-define how much digital devices should support education, but for sure formal education should have students maintain an ongoing engagement with their peers, local communities and the natural world, without the mediation of technology. Framing the problem as a choice between two exclusive options is unhelpful.
Social acceptance
Challenges for a digital transformation of education are partly technical, as mentioned above. However, probably the main overarching challenge is societal. Education policy makers, teachers, parents, and even students, are used to an education standard with very little to no technology. One implication of a digital transformation is that some current practices, which have sometimes taken several decades to become accepted as a fair and normal practice, will be challenged.
An example lies in adaptive assessment. In some countries, parents, teachers and their representative organisations, as well as students, pushed back against the introduction of adaptive assessments. As an analogy, ophthalmologists diagnose which glasses people should wear with adaptive assessments: with the support of their machine, they ask a series of questions to fine tune their diagnosis and (hopefully) provide the right prescription for lenses. Not every patient gets the same questions as it depends on what and when you start seeing things, seeing them blurry, etc. Adaptive assessments do more or less the same with mathematics or reading: they try to provide more fine-tuned assessments by providing questions and exercises that get closer to what students know and understand. As the current standard for a fair assessment is that all students take and are assessed on the same questions, adaptive assessments were considered unfair by society.
While opposition to any use of technology in education (while it is widely accepted in other sectors such as the health sector) should be challenged, naïve endorsement of technology should also be questioned. While digital technology presents many opportunities for the advancement of educational goals, it is neither a panacea nor a poison. Significant challenges for governments lie in transparency about its uses; co-creation and negotiation of its uses; and communication about its benefits and how its potential pitfalls are addressed.
Given the opportunities and challenges laid out before, governments and other stakeholders willing to foster the digitalisation of their education systems have several tasks at hand. The first is to improve their current digital education ecosystem. This section provides a brief overview of the findings presented in the book. The next section will consider the second task of governing digital education to address the challenges and enabling the benefits of a digital transformation of education.
Digital education ecosystems are hybrid human-technology systems. They consist of a mix of human competences, hardware and connectivity, and two types of software: digital tools for system and institutional management, and digital resources for teaching, learning and assessing in the classroom. The pandemic raised a big question: what is the minimal infrastructure a country should provide to its schools, teachers, and students for learning to continue in case of a disruption, but also generally speaking? It also made visible the gap between what would be possible to make education more effective and equitable if teachers and students were augmented by digital education tools, including AI tools, and what countries, educational authorities or schools currently provide.
Having a robust physical digital infrastructure is a pre-requisite for digital education. High quality connectivity and enough quality devices for students and teachers are a moving target that requires continuous investment. Improving the quality of connectivity in school and in their country as well as the availability of digital devices is a priority for many OECD countries (Fragoso, 2023[16]). While these efforts are essential, just providing digital devices and good connectivity will not lead to a digital transformation of education.
Assuming countries’ digital hardware is of good quality, policy makers should consider two big questions:
1. what are the digital tools and resources that could support effective teaching and learning in the classroom and help achieve some of their educational goals (such as making education more inclusive and equitable, making teaching a more attractive profession, providing a holistic education, etc.)?
2. starting from their policy objectives, what digital education ecosystem should they try to build and, in particular, how can they reuse and share collected data so that it helps achieve these policy goals?
In short, the digital ecosystems of educational tools and resources as well as the availability and use of the overall data infrastructure within their education system should be a main concern. Should it not have quality hardware, they should work on improving it and, in the meantime, analyse how they can use the current infrastructure despite its shortcomings.
Let us imagine a country that would want to reduce high school dropout. Policy makers could communicate this policy objective and let education practitioners address it without any technology. Using technology and data collected through their digital infrastructure, they could think of different ways to support education practitioners (who would still have to act at the end of the day). A first way is to create digital tools with reporting systems for absences that trigger a human intervention (e.g. some people go to students’ home to see if they can reengage them). This is a reactive approach: technology enables a faster response than before. Another, more proactive approach is to try to detect the possibility of drop out before it happens (and intervene pre-emptively). This is what early warning systems try to do. What does it take to do so? Typically, countries need to have data about students who dropped out in the past as well as data about their current students – and based on what they collect, they have to figure out how to design strong early warning indicators and then make the relevant data available to teachers and school leaders in real time. A possible challenge in that scenario is that the relevant data to anticipate a dropout risk may be held in different digital systems (e.g. attendance and teacher-given grades in the school system, standardised scores in the national evaluation system, and information about the school or family characteristics in the jurisdictional information system), so that “early warning” is only possible if the relevant data can be accessed, linked, and brought back in a timely manner to the relevant stakeholders dealing with the students at risk of dropping out. A third approach, which can (and often has to) supplement the two first ones is to commission research using the data collected about dropout to better understand who drops out and under what circumstances. While this helps improve policies and better understand the phenomenon, this typically does not help an individual student in real time.
An important message of this hypothetical story is that administrative systems can serve other educational purposes than the administrative processes for which they were designed if the data they collect are used for these other purposes – and if the human policies to reach these benefits are developed (OECD, 2023[24]).
System- and school- level digital management tools
This book provides a comprehensive overview of countries’ digital infrastructure to manage schools and education systems.
The cornerstone of a digital education infrastructure at the system level lies in a longitudinal student information system. Student information systems collect information about the trajectory of each individual student in the system and thus provide the possibility to make the entire education system benefit from information that is gathered at the national/jurisdictional level. This is an important digital tool to turn data into actionable information for local stakeholders in real time. As of 2024, most OECD countries have established a longitudinal student information system, but they still use it mainly for statistical purposes rather than as a way to provide real-time information to stakeholders. A second best is to have a central student register with unique longitudinal identifiers for students (and possibly teachers). The information gathered will allow for generating research evidence that may inform education policies within countries – but cannot quickly be turned into action for individual students (Vincent-Lancrin and González-Sancho, 2023[17]).
Current longitudinal information systems can be described according to four ideal types:
The reporting and research approach. The longitudinal data has allowed countries to enrich their performance cards, but the data and reports produced mainly seek to support policy planning and to inform the public. In some cases, data systems also intend to develop research capacity about educational issues. Most student information systems not only fulfil this function but are also limited to it.
The e-government approach. These student information systems were designed to improve the efficiency of administrative processes (e.g. school transfer, school choice, university application, funds allocation to schools). They contain more data and linkage possibilities than other models, but have a weaker focus on functionalities aimed at improving teaching and personalising education and on reporting learning data to teachers.
The school improvement approach. Putting school improvement at the core of their mission, they can be close to information systems in the reporting and research approach, but typically report data to schools, generally with a visualisation tool. They try to provide information at the individual level and with a granularity that makes it useable by teachers (for example, item-level reporting of assessments).
The expert system approach. Inspired by “expert systems” supporting decision-making, they typically provide rapid and granular feedback to teachers, students and principals, as well as support materials to enhance learning. Beyond mere reporting, they have predictive models and make recommendations, which may be followed or not.
While student information systems are probably where to start for countries establishing a digital education infrastructure, as the digital ecosystem matures, they may become one of its components and may not have to play a central role. While countries should try to move from a mere “reporting and research” to an “expert approach”, this may not have to be a characteristic of their student information system but rather of their entire digital education ecosystem. All depends on the possibility to share and exchange data between the digital tools within the ecosystem (interoperability). Having a robust system-level student information system enables the possibility to receive and send back system-level information to stakeholders (as appropriate, given certain objectives) – this can also be done in different ways, but with usually less ease, which is why it is initially the most importance piece of a system-wide digital education ecosystem.
Countries’ digital education ecosystem is comprised of many other system- and institution-level digital management tools. Learning management systems are the equivalent of student information systems at the school/institution level: they allow schools to manage and track information about individual students, which classes they attend, with which teachers, and, in some cases, to access digital content for teaching/learning. Ideally, learning management systems should be able to “push” and “receive” data to and from their jurisdictional student information system. While most countries report that most of their schools use such learning management systems, at least at some educational level, about half of them are not interoperable with system-level student information systems and require schools to manually provide information to their public authorities/ministries, and in turn are unable to receive any insight from the data collected at the jurisdictional level (Vincent-Lancrin, 2023[25]).
The report shows that most countries provide study/career guidance information through digital means, even though few of them provide tools for more personalised enquiries, and that most national evaluations are digitised or in the process of being so. Digitising actual high-stakes exams for students is a different story, and while some OECD countries are exploring this path, only a few of them have done it (Finland is an example). A few countries have digitised some aspects of the administration of their paper-and-pencil exams as well as their selective admission processes into higher education (and sometimes high school) (Vidal, 2023[26]).
Figure 1.1 provides a picture of the public provision (and use) of digital system- and institution-management level tools.
Number of countries who publicly provide the following system- and institution-level management tools at national or sub-governmental levels
Note: Student Information Systems (EMIS) are available at the central/national level in 19 countries/jurisdictions, and in most subnational levels in 3 more countries/jurisdictions.
Despite ongoing discussions about AI in education, it is noteworthy that relatively few system- and institution-management digital tools use any AI technology such as learning analytics or recommendation tools. The most advanced uses of technology consist of making information available through dashboards or of implementing rule-based algorithms, notably for funding mechanisms or for managing enrolments in or applications for schools.
Number of countries/jurisdictions who publicly provide digital systems with dashboards or some level of rule-based automated recommendations
Note: N=29.
Figure 1.2 shows that in education it is relatively infrequent for digital tools to use even non-AI based algorithms, such as dashboards, or rule-based decision-making or adaptive models. Almost no country reported the use of AI techniques for system-level tools or a common use of them for institution-level tools. There is virtually no “automated” decision- or suggestion-making in education. For example, while most countries have digitised their national standardised evaluations, if not their examinations, almost none use the digital affordances of computer-based testing (such as the use of videos, simulations or adaptive testing). While most countries provide study and career guidance platforms for students, those have relatively little adaptive functionalities that would allow to individualise the study or career suggestions.
Digital ecosystems for teaching and learning
A second question is about the digital learning resources that are accessible by teachers and students. This is another issue that the pandemic has made salient and that led to many new promising initiatives within countries. What is the minimum level of digital learning resources that should be available to any student (and teacher) in an education system? Given the new possibilities of digital education, what are the next-generation digital learning resources that would help students succeed?
Let us think again of a country where preventing school dropout is a priority. Beyond the use of data collected at the system level to power early warning systems and trigger interventions, there could be many instructional ways to keep students engaged in school. Some countries and jurisdictions may be tempted to invest in tools that will help students succeed and ensure they can continue to learn or practice while they are out of school. This could be adaptive learning systems for example. They could also be interested in software that help students remain engaged in their learning. They could provide a variety of resources that will help students find what they are interested in and support teachers in developing their students’ skills in this area, even if not in mathematics and literacy. They could provide teachers with resources to better understand what is of interest to their students, to design more easily engaging lesson plans, etc. All this would assume that an engaging education in subjects of interest to the students would help keep them in school, especially if they are supported and successful. This is already what teachers do, but digital tools can help them to diversify and individualise their teaching. A country making this assumption may want to have a digital ecosystem with some of the teaching and learning tools and resources mentioned above.
As shown by Figure 1.3, most countries are now involved in the provision of digital teaching and learning resources for both students and teachers (Yu, Vidal and Vincent-Lancrin, 2023[27]).
Note: N=29.
The OECD, alongside other international organisations such as UNESCO, have long encouraged countries to develop platforms of open educational resources (OERs). OERs have the advantage of being available free of charge to everyone and thus can support not only students and teachers, but also the general public and lifelong learning. MOOC (massive open online courses) platforms have also expanded that offer, as is the case for TV and radio education and social network channels in selected countries. Most of these offers were boosted by the COVID-19 pandemic and remain available in some countries. Some of these resources are mapped against the national or jurisdictional curriculum. While open educational resources are important, notably to provide an equal baseline to all, the risk is that they get outdated if not continuously updated and modernised. When curated by governmental authorities, one can expect them to be of good quality. In many countries, governmental platforms are supplemented by non-governmental OERs developed by teachers, non-governmental organisations, universities, etc.
Another way for countries to support teachers and students is to license digital learning and teaching resources from commercial education publishers or to enable municipalities or to procure their digital learning resources from them with their public budget. In this case the digital learning resources and tools are provided on a closed access basis, that is, only students and teachers with a recognised role in the education system will be able to access them.
The advantage of a central provision is that central governments have in principle more capacity to quality assure resources and that the resources will be available for teachers and students in the entire education system, regardless of the preferences of their school leaders or choices of subgovernments. Where there is a very uneven provision of digital learning resources and tools, this can be an effective way to level out the playing field. The possible disadvantage of central provision is that the provided resources might not be used. Schools or local governments may be better placed to choose what suits their students. In any event, while having a baseline of “free of charge” or open resources is important to allow all citizens to benefit from public education, private providers remain overall better placed to keep learning resources up to date and should certainly remain part of the public provision/procurement equation.
As of 2024, the majority of digital learning and teaching resources provided by public authorities and used in the classroom remain static, such as (non-interactive) digital textbooks, video content, and past exam questions, which may often merely transpose conventional chalk-and-board teaching methods to a digital format. Static digital resources are useful and will always keep a role in the education process, as is the case for physical, non-digital resources. However, the lack of engagement with AI-based digital learning resources may be a missed opportunity to provide more individualised teaching and learning. Most digital learning resources provided and used in OECD schools are non-adaptive. Interactive digital textbooks are the most widely used “advanced” digital learning resources: they are more interactive and include exercises related to the learnt lessons, etc., but they are still typically not adaptive. Intelligent tutoring systems, which could allow students to overcome some of their misconceptions and master procedural knowledge, are still rarely available and used within countries – not to mention other types of smart technology (OECD, 2021[1]). Most AI in education seems to mainly consist of the use of generative AI, a general purpose AI that is used in practice in all countries and jurisdictions, even if not in the classroom.
Note: N=29.
How do these digital teaching and learning resources fit in an effective digital ecosystem? Each of these digital tools and resources have a value and function for teaching and learning, and fragmentation is not necessarily an issue. It is easier for students and teachers to access those tools and resources without multiplying their access to different platforms, so that being accessible from their school learning management system is an advantage. This is not always possible as students and teachers may have to access them through the platform of a commercial provider (when not provided directly by a public authority). An increasing number of countries offer “single sign-on” solutions to avoid that students manage several access codes (and to protect the privacy of students from the vendors).
As the most advanced digital teaching and learning tools typically collect information about their use by students (and teachers), one could imagine that at least some of the data they collect with public funding and often in public schools could be reused and connected to the overall education data ecosystem. It is also possible that the data collected by one digital learning tool could have value for another tool, which would make it valuable for them to be able to exchange information. In what cases this could be useful and how this could be achieved still need to be imagined.
Digital competences
As mentioned above, a strong digital education ecosystem is a hybrid human-machine system and encompasses students and teachers who are able to use the digital tools and resources at their disposal, provide feedback for their further improvement or competently enforce digitalisation-related regulation. This is also true for school leaders and education administrators as system- and institution-management digital tools get increasingly used. There is no point in providing digital resources that are not effectively used by teachers and students, who should be considered as an integral part of a digital education ecosystem. It is noteworthy that digital competences are just partly about having the skills to use digital devices or find digital resources. These technological competences are important, but digital competences mainly refer to the ability of teachers to use digital tools and resources in their teaching, including advanced technology such as AI. Many countries increasingly emphasise “AI literacy” as an objective for teacher professional learning, which includes both the understanding of the basic functioning of AI models and tools and the use of AI tools (such as specialised educational AI tools or general-purpose tools such as textual or pictural generative AI).
Countries incentivise teachers to develop their pedagogical digital competences in different ways. Most countries (24 out of 29) have some national rules or guidelines on teacher digital competences, but significant differences exist across countries: 14 countries have rules about pre-service teachers compared to only 3 countries for in-service teachers (the latter are more likely to be devolved to lower levels of government). Most of the rules for pre-service teachers are standards that guide teacher training programmes in designing their curricula: their enforcement may be checked when accrediting or recognising those programmes or, more rarely, tested or verified as part of teacher certification/licensing or hiring. Those pre-service standards are often seen as guidelines for in-service teachers, meant to indicate where to put their professional development efforts. As mentioned above, many countries do provide their teachers with digital learning and teaching resources that also encompass the use of digital tools and resources as part of teaching. In general, such rules and guidelines remain high level and as few countries proactively enforce standards by evaluating teachers’ digital competences or linking accreditation processes to the development of digital competences, one may wonder how effective they actually are.
Note: N=29. In 15 countries out of 29, there are regulatory requirements about digital competences to enter the teaching profession, and in 3 countries to maintain those competences while in service. In 19 countries, teachers are incentivised to develop their digital competences by the integration of digital competences as a transversal competence in the student curriculum.
Almost all countries have national rules and guidelines on developing student digital competences. In most cases, these refer to all educational levels (or all levels excluding VET) and these are often integrated across the curriculum. Expecting students to develop digital competences as part of their mandatory education and across all subjects implies being taught by teachers with their own digital competences. However, very few countries formally assess student digital competences, so the incentive structure is also weak.
Countries and jurisdictions could consider different ways to provide stronger incentives for teachers to develop and maintain their digital competences. They could formally assess teacher digital competences, for example as part of teacher qualification processes (e.g. through examinations), as part of teacher trainee evaluations, as part of compulsory or voluntary certification processes, or by strengthening their relevance as part of internal teacher appraisal or external school evaluation criteria.
Where not already the case, countries could link their teacher digital competence expectations to concrete accreditation processes of higher education teacher training programmes (that could also include criteria on the assessment of digital skills). This could help to ensure relevant content in initial teacher education and promote equal opportunities for student teachers to develop their digital competences, while still affording higher education institutions (and other teacher training institutions) with flexibility and autonomy.
Building strong incentive structures to encourage participation in relevant professional development activities as part of career advancement paths can be a powerful way to encourage teachers to maintain and (further) develop their digital competences. These might include new reward and mobility structures within the teaching profession that reflect digital skills, including both vertical (i.e. promotion) and horizontal pathways (e.g. specialised digital roles that come with recognised concessions in teaching responsibilities). Formally recognising digital skills development, for example through certification or micro-credentials, can also incentivise professional development for motivated teachers, although such incentives are unlikely to be effective unless paired with some formal exemption or fulfilment of professional development obligations that matter for career progression and/or compensation (Foster, 2023[28]).
Developing a governance of digitalisation to shape an effective and equitable digital transformation requires focusing both on how to enable the digital transformation and on how to mitigate its risks and challenges. Innovation or digitalisation is not an end in itself. It has to be a means to achieve specific educational objectives: personalisation, inclusion of students with disability or special needs, social diversity in school, etc. The first important step is for countries to identify those purposes and how digital technology as well as a robust data infrastructure could help achieve them, if possible. While most (23 out of 29) countries have published a new or updated a former digital education strategy since 2020, most of these strategies are not structured so much around educational objectives and how they can be achieved using digital tools, but more around big topic areas (digital competences, infrastructure, teaching and learning resources, etc.). A digital transformation of education will require countries to identify more specific purposes of digitalisation.
Once those are specified, governing digital education includes providing access to a digital ecosystem that allows for these objectives to be achieved, that empowers education actors to use digital tools confidently and competently, where trust about the use of personal data is created thanks to privacy and data protection laws and support for relevant staff, that mitigates digitally-induced inequalities and addresses possible systematic biases, and that creates incentives for edtech developers to continue to develop useful and affordable digital tools and resources for education. Several policy levers can be activated for these purpose: incentives to foster interoperability within the system, setting in place risk-management approaches to privacy and data protection, using public procurement, and creating institutions to facilitate the implementation of digital education policies. Rather than being thought of as addressing one specific issue, they can be used to reinforce incentives and address multiple problems.
Interoperability
Interoperability is the capacity to combine and use data from disparate digital tools with ease, coherence and efficiency. It increases the consistency and exchangeability of data collected and maintained by different systems. It reduces the need for ad-hoc processing to re-input, re-format or transform data, so that relevant information can be delivered in a more cost-effective and swift manner to support actions and decisions. In the absence of interoperable digital tools, data linkage and sharing may still be possible but become error prone and time and resource consuming tasks. Interoperability is thus a way to improve efficiency, but also effectiveness of digitalisation (Vincent-Lancrin and González-Sancho, 2023[29]).
Some of the examples of personalisation above require that different systems be able to exchange information. For example, it may not be a problem for standardised assessment evaluations to be stored in a different platform than student information – if those systems are able to communicate and share information easily. If not, it is better to have all the information in the same system (typically the system-level student information system).
At a system level, interoperability requires a widespread adoption of shared standards, including technical specifications for technology tools and applications, data definitions and code sets, and general models for system architecture. In some cases, it may also require a greater alignment in organisational processes and a legal framework supporting legitimate and innovative ways of using education data.
The transition from a fragmented to an interoperable educational technology and data ecosystem builds on some important policy dimensions. These include dealing with legacy systems (that is, the fact that at any point of time an ecosystem encompasses technologies developed at different times and using different standards), increasing awareness of the benefits of interoperability, putting in place an effective mix of incentives and mandates for the adoption of standards, ensuring sustainability and capacity to adapt to changing needs, and taking advantage of international initiatives in this area.
This book highlights many interesting initiatives to enhance interoperability at the country level. It also shows that this is an area where further efforts are needed. While it is difficult to have precise data without a representative school survey, a minority of countries have rules that mandate some interoperability standards (usually with one or more of their system-level administrative systems) or that require some semantic interoperability for digital learning resources. De facto, in less than one third of the reviewed countries do government officials report that most school learning management systems used by schools are interoperable with system-level management systems or other institution-level digital tools. Should a fully effective digital education ecosystem require interoperability, there is still much progress to make.
While regulating technical interoperability is not necessarily a good idea, there are increasingly technical solutions that can facilitate interoperability. Much more can be done by governments on semantic interoperability, both for administrative data and for digital learning resources. About two thirds of countries and jurisdictions recommend the use of some taxonomy for tagging learning resources, but further effort for developing international standards on content (rather than type of resource) could be made.
Note: N=29. The left panel figure shows how many countries use rules or guidelines to encourage different types of interoperability. Ten countries mandate that some systems use specific technical standards (usually to be interoperable with system-level digital systems), while 8 encourage it through guidelines. The right panel figure shows countries where school learning management systems are most commonly interoperable with either system-level digital tools or institution-level digital tools.
Another option to achieve the same result is to integrate most digital systems into one. In some (usually smaller) countries, some digital tools do integrate most of the functionalities of a typical digital education ecosystem (with the system-level student information system also being a school learning management system, a digital resource platform, etc.). These digital tools are usually directly provided by central governments. They have the advantage of allowing easy data linkages and interoperability, but possibly the disadvantage of not providing choice to schools – and incentives for a local EdTech industry. While it can be argued that an integrated system could be easier to manage from a data protection perspective as fewer, possibly more expert people can take care of the issue, the main vulnerability lies in cyber‑security and privacy and data protection (given that all data are held in the “same” place).
Data governance
Strong privacy and data protection is indeed another strong enabler of the digital transformation, both to address objective risks of digital transformation and to create trust in data and AI use in education. Trust in the handling of personal data is necessary to enable the safe and legitimate sharing of data – a necessary condition to harness the opportunities of digitalisation. Privacy and data protection is a multifaceted issue. Two dimensions are considered as a given in this book, but they require strong technical competence and infrastructure and human competence and vigilance: making sure that personal data are protected and not easily hacked (cybersecurity) and making sure that children and students are not exposed to inappropriate content or interactions when using government tools or resources in school (Hooft Graafland, 2018[30]; Ronchi and Robinson, 2019[31]).
Many of the major benefits that a digital transformation of education can bring about rely on the promise of more personalised educational experiences and a stronger knowledge base to design education policies and practices. This usually requires using records stored in silos (if they have shared identifiers). While the sharing and linking of personal education records across different technology tools are central to realising the benefits of digital education, interoperable administrative and instructional systems bring greater privacy and security risks than disparate systems.
Privacy and data protection laws and regulations are essential to prevent privacy breaches and illegitimate uses of personal data. All countries have a privacy and data protection law that applies to education. Most countries that have longitudinal information systems also have specific educational data protection laws or regulations. A risk management framework that recognises a diversity of uses of personal education data, their potential benefits, and their associated privacy risks is best suited to reconcile legitimate privacy concerns with the benefits of using education data to improve educational outcomes.
An important step in the implementation of such a framework is to break away from the expectation of fully eliminating risk in the use of education data. As long as there is an interest in maintaining some analytical value of the collected data, scenarios with zero privacy risks are unrealistic. Another required change is to shift the focus from privacy controls at the stages of data collection and transformation, to a growing emphasis on controls at the stages of data access, sharing and use. Privacy protection should make use of complementary data-focused and governance-focused strategies: data-focused strategies consist of treating data prior to their release or sharing, while governance-focused strategies restrain the interactions of custodians and users with the data both by regulating the conditions for data access and use and by increasing awareness and capacity to address privacy risks. While most countries have published guidelines on the enforcement of their privacy and data protection rules, very few proactively monitor their implementation in school. Privacy awareness campaigns and training programmes have been increasingly implemented as ways of strengthening human safeguards for maintaining the confidentiality of personal data. In Europe, for example, the General Data Protection Regulation requires national data protection authorities to carry out awareness-raising activities for data controllers, processors and individuals, with a special attention to children (Article 57). Providing privacy and data security training to those can help to build a culture of privacy-respectful data use and enhance trust when data are shared.
An increasing number of commercial service providers collect data about students and teachers in formal education. The way they handle data is usually covered by countries’ privacy and data regulations, with specific additional restrictions when handling children’s data. One aspect that remains largely out of sight of current education policies is the possibility to reuse and share some of the data collected by commercial providers – as is the case for data collected by public educational agencies. Many sectors try to incentivise companies to share some of the data they collect (under data protection laws) to allow for a more vibrant supply of digital tools and resources and more innovation. For example, some of the process information collected by adaptive learning systems might have value for other companies and organisations and allow for the quicker development of new types of digital teaching and learning tools.
Technology governance
As advanced technology allows for more automation and systematic impact on human decisions, or for the capture of increasingly sensitive data such as biometric data, the governance of technology itself is becoming a new concern. Technology governance could, for example, consist of setting some obligations when using automated decision-making, forbidding certain types of technology or technology use, requiring the disclosure of the use of automation, requiring that algorithms are explained or that they are “open” and can be examined by experts, etc.
As of 2024, almost no OECD country regulates technology or algorithms used in education. The only case is France, where algorithms used in public decision-making should be explainable and explained and where certain technology uses are forbidden under normal conditions. Typically, the educational organisation providing digital tools or resources are responsible for their results, but as of 2024 no country reported any use of unsupervised automation in education, let alone the use of high stakes decision making. The emergence of generative artificial intelligence has led to countries publishing a number of guidelines, and two countries have rules pending adoption regarding their use in education (France and Korea) (Vidal, Vincent-Lancrin and Yun, 2023[4]). The European Union is also close to passing an AI Act that will regulate the use of AI tools, making some uses illegal and the use of AI tools in some “high risk” sectors such as education undergo specific processes. Most countries deal with advanced technology with guidelines, and a few countries have published some in the past few years.
One common aspect of those guidelines is the need to keep a “human in the loop”. As AI allows for more automated decision-making to happen, this means that while some recommendations or suggestions could be made by AI, human beings should ultimately make the final decision. This is particularly important when AI tools do not have a perfect performance. Most of the time, this is the current situation in education, but the rule can avoid pitfalls. This rule may also mean that a non-digital alternative should be provided, when possible, both for inclusion reasons and to allow the possibility for people to “opt out” (when possible and appropriate) (Vincent-Lancrin and González-Sancho, 2023[32]).
A second aspect of technology governance lies in the avoidance of algorithmic bias, which is particularly important in education. Algorithmic bias refers to cases where an algorithm advantages (or works better for) some populations compared to others (whether the characteristics relate to gender, race and ethnicity, migration status, etc.). The potential of digital education cannot be fully reached if algorithms that may for example support the personalisation of education replicate or even magnify the biases occurring in societies around the world. Research on algorithmic bias focuses on the performance of AI models for different groups of the population. There are other possible forms (and sources) of bias though. Research on algorithmic bias has mainly been undertaken in the United States so far, including for algorithms and systems operating outside of the United States. It has shown the existence of algorithmic bias based on a variety of student characteristics, but the lack of international research limits the understanding of bias. Policy makers should fund research internationally to better identify the various dimensions of bias in different local contexts. Ultimately, they should support the development of toolkits that would make it cheaper to identify bias. An important take-away is that privacy and data protection should take into consideration the need and importance to collect personal (and sometimes sensitive) data to be capable of detecting (and thus addressing) algorithmic bias and unfair technology. This could be done under a variety of arrangements (Baker, Hawn and Lee, 2023[20]).
Procurement
Given that education systems in OECD countries are mainly public, public procurement is a very strong lever to incentivise commercial service and product providers to follow certain guidelines or rules. In an OECD country, public procurement in education represented on average 10.7% of all public procurement in 2021, or 1.4% of a country’s GDP. This is considerable. While the share of digital tools and resources in educational procurement is unknown, one peculiarity of at least some digital devices or tools is that they can be more expensive than usual education materials such as textbooks and benefit more from an aggregated price negotiating power.
Countries already use public procurement as a policy lever to shape their digital ecosystem and foster data protection and security, interoperability, inclusivity and, to some extent, effectiveness. Some countries could however do it in a more proactive way. Countries use multiple, non-exclusive procurement practices. In most countries, governments play a role in the procurement of digital tools and resources, whether for management or teaching and learning. All countries procure digital system-level management tools. Some countries have a mainly centralised approach to procurement (e.g. Czechia, Hungary, Korea, Türkiye) while others leave it to schools to purchase most of their digital educational services and resources (e.g. England, Netherlands) or have a mixed approach (e.g. France, New Zealand). Central procurement supports providing equal access conditions across schools in a country/jurisdiction, greater price negotiation power (thanks to economies of scale), and, in principle, the possibility to rely on more technical competences. More decentralised procurement practices may enable schools (or local education authorities) to choose digital tools and resources that meet their specific needs, even though it might entail higher purchase costs.
For companies, decentralised procurement provides less incentives as it requires a larger sales force, makes the procurement process more varied and complex, and allows less possibilities to scale their offer. This makes entering the market more difficult for small firms. At the same time, a centralised procurement process may limit the number of companies and possibilities to enter the market. Mixing different approaches is thus appropriate for governments to balance access and cost-efficiency, and control of the quality of what is bought with public funds against the provision of market incentives for Edtech suppliers.
Without making the final purchase decision, countries and jurisdictions support their schools and sub-governmental authorities in their procurement through different mechanisms (Figure 1.7). For example, 15 countries negotiate prices with suppliers for some tools; 9 pre-authorise a list of tools and resources to choose from, which gives them the possibility to verify the quality and effectiveness of the resources if they so wish; and 7 countries grant permission on a case-by-case basis, thus allowing for more choice. By attaching product and service criteria to public procurement, governments can help foster a coherent digital ecosystem. As of 2024, 8 countries mandate the public procurement of digital tools and resources according to predefined characteristics: usually tools must meet specific cybersecurity criteria, and less often, interoperability or ecological (sustainability) criteria (Vidal, 2023[33]).
Note: N=29. 15 governments or subgovernments negotiate prices with commercial vendors, 9 have a list from which schools or lower public authorities can choose from for their procurement, etc.
Countries could decide to enact more stringent rules for digital education tools to be put on the education market, beyond data protection and cybersecurity standards. They could require tests of effectiveness, the verification of the lack of algorithmic bias, set interoperability requirements with some specific tools, the use of predefined resource taxonomies when appropriate, etc. However, regulation is not always the best option, and they have to also balance these rules against the incentives for commercial providers to develop digital tools and resources. Despite being a large market, education is not always considered a highly profitable one by technology companies, which tend to focus on the education consumer market rather than the formal one.
Co-creation and multi-stakeholder relations
While one challenge for governments is to ensure that commercial providers have enough market incentives to develop quality digital tools and resources for the education sector, another challenge is to ensure the quality, effectiveness, and usefulness of these tools. Traditional education stakeholders usually do not have the competence to develop digital tools for the education sector. Typically, those are developed by for-profit education technology companies, sometimes specifically for the education sector, often by adapting tools that were developed for other sectors to education. It is rare for education ministries to support commercial companies directly, although education technology may benefit from governmental innovation programmes (e.g. for startups or for research and exploratory development). While a few governments support their education technology industry from an international trade perspective, many engage in a dialogue with them by supporting conferences, etc. Education authorities also collaborate relatively rarely with stakeholders such as parents or students when developing or introducing new digital tools and resources.
Acknowledging that education and computer scientists, education technology companies and governments often work in silo, with relatively little involvement of the teaching profession in the definition and development of AI products, new models of research and development of digital technology should arguably be developed. Several models of research and exploratory development are supported by governments that attempt to involve end users more and take an interdisciplinary approach to the development of AI tools and resources in education (Molenaar and Sleegers, 2023[34]). A first approach reflects and adapts a linear approach to innovation: scientific research is translated and transformed into product development and market applications. An increasing number of initiatives work with end users, although not necessarily from the beginning. A second approach focuses on industry development to help start-ups to improve products (propositions) with scientific insights and enhance the ecosystem for companies to thrive and scale up. This model resorts to diverse business development activities, from supporting prototype development, optimising products in multiple schools, diversifying to new sectors in education, through to validating the effectiveness of products to support an evidence-based development of Edtech tools and resources in schools. Other approaches emphasise international collaboration, with similar coordinated projects across countries, or the relationship with teachers and teacher professional development.
For example, the National Education Lab AI (NOLAI) in the Netherlands starts its development process by questions and needs from educational professionals, which are addressed based on scientific insights and recent industry developments. For example, the “happy readers” project started with a request by primary school teachers to be able to better monitor how students’ technical reading skills develop over time. Based on what university scholars and industry partners know about reading research and current affordances of technology, such as automated speech recognition algorithms, they developed a new approach to digitally-enhanced reading education.
One of the main purposes of this co-creation would be to develop digital technology tools and resources based on teachers and learners’ needs and uses rather than on what is possible given the state of technology.
Support organisations
Governments should not just fund or establish new types of R-D institutions to allow for multi-stakeholder partnerships in tool and resource development. One of their challenges in implementing their digital strategies lies in the difficulty of ensuring staff have the digital competences to deal with the physical digital infrastructure, with digital tools and resources for management, the pedagogical knowledge regarding digital learning resources and tools, and the ability to support teachers in developing the pedagogical competences to embed the use of these tools and resources in their teaching repertoire. These activities require different expertise than those traditionally available at national or sub-national ministries of education since multi-stakeholder partnerships need to be established, managed, and monitored.
What can we learn from the recent history and transformations of support organisations that countries have established to support the digitalisation of education? Dellagnelo (2023[35]) analyses the advantages and disadvantages of different models, and shows that, while driving the digital transformation from within the ministry of education could be the best solution in some countries, many have opted to externalise this function to a public (and sometimes private) agency as it provided more flexibility to bring the required competences together. The roles to be performed by support organisations need to be adapted to the national context since they may vary considerably according to the size of the country, the level of centralisation of the education system, and the existence of other public and private actors in the digital education ecosystem.
To go from the publication of a digital strategy to its actual implementation, governments will need to establish the necessary organisational structure, which will typically mobilise and adapt existing institutions and agencies. However, while there is no foregone conclusion, they may also consider the creation of a specific external support organisation. The trajectory of past support organisations shows that it may be advantageous for them to start by being external before being integrated in the ministry – but all depends on the actual activities involved in the digital strategy and on the competences and capacities already present within the ministry of education.
Monitoring
Finally, while governments sometimes commission research on digital education to their universities or place digital education as a clear priority of their research agenda, it is striking that very few countries actually monitor and evaluate their investments in digital education tools and resources. Information about the physical infrastructure available in schools is missing, not to mention information about uses of digital technology, either as a management tool or as a teaching and learning tool. Nor do countries typically have any research assessing effective versus less effective uses of digital technology at the system, school and classroom levels. It is time for a research effort in this area (OECD, 2023[36]).
This overview of the findings of the OECD Digital Education Outlook 2023 shows that countries have made good progress in digitalising their education systems but that most are embarked on a journey towards a digital transition rather than a digital transformation. Most countries now maintain longitudinal student information systems, which they use to produce education statistics mainly, and other system-level digital management tools that support their educational processes: alert systems to enforce compulsory education, digitalisation of exam administration (but not of the exams themselves), digitalisation of national evaluations, etc. They also provide or support the provision of digital teaching and learning resources through a variety of platforms or support services for school procurement. And they encourage the use of digital tools and resources by providing direct training and support to education stakeholders, by establishing digital competence standards for pre-service teachers and by making students’ digital competences as a transversal objective of their curricula.
However, most of them do not take advantage of the current affordances of advanced digital tools. Very few AI-based educational resources are available in the classroom, and in almost all countries, despite not being designed for educational purposes, AI text generators are the only AI tool that is commonly used by students, with or without the blessing of their teachers. Adaptive learning systems, adaptive assessment systems, adaptive study or careers guidance, and early warning systems are absent from most OECD education systems. Regardless of AI-based digital tools, digitalisation leads to the collection of a significant amount of data across education systems: while those data tend to move up to the national or jurisdictional level, there is much less effort to make this information actionable and used by teachers, students, families, etc.
So far, the governance of digitalisation has mainly focused on avoiding (some of) the possible pitfalls of digital education rather than enabling and unleashing its potential. Countries could take a series of steps to focus on that. Beyond a stronger awareness of the digital education tools and resources that are already available and could be used in their education systems, they should focus on an incremental improvement of their educational processes:
1. Identify use cases. How could digital solutions help achieve some of their education policy objectives? In this chapter we took the example of preventing high school dropout, but there are many other educational goals that digitalisation could help address. What kind of data collection would it take to improve existing interventions about them? Are these data already collected somewhere? How could they be brought back in a timely manner to the right end users?
2. Improve student information systems or their use. Countries that have not yet established a longitudinal information system should consider doing so. Those systems are more effective when schools also have learning management systems that can automatically exchange information with system-level digital tools. While this can take several forms, a major avenue for improving their use is to give back the information that is collected at the education system level to practitioners, in a format that can easily inform their decisions and their thinking.
3. Develop initiatives to enhance the interoperability of the digital education ecosystem. A major way to make a digital ecosystem effective and used is to improve interoperability so that data do not have to be re-entered multiple times and that data collected for different administrative and learning purposes can be reused (under usual privacy and data protection regulations). Improving interoperability is difficult as digital technical standards evolve based on research and development rather than administrative will, but it can be done more easily for semantic interoperability. This is also an incremental enterprise that does not require full interoperability among all digital tools in an ecosystem.
4. Use public procurement practices as a policy lever. Countries already use public procurement in education as a policy lever. However, many procurement practices follow the lines of traditional devolution of responsibilities as if those were set in stone. While interoperability can be achieved in contexts where responsibility is devolved, it requires more organisational and legal initiatives. Constraints on Edtech providers have to be balanced against the vibrancy of the supply side of the market. Over time, expectations and requirements may increase and procurement could be used to ensure minimal levels of performance of digital tools and resources, the demonstrated absence of bias, and include environmental sustainability criteria.
5. Balance different needs when regulating. While regulation is not always the best solution, it is a powerful lever for governments. For example, regulation about privacy and data protection is very important, but it has to be supplemented with training and communication efforts towards staff in schools and administrations, and possibly a more proactive support for its implementation. But it is also important that robust privacy regimes do not become the bedrock of unfairness and discrimination among some population groups by preventing possibilities to identify and address algorithmic bias, for example. Regulation regarding procurement and other matters should also be balanced with incentives for the business sector to develop digital tools and resources for the education sector.
Beyond these policy pointers to address system-level challenges, the OECD Secretariat and Education International, a union federation that brings together organisations of teachers and other education employees from across the world, have developed a series of guidelines to support the adoption of AI and other digital tools in education. The opportunities, guidelines and guardrails are meant to facilitate discussions between governments, local education authorities and the teaching profession (OECD; Education International, 2023[37]). Box 1.1 presents these guidelines, which supplement the policy pointers presented above.
The Digital Education Outlook 2023 and its companion, Country Digital Education Ecosystems and Governance (OECD, 2023[5]), present the first international comparative analysis of countries’ practices and policies. This is a baseline for further international work, highlighting where countries could learn from and get inspired by each other. International work could allow them to move faster towards an effective and equitable digital transformation of education.
The opportunities, guidelines and guardrails for effective and equitable use of AI in education represent positions on the development and use of Artificial Intelligence (AI) and digital education developed by the OECD Secretariat and Education International. This Box focuses on their main headlines.
1. Equitable access to affordable, high quality connectivity. Educational jurisdictions should create digital learning infrastructures at a system level that are accessible to all learners and educators in and outside of school. This strategic physical infrastructure should allow for a quick and equitable shift to remote learning if necessary.
2. Equitable access to and equitable use of digital learning resources. Educational jurisdictions should make available a set of quality digital learning resources to teachers and students, accessible in school and at home. Teachers should be able to use them at their professional discretion within the context of school and jurisdiction policies. Jurisdictions should provide guidance about usage expectations, in consultation with teachers and other education stakeholders, so that all learners, including educators, can have adequate opportunities to develop their digital skills. This soft infrastructure made up of digital learning resources and tools could provide the positive conditions for a quick and equitable shift to remote learning if necessary.
3. Teacher agency and professional learning. The critical and pedagogical uses of up-to-date digital learning resources should become an integral part of teachers, school principals’ and other educators’ professional competences, fostered in initial education but also within continuous professional learning opportunities and professional collaboration. Recognising the importance of teacher agency, efficacy and leadership is key for allowing them to make a critical use of digital learning resources and design rich learning scenarios with their students.
4. Student and teacher wellbeing. The use and development of AI-enabled technology should put learners’ and teachers’ wellbeing and mental health to the forefront, including by keeping a good balance between digital and non-digital activities. Ethical guidelines on digital communications which recognise that learning is a relational and social experience involving human to human interactions should be created in partnership with teachers and their organisations.
5. Co-creation of AI-enabled digital learning tools. Jurisdictions should encourage the involvement of teachers, students and other end users as co-designers in the research and development process of technology to help ensure the usefulness and use of AI-enabled digital tools. An innovation-friendly ecosystem that makes innovation and continuous improvement a culture should allow technology developers to experiment and pilot some tools with the support of teachers and learners.
6. Research and co-creation of evidence through disciplined innovation. Jurisdictions should foster research about the effective use of digital tools in education, including practice-engaged research projects that allow teachers to innovate in their classrooms, co-design the uses of technology with researchers that evaluate and document the conditions under which technology use works and for whom. Researcher-led projects can cast light on the most effective uses of AI-enabled technology. In principle, digital transformation enables quicker feedback and improvement loops than in the past, which education systems should benefit from through an active focus on research.
7. Ethics, safety and data protection. Data protection policies should ensure that the collection of data contributes to securing effectiveness and equity in education while protecting students’ and teachers’ privacy. Educational jurisdictions should provide schools and teachers with clear guidance about data protection and possibly pre-negotiated contracts or guidelines when they resort to commercial solutions. They should ensure that safety or possible algorithmic bias are tested and addressed in their policies. Clear ethical guidelines should also be developed. The ethical use of data about teachers should be negotiated with teachers and their representatives as part of bargaining agreements.
8. Transparency, explainability and negotiation. When using digital tools based on advanced technology that are high stakes for students, teachers, or educational establishments, such as digital forms of evaluation and assessment, educational jurisdictions should be transparent about the objectives and processes by which algorithms reach their recommendations. The uses of high stakes digital tools must be discussed and negotiated with all educational stakeholders.
9. Human support and human alternatives. As AI-enabled digital tools will allow for increased automation of parts of educational processes, from administration through to teaching and learning, jurisdictions should ensure that learners, teachers, and other education stakeholders can receive timely human support when they face a problem, and, when appropriate, a human alternative to the AI-enabled tool.
Note: The full text is available as chapter 16 of this book.
Source: OECD and Education International
References
[20] Baker, R., A. Hawn and S. Lee (2023), “Algorithmic bias: the state of the situation and policy recommendations”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[6] Belpaeme, T. and F. Tanaka (2021), “Social Robots as educators”, in OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, OECD Publishing, Paris, https://doi.org/10.1787/1c3b1d56-en.
[7] Bowers, A. (2021), “Early Warning systems and indicators of dropping out of upper secondary school: the emerging role of digital technologies”, in OECD, Digital Education Outlook: Pushing the frontiers with AI, blockchain, and robots, OECD Publishing.
[14] Bowers, A. and R. Sprott (2012), “Examining the multiple trajectories associated with dropping out of high school: a growth mixture model analysis”, The Journal of Educational Research, Vol. 105/3, pp. 176-195.
[21] Cuban, L. (1986), Teachers and Machines: The Classroom of Technology Since 1920, Teachers College Press.
[18] D’Mello, S. (2021), “Improving Student Engagement in and with Digital Learning Technologies”, in OECD, Digital Education Outlook: Pushing the frontiers with AI, blockchain, and robots, OECD Publishing.
[35] Dellagnelo, L. (2023), “The role of support organisations in implementing digital education policies”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[10] Dillenbourg, P. (2021), “Classroom analytics: Zooming out from a pupil to a classroom”, in OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, OECD Publishing, Paris, https://doi.org/10.1787/336f4ebf-en.
[28] Foster, N. (2023), “Teacher digital competences: formal approaches to their development”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[16] Fragoso, T. (2023), “Hardware: the provision of connectivity and digital devices”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[30] Hooft Graafland, J. (2018), “New technologies and 21st century children: Recent trends and outcomes”, OECD Education Working Papers, No. 179, OECD Publishing, Paris, https://doi.org/10.1787/e071a505-en.
[34] Molenaar, I. and P. Sleegers (2023), Multi-stakeholder collaboration and co-creation: towards responsible application of AI in education, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[8] Murphy, R., J. Roschelle, M. Feng, and C. A. Mason. (2020), “Investigating Efficacy, Moderator, and Mediators for an Online Mathematics Homework Intervention”, Journal of Research on Educational Effectiveness, Vol. 13/2, https://doi.org/10.1080/19345747.2019.1710885.
[19] O’Neil, C. (2016), Weapons of math destruction: how big data increases inequality and threatens democracy, Crown Publishing.
[5] OECD (2023), Country Digital Education Ecosystems and Governance. A companion to Digital Education Outlook 2023, OECD Publishing, https://doi.org/10.1787/906134d4-en.
[24] OECD (2023), “Human resource policies for digital education”, in Shaping Digital Education: Enabling Factors for Quality, Equity and Efficiency, OECD Publishing, Paris, https://doi.org/10.1787/c7e31038-en.
[36] OECD (2023), Shaping Digital Education: Enabling Factors for Quality, Equity and Efficiency, OECD Publishing, Paris, https://doi.org/10.1787/bac4dc9f-en.
[2] OECD (2022), Education at a Glance 2022: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/3197152b-en.
[1] OECD (2021), OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, OECD Publishing, Paris, https://doi.org/10.1787/589b283f-en.
[11] OECD (2020), “The OECD Digital Government Policy Framework: Six dimensions of a Digital Government”, OECD Public Governance Policy Papers, No. 02, OECD Publishing, Paris, https://doi.org/10.1787/f64fed2a-en.
[13] OECD (2019), “Using digital technologies to improve the design and enforcement of public policies”, OECD Digital Economy Papers, No. 274, OECD Publishing, Paris, https://doi.org/10.1787/99b9ba70-en.
[37] OECD; Education International (2023), “Opportunities, guidelines and guardrails for effective and equitable use of AI in education”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[22] Reich, J. (2020), Failure to Disrupt: Why Technology Alone Can’t Transform Education, Harvard University Press.
[31] Ronchi, E. and L. Robinson (2019), “Child protection online”, in Burns, T. and F. Gottschalk (eds.), Educating 21st Century Children: Emotional Well-being in the Digital Age, OECD Publishing, https://doi.org/10.1787/b7f33425-en.
[9] Roschelle, J., M. Feng, R. Murphy, and C. A. Mason (2016), “Online Mathematics Homework Increases Student Achievement”, AERA Open 2, Vol. 4, https://doi.org/10.1177/2332858416673968.
[12] Smolenski, N. (2021), “Blockchain for Education: A New Credentialing Ecosystem”, in OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, OECD Publishing, Paris, https://doi.org/10.1787/6893d95a-en.
[15] Thorn, W. and S. Vincent-Lancrin (2021), Schooling During a Pandemic: The Experience and Outcomes of Schoolchildren During the First Round of COVID-19 Lockdowns, OECD Publishing, Paris, https://doi.org/10.1787/1c78681e-en.
[26] Vidal, Q. (2023), “Digital assessment”, in OECD Digital Education Outlook. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[33] Vidal, Q. (2023), “Public procurement: shaping digital education ecosystems”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[4] Vidal, Q., S. Vincent-Lancrin and H. Yun (2023), “Emerging governance of generative AI in education”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[25] Vincent-Lancrin, S. (2023), “Learning management systems and other digital tools for system and institutional management”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[3] Vincent-Lancrin, S. (2022), How Learning Continued during the COVID-19 Pandemic: Global Lessons from Initiatives to Support Learners and Teachers, OECD Publishing, Paris, https://doi.org/10.1787/bbeca162-en.
[32] Vincent-Lancrin, S. and C. González-Sancho (2023), “Data and technology governance: fostering trust in the use of data”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[17] Vincent-Lancrin, S. and C. González-Sancho (2023), “Education and student information systems”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[29] Vincent-Lancrin, S. and C. González-Sancho (2023), “Interoperability: unifying and maximising data reuse within digital education ecosystems”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.
[23] Vincent-Lancrin, S. et al. (2019), Measuring Innovation in Education 2019: What Has Changed in the Classroom?, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/9789264311671-en.
[27] Yu, J., Q. Vidal and S. Vincent-Lancrin (2023), “Digital teaching and learning resources”, in OECD Digital Education Outlook 2023. Towards an Effective Digital Education Ecosystem, OECD Publishing, https://doi.org/10.1787/c74f03de-en.