OECD Skills Outlook 2019

Thriving in a Digital World

Report

OECD Skills Outlook

9 May 2019

Available in:

English
français

Download PDF

Chapter 5. Learning in a digital environment

Abstract

This chapter examines the opportunities that technology offers for skills development, in schools, higher education and throughout life. It explores the relationship between technology use in schools and students’ performance. It also investigates teachers’ use of new technologies and how policies can unlock the potential of technology for teaching and learning. Outside schools, technology offers new sources of lifelong learning through open education and massive open online courses. This chapter shows that inequalities persist in adults’ participation in online learning activities and discusses how governments can adapt systems for recognising and certifying skills when sources for learning diversify.

The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.

As technology changes, so do the skills people need to thrive in work and life. At the same time, new technologies can enhance learning opportunities and help develop skills for the 21st century. The Internet, videos and applications, have facilitated access to knowledge and have changed the way people learn at home, at work, and in schools. An almost infinite amount of information is available for anybody who browses the Internet. The challenge is learning to select between various sources and make good use of the information they provide.

Open Education Resources, the digital learning resources offered online freely and openly to teachers, educators, students and independent learners, can be used in teaching, learning and research (Orr, Rimini and van Damme, 2015[1]). Beyond Open Education Resources, people learn from networks of those who are debating their ideas online (Weinberger, 2011[2]). They learn from material not specifically designed as educational material. In European countries, more than 50% of individuals read online news, seek health information online or participate in social networks. Close to 50% obtain knowledge from wikis – websites developed collaboratively by communities of users (Figure 5.1).

Figure 5.1. Internet use for activities that can lead to learning

These activities can help people develop their knowledge and learn informally across life but they can also distract individuals, such as students, from their main job or learning activities. If teachers are not well prepared, they may simply loose time by using technology in the classroom.

The Internet and smartphones have changed people’s relationship to knowledge by making information available at any time and often, without any cost. But the Internet is also changing the way people remember and solve problems. Those relying on the Internet to access information are more likely to depend on the Internet than on their memory to access other information (Storm, Stone and Benjamin, 2017[4]). The mere presence of smartphones – for example, when people think about the possibility of using their phones but do not actually use them – can reduce available cognitive capacity (Ward et al., 2017[5]).

As smartphones specifically and technology more generally have entered the workplace, the classroom and everyday life, it is important to understand better how technology changes the way people learn, how it can help develop digital skills and the complementary skills people need, and how policies can make the most of technology for learning.

This chapter discusses how to integrate technology in schools and the extent to which it can provide new opportunities for lifelong learning. It draws from a range of databases, including three maintained by the OECD: the Programme for International Student Assessment (PISA), the Survey of Adult Skills carried out by the Programme for the International Assessment of Adult Competencies (PIAAC) and the Teaching and Learning International Survey (TALIS). The chapter examines how learning and teaching have changed with the development of technology and how policies can help people take advantage of new learning opportunities. Employers need clear signals on a broadening range of skills that workers may have, so it is becoming increasingly crucial to certify skills. As sources of learning diversify, however, this becomes more difficult. The chapter ends by discussing how policies can better recognise and certify skills.

The main findings in this chapter concern three main areas: integrating technology in the classroom, the use of open education and massive open online courses, and the need for better certification.

In schools, mere access to and use of computers is not enough to enhance student performance. The effect of technology on student outcomes depends on how technology is integrated in the classroom and used to support teaching and learning practices:

Access to ICT infrastructure in schools is extensive in most OECD countries and socio-economically disadvantaged students have similar levels of access as advantaged ones.
Student use of school computers, laptops or tablets available in schools is not widespread, however, and the share of students using these tools has decreased in many countries. At the same time, the frequency of digital device use at school has increased, driven by the surge of chatting at school, suggesting that students may simply be using their own mobile devices more during school time for no learning purposes.
Students with very high levels of digital device use at school generally perform less well, whether in mathematics, reading or science. Extensive use of new technologies at school may replace other, more efficient educational practices or simply distract students.
Many types of frequent digital device use at school tend to be found among student who perform poorly in science, mathematics and reading, even when students’ socio-economic status and other characteristics are accounted for. Test scores are higher only for those who browse the Internet for schoolwork regularly. Looking up information may indeed be done more effectively using digital devices.
The digitalisation of economies and societies increases the need to develop a set of digital skills in school. As student assessments rarely measure digital competencies, there is little evidence on how to best develop these skills. Nevertheless, countries need to make sure they implement a consistent approach throughout the school years, focusing on what needs to be learnt, such as computational thinking, rather than on specific computer use or software skills that can quickly become obsolete.
Teachers’ digital competencies are instrumental for their own students’ capacity to make the most out of new technologies. There is a significant positive relationship between teachers’ problem-solving skills in technology-rich environments and students’ performance in computer problem solving and computer mathematics.
At the same time, teachers are less likely than other tertiary-educated graduates to be high performers in problem solving in technology-rich environments. Many teachers specifically report needing professional development in ICT skills for teaching. There is a need to provide quality training to teachers on how to best integrate the technology in their pedagogical practices. More generally, governments’ focus should move from investing in resources to ensuring a tailored approach to technology use, in which teachers have the necessary ICT support and training to rely on digital tools.

Open education and massive open online courses (MOOCs) offer important new sources for knowledge and skills development across life. At this stage, however, they seem to reinforce rather than reduce inequalities in participation in adult learning and little is known on their outcomes in terms of skills development:

The increasing uptake of MOOCs on a broad range of topics – including the development of social and emotional skills and the capacity to learn more – suggests that some people are well aware of the need to adjust skills throughout life and take action to do so. However, little is known about the quality of such courses and it is likely that there are large variations among them. More data are needed to better understand how people may learn through MOOCs.
While open education and MOOCs can generally be accessed for free, patterns of participation seem to reproduce those of standard adult education and training. Highly educated and highly skilled adults are more likely to participate.
Open education is mostly used by those who combine work and formal education, and, to a lesser extent, those who are employed but not in formal education. Hence it seems to be a promising way to facilitate workers’ lifelong learning. Yet the potential that open education and MOOCs can offer to firms to train their workers is not being fulfilled, despite some initiatives in this area.
Governments can work with education and training providers, employers, job-search agencies and MOOC platforms to: i) develop broader participation in open education; and ii) expand the use of MOOCs on the job. In parallel, there is a need to define standards and good practices to better signal the quality of MOOCs.

Better recognition and signalling of skills acquired throughout life would help employers to recruit the right person and provide people with incentives to continue learning. Technology brings some solutions: online certification of a broad range of skills has been developed. Governments can build on this trend to adapt systems of recognition and certification of skills to changing needs:

Employers need clear signals about workers’ and job-seekers’ knowledge and their cognitive, social and emotional skills. That means governments, employers, and education and training institutions need to co-operate to build a competencies-based approach to formal qualification. This would require moving towards a reliable assessment of skills rather than a certification of participation in learning activities.
Governments can work together to harmonise recognition and certification of skills practices at an international level.

Making the most of technology at school

In schools, the use of technology can not only help students develop the skills they need for a digital future but also enable innovative ways of teaching that prevent schools failure. This section investigates the links between technology use in schools and students’ outcomes, and the importance of teaching practices in integrating technology in the classroom.

Access and use of digital technologies in schools

Digital tools have been widely introduced in schools. They include computers, tablets, computer-assisted instruction, and games. By 2015, in OECD countries that participated in the Programme for International Student Assessment (PISA), almost 9 in 10 students had access to computers in schools (Figure 5.2). In some countries, however, the use of such devices in schools remains far from widespread. In Poland, less than half of students reported using desktop computers, laptops or tablets that were available for them at school. In Australia, by contrast, almost all 15-year-olds indicated doing so. On average, around two‑thirds of students made use of computers at school in the OECD countries that participate in the PISA ICT questionnaire.

Figure 5.2. Access and use of computers in schools

Across the OECD, socio-economically disadvantaged students have similar access to ICT devices in schools as advantaged ones and in a few countries, they tend to use these devices more (Figure 5.3). Providing access to the Internet and ICT infrastructure has been a goal of education policies in many OECD countries, to compensate for income inequalities and lower access to computers at home among socio-economically disadvantaged students (OECD, 2015[7]; Bulman and Fairlie, 2016[8]). The digital divide in terms of access to computers in schools appears therefore to have been largely bridged. A notable exception is Mexico, where significantly fewer disadvantaged students report having access to desktop computers, laptops or tablets at school. Moreover, fewer than 40% of socio-economically disadvantaged students in Mexico report using computers available in schools, in comparison to more than 70% of advantaged students. These data do not capture, though, potential differences in the quality of digital infrastructure available in schools for disadvantaged and advantaged students.

Figure 5.3. Computer access and use in schools, by students’ socio-economic status

With few exceptions, the share of students relying on digital devices available at school has been stable or has even declined across OECD countries (Figure 5.4). The progressive introduction of more modern forms of ICT infrastructure, such as laptops and more recently tablets, has not been sufficient to compensate for the decline in the use of desktop computers (Figure 5.5). At the same time, these figures cannot capture whether students make use of their own mobile devices while in class or more generally at school.1

Figure 5.4. Computer use at school in 2015 and before

While fewer students report using the computers, tablets or laptops available in their school, the use of digital devices at school has risen. The index of ICT use summarises the frequency of digital device use for a variety of activities at school, from chatting or playing simulations to doing homework on school computers and practicing skills. Such digital devices may be part of the school infrastructure or may belong to students (e.g. smartphones). Across all OECD countries participating in the PISA ICT questionnaire, students use digital devices at school more often than before and the intensity of use appears to have accelerated in countries where students were already employing digital devices regularly (Figure 5.6). The frequency of digital device use at school is similar for socio-economically advantaged and disadvantaged students, except in Mexico and Australia.

A surge in chatting online at school has triggered the overall rise in the frequency of digital device use at school (Figure 5.7): the share of students reporting that they chat at school at least once per week more than doubled between 2012 (18% of students) and 2015 (42% of students). Among digital activities taking place at school at least once per week, browsing the Internet for schoolwork is most recurrent among students in OECD countries (48% of students report browsing), followed by chatting (41%) and sending emails (28%).

Figure 5.5. Desktop computer, laptop and tablet use in schools in 2015 and before

Figure 5.7. Uses of digital devices at school in 2012 and 2015

These first statistics suggest that technology may not be used to its full potential in schools. The fall in the use of ICT infrastructure available in schools coincides with the rise in the frequency of uses such as chatting, implying that students may simply be using their own mobile devices more for personal uses (e.g. instant messaging with friends) during school time. On the contrary, uses that would more naturally be associated with instruction activities (e.g. doing homework on a school computer or using computers for group work) have experienced only moderate increases.

The potential of new technologies for students’ outcomes

New technologies hold promise for enhancing learning and developing skills that enable people to make the most of the digital society. Digital tools extend the learning universe outside the physical premises of the school. They allow personalised instruction that enables students to progress at their own pace and teachers to spend more time with learners who are lagging behind (Barrow, Markman and Rouse, 2009[11]). Technology is likely to change the content and sources of knowledge: traditional textbooks and curricula may be supplemented by educational software, online courses or digital textbooks. These expand the opportunities young learners have to find information and practice skills (OECD, 2016[12]), including the digital competencies required for sustainable use of new technologies (Box 5.1). At school and education system levels, new digital devices can be used for connected learning and exchange of teaching practices, to collect better student data for more rapid and better-targeted student feedback, and to bring instruction to isolated areas.

Digital competencies

Not all young learners are technological savvy (OECD, 2015[7]; Kennedy et al., 2010[13]) so the use of digital devices in schools can enhance students’ digital skills. Student assessments rarely measure computer competencies, however, so there is little evidence on the impact of technology use in schools on students’ digital skills, although a few studies find positive effects (Bulman and Fairlie, 2016[8]).

Box 5.1. Developing digital skills: Country examples

There is no widely agreed, comprehensive definition of digital skills, in part because technology is constantly advancing.

The Broadband Commission for Sustainable Development – a joint initiative of the International Communication Union and UNESCO – regards digital skills as a continuum from basic to advanced skills:

Basic functional digital skills allow people to access and use digital technologies (e.g. understanding basic ICT concepts, being able to manage computer files, use keyboards or touch-screen devices).
Generic/intermediate digital skills allow people to use technologies in meaningful and beneficial ways (e.g. using work-related software, creating online content, evaluating online risks).
Advanced skills are those needed by ICT specialists (e.g. programming, app development) (Broadband Commission for Sustainable Development, 2017[14]).

Frameworks of digital competence

Generic/intermediate digital skills are often at the core of national digital strategies or policies that seek to develop the population’s digital literacy or competence. Frameworks of digital competence can help assess not only the levels and types of skills, but also the attitudes and knowledge individuals have or should develop in the digital area (Broadband Commission for Sustainable Development, 2017[14]).

The European Commission has designed a digital competence framework that broadly defines digital competence as “the confident, critical and creative use of ICT to achieve goals related to work, employability, learning, leisure, inclusion and/or participation in society” (Ferrari, 2013[15]). Its typology identifies five areas of digital competence (each of which contain knowledge, skills and attitude dimensions): information and data literacy, communication and collaboration, digital content creation, safety and problem solving. Proficiency levels for these areas are assessed based on the complexity of tasks, the autonomy with which the individual can perform these tasks and the cognitive domain (remembering, understanding, applying, and creating).

At the school level, an example of a framework for digital competence is the one put forward by the Australian Curriculum, Assessment and Reporting Authority (ACARA). Students who develop an ICT capability are students who “learn to use ICT effectively and appropriately to access, create and communicate information and ideas, solve problems and work collaboratively in all learning areas at school and in their lives beyond school” (ACARA, n.d.[16]).

For ACARA, ICT capability development is organised around several dimensions: managing and operating ICT (e.g. managing data, selecting and using software), communicating with ICT, creating with ICT (e.g. using ICT to generate ideas or manage digital solutions for issues arising in learning activities), investigating with ICT (e.g. finding and analysing information, verifying sources and reliability of digital data), and applying social and ethical protocols and practices when using ICT (e.g. recognising intellectual property, applying personal security protocols).

Students’ proficiency is assessed in all these dimensions and across all school years, since the development of ICT capability is considered as a learning continuum (ACARA, n.d.[17]). At the same time, ICT capability supports student learning in all subjects covered by the curriculum, for instance by using digital tools to create artworks, looking for and critically analysing online information about historical events, or investigating mathematical concepts using multimodal technologies. A Digital Technologies learning area is also part of the curriculum, focusing specifically on “understanding the characteristics of data, digital systems, audiences, procedures and computational thinking” (ACARA, n.d.[16]).

Developing digital skills and competence in schools

In many countries, the development of digital skills in schools has relied primarily on ICT or computational science classes. The framework developed by ACARA is an example of a progressive move from developing digital skills as part of stand-alone ICT classes, to a more comprehensive approach in which digital skills are also fostered in other learning areas. There is a risk, however, that developing digital skills by integrating technology across different subjects may result in uneven levels of technology use across classes or schools (Praxis, 2017[18]).

In France, a mandatory course on computational sciences and technology will be introduced in 2019, with the objective not only of teaching ICT as a science but also of discussing the role of digital technologies in society (Ministère de l'Éducation nationale et de la Jeunesse, 2018[19]). The government is also encouraging the creation of coding workshops outside classes and will progressively introduce a certification of digital skills for students in their last secondary school year.

In Canada, several provincial governments have adopted a comprehensive approach to digital competence (Hoechsmann and DeWaard, 2015[20]). For example, the government of Manitoba has put the focus on developing “literacy with ICT”, which spans all curricular areas. In a similar vein to the ACARA framework, literacy with ICT requires “thinking critically and creatively, about information and about communication, as citizens of the global community, while using ICT safely, responsibly and ethically” (Manitoba Education and Training, n.d.[21]). Students are assessed based on a developmental learning continuum.

Between 2012 and 2016, Estonia implemented the ProgeTiger programme, aimed at preschool, primary and vocational education students (HITSA Information Technology Foundation in Education, n.d.[22]). The programme’s aim was to enhance the digital competence of students by integrating technology education in the curriculum, by training teachers and by financing ICT infrastructure acquisition by schools (Conrads et al., 2017[23]). The programme required teachers to integrate technology in different subjects, allowing them to choose the type of technology they would use. Teachers had access to face-to-face and online training, and benefited from the support of local networks related to the programme.

Sources: Broadband Commission for Sustainable Development (2017[14]), Working Group on Education: Digital Skills for Life and Work, https://unesdoc.unesco.org/ark:/48223/pf0000259013 (accessed on 13 December 2018); Ferrari, A. (2013[15]), DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe, http://dx.doi.org/10.2788/52966; ACARA (n.d.[16]), Information and Communication Technology (ICT) Capability, https://www.australiancurriculum.edu.au/f‑10‑curriculum/general-capabilities/information-and-communication-technology-ict-capability/ (accessed on 17 May 2018); ACARA (n.d.[17]), Information and Communication Technology Capability Learning Continuum, https://www.australiancurriculum.edu.au/media/1074/general-capabilities-information-and-communication-ict-capability-learning-continuum.pdf (accessed on 14 December 2018); Praxis (2017[18]), ICT Education in Estonian Schools and Kindergartens, http://www.praxis.ee/en/works/ict-education-in-estonian-schools-and-kindergartens/ (accessed on 14 December 2018); Ministère de l'Éducation nationale et de la Jeunesse (2018[19]), Le numérique au service de l'École de la confiance, http://www.education.gouv.fr/cid133192/le-numerique-service-ecole-confiance.html (accessed on 14 December 2018); Hoechsmann, M. and H. DeWaard (2015[20]), Mapping Digital Literacy Policy and Practice in the Canadian Education Landscape, http://mediasmarts.ca/teacher-resources/digital-literacy-framework/mapping-digital-literacy-policy-practice-canadian-education-landscape (accessed on 14 December 2018); Manitoba Education and Training (n.d.[21]), Literacy with ICT - What is LwICT?, https://www.edu.gov.mb.ca/k12/tech/lict/what/index.html (accessed on 14 December 2018); HITSA Information Technology Foundation in Education (n.d.[22]), ProgeTiger Programme, https://www.hitsa.ee/it‑education/educational‑programmes/progetiger (accessed on 14 December 2018); Conrads, J. et al. (2017[23]), Digital Education Policies in Europe and Beyond: Key Design Principles for More Effective Policies, http://dx.doi.org/10.2760/462941.

As technologies evolve at an ever-faster pace, it is essential for people to acquire more general digital literacy skills rather than specialised ones that risk rapidly becoming obsolete. In an increasingly digitalised society, individuals should be able to interpret the information provided by digital tools in specific contexts, adapt to an expanding number and types of tools, protect their data and privacy, and develop their own digital content (Carretero, Vuorikari and Punie, 2017[24]). Schools can help develop such skills from an early age and empower students to become not only critical users of new technologies, who understand the inherent mechanisms and risks of technology, but also creators of digital material and maybe of tools that serve their purposes (Bell, 2016[25]).

This translates into a need to move beyond traditional ICT classes, which teach students how to use specific software, and into the domain of computational thinking (Box 5.2) (Bocconi et al., 2016[26]).

Box 5.2. Computational thinking, computer programming and coding

Computational thinking frames problems in ways that computers can help solve them (Wing, 2006[27]; CSTA/ISTE, 2011[28]; Paniagua and Istance, 2018[29]). It requires algorithmic thinking, problem decomposition, logical reasoning and abstraction (Voogt et al., 2015[30]; Bell, 2016[25]; Paniagua and Istance, 2018[29]). In this respect, it is closely related to mathematics and computer science reasoning. Early research on computational thinking presented it as a “universally applicable attitude and skill set everyone, not just computer scientists, would be eager to learn and use” (Wing, 2006[27]). The focus of computational thinking is thus not on technology use per se, but rather on understanding the underlying notions and mechanisms of digital technologies (Bocconi et al., 2016[26]).

Computational thinking can be taught through computer programming, the process of instructing a computer to carry out specific tasks (Balanskat and Engelhardt, 2015[31]). While programming, students are exposed to computational thinking and solve problems with the help of computers (Lye and Koh, 2014[32]). One of the first programming experiences in schools was that of Logo programming for teaching mathematics in the 1960s (Feurzeig, Papert and Lawler, 2010[33]) and many newer programming languages (e.g. Alice, Scratch) are based on Logo (Lye and Koh, 2014[32]).

Computer programming and coding are often used as similar notions. However, coding refers more precisely to the writing in a specific programming language of instructions the computer has to perform (Balanskat and Engelhardt, 2015[31]). Programming is therefore a wider concept than coding, since it involves the more general analysis, development and implementation of a solution to problems using a computer (Lye and Koh, 2014[32]; Bocconi et al., 2016[26]).

Sources: Wing, J. (2006[27]), Computational Thinking, https://www.cs.cmu.edu/~15110-s13/Wing06-ct.pdf (accessed on 09 April 2018); CSTA/ISTE (2011[28]), Computational Thinking. Teacher Resources, http://www.iste.org/docs/ct‑documents/ct‑teacher‑resources_2ed‑pdf.pdf?sfvrsn=2 (accessed on 27 March 2018); Paniagua, A. and D. Istance (2018[29]), “Teachers as Designers of Learning Environments: The Importance of Innovative Pedagogies”, https://www.oecd-ilibrary.org/education/teachers-as-designers-of-learning-environments_9789264085374-en; Voogt, J. et al. (2015[30]), “Computational thinking in compulsory education: Towards an agenda for research and practice”, http://dx.doi.org/10.1007/s10639-015-9412-6; Bell, T. (2016[25]), What’s All the Fuss About Coding?, https://research.acer.edu.au/cgi/viewcontent.cgi?article=1288&context=research_conference (accessed on 27 March 2018); Bocconi, S. et al. (2016[26]), Developing Computational Thinking in Compulsory Education – Implications for Policy and Practice, http://dx.doi.org/10.2791/792158; Balanskat, A. and K. Engelhardt (2015[31]), Computing Our Future. Computer Programming and Coding. Priorities, School Curricula and Initiatives across Europe, http://fcl.eun.org/documents/10180/14689/Computing+our+future_final.pdf/746e36b1-e1a6-4bf1-8105-ea27c0d2bbe0 (accessed on 29 March 2018); Lye, S. and J. Koh (2014[32]), “Review on teaching and learning of computational thinking through programming: What is next for K‑12?”, http://dx.doi.org/10.1016/J.CHB.2014.09.012; Feurzeig, W., S. Papert and B. Lawler (2010[33]), “Programming-languages as a conceptual framework for teaching mathematics http://dx.doi.org/10.1080/10494820903520040.

Computational thinking does not necessarily imply the use of computers, but it can occur in the context of programming. In a similar vein, computational thinking may be taught as a subject in itself or it may be incorporated as a tool for the study of other subjects.

When students are exposed to computational thinking through programming, they can increase both their problem-solving and digital competencies, as well as acquire a deeper understanding of the underlying mechanisms and concepts of new technologies. Promising research shows that computational thinking activities have the potential to develop both specific academic skills (e.g. in mathematics) as well wider 21^st century skills, including creativity, digital literacy or critical thinking (Lye and Koh, 2014[32]; Paniagua and Istance, 2018[29]).

Academic performance

Simply providing access to digital tools or using them in the classroom does not automatically lead to better academic results, even if investment in ICT does not crowd out resources allocated to other inputs (Bulman and Fairlie, 2016[8]; Escueta et al., 2017[34]). This suggests that programmes aiming at merely increasing availability of digital devices for students do not increase instruction time, but rather substitute more efficient traditional instruction with time devoted to computer use (Angrist and Lavy, 2002[35]; Leuven et al., 2007[36]; Cristia et al., 2017[37]). When academic performance did improve, this was mostly at to schools that had benefitted from the largest increases in ICT investment and were also already able to use ICT infrastructure more efficiently (Machin, Mcnally and Silva, 2007[38]). Such schools already had Internet access and hence may have concentrated the additional investment on teacher training and support.

In a similar vein, the impact of computer-assisted instruction (or educational software) on academic performance depends on whether such technology is used as a substitute or a complement to traditional instruction, and if it is used as a substitute, of the quality of the traditional method it substitutes. The use of such technology in schools may improve students’ performance more in developing countries than in developed ones if it replaces traditional instruction of lower quality or compensates for a lack of teachers (Banerjee et al., 2007[39]; The Economist, 2018[40]).

Focusing on specific school subjects, computer-assisted instruction technologies that help engage students in practicing their mathematics skills have shown more promising results (Barrow, Markman and Rouse, 2009[11]; Roschelle et al., 2010[41]; Roschelle et al., 2016[42]). For reading, traditional computer-assisted programmes have only a moderate impact but programmes that combine computer and non-computer instruction with teacher professional development appear to be more effective (Cheung and Slavin, 2012[43]).

The role of pedagogies

Irrespective of the subject in which it is used, technology has the most positive effects when used as an amplifier for teaching, enabling teachers and students to relate the knowledge and skills developed in traditional and non-traditional instruction (OECD, 2015[7]; Paniagua and Istance, 2018[29]; Peterson et al., 2018[44]). When technology is blended in innovative teaching and learning methods, it can boost student performance and enhance student motivation (Fleischer, 2012[45]; Paniagua and Istance, 2018[29]; Peterson et al., 2018[44]).

Technology cannot achieve its full potential in classrooms if it is used merely to reproduce traditional practices and pedagogies. If such practices are already insufficient to raise student outcomes, then relying on technology only replicates the same results. Technology can even have detrimental effects if it results in distraction or cognitive overload or otherwise frustrates students’ learning needs (Paniagua and Istance, 2018[29]; Peterson et al., 2018[44]) .

Innovative uses of digital tools and devices show great promise for teaching and re-engaging those who face difficulties at school. There are many examples of such pedagogical methods, including gamification, which integrates the pedagogical principles of play and games (including video games) in formal learning, or flipped classes, in which students are required to attain content, usually provided by ICT material, before the class.

Pedagogies are therefore of crucial importance for making the most of new technologies in schools (Table 5.1). They should rely on technology as a tool for enhancing student motivation and learning, rather than treat technology use as an objective per se. Pedagogies ensure that digital uses correspond to learners’ needs, prior competencies and digital literacy, following clear instructional designs. They encourage active learning, with teachers acting as mentors who guide students and help them remain focused on the learning elements of tasks. Finally, innovative pedagogies related to technology use put forward new ways of collaboration and learning (e.g. through the use of social networks), extending the learning process outside the school environment (Paniagua and Istance, 2018[29]; Peterson et al., 2018[44]).

Technology type	What pedagogies can do
General ICT	Use ICT as a complement to teaching practices	Enhance motivation “through” and not “to” technology	Promote digital literacy and ensure students have prior competences to use digital tools	Encourage active learning and collaboration
Multimedia materials	Use sound instructional designs	Encourage multimedia authoring as a tool for thinking skills, communication and self-expression development	Accompany students, scaffolding the use of materials	Ensure contents can be understood and learners can stay focused
Multi-tasking and interactive environments	Enhance awareness of multi-tasking and of its consequences	Design and implement environments based on sound pedagogical approaches	Address harmful multi-tasking	Promote the use of knowledge frameworks to help students connect new information with prior knowledge
Gaming	Ensure the integration of video games into the instructional context	Ensure exploration and manipulation of realistic scenarios	Ensure students focus on the learning elements of games	Provide feedback to students and align games to their learning capacity
Collaborative and Web 2.0 environments	Ensure Web 2.0 principles are followed (e.g. student-generated content, interaction and collaboration)	Avoid transmission of content by the teacher/ relegation of students to passive roles	Enhance students’ capacities to self-regulate and remain focused	Put forward new ways of collaboration and learning based on Web 2.0 tools, extending learning outside the classroom

Note: General ICT refers to digital technology (e.g. computers, smartphones, and software). Multimedia materials refer to combinations of verbal and non-verbal technology-based content (e.g. video clips, e-books, and PowerPoint slides). Multi-tasking and interactive environments refer to the performance of different tasks at the same time (e.g. watching videos, reading online, and sending messages) in environments that are responsive to users’ actions. Gaming environments refer to the use of video games in school settings. Collaborative and Web 2.0 environments refer to the use of digital tools for collaborative and social activities (e.g. blogs, social networking sites, and wikis).

Source: Peterson et al. (2018[44]), "Understanding innovative pedagogies: Key themes to analyse new approaches to teaching and learning", https://www.oecd-ilibrary.org/education/understanding-innovative-pedagogies_9f843a6e-en.

In practice: The impact of technology use in schools

Data from PISA 2015 show that when levels of ICT use at school are very high, student performance tends to be lower, whether in science, mathematics or reading (Figure 5.8). The exception to this pattern is Australia, where students in the top quarter score perform better than those at the lower end of the distribution of ICT use. In the Australian curriculum, ICT use and the development of ICT capabilities are embedded at all learning levels and in all curriculum areas, not only in the technology-specific ones (Box 5.1). The objective is that students apply ICT knowledge and skills to meet learning requirements across a large range of subjects, from mathematics, to humanities, health and physical education (ACARA, n.d.[16]).

Results in collaborative problem-solving student assessments match these findings (OECD, 2017[46]), suggesting that extensive use of technologies at school may replace other, more efficient educational practices or may simply distract students. In many countries, disadvantaged schools have benefitted from substantial investments in ICT, but overall, this effect does not seem to explain the negative relationship between highly frequent ICT use and students’ performance (Figure 5.9).

Figure 5.8. Index of ICT use at school and student performance in school subjects

Figure 5.9. Index of ICT use at school and student performance in school subjects in socio‑economically disadvantaged schools

The effect of technology on student performance depends on how devices are used in the classroom. Some activities may be more effective in class when computers are used instead of traditional instruction techniques. The lack of any visible correlation between computer use and overall student performance would thus simply be the result of positive effects of computer use in some activities being offset by negative effects in others (Falck, Mang and Woessmann, 2015[47]; Comi et al., 2017[48]).

Analyses based on PISA data show that many types of frequent digital device use at school tend to accompany lower students’ performance, whether in science, mathematics or reading (Figure 5.10). Playing simulations, posting work on the school website, using school computers for homework or group work at least once per week are all associated with lower test scores, even when socio-economic status and several other individual and school characteristics are accounted for. Test scores appear to be higher only for students who browse the Internet regularly for school work. Looking up for information may indeed be more efficiently done using a computer, while other activities – such as the practice of skills or even group work – can be performed equally well without the help of technology (Falck, Mang and Woessmann, 2015[47]).

Many digital device uses are likely to displace more efficient instructional activities when done with high frequency. Spending several hours per week posting work on the school website or working in groups on a computer is more likely to replace other more productive tasks than devoting ten minutes to browsing the Internet for school work. Moreover, effects are also likely to vary depending on whether specific uses are made within the class or not. Chatting appears to be less negatively related to school performance than playing simulations, but chatting may also be done during break time and hence not interfere with actual instruction. Additional information on teaching practices would help further refining these findings, as well as more data on the amount of time students devote to digital device uses at school, and more specifically within the classroom.

In addition, reliance on ICTs appears to be most recurrent in subjects that are already traditionally associated with the use of technology (Figure 5.11). In countries participating in TALIS, more than half of teachers in technology and almost half of those who teach practical and vocational skills classes rely on ICTs frequently for students’ class work or projects. On the contrary, foreign languages and even science or mathematics teachers are less likely to do so, whether at the lower-secondary or upper-secondary levels. Digital tools are used mainly in subjects where they would be expected to be present, suggesting that innovative methods based on ICTs are still not common in schools. Many teachers still report using technologies primarily for administrative tasks and the preparation of lessons rather than as an integral part of their in-class teaching (European Commission, 2013[49]).

In this respect, technology use should be seen as an integrated tool in wider teaching and learning activities rather than an objective in itself or a direct route to academic improvement. ICT investments dedicated to teachers tend to accompany higher student performance than increases in the number of available computers for students (Denoël et al., 2017[50]).

For technology to improve students’ academic performance, both its quality and its co-ordination with other teaching practices and the curriculum are essential. Access to digital devices in schools is widespread in OECD countries, but the tools may not be adequate, sufficiently up-to-date or optimally used (Chatterji, 2017[51]).

Figure 5.10. Uses of digital devices at school and performance in school subjects, accounting for students’ socio-economic status

The results in this section emphasise the need to fundamentally rethink the use of new technologies for young learners in schools. Teachers are the people who are most aware of their students’ needs. When decisions are made to adopt given technologies in schools, teachers could be consulted or allowed to choose between various types of technologies or digital tools. Professional development programmes in ICT use for teaching could be combined with higher availability of ICT support in schools. Assessing the efficiency of software or tools before adopting them at a large scale also need to be considered. More generally, governments should shift their focus from simply investing in resources to ensuring a tailored approach to technology use, in which teachers have the necessary ICT support and training to rely on digital tools.

Figure 5.11. Teachers with high ICT frequency use for students’ projects or class work, by subject

Teachers’ use of new technologies

To ensure that students develop the skills they need for the future, it is vital that teachers are able to use appropriate and innovative pedagogical tools. Many of these tools and methods in turn rely on technology, making it crucial that teachers themselves are equipped with the skills required to use new technologies effectively (Paniagua and Istance, 2018[29]; Peterson et al., 2018[44]).

Teachers also have a role to play in raising students’ awareness of the risks associated with new technologies and how to avoid them. As in other professions, teachers are also expected to have the necessary skills to make use of digital devices as part of their regular work, even outside the classroom.

Teachers’ skills, motivations and attitudes are instrumental for the way ICTs are implemented in the classroom (Voogt et al., 2013[53]; European Commission, 2013[49]) and hence, for their own students’ ability to make the most out of new technologies. To properly integrate ICTs in the classroom, teachers need not only basic digital skills that allow them to use a computer but also more complex digital skills that enable them to tailor the use of technology to their own teaching.

The Survey of Adult Skills (PIAAC) measures how adults, including teachers, can “use ICT tools and applications to assess, process, evaluate and analyse information in a goal-oriented way” (OECD, 2016[54]). The share of teachers with low problem-solving skills in technology-rich environments varies from less than 5% in Australia to around 20% or more in Chile and Turkey (Figure 5.12). Teachers appear to be as likely as other workers with a tertiary degree to have low skills in this area but less likely to have high skills. Australia displays the highest share of top performing teachers in problem solving in technology-rich environment (63.5%). It is also in Australia that high levels of ICT use in schools tend to be accompanied by high student performance (Figure 5.8).

Figure 5.12. Teachers’ problem solving in technology-rich environment proficiency

Box 5.3. Teachers’ problem-solving skills in technology-rich environments and students’ digital performance: methodology

The analysis here is based on the methodology of (Hanushek, Piopiunik and Wiederhold, 2014[57]) who examined whether differences in teachers’ cognitive skills (literacy and numeracy) among developed countries correspond with differences in student performance (in reading and mathematics). This section investigates the relationship between teachers’ problem-solving skills in technology-rich environments (as measured in PIAAC 2012 and 2015) and students’ digital performance (as measured by test scores in PISA 2012). It assumes that students assessed in PISA (2012) are likely to have been taught by teachers assessed in PIAAC (2012, 2015).

Students’ and teachers’ skills

Teachers in this analysis are adults who report that they are primary school teachers, secondary school teachers or other teaching professionals (e.g. special needs teachers, other music teachers). In a similar vein to (Hanushek, Piopiunik and Wiederhold, 2014[57]), the analysis excludes vocational education teachers and university professors since 15-year-olds assessed in PISA are unlikely to have been taught by these teachers.

Students’ digital performance is measured through two assessments in PISA 2012: computer problem solving and computer mathematics. In PISA 2012, the assessment of computer problem solving focused on the fundamental cognitive processes that are essential for successful problem solving; only foundational ICT skills are required to take the test (OECD, 2013[58]). On the contrary, the computer-based assessment of mathematics included a variety of computer-based mathematics tools (e.g. statistical software, geometric construction and visualisation utilities, and virtual measuring instruments) among the assessment items (OECD, 2013[58]).

Empirical analysis

The analysis examines the extent to which differences in students’ test scores in computer problem solving and computer mathematics can be explained by cross-country differences in teachers’ problem-solving skills in technology-rich environments, when individual and school characteristics are accounted for. Teacher skills are the median of teachers’ scores in problem solving in technology-rich environments at the country level. Both teachers’ and students’ scores are standardised across countries.

An ordinary least squares regression is estimated. One standard deviation increase in teachers’ problem-solving skills in technology-rich environments is associated with higher student performance by 0.166 standard deviation in computer problem solving and by 0.175 in computer mathematics. Coefficients are statistically significant at the 1% level. Countries are given equal weights and robust standard errors are clustered at the country level. The estimation accounts for student, parent and school characteristics. Student characteristics include age, gender, migrant status, language spoken at home, number of books at home and an index of ICT availability at home. Parent characteristics include parents’ labour market status, education levels and the ISCO-08 occupation code of the father. School characteristics include a dummy for whether the school is private or public, a dummy for whether the school is in a rural or in an urban area, the ratio of students and computers in school, the number of students in the school, an index for the degree of school autonomy and an index for teacher participation in school decisions. The index for the degree of school autonomy and the index for teacher participation in school decisions are defined in (OECD, 2014[59]).

Sources: Hanushek, E., M. Piopiunik and S. Wiederhold (2014[57]), “The value of smarter teachers: International evidence on teacher cognitive skills and student”, http://www.nber.org/papers/w20727 (accessed on 13 April 2018); OECD (2012[55]) and OECD (2015[56]), Survey of Adult Skills (PIAAC), www.oecd.org/skills/piaac/publicdataandanalysis; OECD (2013[58]), PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy, http://dx.doi.org/10.1787/9789264190511-en; OECD (2014[59]), PISA 2012 Technical Report, https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf (accessed on 09 April 2018).

Student achievement is closely tied with the quality of teachers (Barber and Mourshed, 2007[60]; Chetty, Friedman and Rockoff, 2014[61]; Hanushek, Piopiunik and Wiederhold, 2014[57]). Students’ scores in computer problem solving and computer mathematics are related to teachers’ problem-solving skills in technology-rich environments (Box 5.3). Many OECD countries would experience large increases in their students’ digital performance were their teachers’ problem-solving skills in technology-rich environments raised to the level of Australian teachers, the highest-performing ones in the sample (Figure 5.13). The magnitude of the relationship between teachers’ and students’ digital performance is similar to that between teachers’ cognitive skills and students’ scores in mathematics (Hanushek, Piopiunik and Wiederhold, 2014[57]).

The link between teachers’ problem-solving skills and students’ computer problem solving is likely to capture students’ general capacity to solve problems rather than the ICT skills required for such tasks, since in PISA 2012 only basic ICT skills are required for this assessment. However, in the computer-based mathematics assessment, students are required to make use of a variety of digital equipment and software for mathematics. Hence, the relationship between students’ scores in computer-based mathematics and teachers’ problem-solving skills in technology-rich environments does capture the capacity to solve problems in a digital environment.

At work, teachers use ICTs with the same intensity as other high-skilled workers in many OECD countries. In countries where teachers’ use of ICTs is low, it is below the use of ICTs by non-teacher workers with a similar level of education (Figure 5.14). Overall, teachers are required to make a sustained use of digital devices as part of their work. However, in 2013, only one third of teachers across countries included in the TALIS database used ICTs frequently as part of their regular teaching activities (Figure 5.15).

On average, teachers do not make a lot of use of technology for teaching activities but data from PIAAC show that this is not because their use of ICT decreases with age (Figure 5.16). However, these data do not distinguish between uses of digital devices inside the classroom and elsewhere at school (e.g. for administrative work). Data from TALIS focusing on the use of ICTs for students’ projects or class work give a similar picture: the share of teachers using ICTs with high frequency in the classroom is almost constant across ages and experience levels. Upper secondary-level teachers make the biggest use of ICTs, as expected, reflecting either the capacity of students at these ages to make more mature uses of digital devices or the higher frequency of school subjects related to practical skills acquisition, which may rely more heavily on the use of technologies (Figure 5.11).

In most countries, more than 30% of teachers said they needed further training to perform their duties. In countries where the need for training among high-skilled workers is the largest (Austria, Chile, Germany, Lithuania, Slovenia), teachers are more likely than non-teachers to be in need of training (Figure 5.17).

Figure 5.13. Potential increase in computer problem solving and mathematics student scores linked to an increase in teachers’ skills to the level of top performers

Figure 5.14. ICT intensity at work of the teaching profession

Figure 5.15. Teachers’ use of ICTs in the class

Teachers’ demand for training may be greater because they are more ready and willing to learn in general, but many teachers specifically report needing professional development in ICT skills for teaching. Across OECD countries participating in TALIS, around 1 in 5 teachers indicate having a high level of need for such training (Figure 5.18). Together with professional development for teaching students with special needs, training in ICT skills for teaching is the most needed type of professional development reported by teachers in TALIS. At the same time, professional development programmes that focus on ICT translate into additional workload for teachers, since training is often provided outside of school hours. Training options need to be flexible and account for the potential impact such programmes might have on teachers’ well-being.

Figure 5.16. Teachers’ intensity of ICT use at work and as part of their classes, by age

Figure 5.17. Share of teachers and non-teachers reporting needing training

The teaching population is getting older, particularly at higher levels of education, while technology is increasingly entering schools and universities, which may partly explain why training needs are high (OECD, 2017[63]). On average across OECD countries, 37% of primary to secondary teachers were at least 50 years old in 2015, up from 31% in 2005. At the same time, the teaching profession is increasingly unattractive to students. Teachers’ salaries are lower than those of other, similarly educated full-time workers. Making the teaching profession attractive to students and developing high quality training for teachers, both initial and continuous, are important steps to ensure education systems adapt to new needs.

Figure 5.18. Share of teachers needing training in ICT skills for teaching

Learning in higher education and throughout life: The role of open education

The digital transformation is playing a major role in opening up higher education and knowledge to more students and to a broader range of socio-economic groups (Vincent-Lancrin, 2016[64]). Open universities, initially designed to serve older students, provide distance education to students who do not necessarily have an upper secondary education degree. Anybody can easily access for free an increasing amount of learning materials such as text, images, video and games. In recent years, massive open online courses (MOOCs) have developed. They enable anyone at any age to take a course provided by top universities, the business sector or independent experts.

The potential of open education for lifelong learning

People have to keep learning as the skills required on the job change. The question is whether and how open education, including MOOCs, can become a cornerstone of lifelong learning. Ideally, open education could not only help workers adapt their skills mix and knowledge to evolving labour market needs but also enable those who have left education without an appropriate level of skills to catch up with labour market needs. For this to happen, open education needs to: i) be adopted broadly by employers, workers and individuals; ii) benefit all individuals, and iii) provide high-quality learning material aligned with labour market needs.

Open education courses, and MOOCs in particular, are often presented as a way to improve access to higher education across socio-economic groups as they are offered for free or at a low cost. However, if high-skilled people from the most advantaged socio-economic groups participate the most in open education, this may reinforce rather than diminish inequalities in participation in education and training.

The Survey of Adult Skills includes questions on participation in courses conducted through open or distance education that do not lead to formal qualification. This covers courses “which are similar to face-to-face courses but take place via postal or correspondence or electronic media, linking together instructors, teachers and tutors or students who are not together in the classroom”. Since most countries were surveyed in 2012, when MOOCs were in their infancy, it is likely that responses mainly capture more traditional forms of open education such as courses or other material available online to learners.

Among countries covered by the Survey of Adult Skills in 2012 or 2015, 10% of the population participated in open education on average, but participation varied a lot, from almost 20% in Korea, a country with a lengthy and considerable experience with open education, to less than 2% in France (Figure 5.19). In most countries, young people are more likely to participate in open education than older adults but in Canada, Denmark, Finland and the United States, participation among prime-age adults is high (Figure 5.20).

Figure 5.19. Participation in open education

Figure 5.20. Participation in open education by age

Open education offers flexible ways to learn. People already in formal education participate the most in open education. Among them, those who combine work and study, and to some extent, those in formal education who are looking for a job (unemployed) are more likely to participate than those who study but are out of the labour force (not looking for a job) (Figure 5.21). These results suggest that open education provides flexibility to people combining work and study and is used as a way to transition to the labour market. Among people who are not in education anymore, those who participate the most are also the employed, and to a lesser extent the unemployed. Open/distance education does not seem to be successful in reaching those out of the labour force who are not studying.

People participate in open/distance education mainly to improve job performance or prospects and to a lesser extent, to develop knowledge or skills more generally (Figure 5.22). Few participants aim to gain a certificate through their participation, perhaps because at the time of the survey, certificates were rarely attached to open education programmes. Another important reason may be that since most participants are in formal education, they aim to obtain a qualification through the formal education programme, not through participation in open education. More than 40% of participants find the experience very useful.

Figure 5.21. Participation in open education by employment and education status

Figure 5.22. Reasons for and usefulness of participation in open/distance education

Online delivery of education and training can also provide large geographic and time flexibility for learners to pursue education. Some examples of formal education programmes suggest that online delivery may expand the number of people pursuing education (Goodman, Melkers and Pallais, 2018[65]). The Georgia Institute of Technology’s online master’s degree in Computer Science, introduced in 2014, enables mid-career individuals who would not otherwise pursue education to obtain a degree. This programme is offered online at a significantly lower cost than in-person for a degree that is not signalled as having been obtained on line and is fully equivalent to the in-person degree. The online courses are versions of the same courses students take in person, designed by the same faculty, and graded using the same standards. In the first year, the course was taken by mid-career individuals whose average age was 34, compared with an average of 24 for in-person students.

One of the promises of open education is to expand access to tertiary education for disadvantaged students by lowering costs of delivery. However, as for other types of training, highly skilled and educated people are more likely to participate in open education (Figure 5.23). Hence, open education may tend to reinforce rather than close the gap in participation in adult education between low-skilled and high-skilled individuals. This is not surprising as most of these programmes are at a tertiary education level. Moreover, skilled and more privileged individuals have better access to the technology itself and to the enabling conditions – the time, the skills, and the motivation. Nonetheless, around 20% of the population without a tertiary degree had participated in open education at the time of the survey.

Recent opportunities brought by MOOCs

The recent development of MOOCs has boosted open education. Courses are generally provided by universities, including many top ones, but the business sector and independent experts also propose some MOOCs. Participation is generally free, but learners now have to pay to get a certificate. Contrary to traditional support provided online, participants in a MOOC take the course at the same period of time and can communicate with other participants through course forums.

In theory, MOOCs can help reduce the skills gap that has emerged as the digital transformation has changed skills needs (Music, 2016[66]). Learners can access courses from top universities in multiple fields with a large choice of when and what to learn. Such flexibility facilitates participation by various groups: the employed, those living in remote areas, and those who cannot afford to return to formal education. It also helps those who combine work and study to complete programmes by allowing them to complement regular courses with other courses. Learners can expand professional and personal networks around the world by participating in discussion forums. Such interactions and exchanges are increasingly needed in a globalised and digital world. Overall, MOOCs have the potential to better align education with employers’ needs.

Due to data limitations, it has not been possible so far to get a broad view of the quality of MOOCs, and their impact on skills development and on equality in access to and participation in education. In particular, there are no data covering a large numbers of participants that show how MOOCs affect their skills and knowledge development. Some data on MOOCs proposed by the MIT and Harvard University covering 290 courses and 4.5 million participants between 2012 and 2016 give some indication on how MOOCs are used and by whom (Chuang and Ho, 2016[67]).

Figure 5.23. Participation in open/distance education by educational attainment and skills proficiency

These data suggest that most participants in MOOCs are highly educated, with a bachelor’s degree, confirming results on early phases of open education coming from the Survey of Adult Skills (Figure 5.24). Women are less likely to participate in MOOCs than men, for all disciplines covered in the sample. Participants from the United States do not represent the majority of learners, suggesting that MOOCs are efficient in breaking geographical frontiers. Forty percent of participants live in developing countries and among those who complete courses, participants in developing countries were more likely to report career or educational benefits (Zhenghao et al., 2015[68]).

Figure 5.24. Characteristics of participants in MOOCs, for a sample of MOOCs

Data on this limited sample of MOOCs show that most participants access less than 50% of the content of the course and do not earn a certificate (Figure 5.25). When considering only engaged participants who look at more than 50% of the content of the course, the certification rate increases to 30%. Participation in MOOCs is less intense in computer sciences or science, technology, engineering and mathematics than in the humanities and social sciences, perhaps because of the higher degree of specialisation of the former. Participation in forums ranges from 5% to 12% of participants, on average, depending on the field of study.

Completion rates of MOOCs are low. This has often been put forward as one of their main limits, but participants in MOOCs have different learning goals. Some want to learn about a topic without intending to complete the course. Others would like their participation to be recognised by employers or education institutions as some form of extra education.

Figure 5.25. Patterns of participation in a sample of MOOCs

A case study on the MOOC “Big Data in Education” delivered via Coursera suggests that completers and non-completers differ by their objectives but not by their readiness to learn (Wang and Baker, 2015[69]). Non-completers are more likely than completers to take the course because they are curious to take an online course, they use it as a supplement or complement to other courses, or they cannot afford to pursue formal education. A range of questions aiming to capture participants’ goal orientation and readiness to learn show no significant difference between completers and non-completers. Another study finds that many participants who may be classified as non-completers are in fact still participating in the course in their own preferred way, either at a slower pace or with a selective approach to the material of the course engagement (Onah, Sinclair and Boyatt, 2014[70]).

More data are needed to understand how and what participants in MOOCs learn, both completers and non-completers. For MOOC providers, the diversity of objectives and backgrounds of participants makes it difficult to develop MOOCs that are appropriate for all learners.

The ranking of MOOCs according to their popularity gives an indication of what participants try to learn. Most popular courses are in computer sciences but also in social and emotional skills development and traditional topics such as finance and English (Box 4.1). The popularity of courses to develop social and emotional skills such as the MOOC “Learning how to learn: Powerful mental tools to help you master tough subjects”, which has attracted more than 1 million participants (Class Central, 2017[71]), suggests that participants in MOOCs worry about their capacity to learn and adapt their skills to changing requirements.

Box 5.4. Most popular MOOCs as of 2017

Based on statistics on185 MOOCs provided by Coursera and EdX, Class Central has ranked courses according to their enrolment number, considering all sessions of each MOOC as of 2017. The 15 most popular MOOCs according to this ranking are:

1. Learning How to Learn: Powerful mental tools to help you master tough subjects / University of California San Diego

2. Machine Learning: Master the Fundamentals / Stanford University

3. R Programming / Johns Hopkins University

4. Introduction to Finance / University of Michigan

5. The Data Scientist’s Toolbox /Johns Hopkins University

6. Think Again: How to Reason and Argue / Duke University

7. Algorithms: Part 1 / Princeton University

8. Developing Innovative Ideas for New Companies: The First Step in Entrepreneurship / University of Maryland, College Park

9. Understanding IELTS: Techniques for English Language Tests / British Council

10. Programming Mobile Applications for Android Handheld Systems – Part 1 / University of Maryland

11. Cryptography I / Stanford University

12. Programming for Everybody (Getting Started with Python) / University of Michigan

13. Social Psychology / Wesleyan University

14. Introduction to Public Speaking / University of Washington

15. Model Thinking / University of Michigan

Source: Class Central (2017[71]), The 50 Most Popular MOOCs of All Time, https://www.onlinecoursereport.com/the-50-most-popular-moocs-of-all-time/ (accessed on 20 February 2018).

MOOCs can break down the boundaries between knowledge areas and help develop multidisciplinarity. When learners enter a MOOC platform, they may be tempted to start other courses in other disciplines, which can be done more easily than at universities on site. Data on MOOCs from Harvard University and the MIT show that multiple enrolments are frequent (Chuang and Ho, 2016[67]). Many participants start with courses in computer science and continue with courses in other subjects, such as government, health and social science (Figure 5.26).

Figure 5.26. Multiple enrolments in MOOCs

MOOCs can provide simple, flexible and low-cost options for companies to train their workers. Employers can simply encourage or allow their workers to take a course on their working time. Given the wide range of courses, employers in many industries could adopt this option. Examples of use of MOOCs by employers mostly concern large firms, some of which have developed their own content on topics such as management, computer science and finance (Hamori, 2018[72]). However, most MOOCs proposed by universities may be too general to meet the needs of firms. Many MOOC platform providers have started exploring MOOCs for professional development and there are already some successful examples of MOOCs in this area (Music, 2016[66]). When employers take part in the design of a MOOC, they can use this experience to attract new employees.

In general, however, firms have not fully used the potential of MOOCs to develop their employees’ skills. A reason might be the lack of information and the organisation of the provision of training (Hamori, 2018[72]). Employers may not consider MOOCs as a substitute to other forms of training. Line managers, who know the domain of expertise, are well placed to initiate this type of training and manage workloads to enable workers undertake courses. Human resources managers, who are in charge of training policies, are less likely to do so. For workers, it is easier to look for MOOCs and try them out if they think this is going to be valued by employers. Human resources managers and employers could help employees choose MOOCs by looking at the quality of providers, whether courses have a clear description, and learning outcomes. As MOOCs and open education mostly seem to attract people who are already employed, governments could try to foster the use of MOOCs by employers as a way to develop adult learning. They could raise employers’ awareness of the potential of MOOCs and help them partner with MOOC platforms to find or develop courses aligned with their firms’ skills needs.

MOOCs also help diffuse knowledge that can be used by the teaching profession, either as material to improve their own courses or as a pedagogical tool. Many MOOC participants are teachers (Seaton et al., 2014[73]). In the United States, some universities2 have partnered with MOOC platforms to propose preparatory courses to high school students and teachers in Advanced Placement courses. These offer tertiary-level curricula and examinations to high school students; top-scoring participants may obtain placement in universities or course credit (Seaton, 2016[74]). Students using MOOCs rather than standard material tend to achieve slightly better results.

Figure 5.27. Teachers participating in MOOCs, for a sample of MOOCs

MOOCs and open education can indirectly raise the quality of education and thereby ensure that students are better prepared for changing skills needs. MOOCs and open education may also increase competition between universities, but this depends on the quality of MOOCs, which remains to be better understood. Governments need more information on the quality of MOOCs before supporting their integration in the education system and adult learning. Up to now, the absence of pedagogical and technological standards and a lack of government expertise and reactivity on this subject have made this type of public investment risky (Music, 2016[66]). Adapting MOOCs to a larger group of participants – including those with few computer skills, few skills to know how to learn and little capacity to motivate themselves – remains another important challenge.

Recognising and certifying skills as sources for learning diversify

The way skills are recognised and certified needs to evolve as the learning environment and the world of work are affected by the digital transformation. As digitalisation of learning increases, more people are likely to acquire skills outside formal education. As a result, qualifications obtained through initial education may reflect less and less well the skills people have. At the same time, the digitalisation of the world of work is changing skills needs, so employers may need broader and more up-to-date information on workers’ skills than standard qualifications can supply.

Rationale

The expansion of skills acquisition throughout life and through diverse sources, including open education and MOOCs, poses the question of how to formally recognise and certify these newly acquired skills. Technology enables people to develop skills through non-formal learning (structured classes that do not lead to a formal degree) and informal learning (learning that takes place as part of other activities) that are not always reflected in qualifications.

The need to ensure that qualifications better reflect skills is also being felt at universities, where the increasing share of students has been accompanied by more variability in the skills of young graduates (Paccagnella, 2016[75]).

In parallel, employers need different skills that are not reflected in most qualifications. More and more they are valuing social and emotional skills (Deming, 2017[76]). As technology makes knowledge and skills date more quickly, employers need workers with the capacity to learn and adapt to new tools and methods.

When part of the chain can be automated, the economic value of the remaining tasks performed by workers increases and poor performance greatly reduces the value of output (Autor, 2015[77]; OECD, 2017[78]). Hence, employers increasingly need clear signals about workers skills. When diplomas reliably reflect what workers can do, workers are more likely to be able to perform well the tasks for which they have been recruited. In addition, the world of work is increasingly global, requiring transparency and standardisation of skills and qualifications acquired abroad.

The Survey of Adult Skills (PIAAC) shows how countries differ in their skills dispersion, for instance in terms of literacy skills (OECD, 2017[78]). Some observable characteristics, such as the level of education, participation in training, age, and gender explain this dispersion. However, part of the dispersion cannot be explained by differences in observable characteristics; this is called the unobservable skills dispersion. In countries with high unobservable skills dispersion, employers face greater difficulties in recruiting individuals who perform at the level expected given their education level and other observable characteristics. An indicator based on the Survey of Adult Skills (PIAAC) shows that this dispersion is low in countries like the Czech Republic, Korea, Japan, and the Slovak Republic.

Policies

A better signalling of skills requires recognising and certifying: 1) skills acquired throughout life and therefore outside formal education, for instance skills acquired by learning on the job and those acquired through non-formal education such as MOOCs; 2) a broader range of skills than those included in standard diplomas, such as social and emotional skills.

Figure 5.28. Dispersion in unobservable component of literacy skills

There are two major non-exclusive approaches for certifying a broader range of skills acquired throughout life. The first approach consists in facilitating the recognition of skills acquired outside formal education, such as through learning on the job, within the formal qualification framework. For instance, in recognition of pre-existing skills, the duration of a formal education and training programme can be reduced, or a person can obtain direct access to the final qualifying examination. This approach exists in many countries and is well-suited to recognising and certifying specific skills acquired through work‑based learning, such as through apprenticeship (Kis, 2018[79]). This approach leads to a real assessment of skills and could therefore be expected to be fully recognised by employers. Expanding this approach would be costly, however, for learners and for education and training institutions. Apart from those in regulated occupations, most learners would not always need the additional qualification for their career progression. Finally, while this approach can help in certifying skills acquired throughout life, it is unlikely to lead to the recognition of a broader range of skills than those already recognised in qualifications.

The second approach for certifying a broader range of skills acquired throughout life consists in developing certification of skills acquired through non-formal or informal learning as a complement to formal qualification. Digitalisation has boosted skills certification through the emergence of open badges and credentialing platforms (Box 5.5). In addition to enabling certification of skills acquired throughout life, these programmes aim to signal a broader range of skills than qualifications do, including not only professional and technical skills but also social and emotional skills, such as leadership qualities.

At the moment, certification mechanisms rely on proof of participation in learning activities, work experience or other types of activities but do not test the skills of individuals. For certificates of non-formal and informal learning to a play a bigger role in education and work trajectories, such certification would need to be based on valid and reliable assessments.

As sources for learning diversify and lifelong learning becomes increasingly important, it will become crucial to separate the assessment of skills from the provision of education and training. Some large firms, including in the ICT industry, test skills on their own and rely less on diplomas. However, this approach is not suited to all firms and all occupations. Assessing practical technical skills directly, in an authentic working environment, can be very costly because of the material and equipment involved (Kis, 2018[79]). Technology may enable cheaper ways of assessing practical skills. Apart from providers of formal education and training, however, no institutions have proved yet that they have the capacity to develop reliable assessments of skills on a large scale.

Box 5.5. Online certificates, badges and portfolios

Online certification has proliferated over the last years. Open badges, introduced in 2011 by the Mozilla Foundation, aim to recognise skills from various activities, especially those developed outside formal education. They are issued through an open badges platform by education and training providers, employers and many other organisations that propose non-formal and informal learning. Badges may enable individuals to present their skills in a more flexible way than full qualifications, or to signal specific interests or knowledge.

In the United States, several large firms have adopted open badges, including IBM, Microsoft and Oracle (Fong, Janzow and Peck, 2016[79]). ACE CREDIT, the US organisation in charge of validating non-formal and informal learning, has partnered with Credly, an open badges platform, to enable education and training providers, employers, and other participants to issue open badges.

Outside the United States, RMIT University in Melbourne is also working with Credly to issue badges for skills that firms value and that are not tested in exams.

Some platforms propose direct certification of skills, in which certificates are issued on the request of individuals. Degreed, launched in 2012, proposes the certification of more than 1 500 skills. The site validates work experience and other learning events and breaks down these learning experiences into skills categories. Validation is based on the firm’s expertise and does not involve exams or tests.

Students, workers and job seekers often share the open badges and digital certificates they have acquired on social media platforms like LinkedIn, Facebook and Twitter. LinkedIn is also an example of online portfolios that have developed in parallel with online certification. While portfolios do not certify skills, workers use them to showcase their experience and signal their skills to employers, who may use this information in the recruitment process.

Source: Fong, J., P. Janzow and K. Peck (2016[79]), “Demographic Shifts in Educational Demand and the Rise of Alternative Credentials”, https://upcea.edu/wp‑content/uploads/2017/05/Demographic‑Shifts‑in‑Educational-Demand-and-the-Rise-of-Alternative-Credentials.pdf (accessed on 16 April 2018).

Recognising skills acquired through open education poses a particular challenge, especially in the case of MOOCs, as they resemble formal education. Most open education and MOOC learners already have a tertiary degree. For these participants, obtaining an additional certification may be less important than evidence of participation in a learning or skill acquisition activity (Figure 5.21). For high-school or university students, however, it might be important to gain credits that are recognised within the formal education system.

MOOCs have already moved a long way towards certification. Most MOOCs lead to a certificate issued by MOOC platforms or jointly by the MOOC platform and the provider, such as open badges or other types of digital badges. Recently, MOOC platforms have developed “nanodegrees” (Udacity), “micromasters” (edX) or “specialisations” (Coursera), comprised of a bundle of around five courses on a specific topic. They may constitute good skills signals for employers as they encapsulate a range of competences that are necessary for that specific discipline. In addition, they can sometimes enable students to apply for an accelerated on-site programme.3

In most cases, certificates earned through MOOCs are not understood as part of larger qualifications. In the United States, however, ACE CREDIT, the organisation in charge of validating non-formal and informal learning (part of the American Council on Education), has included MOOC certificates in its credit recognition programme, although only a small number of MOOCs have been certified so far (Box 5.5). Higher education institutions and employers can use recommendations from this organisation to make their validation decisions. Some institutions in Europe offer formal accreditation in terms of the European credit transfer and accumulation system but accumulation of these credits does not entail the award of a degree.

Assessing what someone has learned from a MOOC requires making sure that the person who takes the test is the one who took the online course. In 2013, Coursera launched a verified certificate system that considers the typing pattern of the students to link them to their ID and deliver a nominative course completion certificate. Half of the courses offered by Coursera were eligible for this type of certificate in 2016.

Improving recognition and certification of skills to respond to employers’ changing needs and evolving ways of learning requires strong co-operation between governments (including national accreditation agencies), education and training providers, and employers. Options to better recognise and certify skills include:

Moving to a competencies-based approach to formal qualification, to improve transparency and homogeneity of diplomas issued by different education institutions. A competencies-based approach has developed over the last decades in higher education (Nodine, 2016[80]). Participation of employers in the design and review of qualification frameworks is important to ensure the qualifications are recognised.
Encouraging the development of certificates for skills acquired through non-formal and informal learning. In parallel, governments, education and training providers, and employers can co-operate to define standards and good practices for certification, to move towards a more reliable assessment of the skills people really have.
Integrating certificates earned through non-formal and informal learning in national qualification frameworks. This would need to be done on a case by case basis, respecting all relevant standards, to secure the trust of employers and education providers. Whether certificates can lead to credits or other routes to a formal qualification would be for education providers to decide.
Governments can work together to harmonise recognition and certification of skills practices at an international level.

Summary

The digital transformation offers many new sources and forms of learning in schools, in jobs and at home. However, the benefits of these new forms of learning cannot be taken for granted.

In schools, technology needs to be carefully integrated so that it amplifies teaching and learning. Teachers need to be trained to use technology to improve teaching practices and students’ results. Technology can enable more individualised teaching, in which students can progress at their own pace. Instead of spending a large amount of time delivering traditional classes, teachers can devote more time to teaching complex skills, such as critical thinking and team work, and have a computer provide routine information. In practice, few countries seem to have realised the potential of technology as a teaching and learning asset on a large scale. As well as learning via technology at school, students need to learn about technology, including digital skills, such as browsing safely and effectively on the Internet, computational thinking, and digital critical thinking. Countries need to adopt strategies for introducing technology in schools that go beyond quantitative aspects such as the number of tablets per children.

Training teachers to help them make the most of technology at school is crucial. Students from various OECD countries face the same needs in digital skills, but there are large variations among countries in teachers’ problem-solving skills in technology-rich environments.

Countries need to assess regularly the effect of technology in schools to make sure that it helps and does not hinder students’ learning. As technology evolves, the way students make use of it and the time they devote to it keep on changing. The development of the use of smartphone and chatting in schools is a good example of these changes.

Policies on the integration of technology need to be adjusted regularly to make the most of the positive effects while limiting negative ones.

Outside schools, technology also offers potential for learning via open distance resources, particularly MOOCs. However, these new learning opportunities are mostly benefiting highly skilled people, even though they are accessible to everybody. Governments can co-operate with education and training providers, employers, job-search agencies and social policy institutions to realise the full potential of open education as a universal learning tool.

References

[16] ACARA (n.d.), Information and Communication Technology (ICT) Capability, The Australian Curriculum, Australian Curriculum, Assessment and Reporting Authority, https://www.australiancurriculum.edu.au/f-10-curriculum/general-capabilities/information-and-communication-technology-ict-capability/ (accessed on 17 May 2018).

[17] ACARA (n.d.), Information and Communication Technology Capability Learning Continuum, Australian Curriculum, Assessment and Reporting Authority, https://www.australiancurriculum.edu.au/media/1074/general-capabilities-information-and-communication-ict-capability-learning-continuum.pdf (accessed on 14 December 2018).

[35] Angrist, J. and V. Lavy (2002), “New evidence on classroom computers and pupil learning”, The Economic Journal, Vol. 112/482, pp. 735-765, http://dx.doi.org/10.1111/1468-0297.00068.

[77] Autor, D. (2015), “Why are there still so many jobs? The history and future of workplace automation”, Journal of Economic Perspectives, Vol. 29/3, pp. 2015-3, http://dx.doi.org/10.1257/jep.29.3.3.

[31] Balanskat, A. and K. Engelhardt (2015), Computing Our Future. Computer Programming and Coding. Priorities, School Curricula and Initiatives across Europe, European Schoolnet, http://fcl.eun.org/documents/10180/14689/Computing+our+future_final.pdf/746e36b1-e1a6-4bf1-8105-ea27c0d2bbe0 (accessed on 29 March 2018).

[39] Banerjee, A. et al. (2007), “Remedying education: Evidence from two randomized experiments in India”, The Quarterly Journal of Economics, Vol. 122/3, pp. 1235-1264, http://dx.doi.org/10.1162/qjec.122.3.1235.

[60] Barber, M. and M. Mourshed (2007), How the World’s Best-performing School Systems Come Out on Top, https://www.mckinsey.com/~/media/mckinsey/industries/social%20sector/our%20insights/how%20the%20worlds%20best%20performing%20school%20systems%20come%20out%20on%20top/how_the_world_s_best-performing_school_systems_come_out_on_top.ashx (accessed on 22 February 2018).

[11] Barrow, L., L. Markman and C. Rouse (2009), “Technology’s edge: The educational benefits of computer-aided instruction”, American Economic Journal: Economic Policy, http://dx.doi.org/10.1257/pol.1.1.52.

[25] Bell, T. (2016), What’s All the Fuss About Coding?, Australian Council for Educational Research, https://research.acer.edu.au/cgi/viewcontent.cgi?article=1288&context=research_conference (accessed on 27 March 2018).

[26] Bocconi, S. et al. (2016), Developing Computational Thinking in Compulsory Education – Implications for Policy and Practice, European Commission Joint Research Centre, Brussels, http://dx.doi.org/10.2791/792158.

[14] Broadband Commission for Sustainable Development (2017), Working Group on Education: Digital Skills for Life and Work, https://unesdoc.unesco.org/ark:/48223/pf0000259013 (accessed on 13 December 2018).

[8] Bulman, G. and R. Fairlie (2016), “Technology and education: Computers, software, and the Internet”, in Handbook of the Economics of Education, http://dx.doi.org/10.1016/B978-0-444-63459-7.00005-1.

[24] Carretero, S., R. Vuorikari and Y. Punie (2017), DigComp 2.1. The Digital Competence Framework for Citizens, Publications Office of the European Union, Luxembourg, http://publications.jrc.ec.europa.eu/repository/bitstream/JRC106281/web-digcomp2.1pdf_(online).pdf (accessed on 28 March 2018).

[51] Chatterji, A. (2017), “Innovation and American K-12 education”, No. 23531, NBER, http://www.nber.org/papers/w23531 (accessed on 20 December 2017).

[61] Chetty, R., J. Friedman and J. Rockoff (2014), “Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood”, American Economic Review, Vol. 104/9, pp. 2633-2679, http://dx.doi.org/10.1257/aer.104.9.2633.

[43] Cheung, A. and R. Slavin (2012), “How features of educational technology applications affect student reading outcomes: A meta-analysis”, Educational Research Review, Vol. 7/3, pp. 198-215, http://dx.doi.org/10.1016/J.EDUREV.2012.05.002.

[67] Chuang, I. and A. Ho (2016), “HarvardX and MITx: Four years of open online courses - Fall 2012-Summer 2016”, SSRN Electronic Journal, http://dx.doi.org/10.2139/ssrn.2889436.

[71] Class Central (2017), The 50 Most Popular MOOCs of All Time, https://www.onlinecoursereport.com/the-50-most-popular-moocs-of-all-time/ (accessed on 20 February 2018).

[48] Comi, S. et al. (2017), “Is it the way they use it? Teachers, ICT and student achievement”, Economics of Education Review, Vol. 56, pp. 24-39, http://dx.doi.org/10.1016/J.ECONEDUREV.2016.11.007.

[23] Conrads, J. et al. (2017), Digital Education Policies in Europe and Beyond: Key Design Principles for More Effective Policies, Publications Office of the European Union, Luxembourg, http://dx.doi.org/10.2760/462941.

[37] Cristia, J. et al. (2017), “Technology and child development: Evidence from the One Laptop per Child program”, American Economic Journal: Applied Economics, Vol. 9/3, pp. 295-320, http://dx.doi.org/10.1257/app.20150385.

[28] CSTA/ISTE (2011), Computational Thinking. Teacher Resources, http://www.iste.org/docs/ct-documents/ct-teacher-resources_2ed-pdf.pdf?sfvrsn=2 (accessed on 27 March 2018).

[76] Deming, D. (2017), “The growing importance of social skills in the labor market”, The Quarterly Journal of Economics, Vol. 132/4, pp. 1593-1640, http://dx.doi.org/10.1093/qje/qjx022.

[50] Denoël, E. et al. (2017), Drivers of Student Performance: Insights from Europe, McKinsey & Company, https://www.mckinsey.com/~/media/mckinsey/industries/social%20sector/our%20insights/drivers%20of%20student%20performance%20insights%20from%20europe/drivers-of-student-performance-insights-from-europe-the-book.ashx (accessed on 20 February 2018).

[34] Escueta, M. et al. (2017), “Education technology: An evidence-based review”, http://www.nber.org/papers/w23744.

[49] European Commission (2013), Survey of Schools: ICT in Education, Publications Office of the European Union, Luxembourg, http://dx.doi.org/10.2759/94499.

[3] Eurostat (2017), Digital Economy and Society Statistics, Comprehensive Database, http://ec.europa.eu/eurostat/web/digital-economy-and-society/data/comprehensive-database.

[47] Falck, O., C. Mang and L. Woessmann (2015), “Virtually no effect? Different uses of classroom computers and their effect on student achievement”, IZA DP No. 8939.

[15] Ferrari, A. (2013), DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe, European Commission. Joint Research Centre, Luxembourg, http://dx.doi.org/10.2788/52966.

[33] Feurzeig, W., S. Papert and B. Lawler (2010), “Programming-languages as a conceptual framework for teaching mathematics”, Interactive Learning Environments, http://dx.doi.org/10.1080/10494820903520040.

[45] Fleischer, H. (2012), What Is Our Current Understanding of One-to-one Computer Projects: A Systematic Narrative Research Review, http://dx.doi.org/10.1016/j.edurev.2011.11.004.

[79] Fong, J., P. Janzow and K. Peck (2016), “Demographic Shifts in Educational Demand and the Rise of Alternative Credentials”, https://upcea.edu/wp-content/uploads/2017/05/Demographic-Shifts-in-Educational-Demand-and-the-Rise-of-Alternative-Credentials.pdf (accessed on 16 April 2018).

[65] Goodman, J., J. Melkers and A. Pallais (2018), “Can online delivery increase access to education?”, Journal of Labor Economics, Vol. 37/1, pp. 1-34, http://dx.doi.org/10.3386/w22754.

[62] Grundke, R. et al. (2017), “Skills and global value chains: A characterisation”, OECD Science, Technology and Industry Working Papers, No. 2017/05, http://dx.doi.org/10.1787/cdb5de9b-en.

[72] Hamori, M. (2018), “Can MOOCs solve your training problem?”, Harvard Business Review, https://hbr.org/2018/01/can-moocs-solve-your-training-problem (accessed on 7 May 2018).

[57] Hanushek, E., M. Piopiunik and S. Wiederhold (2014), “The value of smarter teachers: International evidence on teacher cognitive skills and student”, NBER Working Paper Series Working Paper 20727, http://www.nber.org/papers/w20727 (accessed on 13 April 2018).

[22] HITSA Information Technology Foundation in Education (n.d.), ProgeTiger Programme, https://www.hitsa.ee/it-education/educational-programmes/progetiger (accessed on 14 December 2018).

[20] Hoechsmann, M. and H. DeWaard (2015), Mapping Digital Literacy Policy and Practice in the Canadian Education Landscape, MediaSmarts, http://mediasmarts.ca/teacher-resources/digital-literacy-framework/mapping-digital-literacy-policy-practice-canadian-education-landscape (accessed on 14 December 2018).

[13] Kennedy, G. et al. (2010), “Beyond natives and immigrants: Exploring types of net generation students”, Journal of Computer Assisted Learning, Vol. 26/5, pp. 332-343, http://dx.doi.org/10.1111/j.1365-2729.2010.00371.x.

[36] Leuven, E. et al. (2007), “The effect of extra funding for disadvantaged pupils on achievement”, The Review of Economics and Statistics, Vol. 89(4), pp. 721–736, https://www.mitpressjournals.org/doi/pdf/10.1162/rest.89.4.721 (accessed on 25 January 2018).

[32] Lye, S. and J. Koh (2014), “Review on teaching and learning of computational thinking through programming: What is next for K-12?”, Computers in Human Behavior, Vol. 41, pp. 51-61, http://dx.doi.org/10.1016/J.CHB.2014.09.012.

[38] Machin, S., S. Mcnally and O. Silva (2007), “New technology in schools: Is there a payoff?”, The Economic Journal 117 (July), pp. 1145–1167.

[21] Manitoba Education and Training (n.d.), Literacy with ICT - What is LwICT?, https://www.edu.gov.mb.ca/k12/tech/lict/what/index.html (accessed on 14 December 2018).

[19] Ministère de l’Éducation nationale et de la Jeunesse (2018), Le numérique au service de l’École de la confiance, http://www.education.gouv.fr/cid133192/le-numerique-service-ecole-confiance.html (accessed on 14 December 2018).

[66] Music, A. (2016), Massive Open Online Courses (MOOCs): Trends and Future Perspectives, OECD, Berlin.

[80] Nodine, T. (2016), “How did we get here? A brief history of competency-based higher education in the United States”, The Journal of Competency-Based Education, Vol. 1/1, pp. 5-11, http://dx.doi.org/10.1002/cbe2.1004.

[63] OECD (2017), Education at a Glance 2017: OECD Indicators, OECD Publishing, Paris, http://dx.doi.org/10.1787/eag-2017-en.

[78] OECD (2017), OECD Skills Outlook 2017: Skills and Global Value Chains, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264273351-en.

[46] OECD (2017), PISA 2015 Results (Volume V): Collaborative Problem Solving, PISA, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264285521-en.

[12] OECD (2016), Innovating Education and Educating for Innovation: The Power of Digital Technologies and Skills, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264265097-en.

[54] OECD (2016), Skills Matter: Further Results from the Survey of Adult Skills, OECD Skills Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264258051-en.

[6] OECD (2015), PISA database 2015, OECD, Paris, http://www.oecd.org/pisa/data/2015database/.

[7] OECD (2015), Students, Computers and Learning: Making the Connection, PISA, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264239555-en.

[56] OECD (2015), Survey of Adult Skills (PIAAC), PIAAC, OECD, Paris, http://www.oecd.org/skills/piaac/publicdataandanalysis.

[59] OECD (2014), PISA 2012 Technical Report, OECD, Paris, https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf (accessed on 9 April 2018).

[58] OECD (2013), PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy, PISA, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264190511-en.

[52] OECD (2013), TALIS database 2013, OECD, Paris, http://www.oecd.org/education/school/talis-2013-results.htm.

[9] OECD (2012), PISA database 2012, OECD, Paris, http://www.oecd.org/pisa/pisaproducts/pisa2012database-downloadabledata.htm.

[55] OECD (2012), Survey of Adult Skills (PIAAC), PIAAC, OECD, Paris, http://www.oecd.org/skills/piaac/publicdataandanalysis.

[10] OECD (2009), PISA database 2009, OECD, Paris, http://www.oecd.org/pisa/pisaproducts/.

[70] Onah, D., J. Sinclair and R. Boyatt (2014), “Dropout rates of massive open online courses: Behavioural patterns”, https://warwick.ac.uk/fac/sci/dcs/people/research/csrmaj/daniel_onah_edulearn14.pdf (accessed on 10 April 2018).

[1] Orr, D., M. Rimini and D. van Damme (2015), Open Educational Resources: A Catalyst for Innovation, Educational Research and Innovation, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264247543-en.

[75] Paccagnella, M. (2016), “Literacy and Numeracy Proficiency in IALS, ALL and PIAAC”, OECD Education Working Papers, No. 142, OECD Publishing, Paris, http://dx.doi.org/10.1787/5jlpq7qglx5g-en.

[29] Paniagua, A. and D. Istance (2018), Teachers as Designers of Learning Environments: The Importance of Innovative Pedagogies, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/9789264085374-en.

[44] Peterson, A. et al. (2018), “Understanding innovative pedagogies: Key themes to analyse new approaches to teaching and learning”, OECD Education Working Papers, No. 172, OECD Publishing, Paris, https://dx.doi.org/10.1787/9f843a6e-en.

[18] Praxis (2017), ICT Education in Estonian Schools and Kindergartens, http://www.praxis.ee/en/works/ict-education-in-estonian-schools-and-kindergartens/ (accessed on 14 December 2018).

[42] Roschelle, J. et al. (2016), “Online mathematics homework increases student achievement”, AERA Open, Vol. 2/4, p. 233285841667396, http://dx.doi.org/10.1177/2332858416673968.

[41] Roschelle, J. et al. (2010), “Integration of technology, curriculum, and professional development for advancing middle school mathematics: Three large-scale studies”, American Educational Research Journal, Vol. 47/4, pp. 833-878, http://dx.doi.org/10.3102/0002831210367426.

[74] Seaton, D. (2016), Complementary Models of MOOC Instruction for Advanced Placement High School Courses, edX Blog, https://blog.edx.org/complementary-models-mooc-instruction-advanced-placement-high-school-courses (accessed on 3 April 2018).

[73] Seaton, D. et al. (2014), “Teacher enrollment in MIT MOOCs: Are we educating educators?”, SSRN Electronic Journal, http://dx.doi.org/10.2139/ssrn.2515385.

[4] Storm, B., S. Stone and A. Benjamin (2017), “Using the Internet to access information inflates future use of the Internet to access other information”, Memory, Vol. 25/6, pp. 717-723, http://dx.doi.org/10.1080/09658211.2016.1210171.

[40] The Economist (2018), “In poor countries technology can make big improvements to education”, https://www.economist.com/international/2018/11/17/in-poor-countries-technology-can-make-big-improvements-to-education (accessed on 12 December 2018).

[64] Vincent-Lancrin, S. (2016), Open Higher Education: What Are We Talking About?.

[30] Voogt, J. et al. (2015), “Computational thinking in compulsory education: Towards an agenda for research and practice”, Education and Information Technologies, Vol. 20, pp. 715-728, http://dx.doi.org/10.1007/s10639-015-9412-6.

[53] Voogt, J. et al. (2013), “Under which conditions does ICT have a positive effect on teaching and learning? A call to action”, Journal of Computer Assisted Learning, http://dx.doi.org/10.1111/j.1365-2729.2011.00453.x.

[69] Wang, Y. and R. Baker (2015), “Content or platform: Why do students complete MOOCs?”, MERLOT Journal of Online Learning and Teaching, Vol. 11/1, http://jolt.merlot.org/vol11no1/Wang_0315.pdf (accessed on 21 March 2018).

[5] Ward, A. et al. (2017), “Brain drain: The mere presence of one’s own smartphone reduces available cognitive capacity”, Journal of the Association for Consumer Research, Vol. 2/2, pp. 140-154, http://dx.doi.org/10.1086/691462.

[2] Weinberger, D. (2011), Too Big To Know: Rethinking Knowledge Now That the Facts Aren’t the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room, Basic Books.

[27] Wing, J. (2006), Computational Thinking, https://www.cs.cmu.edu/~15110-s13/Wing06-ct.pdf (accessed on 9 April 2018).

[68] Zhenghao, C. et al. (2015), Who’s Benefiting from MOOCs, and Why, Harvard Business Review, https://hbr.org/2015/09/whos-benefiting-from-moocs-and-why (accessed on 3 April 2018).

Notes

← 1. Between 2009 and 2015, the largest drops in students’ use of ICT infrastructure available in schools were observed in Denmark (25 percentage points) and Poland (20 percentage points). In Denmark, more than 80% of students at grade 11 reported using their own laptops in class for learning at least once per week (European Commission, 2013[49]).

← 2. In 2015, Davidson College in the United States launched a series of online test preparation modules through the MOOC provider edX for high school students and teachers in Advanced Placement courses.

← 3. For instance, learners who pass an integrated set of MITx graduate-level courses on edX.org, and one or more proctored exams, will earn a MicroMasters credential from MITx, and can then apply for an accelerated on-campus master’s degree programme at MIT or other top universities.

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment

Governance

Health

Industry, business and entrepreneurship

Regional, rural and urban development

Science, technology and innovation

Society

Taxation

Trade

Energy

Nuclear energy

Transport

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment

Governance

Health

Industry, business and entrepreneurship

Regional, rural and urban development

Science, technology and innovation

Society

Taxation

Trade

Energy

Nuclear energy

Transport

Countries A - C

Countries D - I

Countries J - M

Countries N - R

Countries S - T

Countries U - Z

Regional and global engagement

Countries

Countries A - C

Countries D - I

Countries J - M

Countries N - R

Countries S - T

Countries U - Z

Regional and global engagement

Publications

Publications

Featured publications

Data

Data

Featured data

News & Events

News & Events

Featured Events

About OECD

About

Engage with us

Work with us

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment