Abstract: This chapter gives an overview of why and how we measure innovation in education, relates the methodology used to other existing measures or approaches, and provides a summary of the main findings of the book. It ends by pointing to some possible next steps for strengthening the measurement of innovation in the education sector.
Measuring Innovation in Education 2019
What Has Changed in the Classroom?
Educational Research and Innovation
Chapter 1. Overview
Abstract
The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.
Why measure innovation in education?
The understanding of innovation is essential to the improvement of education. Developing the ability to measure it as well as its drivers and effects is a first step to refining this understanding.
Monitoring systematically whether, and how, practices are changing within classrooms and educational organisations, how teachers develop professionally and use learning resources, how schools communicate with their communities, and to what extent change and innovation are linked to better educational outcomes would provide a substantial increase in the international education knowledge base. Policy makers would be able to better target interventions and resources, get quick feedback on whether reforms changed educational practices as expected, and we would better understand the conditions for and impact of innovation in education.
The OECD project Measuring Innovation in Education uses three perspectives for addressing these issues: 1) comparing innovation in education to innovation in other sectors (see OECD, 2014); 2) identifying meaningful innovations across educational systems; and 3) constructing metrics in order to examine the relationship between educational innovation and changes in educational outcomes. This publication mainly focuses on the two latter points.
The work also aims to set the basis for cumulative work on educational innovation and educational innovation policy by providing countries with indicators that can be regularly updated over time (and a methodology to do so). While this can partially rely on the use of existing international data sets, the work also aims to analyse and better understand the drivers of innovation in the education sector (see Vincent-Lancrin, 2017), where countries stand in this area, and to expand the methodologies and data sources to measure innovation in an accurate and comprehensive way.
How do we define innovation in education?
In accordance with international practice, we start with the definition of innovation as “a new or improved product or process (or combination thereof) that differs significantly from the unit’s previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process)” (OECD/Eurostat, 2018). Simplifying the previous edition of the Oslo Manual, which categorised innovation into product, process, marketing and organisation innovation, the new edition distinguishes between two main types of innovation: “product” innovation and “process” innovation. These two categories can easily be mapped against the four previous types of innovation.
Product innovation refers to innovation in goods and services, two categories that are sometimes intertwined, especially in the context of digitalisation. Process innovation refers to innovation in production processes or activities, that is, “all activities under the control of an institutional unit that use inputs of labour, capital, goods and services to produce outputs of goods and services”. In brief, process innovation mainly refers to innovation in organisational processes, even though processes can be broader: “processes include policies that provide an overall strategy that drives a unit’s activities, activities that transform inputs into outputs, and procedures that govern the detailed steps for activities to transform inputs into outputs” (OECD/Eurostat, 2018).
Educational organisations (e.g. schools, universities, training centres, education publishers) contribute to product innovation when they introduce new or significantly different products and services, such as new syllabi, textbooks or educational resources, or new pedagogies or educational experiences (for example e-learning or new qualifications). They contribute to process innovation when they change significantly their organisational processes for producing their educational goods or services. For example, they may change how teachers work together, how they group students and manage other aspects of their learning experience; they may collaborate with other entities, use new marketing and external relations methods, new forms of communication with students and parents, etc. In the case of services such as education, products and processes may also be difficult to tell apart.
New or significantly changed practices aim at improving the provision of education in one way or another, and should therefore be regarded as intended “improvements” (rather than proven ones). While the definition of innovation of the Oslo Manual refers to new or “improved” products and processes, the main emphasis lies in establishing shared standards about how “significantly different” or “novel” the products or processes are (rather than demonstrating they are improvements). For some goods and services, notably manufactured products, technical or cost improvements may be easy to observe and document. This is not the case for all, though, and more difficult for processes. While innovation usually aims at improving something, for example a firm’s bottom line or the performance of a good, there is no guarantee that it achieves its goal. Innovation is in fact merely a new or significantly changed (or different) product or process, and measured as such, whether it is an improvement or not. As noted in the Oslo Manual, innovation does not necessarily result in desirable outcomes for all parties. Specific innovations may also prove to be good or bad for society. It usually takes time to find out with some level of certainty whether specific innovations are improvements or not…
What are the different ways to measure innovation in education?
Two broad approaches to measuring innovation in education have been used so far, aligned with existing approaches to measuring innovation in the public sector.
The first broad approach to measuring innovation in education is the adaptation to education of national innovation surveys’ methodology (e.g. the EU Community Innovation Survey). Such surveys offer well-established tools for measuring innovation, and have been used for several decades in the private sector. In recent years, there were some efforts to adapt them for a use in the public sector (e.g. Bloch and Bugge, 2013).
Measuring Innovation in Education: A New Perspective (OECD, 2014) explored this approach and presented indicators based on the analysis of two surveys asking graduate questions about innovation in their work environment, in line with the methodology of the Community Innovation Survey. Rather than firm representatives, as is usually the case in innovation surveys, it was employees working in different sectors of the economy who were surveyed.
This “innovation survey” approach has recently been implemented to measure innovation in education in Hungary. An innovation survey was designed and administered to 5 000 educational units from all sub-systems (from pre-school to higher education) and connected to pupil performance thanks to the regular national evaluations (Halász, 2018). The survey exhibited good levels of innovation in all systems, and exhibited strong associations between innovation and performance in the case of low-performance schools. In Australia and New Zealand, a survey of management and service innovations within universities was carried out with a similar methodology (Arundel et al., 2016). A similar approach was also used in the Netherlands to analyse innovation in secondary education (Haelermans, 2010).
The second broad approach that has been used to measure innovation in the public (and business) sector is inspired by surveys of organisational change. These surveys typically measure the dissemination of specific innovations in work practices, for example computers or organisational practices (e.g. Greenan and Lorenz, 2013; MEADOW Consortium, 2010).
Measuring Innovation in Education: A New Perspective (OECD, 2014) also adapted this approach and measured innovation as a new or significantly changed process, practice, organisational or marketing method observed at the education system level through micro-data collected within schools. The emphasis is particularly placed on change in practices. Contrary to the “organisational change” surveys, change was measured by comparing reports on similar practices at different points in time. This publication also adopts this approach.
Other approaches to identify (rather than measure) innovation have also contributed to the better understanding of what innovations may transform education. Examples are the annual New Horizon reports by EDUCAUSE and formerly the New Consortium Media (Adams Becker et al., 2018).
How do we measure innovation in education in this publication?
We define innovation as a significant change in selected key practices in education (and mix thereof). We use the Programme on International Student Assessment (PISA), Trends in International Mathematics and Science Study (TIMSS) and Progress in International Reading Literacy Study (PIRLS) databases to cover and identify these key practices at the classroom or school levels. The repeated cross-sectional nature of these surveys makes it possible to map trends over time. For this reason, we focus on questions that were asked in at least two waves of these surveys and build indicators that allow identifying how much change students in a given country were exposed to.
Our indicators measure “systemic innovation”. They tell what percentages of students in a system are exposed to a given practice at more or less 10 years of interval (depending on our data source). We identify whether and to what extent some practices have gained or lost ground within a system – in the literal sense that more or less students have been exposed to them. If a given practice has increased significantly in a country, for example the use of computers in maths lessons, there has been innovation: observers waking up from a decade-long sleep would find that students are experiencing significantly different instruction methods than when they fell asleep. The same is true if the practice has significantly lost ground. Should they be significant in magnitude, both the spread and contraction of a practice correspond to an innovation for a given system and its students.
How much change counts as a significant change? There is no definitive answer to this question. The Oslo Manual acknowledges this as a key comparability challenge within and across countries and suggests that innovation survey respondents should be given a same reference point to identify what to report as innovation. Our methodology makes the challenge different. Given that innovation is not directly reported by one individual in a retrospective manner, but inferred from the reporting on the prevalence of the same practice at two different points in time by a representative sample of students, teachers or school principals, the challenge does not lie with the respondents but with those interpreting the observed change. For example, the degree to which the adoption of a teaching practice by 10% more teachers can be considered innovative depends on the context: it may be considered a more significant change in a country in which 10% of teachers used the practice than in a country in which 70% of teachers already used it. For that reason, while we focus on the change and its magnitude, we also provide readers with the actual prevalence of the practice.
We also translate these changes from percentage points to effect sizes in order to assist the readers in making their judgment about the significance of the difference. Effect sizes give a standardised measure of these changes and help interpret their relative magnitude across all indicators: the greater the effect size, the higher the magnitude (and likelier the “significance”) of change over time. In line with common practice, we refer to effect sizes below 0.2 to “small”, from 0.2 to 0.4 to “moderate”, and over 0.4, to “large”. This is a continuum though, and readers can choose their own thresholds.
What educational practices do we cover?
This edition of Measuring Innovation in Education focuses on pedagogical innovation in primary and secondary education. The publication covers 158 educational practices. Most of them (107) are pedagogical practices used by teachers during their reading, mathematics and science instruction in primary and secondary education. These pedagogical innovations cover a large number of teaching and learning strategies in reading, mathematics and science, including information about the use of homework and assessment.
The book covers three other areas of interest: the availability of learning resources (books and ICT), teacher professional development practices (formal training and peer learning), external relations with stakeholders (parents, the public at large, other education agencies). All the practices covered in this edition can thus be considered to be “business process” operations. At the same time, in the case of services, “services” and “business process” can overlap, and the distinction is more clear-cut between “product” and “business process” innovations.
Because we rely on international data that were collected to contextualise international assessments of students, the coverage of practices is not as comprehensive as one might have wished to assess innovation in all its dimensions, nor targeting enough emerging practices. Given our methodology, only practices that experts and policy makers deemed important to document 10 years ago could be covered. Given the limited comparative information available on tertiary education, we cover only primary and secondary education.
Notwithstanding these limitations, the covered practices correspond to key teaching and learning practices that countries and a community of international experts deemed important enough to be repeatedly documented to understand the performance of education systems in terms of learning outcomes.
One strength of our methodology is to clearly know which practices contribute to innovation within a country, whereas most innovation surveys identify innovation in generic terms (broad types of innovation), leaving the innovations unnamed. One other strength is that our innovation indices synthesise a large number and types of practices rather than just a few, as is usually the case with composite indices. This is particularly important when one focuses on one particular sector (education). Innovation surveys usually aim to compare different sectors of the economy, which makes the identification of relevant practices more difficult.
Being aware of the change in the key educational practices covered in the publication is important regardless of whether one is interested in innovation or not. The measurement of their level and change over time gives policy- and other decision-makers a state of the educational practices their students are exposed to. Without this visibility, they cannot know whether ongoing pedagogical practices correspond to those they would like to see in their system’s classrooms.
Is innovation necessarily “innovative”?
Can there be systemic innovation in traditional practices? Of course. Many of the practices covered in the book are not necessarily those that would come to mind when thinking about educational innovation. An emerging practice such as the flipped classroom is for instance not covered. While it would certainly be worthwhile to measure the prevalence of practices that were recently introduced in the education sector, there is no international (and perhaps even national) dataset covering the uptake of these practices. Moreover, identifying relevant practices internationally may not be trivial. Such an approach was beyond the scope and budget of this project.
Measuring the diffusion or disappearance of educational practices remains an important and valid measure of systemic innovation, even though it does not cover the entire spectrum of educational innovation. Given that teaching and learning is a mix of different practices, the appearance of new or “innovative” practices are not necessarily what changes the most significantly the educational process within a country. While learning by memorisation is an old pedagogical strategy, its disappearance from formal education would be a noticeable innovation to students in most systems. Its significant increase would also be an innovation: students would then be exposed to a significantly different teaching and learning process. In short, what is innovative may not be the practice itself.
The word “innovative” can be particularly misleading in our context. What we measure in this book is how much change students have experienced in their learning environment over a decade. Where we observe significant change, there is (systemic) innovation. This does not imply that the new practices (or mix thereof) are more innovative than the previous ones. Neither does this imply that the countries where more innovation has been observed in the past decade are intrinsically more “innovative”: they have in fact just experienced more innovation in the way education is delivered over the past decade. This may have been different in the past and may be different in the future given that innovation is often governed by cycles. The situation may also be different for other types of innovation.
We do not assume that innovation is necessarily an improvement, but it should be noted that almost all of the practices covered in this publication are “good” practices according to the research literature – although they are usually too narrow to be looked at in isolation. Education is a mix of all those instructional practices. Our comments on each practice are based on the existing research literature, for example evidence from meta-analyses (e.g. Hattie, 2008; OECD, 2010; Education Endowment Foundation, 2018). We signal the few practices that are inherently to be avoided.
Innovation can also be conceived as a mix of “alternative” practices that remain at the margins of education systems, or whose uptake remains limited (OECD, 2013). The indicators provided in this edition (as well as in the 2014 edition) give readers information about some of those practices – and allow readers to identify which practices are “mainstream” and which are “alternative”.
Country coverage
Education systems covered in this edition
Australia, Austria, Belgium, Brazil, Canada (Quebec and Ontario), Colombia, Chile, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hong Kong, China, Hungary, Iceland, Indonesia, Ireland, Israel, Italy, Japan, Korea, Latvia, Lithuania, Luxembourg, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Russian Federation, Singapore, Slovak Republic, Slovenia, South Africa, Spain, Sweden, Switzerland, Turkey, United Kingdom (England), United States (including Massachusetts and Minnesota).
Education systems covered in the online tables
Argentina, Armenia, Bahrain, Bulgaria, Croatia, Egypt, Georgia, Jordan, Kazakhstan, Kuwait, Islamic Republic of Iran, Lebanon, Malaysia, Malta, Morocco, Oman, Qatar, Montenegro, Romania, Saudi Arabia, Chinese Taipei, Thailand, Tunisia, Ukraine, Uruguay.
Figure 1.1. Education systems covered in this edition
Note: Education systems covered in blue are part of the main report, while those in grey are included in the online tables.
Systemic innovation in primary and secondary education
On average there has been a moderate level of innovation in educational practices in primary and secondary education in the OECD area. At the education system level, students have experienced a different mix of teaching and learning practices compared to their peers at the same level 10 years ago. As already noted in the 2014 edition of Measuring Innovation in Education, there is a fair level of innovation in primary and secondary education and depicting it as devoid of innovation is certainly ungrounded. However, as innovation has remained moderate rather than large in the past decade, while education is not quite the same as what it used to be, it is still easily recognisable.
The average innovation index for OECD countries has been a bit greater between 2005-2016 than it was between 2001-2011, pointing to increased changes in educational practices in recent years. Comparisons between the two editions of Measuring Innovation in Education should be taken with caution though, given changes in the methodology and country coverage. That being said, over time repeated measures of innovation could give us an accurate idea of whether innovation has intensified or slowed down in the OECD area or within specific education systems over a certain period of time.
The innovation intensity has not varied much across countries overall: most of them are close to the OECD average. There are some differences though: some education systems such as Japan, Ontario (Canada) or the United States have had more stable educational practices over the past decade, while others, such as Quebec (Canada) or Slovenia, have experienced more innovation. As was the case in the previous edition, innovation has not necessarily concerned the same educational practices across countries. Apart from the increase in peer learning for teachers, the increase in the use of ICT in school work, and the slight decrease in access to computers, changes in educational practices have not been consistent across countries. In spite of stronger international learning across countries, there is no convergence in the adopted changes.
Systemic innovation was also measured separately in primary and secondary education, as well as in relation to different disciplines. The average level of innovation in educational practices is about the same in primary and secondary education, so that the two levels contribute equally to the overall innovation index. The variation across countries is also similar in primary and in secondary education, ranging from countries that have experienced moderate-small levels of innovation in their system practices to others with large or moderate-large levels. Countries for which innovation indices could be computed at both levels experienced similar levels of innovation in primary and secondary education, suggesting that innovation might have come from similar forces within country (or at least gone hand in hand in primary and secondary education).
Pedagogical innovation
Pedagogical innovation in mathematics, science and reading lessons is the main focus of the book. On average, it has been moderate in the last decade. What are the practices that have significantly spread (or receded)? The largest innovation lay in independent knowledge acquisition and homework practices, followed by both rote learning and active learning practices.
The main innovation in independent knowledge acquisition lay in the use of computers during lessons to look up for ideas and information. In science and reading, the practice has increased by around 20 percentage points on average in the OECD area, but already concerned 20 to 30% students at the beginning of the studied period. The real novelty is in mathematics for which it was hardly used 10 years ago: the share of students using computers during lessons to search ideas and information in maths went up from 3 to 31% in primary education, and from 5 to 23% in secondary education. In some countries, such as the United States, Australia or New Zealand, the increase was even more spectacular.
Practices around homework represented a second big domain of pedagogical innovation. Whereas there was virtually no change on average in the frequency of homework, the main consistent change among countries occurred in teachers discussing their homework in class in secondary education: the share of students that experienced this practice systematically increased from 22 to 58% on average in maths, and from 25 to 55% in science between 2007 and 2015. In Hungary or Lithuania, the practice was nascent in 2007, and almost universal in 2015.
Learning by memorising and drilling is often opposed to active learning. However, they can also go hand in hand. The spread of both types of practices has been moderate, but has gone in the same upward direction. Memorising rules, procedures and facts in at least half of the maths and science lessons has gained ground. The share of students concerned expanded from 22 to 43% in primary maths education, and increased by about 15 percentage points in primary and in secondary science lessons to reach about one student in two. As for active learning in science, it has mainly spread in primary science lessons. For example, the share of students asked to plan or design an experiment in at least half of their lessons increased from 19 to 37% in primary education (and 19 to 31% in secondary education).
Interestingly, in spite of the enhanced awareness of the need to develop students’ higher order skills, there has been relatively little expansion in the practices trying to foster them. Only practices fostering observation skills in science have increased significantly, while opportunities given to students to explain their ideas, draw conclusions or make inferences remained stable and concerned relatively few students.
Technology-related innovation
Most people associate innovation to (information and communication) technology, perhaps because this is the most visible form in an increasingly digital world. While innovation in educational practices is not necessarily related to technology, innovation in the availability of computers and in the use of ICT in student’s school work have actually been strong drivers of change over the past decade.
In almost all countries, students have experienced small decreases in the availability of desktop computers and tablets for use in their reading, maths and science lessons, as well as less desktop computers available in school. The Russian Federation and New Zealand are the only two exceptions. This is a paradoxical trend, confirmed by several country-level studies. However, access to desktop remains very high: 80% of secondary students on average still have access to desktop computers at school, and an increasing share have had access to laptop computers. In Sweden and Denmark where the share of secondary students having access to desktop computers in school has dropped to around 65%, 85-90% of students could access laptops in school in 2015. By contrast, students having access to desktops decreased significantly in Poland and Japan (to about 65%) without any notable increase in access to laptops, so that there was arguably no strong substitution effect. The availability of computers and tablets during lessons has decreased (be it in mathematics, science or reading). This downward trend may be explained by a variety of reasons: the greater availability of computers at home may have changed the role of computers in school, mobile phones and personal computers may be used under a “bring your own device” policy, etc.
The decrease in the availability of computers has been accompanied by an intensified use of computers and information technology. This is the case in all covered countries, except Portugal, Chile, and to a lesser extent Ireland. A greater percentage of students having access to computers use them in their lessons and for their schoolwork. On average, in an OECD education system the share of students using computers to practice their maths skills and procedures at least once a week has increased by 42 percentage points in primary education (to 51%) and by 23 percentage points (to 32%) in secondary education. The average share of students using computers to practice their science skills and procedure at least once a week increased by 17 percentage points in primary education and 15 percentage points in secondary education (to 22% and 26% respectively). And in reading, the average share of students using computers to write stories and texts at least once a week increased by 10 percentage points (to 34%). Looking up for ideas and info on computers in mathematics, science and reading is a new practice that has spread quickly over the past decade, with a significant increase by 27 percentage points in primary mathematics (from 3 to 31% of students doing it on average), and by around 20% in secondary mathematics (from 5% to 23%), primary and secondary science (22 to 39% and 17 to 38%, respectively), and primary reading (30 to 52%). The use of computers to access information has thus continued to spread across systems, and emerged and diffused quickly in mathematics.
Innovation in teacher professional development
One of the most remarkable innovations for students lay in how their teachers developed their professional knowledge. In brief, the share of students taught by teachers who took part in peer learning increased considerably, while those taught by teachers who attended a formal teacher training in the past two years remained stable. Given the importance of peer learning for professional development, this is good news. In some countries, a strong increase in peer learning seemed to have been accompanied by a strong decrease in formal teacher training – an innovation which is more difficult to assess as such.
On average, the share of students with teachers who participated in a formal teacher training programme remained relatively stable. The OECD average usually points to a small decrease that rarely exceeds 10 percentage points. There are a few exceptions, but only training about the integration of IT in mathematics has increased by more than 4 percentage points (7 percentage points). Overall, this consistent downward change represented a small innovation for students. However, average stability sometimes hides contrasting directions of change within countries. For example, during the past decade, the share of Slovenian students whose primary teachers had a training in mathematics, in science, in maths pedagogy or in science pedagogy dropped significantly (from 43 to 20%, 63 to 24%, 35 to 17%, and 57 to 15% respectively). In Hungary, Turkey, the Slovak Republic, there has also been an important decrease in some if not all of these teacher trainings. By contrast, in Poland teacher training increased significantly between 2011 and 2015: the share of students with a teacher who took a training in the past two years went up from 32 to 56% for maths content, from 34 to 74% for science content, from 31 to 69% for maths pedagogy and from 19 to 49% for science pedagogy. Some countries also had big changes in one or more of these domains (for example Australia, Sweden, and New Zealand).
The diffusion of teacher professional development through peer learning has been (on average) the largest innovation experienced by students in the OECD area, notably in secondary education. The share of secondary students having a teacher discussing how to teach a particular maths or science topic has increased in all covered countries, and by 21% on average (from 41 to 62% in maths, and 39 to 60% in maths). In Israel, the practice has become almost universal during the last decade (going up from 35 to 78% in maths and 83% in science). Collaboration in planning and preparing lessons has also become more prevalent, with the OECD average increasing from 40 to 56% in maths and from 37 to 55% in science. In Israel, Italy and New Zealand, this has represented a major change in the system. Finally, even though only 18% of secondary students had a teacher visiting a colleague’s classroom in an average OECD country, there was significant spread of this practice in the last decade: in 2007, only 3-4% of secondary students had a maths or science teacher in this case. The largest increases occurred in Korea (38 percentage points in maths and 35 in science), Turkey (37 in maths and 35 in science), and the Russian Federation (40 in maths and 34 in science).
Innovation and education systems’ performance
Ultimately, innovation should be about improvement, and the main reason why countries should monitor changes (or lack thereof) is to understand and monitor whether changes in educational practices lead to progress, to identify which changes or combination of changes lead to improvement of specific outcomes. At the very least, it helps monitor whether intended changes did translate into actual change in practices – and to see whether innovation policies in education, where they exist, produce the expected levels and types of innovation.
At this early stage of our measurement effort, we can assess the strength of associations between innovation and certain educational outcomes, and, more importantly, start raising some questions and assumptions about the relationship between innovation and educational outcomes. Any deeper analysis would require more granular analysis using longitudinal data that allow for the tracking of students over time, of their outcomes and of their corresponding teaching and learning environment. Part of this work is done by specific evaluation or “scaling up” studies, but very few still have sufficient scale to tell us much about innovation at the system level. Many assumptions about the possible effect of educational innovation in general or specific innovation on various educational outcomes remain to be proven or more carefully examined.
In the past decade, innovation in education has been associated with the improvement of academic learning outcomes, both in primary and secondary education. In countries where there has been more change in educational practices, students’ scores to international assessments have improved more on average. This is also generally true at the disciplinary level. More innovation in science education is associated with more improvement in science scores in primary and secondary education; countries where primary reading lessons have changed the most have also usually had more improvement in reading. A positive association also exists in maths education, but only at the primary education level. Other outcomes such as student satisfaction or the enjoyment of science have also increased more where there was more innovation. Innovation is not always accompanied by better outcomes though. In secondary education, countries that experienced more innovation have not improved their learning outcomes the most in mathematics, and no relationship with student satisfaction could be found.
Innovation in education should in principle only be encouraged when its benefits outweigh its costs – and if it is an improvement compared to the status quo. While in practice this is not feasible, because the generation of evidence and cost-benefit analysis is too slow (and relatively uncertain), this remains an important objective, and more research on the effects of specific educational practices and of their combination should be encouraged at the local, regional, national and international levels. In our report, there is a weak or inexistent association between innovation in the past 10 years and educational expenditure (per student). While it would be hasty to generalise that innovation does not require additional budget, it shows that many innovations, notably when they are pedagogical in nature, may be implemented within existing resources.
What are the drivers of innovation?
Innovation can be the result of different processes, especially when it happens in the classroom. It can be mandated or incentivised by local authorities or central governments as part of reforms or regulatory measures. It can be willingly adopted with no hierarchical incentives or mandates as part of the circulation of knowledge (training, peer learning, independent learning), the perceived demands of students and parents, feedback loops from data, the persuasiveness of “evidence”, the introduction of new products on the education market, etc.
Key drivers of innovation and improvement in education are as follows:
Human resources: the skills for and openness to innovation of actors within the education sector, notably teachers and faculty, are key aspects of a good innovation ecosystem.
Learning organisations: innovation and improvement are strongly related to how work is organised and whether education establishments and professionals are able to both absorb and generate improved knowledge and practice.
Technology: the application of general purpose technologies to the education sector, and notably of digital technologies, is a key promise for innovation and improvement. In particular, the development and use of longitudinal information systems (and their “big data”) holds key promises for innovating the education sector.
Regulation and system organisation: innovation and improvement only thrives where good ideas can be implemented and are not hidden by too risk-averse regulations on curriculum, assessment, etc. It also depends on the entrepreneurialism of the actors, on incentives, and on the availability of funds for educational innovation.
Educational research: the investment in and use of research and evaluation are key elements in an educational innovation ecosystem.
Educational Development: as in other sectors, an education industry should develop innovative tools, organisations and processes to improve and change the practices in the education sector.
Some of these different pillars of innovation could be measured and monitored over time at the country level and thus pave the way towards an “innovation capacity index” in education. In any event, it would give countries a better understanding of their strengths and weaknesses in the further of their education systems.
Towards an international survey instrument on innovation in education
While existing international datasets already provide us with important information about systemic innovation, improved measures of innovation in education would entail more specific studies. Our preferred approach to measuring innovation in education would be to develop a dedicated international survey – or at least survey instrument. This survey would ideally:
Adopt and adapt the “organisational change” approach using matched employer-employee-user surveys.
Be administered to the central educational administration (ministries or relevant local authorities) and to educational establishments in primary, secondary and tertiary education.
Question three levels of stakeholders (principal/president, teachers/faculty and students) about the state and changes in their work practices and work environment.
Infer innovation by comparing whether the investigated practice was used (or used to the same extent) at the time of the survey and, say, three years before.
Ask respondents their opinion about the impact of these practices (or change in these practices) on different educational goals (e.g. learning outcomes, equity, access, cost-efficiency).
Capture the sources and objectives of planned innovations, to what extent these planned improvements are implemented and perceived on the ground, and the extent of unplanned innovations.
Cover the broad innovation areas: products and services offered by educational organisations to their users/clients (e.g. textbooks, study programmes); pedagogic practice (e.g. pedagogies, introduction of new teaching or administrative equipment); organisational practice (e.g. organisational routines, human resource practices, knowledge management practices; support for the introduction of new ideas and practices, participation in training and retraining courses); external relations (e.g. relationships with parents, employers, research organisations, other academic institutions, advertisement practices).
Collect information about the broader environment in which these practices take place, such as information about size of establishment and classrooms, number of classes, competition with other schools in the neighbourhood, regulation and regulatory changes.
With support from the European Commission, the OECD Centre for Educational Research and Innovation plans to continue to develop new methodologies and instruments to address this important measurement gap for policy making and provide countries to monitor their innovation ecosystem in education.
References
Arundel, A., D. Bowen Butchart, S. Gatenby-Clark, and L. Goedegebuure (2016), Management and Service Innovations in Australian and New Zealand Universities – Preliminary report of descriptive results, June 2016. Australian Innovation Research Centre, Hobart and LH Martin Institute, Melbourne.
Adams Becker, S., M. Brown, E. Dahlstrom, A. Davis, K. DePaul, V. Diaz, and J. Pomerantz (2018), NMC Horizon Report: 2018 Higher Education Edition, EDUCAUSE.
Bloch, C. and M.M. Bugge (2013), “Public sector innovation – from theory to measurement”, Structural Change and Economic Dynamics, 27, 133-145.
Dumont, H., D. Istance and F. Benavides (2010), The Nature of Learning: Using Research to Inspire Practice, OECD Publishing.
Education Endowment Foundation (2018), Teaching and Learning Toolkit, https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit.
Greenan, N. and E. Lorenz (2013), “Developing harmonized measures of the dynamics of work organizations and work”, in F. Gault Ed., Handbook of Innovation Indicators and Measurement, Edward Elgar.
Haelermans, C. (2010), “Innovative power of Dutch secondary education”, Innovation: management, policy & practice, 12: 154–165.
Halász, G. (2018), “Measuring innovation in education: The outcomes of a national education sector innovation survey “, European Journal of Education, 1-17, https://doi.org/10.1111/ejed.12299.
Hattie, J. (2008), Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, Routledge.
MEADOW Consortium (2010), The MEADOW Guidelines, Grigny, France, http://meadow-project.eu/images/2013/meadowguidelines.pdf
OECD/Eurostat (2018), Oslo Manual 2018: Guidelines for Collecting, Reporting and Using Data on Innovation, 4th Edition, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris/Eurostat, Luxembourg. https://doi.org/10.1787/9789264304604-en
OECD (2013), Innovative Learning Environments, OECD Publishing, Paris.
OECD (2014), Measuring Innovation in Education: A New Perspective, OECD Publishing, Paris.
Vincent-Lancrin, S. (2017), “Understanding innovation in education: where do we stand?”, in G. Johnes, J. Johnes, T. Agasisti and López-Torres (Eds.), Handbook of Contemporary Education Economics, 162-183, Edward Elgar.