Measuring Innovation in Education 2019

What Has Changed in the Classroom?

Report

Educational Research and Innovation

5 March 2019

Chapter 15. Innovation and educational outcomes

Abstract

This chapter examines the association between innovation and some educational outcomes at the country level: academic learning outcomes in primary and secondary education, the enjoyment of science, student satisfaction, equity, and educational expenditures. Beyond presenting some information about the past trends, the chapter aims to raise some questions that could be explored over time or with more granular data on innovation in education.

The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law.

Linking innovation to educational outcomes

Innovation in education is not a goal in itself, but a means to achieve other educational objectives: improving learning outcomes, including students’ wellbeing, improving cost-effectiveness and cost-efficiency, closing the achievement gap, improving teachers’ learning and work satisfaction, etc. Measuring innovation in education is critical to see to what extent reforms (a top-down driver of innovation) and incentives for innovation are translating into actual change in classrooms and schools. This allows decision makers to assess whether their innovation policies and other policy reforms lead to the intended changes. It also allows them to get a better understanding of current practices and think about the mechanisms through which intended changes could actually occur.

Another key reason to measure innovation is to assess whether some innovations are good or bad. Ultimately, monitoring innovations, preferably at the micro level and with longitudinal data, should be a way to assess and identify what improves (or worsens) educational outcomes. This chapter aims to highlight this key objective in a heuristic way. While correlations at the macro-level of countries do not allow one to establish the direction of causality, they show whether there is an association between two variables and highlight the kind of questions one could better discuss and answer with more granular data.

This chapter looks at the links between pedagogical innovation and students’ academic learning outcomes in primary and in secondary education, educational equity, students’ satisfaction and enjoyment, but also the association between innovation and educational expenditures or teacher satisfaction. Some of the questions that one may want to be able to answer are as follows:

Have past pedagogical innovations led to better learning outcomes? What are the drivers of positive change in education systems? Do some types of pedagogical innovation work better for some students than others and lead to close the achievement gap? Is innovation more likely in some contexts than others (for example where learning outcomes are lower or are declining)?
How does innovation relate to educational expenditures? Most of the pedagogical innovation captured here does not require more expenditure. Some of it does though, for example teacher training or ICT devices. When or in which areas are increased or maintained educational expenditures a condition of educational innovation? When is it not the case? In some instances, one could imagine that innovation is a response to decreased educational expenditures. What are the links between available funds and practices within the classroom? Here is a second set of questions for policy makers.
Innovation is a source of professional development for teachers. Is it also a source of satisfaction and wellbeing? How does it relate to their teaching efficacy and self-efficacy? When is it a source of stress? Is there a good level of innovation? While we can only glance at this issue, this is also an area to investigate.

Innovation and academic outcomes in primary education

Is innovation in primary education associated with an improvement of academic learning outcomes? This is what one would hope. Although innovations may sometimes have other objectives (for example budget savings), one would expect innovation in mainly pedagogical practices to be associated with an improvement of students’ academic learning outcomes. Of course, the expectation may not necessarily be met in reality.

In primary education the same teacher usually teaches all disciplines, so that innovation might have a cross-disciplinary effect and be linked to all learning outcomes. Innovation in all primary education practices and the average change in the learning outcomes for the three disciplines covered by the book are indeed positively associated. Innovation and improved learning outcomes have gone hand in hand.

At the disciplinary level, there was also a positive relationship between innovation in reading and positive change in reading scores, as well as between innovation in science education and positive change in science scores. (Due to a too small number of indicators on maths practices, we did not compute a separate maths innovation index for primary education.)

In most cases, higher levels of innovation are associated with stability or increases in students’ learning outcomes, suggesting that innovation was not detrimental and sometimes beneficial to the systems where teachers innovated the most in their educational practices over the past decade. An alternative explanation may be that teachers in countries making the most progress in learning outcomes felt more secure to innovate and change their teaching and learning practices. That being said, in a few cases, above-average levels of innovation were associated with declining learning outcomes, reminding us that innovation might also be detrimental.

Figure 15.1. Innovation in primary education and average change in primary science, maths and reading learning outcomes (2006-2016)

Figure 15.2. Innovation in primary reading education and change in reading learning outcomes (2006-2016)

Figure 15.3. Innovation in primary science education and change in science learning outcomes (2007-2015)

Innovation and academic outcomes in secondary education

Is innovation in secondary education associated with improved academic learning outcomes? This is one what would hope, especially when innovation is mainly pedagogical. Innovation may sometimes have other objectives than the improvement of learning outcomes though, and even when it is their objective, past innovation may have just failed to achieve this goal.

There is a small positive association between innovation in our secondary education practices and the average change in the learning outcomes in maths and science. Given that in secondary education different teachers usually teach science and maths, there is less chances of cross-fertilisation between innovation in maths and science education. However, change in practices at the school or system level may have an impact.

Innovation in science education has been positively associated with the improvement of science learning outcomes in the last decade, whereas innovation in maths education has been negatively correlated with the improvement of maths outcomes. This reminds us that innovation does not necessarily lead to an improvement in the desired outcomes, exactly like policy reforms sometimes fail. This also raises the question of the lag time for innovation produce its effects, another question that the continuous study of innovation would allow one to answer.

The other direction of causality should also be taken seriously. In the case of mathematics, another possible interpretation could be that where teachers felt their students’ learning outcomes decrease, they have changed their practices more, but perhaps not yet with observable success.

Figure 15.4. Innovation in secondary education and average change in science and maths learning outcomes (2007-2015)

Figure 15.5. Innovation in secondary maths education and change in maths learning outcomes (2007-2015)

Figure 15.6. Innovation in secondary science education and change in science learning outcomes (2007-2015)

Innovation and student enjoyment in science education

One of the strong pleas for innovation in education is that existing teaching and learning practices would often be irrelevant to students, who get bored in class and do not engage in their learning. Many feel that this is particularly true in science – and also particularly problematic given the (alleged) lack of interest of students for science careers and studies. Is there an association between innovation in science educational practices and students enjoyment of their science lessons?

From another perspective, the emotional nature of learning has become more strongly acknowledged, and many teachers realise that the enjoyment of learning need not be an oxymoron. It also contributes to students’ wellbeing. One would thus expect or hope that innovation in educational practices lead to enhanced student enjoyment of their learning in general, and in our case, in science in particular.

Both in primary and secondary education, there was a positive association between innovation in science education and the increase of students’ enjoyment of their science lessons. We measure the enjoyment of science as the share of students in an education system reporting that they enjoy learning science at least a little. In primary education, all systems but Italy have had an increase in the enjoyment of science education, and this has been more often the case where innovation in science education practices has been more intense. Countries such as the Russian Federation or Norway have experienced both moderate-large innovation and greater enjoyment of science. The association is still positive, though not as strong in secondary education.

The direction of the causality may also run in the other direction. One could indeed imagine that, where a greater share of students start enjoying science (perhaps for reasons not captured in our book), it motivates teachers to change their teaching and learning practices. The (moderate) increase in active learning practices in science education could then be the outcome of a better learning climate as much as its cause.

While our aggregate data do not allow for any definitive conclusion, they show the kinds of questions that policy- and other decision-makers could answer with more systematic and refined data collections monitoring innovation and how education systems change over time. More granular data would make it possible to identify whether a mix of practices are associated to stronger increases in students’ enjoyment of science and other disciplines. This could be true for a series of educational outcomes and skill acquisition.

Figure 15.7. Innovation in science education and change in student enjoyment of science lessons in primary education (2007-2015)

Figure 15.8. Innovation in science education and change in student enjoyment of science lessons in secondary education (2007-2015)

Innovation and student satisfaction

Wellbeing is both a skill that can be developed, and a function of the learning environment. Depending on their age, whatever they really think, students might find it not socially desirable to say that they like school. But sometimes they really do not like it... While this was seen as irrelevant in a not so distant past, and may still be seen as such in some schools or classes, most education systems now also aim to develop positive attitudes towards education and learning, both because this might lead to better academic outcomes but also just because it contributes to children’s wellbeing (and possibly their likelihood to engage in lifelong learning). One can thus hope that past innovation has improved student satisfaction at school.

Perhaps because “innovation” is (usually) positively connoted, people often claim that innovation leads to greater student satisfaction. If nothing else, new pedagogical practices should make schooling more exciting and satisfactory. In fact, greater student satisfaction is a common finding of impact studies focusing on pedagogical interventions. Change itself may be an important element of satisfaction, and innovation may be useful for this sole reason: making people happier.

As innovation is not necessarily an improvement, it can also happen that learning conditions become worse than before, or that students enjoy less their learning environment. In that respect, policy makers and practitioners should be interested in how specific educational changes affect students’ wellbeing in school – and to what extent certain levels of overall innovation have an impact on it.

In primary education, we find a positive association between educational innovation and student satisfaction, whereas in secondary education there is no association. We measure student satisfaction as the share of students reporting that they like being in school at least a little. The association in primary education supports the ideas that the change in the mix of educational practices has possibly gone in the right direction of improving student’s liking of school, or even possibly driven this satisfaction given that, on average, greater levels of innovation were accompanied by greater increases of the share of satisfied students.

The lack of association in secondary education highlights that those assumptions are not self-evident and would require further investigation. The difference between secondary and primary education also lies in the fact that we only cover innovation in science and maths practices in secondary education, while our primary innovation index is more comprehensive and representative of what is learnt in primary education. More comprehensive measures may lead to different associations. Another possibility is that student satisfaction in secondary education depends on different factors than in primary education.

Longitudinal data concerning the same individuals and the mix of teaching and learning practices they experience would allow us to cast light on these issues.

Figure 15.9. Innovation in primary education and change in 4th grade student satisfaction (2007-2015)

Figure 15.10. Innovation in secondary education and change in 8th grade student satisfaction (2007-2015)

Innovation and equity in education

One concern with innovation is that it increases the achievement gap between students from different socio-economic backgrounds. Assuming innovation leads to an improvement of educational practices, this is indeed a very plausible outcome. This is for example why many observers worried about a “digital divide” when computers were just being first introduced in schools.

On the other hand, one can also hope that innovation will close the achievement gap and decrease inequity in education. In fact, this could only happen through innovation given that existing practices are still associated with relatively high levels of inequity (even though inequity has decreased in most countries over the past decades). Reducing inequity would come from a better dissemination of good and effective practices as much from a better tailoring of those practices to the learners. Some practices, such as mixed ability groups, are also believed to work particularly well for students from less advantaged backgrounds (while making little differences for the others).

In secondary education, we don’t find any relationship between innovation levels in maths and science and the change in the score gap between students from higher and lower socio-economic backgrounds. (We do not show the corresponding graphs, but the coefficients of correlation are -0.07 and -0.14 for maths and science, respectively.)

In primary education, there was no consistent trend. In the past decade, inequity in the reading scores has increased in almost all countries covered. Countries that have experienced more innovation in teaching and learning practices in reading have also had less increase of educational inequity. In science education it was the opposite. Where there has been more innovation in science education practices, there has also been an increase in educational inequity.

Are there specific practices that explain more the association in one direction or in the other? While we cannot answer this question with aggregated data, this is again a question that needs to be investigated within country with more granular data.

Figure 15.11. Innovation in primary reading education and trends in equity of primary reading scores (2006-2016)

Figure 15.12. Innovation in primary science education and trends in equity of primary science scores (2007-2015)

Innovation and teachers’ collective self-efficacy

Innovation leads to and results from teacher professional development. Trying out new practices makes teachers pause and reflect about their teaching. Regardless of whether their attempts translate into success or not, this is an occasion to try to improve their teaching. Innovation also comes from the awareness that some changes in their pedagogical practices may be beneficial, either because they have acquired some new knowledge in a formal training, by discussing or observing colleagues or through any other way.

When it is implemented at scale, one should hope that, beyond individual learning, innovation leads to collective learning at the school level. Facing similar challenges at the same time conducts teachers to work collaboratively and reflectively with their peers. In that case, innovation both results from and induces the emergence of learning organisations as a form of work organisation.

One misconception about innovation and innovative teaching is that it may challenge the implementation of the national (or local) curriculum. This needs not be the case, and can actually be the contrary. If teachers develop professionally when they innovate, we can assume that they will be more successful in the delivery of the curriculum at some point, in spite of the possible decrease in efficacy for a while.

We approximate collective self-efficacy within school as the share of teachers who report that, within their school, teachers are highly or very highly successful in implementing the school’s curriculum. Does this feeling of collective success in a defined community of practitioners increase when there is more innovation, or does it on the contrary disrupt it?

Overall, there is a small positive association in secondary education. The direction of causality could come from both directions. On the one hand, innovation may make teachers work collaboratively as they try to implement new pedagogies, which increases the collective belief that they are collectively successful within a school. On the other hand, the feeling of being successful in implementing the curriculum may contribute to the adoption of new pedagogical practices as teachers feel more self-confident, and also to greater levels of innovation as this could speed up the dissemination of the practices.

Figure 15.13. Innovation and change in teachers’ collective self-efficacy at the primary level (2007-2015)

Figure 15.14. Innovation and change in teachers’ collective self-efficacy at the secondary level (2007-2015)

Innovation and teachers’ collective ambition for their students

Innovation can be driven by many different objectives. A search for improving students’ learning outcomes, a quest for reducing inequity or achieving collective wellbeing, a response to budget cuts (or increases), an adjustment to parental or social demand, an answer to individual or collective learning, or even a response to lower salaries, to worsening working conditions or social status, etc.

One would hope that innovation relates to teachers’ and other actors’ willingness to improve students’ education and wellbeing. In some cases, limited educational improvement comes from teachers’ lack of ambition for their students or from the belief that some of their students cannot make progress. Educational improvement can also come from the opposite belief. While there has recently been many discussion around the “growth” mindset of students, this should also apply to teachers.

Whether teachers report that teachers in their schools have high or very high expectations for student achievement is one measure of teachers’ ambition for their students within a school.

Is innovation related to high expectations for achievement at the school level? This seems to be the case at the country level. In both primary and secondary education, teachers’ collective expectations for their student achievement have increased. This is a trend that was witnessed in virtually all the covered countries. On average, the more innovation there has been in a country, the more teachers’ expectations for their students’ achievement have increased. While we cannot claim that there is a causal association, it is more plausible that innovation was driven by these high achievement expectations rather than the opposite.

Figure 15.15. Innovation and change in teachers’ expectations for student achievement at the primary level (2007-2015)

Figure 15.16. Innovation and change in teachers’ expectations for student achievement at the secondary level (2007-2015)

Innovation and change in educational expenditures

There are different assumptions as to how innovation relates to expenditure and budget. Innovation surveys in the business sector consistently show that the lack of funding within and outside a company is reported as the top hurdles to innovation. Depending on the type of innovation this could be true in the education sector as well. Typically, a lack of budget (or of investment capability) may lead to the slowing down or postponement of innovation activities. This is what happens during an economic downturn or a restrictive budgetary policy. On the other hand, some believe that innovation can be triggered by adversity, and that people will innovate more in difficult budgetary situations that force them to be more creative. “Frugal” or “inclusive” innovation is partly a response to the lack of financial resources.

Innovation in education may be frugal or expensive. Some pedagogical innovations require budget. Enhanced access to ICT and sometimes proper use of ICT during class rely on a certain level of equipment and infrastructure. The mere maintenance of ICT has a cost; including the support of technical staff. Given the stability and even small decrease in access to ICT that we observed, one should not expect much expenditure on that front, except in very few countries. Some innovations may require some formal teacher training, but also the participation in more informal learning opportunities, which also involve some cost (such as staff time).

Innovation in the practices covered in the book does not require a specific budget in terms of implementation: most of those changes translate into a different use of students’ and teachers’ time. For example, discussing homework systematically in class implies that homework becomes a more integral part of students’ instruction, but also that more class time will be devoted to it (as opposed to other practices). This should not change educational expenditure inasmuch as class time remains stable, but only how the existing budget is used.

Changing one’s teaching and learning practices may require a change in knowledge, beliefs or attitudes that may have require some investment: new knowledge production, communication, facilitation of peer learning through a variety of means, from blog posts to systematic reviews of existing evidence, from internal school meetings to participation in conferences or visits abroad.

In the past decade, innovation in primary education has taken place in countries were educational expenditures per students were slightly on the rise. There was no association between educational expenditures and innovation in secondary education. (One caveat of this indicator is that expenditure per student can vary based on school demography, while the main educational expenditure (wages) typically remains more stable. It remains the most appropriate to use though.) One would learn more from studying the link between innovation and some sub-categories of budget, such as systems’ innovation budget, training budget, etc. These educational expenditures are not available at the international level.

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment

Governance

Health

Industry, business and entrepreneurship

Regional, rural and urban development

Science, technology and innovation

Society

Taxation

Trade

Energy

Nuclear energy

Transport

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment

Governance

Health

Industry, business and entrepreneurship

Regional, rural and urban development

Science, technology and innovation

Society

Taxation

Trade

Energy

Nuclear energy

Transport

Countries A - C

Countries D - I

Countries J - M

Countries N - R

Countries S - T

Countries U - Z

Regional and global engagement

Countries

Countries A - C

Countries D - I

Countries J - M

Countries N - R

Countries S - T

Countries U - Z

Regional and global engagement

Publications

Publications

Featured publications

Data

Data

Featured data

News & Events

News & Events

Featured Events

About OECD

About

Engage with us

Work with us

Featured topics

Agriculture and fisheries

Climate change

Development

Digital

Economy

Education and skills

Employment

Environment

Finance and investment