This chapter summarises students’ overall performance in creative thinking by country/economy and the variation in creative thinking performance across countries. It also examines the association between student performance in creative thinking and the three PISA core domains (mathematics, reading and science), including countries’ and economies’ relative performance in creative thinking. It concludes by describing differences in students’ creative thinking competencies across countries and economies.
PISA 2022 Results (Volume III)
Creative Minds, Creative Schools
PISA
2. Student performance in creative thinking
Abstract
For Australia*, Canada*, Denmark*, Hong Kong (China)*, Jamaica*, Latvia*, the Netherlands*, New Zealand* and Panama* caution is advised when interpreting estimates because one or more PISA sampling standards were not met (see Reader’s Guide, Annexes A2 and A4).
For Albania** and the Dominican Republic**, caution is required when comparing estimates with other countries/economies as a strong linkage to the international PISA creative thinking scale could not be established (see Reader's Guide and Annex A4).
“Everyone has huge creative capacities. The challenge is to develop them.”
Ken Robinson(2011[1])
Every child has the potential to think creatively. But how well are education systems worldwide developing this potential? To what extent are different education systems preparing students to think outside of the box and to express their own original or diverse ideas? The previous chapter introduced the construct of creative thinking and its measurement framework in PISA 2022. It also provided initial insights into the kinds of tasks that students at different levels of proficiency in creative thinking can complete. This chapter presents the international results from the PISA 2022 Creative Thinking assessment and examines the relationship between student performance in creative thinking and in the core PISA domains of mathematics, reading and science.
What the data tell us
Singapore, Korea, Canada* and Australia* are the highest-performing systems in creative thinking, with an average student score of 37 score points or above – significantly higher than the OECD average (33 score points). Students in Singapore achieved an average score of 41 points.
Most of the countries and economies that scored above the OECD average in creative thinking also outperformed the OECD average in mathematics, reading and science. Only Portugal performed above the OECD average in creative thinking (34 points) but not significantly different from the OECD average in all three PISA core domains. Czechia, Hong Kong (China)*, Macao (China) and Chinese Taipei performed at or below the OECD average in creative thinking but scored above the OECD average in mathematics, reading and science.
The performance gap in creative thinking between the highest-performing and lowest-performing country is very large, around 28 score points. Less than 3 in every 100 students in the five best-performing countries (Singapore, Korea, Canada*, Australia* and New Zealand*, in descending order) perform around or below the mean of the five lowest performing countries (Albania**, the Philippines, Uzbekistan, Morocco and the Dominican Republic**, in order).
In general, variation in creative thinking performance is not strongly related to mean performance. Students in Belgium, Denmark*, Estonia, Korea, Latvia*, Portugal and Singapore combine high levels of creative thinking proficiency overall with small variations in performance.
Only around 28% of the total variation in creative thinking performance can be uniquely associated to student performance in mathematics, on average across the OECD.
Students who performed at the highest (or lowest) level in creative thinking tended to also perform at the highest (or lowest) level in mathematics. However, similar proportions of students within the third quintile of creative thinking (over one-quarter of students, OECD average) scored within the second, third and fourth quintiles, respectively, in mathematics; and around 14% of students (OECD average) within the third quintile of mathematics scored in the upper quintile of creative thinking. This implies that academic excellence is not a pre-requisite for excellence in creative thinking.
In Chile, Mexico, Australia*, New Zealand*, Costa Rica, Canada* and El Salvador (in descending order), students scored over 4.5 points higher than expected in creative thinking after accounting for their mathematics performance. In Singapore, Australia*, Canada*, Latvia*, Korea, Belgium, Finland and New Zealand* (in descending order), students scored around 3 points or more higher than expected after accounting for their reading performance.
On average across OECD countries, around 1 in 2 students could think of original and diverse ideas in the context of simple imagination tasks or everyday problem-solving situations (i.e. they performed at or above Level 4). In Singapore, Korea and Canada*, over 70% of students performed at this level.
In Singapore, Latvia*, Korea, Denmark*, Estonia, Canada* and Australia* (in descending order), more than 88% of students reached a baseline level of creative thinking proficiency (Level 3), with the OECD average being 78% of students. However, in 20 countries and economies, more than 50% of students did not reach proficiency Level 3. This means that many students in these countries/economies struggled to think of appropriate ideas for a range of tasks, and few could suggest original ideas for familiar problems.
Australia*, Canada*, Finland and New Zealand* combined high levels of mean performance and overall relative performance in creative thinking (i.e. after accounting for students’ mathematics and reading scores, respectively) with at least 75% of students who reached proficiency Level 3.
Mean performance in creative thinking across countries and economies
Students across OECD countries scored 33 points on average on the PISA 2022 Creative Thinking assessment (Table III.B1.2.1). 12 countries (Singapore, Korea, Canada*, Australia*, New Zealand*, Estonia, Finland, Denmark, Latvia*, Belgium, Poland and Portugal, in descending order) significantly outperformed the OECD average in creative thinking, with mean scores between 41 and 34 points on the creative thinking scale (Table III.2.1). Box III.2.1 explains how to interpret differences in creative thinking performance across countries.
Singapore is the top-performing education system in creative thinking, with a student average score of 41 points, and it is the only non-OECD country to score significantly above the OECD average. Most of the 12 countries that outperformed the OECD average in creative thinking also outperformed the OECD average in mathematics, reading and science. Only Portugal performed above the OECD average in creative thinking (34 points) but not significantly different from the OECD average in the three PISA core domains.1 Four countries and economies – Czechia, Hong Kong (China)*, Macao (China) and Chinese Taipei – performed at or below the OECD average in creative thinking despite being high-performing systems (i.e. performing above the OECD average) in mathematics, reading and science.
Table III 2.1. Comparing countries’ and economies’ performance in creative thinking
Note: Only countries and economies with available data are shown.
Countries and economies are ranked in descending order of the mean performance in creative thinking.
Source: OECD, PISA 2022 Database, Table III.B1.2.1. The StatLink URL of this table is available at the end of the chapter.
Box III 2.1. Interpreting differences in creative thinking performance
The performance of students in creative thinking is summarised on a single creative thinking scale (as described in Box III.I.2, Chapter 1) that provides an overall estimate of their creative thinking proficiency. The performance of students within each country and economy can then be summarised according to the average student creative thinking score.
Interpreting differences in rankings
Many countries and economies score at similar levels in creative thinking. Small differences that are not statistically significant or practically meaningful should not be considered (see Reader’s Guide at the beginning of this volume). Because mean-score estimates are derived from samples and are thus associated with statistical uncertainty, it is often not possible to determine an exact ranking for all countries and economies. However, it is possible to identify the range of possible rankings for a country or economy’s mean performance (see Table III.A7.1 in Annex A7). This range of rankings can be relatively wide, particularly for countries/economies whose mean scores are similar to those of many other countries/economies.
Interpreting differences in scores: How large is a “large” score difference in creative thinking?
The creative thinking data are summarised according to a different PISA scale than the assessments of mathematics, reading and science, with which readers may be more familiar. In this report, the following three benchmarks provide guidance for interpreting test-score differences in creative thinking.
A first benchmark, which defines a “large” change in creative thinking, is 3 score points. Typically, in the PISA core domain assessments, a “large” difference is defined as a change of 20 score points or more. This is approximately equivalent to the typical annual learning gain by students around the age of 15 and is around one-fifth of the OECD standard deviation in performance. Given the broader “grain” size of the creative thinking scale (see Annex A5 for more), a change of 3 score points is approximately equivalent to one-quarter of the OECD standard deviation in creative thinking performance.
A second benchmark, which defines a “small” change in creative thinking, is 1 score point. Changes of up to 1 score point correspond to just under one-tenth of the OECD standard deviation in creative thinking performance. Given the first two benchmarks discussed here, score changes of between 1 and 3 points can thus be considered “moderate” differences in performance.
The third benchmark for interpreting differences in performance is statistical significance (see Reader’s Guide). This confidence interval needs to be taken into account when making comparisons between estimates so that differences that may arise simply due to the sampling error and measurement error are not interpreted as real differences.
There are substantial differences in creative thinking performance across countries and economies. The lowest-performing system (Albania**) has an average student score of 13 points on the creative thinking scale – around 28 points lower than the average student in Singapore (the highest-performing system). This gap between average student performance in the highest- and lowest-performing systems is more than 2.5 times larger than the OECD average standard deviation in performance (10.8 points), or roughly equivalent to between three and four proficiency levels in creative thinking. Less than 3% of students in the five best-performing countries and economies (Singapore, Korea, Canada*, Australia* and New Zealand*, in descending order) perform at or below the mean of the five lowest-performing countries and economies (Albania**, the Philippines, Uzbekistan, Morocco, the Dominican Republic**, in order).
Eight countries (Lithuania, Spain, Czechia, Chinese Taipei, Germany, France, the Netherlands* and Israel, in descending order) score around the OECD mean in creative thinking (33 points). The lowest-performing OECD country in creative thinking, Colombia, has a mean score of 26 points (7 points below the OECD average).
Variation in creative thinking performance across countries and economies
Comparing the mean performance of students in the 90th percentile in each country/economy with those in the 10th percentile in each country/economy is one way of examining the variation in student performance across countries and economies. Across the OECD, the average difference in creative thinking performance between these two groups of students is very large – around 29 score points (Table III.B1.2.1). This is roughly the equivalent to a difference of between three to four proficiency levels in creative thinking.
Significant differences in the variation in student performance exist across countries and economies (Figure III.2.1). The smallest variation in creative thinking performance between students in the 90th percentile and the 10th percentile is found in Latvia* and Uzbekistan (22 points), the Dominican Republic** (23 points) and Denmark* (24 points), although the average (mean) performance of students in these countries differs significantly. Among high-performing countries, in addition to Latvia* and Denmark*, Singapore has a relatively narrow spread of performance (25 points). At the other extreme, the United Arab Emirates, Jamaica* and Qatar (in descending order) have the largest variations in creative thinking performance, with a difference of more than 35 points between students in the top and bottom deciles. A large diversity in performance within a country/economy means that there are more students who tend to perform both relatively better and relatively worse than the average student in those countries/economies, compared to the average variation in performance observed across OECD countries.
Figure III 2.1. Variation in creative thinking performance across countries and economies
Differences in mean score between students in the 90th percentile and the 10th percentile
Notes: Only countries and economies with available data are shown.
All differences between the 90th and the 10th percentiles are statistically significant (see Annex A3).
Countries and economies are ranked in descending order of the median performance in creative thinking.
Source: OECD, PISA 2022 Database, Table III.B1.2.1. The StatLink URL of this figure is available at the end of the chapter.
In general, variation in creative thinking performance (as measured by standard deviation) is not strongly related to mean scores across participating countries and economies (Figure III.2.2).2 Students in Belgium, Denmark*, Estonia, Korea, Latvia*, Portugal and Singapore combine high levels of average creative thinking proficiency with small variations in performance. This means that relatively more students in those countries and economies score close to the country/economy mean performance, which is above the OECD average. While students in Finland also score well above the OECD average, there is a much wider variation in student performance in the country, meaning that many students score significantly above and below the country mean.
Figure III 2.2. Average performance in creative thinking and variation in performance across countries and economies
Note: Only countries and economies with available data are shown.
Source: OECD, PISA 2022 Database, Table III.B1.2.1. The StatLink URL of this figure is available at the end of the chapter.
How performance in creative thinking compares to performance in mathematics, reading and science
To what extent does the PISA creative thinking assessment measure a different set of skills with respect to those measured in the core assessment domains? Generating, evaluating and improving upon ideas are fundamental cognitive processes in every curricular subject area. It should therefore be expected that student performance in creative thinking correlates positively with performance in mathematics, reading and science – even if the PISA 2022 Creative Thinking assessment places more emphasis on students’ ability to generate original or qualitatively different ideas than in those domains. Students who perform well in creative thinking are likely to perform well in other subject areas, just as students who do not achieve high scores in mathematics, reading and science are likely to achieve low scores in creative thinking. This section examines these associations in more detail, including the relative performance of students in creative thinking given their performance in mathematics and reading.
Correlation with performance in mathematics, reading and science
Student performance in creative thinking correlates positively to performance in mathematics, reading and science respectively – but the strength of this association is weaker than the associations amongst mathematics, reading and science. Table III.2.2 shows the OECD average within-country correlations between student performance in creative thinking, mathematics, reading and science. The correlation between creative thinking, on the one hand, and each of the other three PISA domains on the other, is almost identical: 0.67 with mathematics, 0.66 with reading and 0.66 with science.3 In comparison, performance scores among the three core PISA domains are more strongly associated, especially the correlation between performance in the mathematics and science assessments (0.87).4 These results support the assertion that the creative thinking assessment measures a different subset of skills with respect to those measured in the mathematics, reading and science assessments.
Table III 2.2. Correlation in performance among creative thinking, mathematics, reading and science
OECD average
Source: OECD, PISA 2022 Database, Table III.B1.2.3. The StatLink URL of this table is available at the end of the chapter.
A different way of evaluating whether the skills measured in the PISA 2022 Creative Thinking assessment are unique is to examine the variation in student performance in creative thinking that can be associated with their performance in another assessment.5 On average across the OECD, only around 28% of the variation in creative thinking performance can be uniquely associated to student performance in mathematics (the major domain focus in PISA 2022) (Figure III.2.3). This means that relatively little of the variation in performance across OECD countries can be accounted for simply by student performance in the mathematics assessment. A smaller proportion of the variation in student performance – around 20% – can be explained by student gender and student and school socio-economic profile (i.e. student background variables), and factors common to both student background and mathematics performance.
In four countries and economies (Slovenia, Panama*, Kazakhstan and Jamaica*, in order of smallest to largest), less than 23% of the variation in students’ creative thinking performance can be associated with their performance in mathematics uniquely. In these countries and economies, more than in others, performance differences in creative thinking do not necessarily match those found in mathematics. For example, some students who perform at a high level of proficiency in mathematics achieve relatively low scores in creative thinking (and vice versa) in these countries/economies. In contrast, in the Philippines, Morocco, Uzbekistan, Malta, Malaysia, and the United Arab Emirates (in descending order), around 40% or more of the variation in creative thinking performance reflects performance differences captured uniquely in the mathematics assessment.
While the relative strength of the association between creative thinking and mathematics performance, as well as certain student and school characteristics, varies across countries and economies, in only one country do these factors account for more than two-thirds of the total variation in creative thinking performance (Brunei Darussalam, 67%). Again, these findings support the assumption that the PISA 2022 Creative Thinking assessment measures a different subset of skills with respect to the other PISA assessments.
Figure III 2.3. Variation in creative thinking performance associated with performance in mathematics
Notes: Only countries and economies with available data are shown.
Explained variation is the R squared coefficient from a regression of creative thinking score on mathematics, gender and students' and schools' socio-economic profile (ESCS).
Countries and economies are ranked in descending order of the percentage of variation in creative thinking performance explained uniquely by performance in mathematics.
Source: OECD, PISA 2022 Database, Table III.B1.2.3. The StatLink URL of this figure is available at the end of the chapter.
Is the correlation between creative thinking performance and performance in mathematics linear across the entire performance scale? Or is there more variability in creative thinking performance at different skill levels in the core domains? Figure III.2.4 shows the proportion of students (OECD average) within each quintile on the mathematics scale who score within each quintile on the creative thinking scale. At the upper and lower ends of both scales, there is less variability in student performance: around half of all students in the top quintile of performance in mathematics (and over half of all students in the bottom quintile) are in the top quintile of performers in creative thinking (or bottom quintile, respectively). In other words, students who performed at either the highest or lowest levels in mathematics tended to also perform at the highest or lowest levels in creative thinking. Yet far greater variability in creative thinking performance was observed amongst students in the second, third and fourth quintiles of mathematics performance. For example, just over one-quarter of all students within the third quintile in creative thinking also performed within the third quintile in mathematics, with similar proportions of students performing within the second and fourth quintiles in mathematics; and around 14% of all students within the third quintile in mathematics performed within the upper quintile in creative thinking. Similar patterns were observed when examining the distribution of students across quintiles of performance in creative thinking and reading.6
Figure III 2.4. Distribution of students across quintiles of performance in creative thinking and mathematics
OECD average
Source: OECD, PISA 2022 Database. The StatLink URL of this figure is available at the end of the chapter.
These data imply that performance in creative thinking and academic performance go hand in hand to some extent, particularly at the upper and lower ends of the proficiency scales. Yet academic excellence is not a pre-requisite for excellence in creative thinking. While creative thinking performance and academic performance complement one another, PISA data show that it is possible for many students to be strong creative thinkers. While all students have the potential to excel in creative thinking, this is especially the case for those who reached at least a baseline level of proficiency in mathematics, reading and science.
Countries’ and economies’ relative performance in creative thinking
In general, the positive correlation between performance in creative thinking and performance in mathematics, reading and science means that students who perform well in the PISA core domains will likely perform well in creative thinking. However, in some countries and economies, students may have performed relatively better or worse than expected in creative thinking given their scores on the other PISA assessments. In this section, “relative performance” refers to students’ performance in creative thinking after accounting for their mathematics or reading performance.7 In other words, relative performance describes how well students performed in creative thinking compared to other students with similar mathematics or reading scores.
In seven countries (Chile, Mexico, Australia*, New Zealand*, Costa Rica, Canada* and El Salvador, in descending order), students scored over 4.5 points higher than expected in creative thinking – a large relative performance advantage – after accounting for their mathematics performance (Figure III.2.5). These seven countries include education systems across the entire performance range of creative thinking, including both high- and low-performing systems, meaning it is possible for students with both stronger and weaker mathematics proficiency to perform relatively well in creative thinking. By some margin, the country with the weakest relative performance after accounting for students’ mathematics performance is Albania**, with students scoring nearly 8 points lower in creative thinking than expected, followed by Uzbekistan and Macao (China) (both around -6 points). Among OECD countries, Slovenia (-2.5 points) and the Slovak Republic (-1.2 points) had the largest relative performance deficit. All countries with a negative relative performance in creative thinking after accounting for their mathematics performance scored significantly below the OECD average in creative thinking, except for Chinese Taipei (whose mean score was not statistically different to the OECD average).
Figure III 2.5. Countries’ and economies’ relative performance in creative thinking
Score-point difference between actual and expected performance in creative thinking
Note: Only countries and economies with available data are shown. The actual mean score in creative thinking is shown next to the country/economy name.
Statistically significant score-point differences are shown in a darker tone (see Annex A3).
A student's relative performance in creative thinking is defined as the residual obtained upon a cubic polynomial regression of the student's performance in creative thinking over his or her performance in mathematics or reading. The regression is performed at an international level, pooling data from all countries and economies that participated in the creative thinking assessment.
Countries and economies are ranked in descending order of the relative performance in creative thinking based on performance in mathematics.
Source: OECD, PISA 2022 Database, Table III.B1.2.4. The StatLink URL of this figure is available at the end of the chapter.
Are similar patterns observed when examining students’ relative performance in creative thinking given their reading scores? In general, students in countries/economies who performed relatively better in creative thinking after accounting for their mathematics performance tended to also perform relatively better after accounting for their reading performance, and vice versa (Figure III.2.5). In Singapore, Australia*, Canada*, Latvia*, Korea, Belgium, Finland and New Zealand* (in descending order), students had the largest relative performance in creative thinking after accounting for their reading performance – scoring around 3 or more points higher in creative thinking than expected.
When considering both relative performance measures, students in four countries demonstrated a large relative strength in creative thinking overall (Figure III.2.6): in Australia*, Canada*, Finland and New Zealand*, students scored around 3 points higher in creative thinking after accounting for both their mathematics performance and after accounting for their reading performance, respectively. Other clusters of countries also demonstrated a moderate to strong relative performance in creative thinking overall. For example, in the Latin American region, Chile, Costa Rica, El Salvador, Mexico and Uruguay all demonstrated a strong relative performance in creative thinking (i.e. scored around 3 or more points higher than expected) given their mathematics performance, and a moderate relative performance (scoring between 1 and 2.75 points higher) given their reading performance. A cluster of European countries also demonstrated a moderate relative performance in creative thinking given their mathematics scores, combined with either a strong (Belgium, Latvia*) or moderate (Denmark*, Estonia, Lithuania, Poland, Portugal and Spain) relative performance in creative thinking given their reading performance. Students in Israel, Qatar and the United Arab Emirates also demonstrated a moderate strength in creative thinking with respect to both reading and mathematics scores. Only Singapore and Korea showed a relative strength in creative thinking overall amongst East Asian countries, combining a large relative performance in creative thinking after accounting for reading scores with a moderate relative performance in creative thinking after accounting for mathematics scores.
In several of these countries/economies with a notable relative strength in creative thinking, system-level reforms of curricula and assessment practices over the past decade have focused on furthering the importance of creative thinking in education (see Box III.2.2).
Figure III 2.6. Countries and economies that perform better than expected in creative thinking
Note: Countries/economies with a "strong" relative performance in creative thinking are those whose students score significantly higher than 2.75 score-points in creative thinking after accounting for mathematics/reading performance (see Annex A3). Countries/economies with a "moderate" relative performance in creative thinking are those whose students score significantly higher than 1 score-point but less than 2.75 score-points higher in creative thinking after accounting for mathematics/reading performance.
Source: OECD, PISA 2022 Database, Table III.B1.2.4. The StatLink URL of this figure is available at the end of the chapter.
Students in other countries/economies demonstrated a mixed or negative overall relative performance in creative thinking compared to their performance in the PISA core domains. In Baku (Azerbaijan), Iceland, Malaysia, Mongolia and the Netherlands*, students scored as or lower than expected in creative thinking after accounting for their mathematics performance but scored at least 1 point higher than expected in creative thinking after accounting for their reading performance. Conversely in Brazil, Colombia, Jamaica* and Panama*, students scored relatively lower in creative thinking after accounting for their reading performance despite scoring at least 1 point higher than expected after accounting for their mathematics scores. In both cases, the mixed direction of relative performance results in creative thinking across the two measures likely reflects relative weaknesses in either reading (for Baku (Azerbaijan), Iceland, Malaysia, Mongolia and the Netherlands*) or mathematics (for Brazil, Colombia, Jamaica* and Panama*) performance, rather than a relative strength in creative thinking overall.
Students in Albania**, Brunei Darussalam, Bulgaria, Macao (China), Morocco, the Philippines and Uzbekistan demonstrated the largest overall relative weaknesses in creative thinking, scoring at least 3 points lower than expected after accounting both for mathematics and for reading performance, respectively. Students in the Dominican Republic**, Hong Kong (China)*, Indonesia, Moldova, North Macedonia, the Palestinian Authority, Slovenia, Chinese Taipei and Thailand also demonstrated a moderate overall relative weakness in creative thinking.
Box III 2.2. System-level efforts to integrate creative thinking into the curriculum and assessment
Promoting the development of creative thinking consistently and effectively in education requires educators, curriculum developers and assessment designers to have a shared understanding of what creative thinking is, how students can develop creative thinking skills, and how their progress can be measured. Redefining curricula and learning progressions with these goals in mind can facilitate the development of creativity-supportive teaching and learning. A recent publication providing a snapshot of progress in integrating creative thinking in schools around the world concluded that while creative thinking is increasingly specified in curricula, only a small number of jurisdictions provide strategic leadership and clear guidance in practice (Lucas, 2022[2]). However, there are a growing number of promising examples.
From high-level strategy to curriculum reform
In 2019, Norway published a new strategy recognising the link between creative work and learning in all subjects (Ministry of Education and Research, 2019[3]). The strategy’s goals were to reduce curriculum content to facilitate deeper learning experiences, strengthen the practical and aesthetic aspects of the curriculum (including arts, crafts, music, and food and health education), and develop new guidelines to support teacher practice. A new national curriculum was introduced in 2020, and accompanying resources for teaching and learning in the practical-aesthetic subjects were developed. Similarly, in Denmark, a cross-party strategy published in 2018 stressed that pupils should have more opportunities to develop imaginative and creative skills in primary and lower secondary school (Government of Denmark, 2018[4]). Among other aspects, a two-year practical/musical elective subject was made a compulsory element in the curriculum that must be completed with an exam.
In Korea, an important goal of the national curriculum reform back in 2009 was “raising a creative person” and the government has since made several efforts to integrate creative thinking into the education system (So, Hu and Park, 2017[5]). As a result, Korea reduced the number of compulsory curriculum areas and established “creative experiential learning activities” at both primary and secondary education levels as a holistic accompaniment to compulsory and elective subjects. These creative experiential activities are units in which teachers and students can choose a topic of interest to study in any way they wish. Although elementary and secondary schools are required to allocate three to four units to these activities, they remain limited to extra-curricular offerings. However, in 2016, an exam-free semester was also introduced in middle schools throughout the country to create more flexible space in the curriculum for creative projects and to relieve students of the pressures of regular mid- and end-of-term examinations. Other countries have also introduced similar exam-free-semester initiatives (see Box III.4.4 in Chapter 4).
Developing detailed learning progressions
Several jurisdictions have gone a step further by developing detailed guidance and learning progressions to help teachers understand the types of outcomes that ought to be expected of students and the learning trajectories they typically follow to reach them. In Australia, for example, the Assessment and Reporting Authority (ACARA) developed a “critical and creative thinking learning continuum” that maps progression in creative thinking from Level 1 to Level 6. The continuum supports the 2010 curriculum reform which included “critical and creative thinking” as one of seven general capabilities that intersect with the eight subjects or learning areas. The continuum describes, in concise and simple statements, the behaviours of students at each level and for each sub-element that evidences critical and creative thinking. Some states, for example Victoria (Australia), have subsequently adapted their own learning progressions connected to years of schooling and standards describing expected student outcomes at each two-year interval.8
In Canada, several provinces have also been active in this space. In Alberta (Canada), creative thinking is framed as a competency that intersects with knowledge and skills across all disciplines, and as such, it has been mapped across the content of all subject areas in the curriculum. An online platform provides teachers with competency progressions across grades and ages.9 In Ontario (Canada), guidelines on assessment, evaluation and reporting for schools describe performance standards related to creative and critical thinking processes across subjects, with example criteria in the arts, sciences and English subject areas (Ministry of Education, Ontario, 2010[6]).
Integrating creative thinking in system-level assessments
While progress has been made to better integrate creative thinking in curricula and support educators in recognising how creative thinking can be developed and evidenced, very few systems assess creative thinking in a standardised way. One exception is in Victoria (Australia): since 2016, the VCAA has administered Critical and Creative Thinking (CCT) assessments annually to a sample of schools. The CCT assessments support its commitment to measure its Education State Targets, one of which aims for 25% or more Year-10 students to have developed excellent critical and creative thinking skills.10
Differences in student competencies across countries and economies
What can students do in terms of their creative thinking proficiency? On average across OECD countries, around 1 in 2 students could think of original and diverse ideas in familiar contexts (i.e. they performed at proficiency Level 4 or above). This means that around half of the OECD student population struggled to think of original and different ideas for different types of tasks, even in the context of simple imagination tasks or everyday problem-solving situations. In Singapore, Korea and Canada*, over 70% of students performed at or above Level 4.
Figure III.2.7 shows the distribution of students at each proficiency level within each country/economy, where Level 6 is the highest proficiency level and Level 1 is the lowest described level.11 There is significant variation across participating countries and economies in the distribution of students across proficiency levels. In most countries and economies, the largest proportion of students performs at Level 4 or at Level 3, with Level 3 considered to be a baseline level of proficiency in creative thinking (Box III.2.3). In Singapore, Latvia*, Korea, Denmark*, Estonia, Canada* and Australia* (in descending order), more than 88% of students were proficient at Level 3 or above – with the OECD average just over 78% of students. However, in 20 countries and economies, more than 50% of students did not reach a baseline level of proficiency in creative thinking.12 This means that many students in these countries/economies struggled to think of appropriate ideas for a range of tasks, and few were able to suggest original ideas for familiar problems.
Box III 2.3. Creative thinking proficiency: What is a baseline target for education systems?
In the PISA core domain assessments of mathematics, reading and science, Level 2 is considered as a baseline level of proficiency that students need to fully participate in society. In the PISA assessment of creative thinking, Level 3 can be interpreted as the baseline level of skills that all systems should target. In creative thinking, students at Proficiency Level 3 demonstrated the capacity to generate appropriate ideas for simple to moderately complex expressive and problem-solving tasks. At this level, they also began to demonstrate the ability to generate original ideas or solutions in familiar task contexts (see Tables III.1.1 and III.1.2 in Chapter 1).
A different baseline proficiency level is used for creative thinking, given the PISA creative thinking assessment differs with respect to the PISA assessments of mathematics, reading and science in two significant ways. First, students engaged with open-ended, constructed-response tasks in the creative thinking assessment. To successfully engage with the tasks and demonstrate proficiency, students needed to utilise both divergent and convergent thinking processes and have some level of task engagement (OECD, 2023[8]). Second, in the creative thinking test, partial credit responses demonstrated that students could generate appropriate and relevant ideas but not that they could generate original or diverse ideas – in other words, only full credit responses differentiated creative ideas from those that were simply appropriate, common and/or alike one another. At Level 3, students began to achieve full credit in some task contexts.
Very few students in OECD countries performed at the lowest described level of creative thinking (Level 1). However, in Colombia, the Slovak Republic, Greece, Israel and Costa Rica (in order of larger share to smaller), over 1 in 10 students performed at Level 1 or below; in Colombia, it was around 2 in 10 students. For many low-performing countries/economies in creative thinking overall, a relatively large share of students performed at Level 1 or below – possibly reflecting poor levels of student engagement with the creative thinking items (Box III.2.4).
Figure III 2.7. Students’ proficiency level in creative thinking, by country/economy
** Caution is required when comparing estimates with other countries/economies as a strong linkage to the international PISA creative thinking scale could not be established (see Reader's Guide and Annex A4).
Notes: Only countries and economies with available data are shown.
The mean score in creative thinking is shown next to the country/economy name.
Countries and economies are ranked in descending order of the mean performance in creative thinking.
Source: OECD, PISA 2022 Database, Table III.B1.2.2. The StatLink URL of this figure is available at the end of the chapter.
Top-performing students
In this report, students performing at Proficiency Levels 5 or 6 are referred to as “top performers” in creative thinking. Top performers demonstrated the ability to generate, evaluate and improve creative ideas in diverse and complex tasks, including abstract design tasks or more constrained/unfamiliar scientific and social problem-solving scenarios. Over 1 in 4 students on average across the OECD are top performers in creative thinking (Figure III.2.7). Some high-performing countries have a large share of top-performing students: around 40% of students in Australia*, Finland and New Zealand* and 45% in Canada* and Korea. Singapore has the largest proportion of top performers in creative thinking by far of all countries and economies (58% of all students).
The share of top performers within a country/economy largely reflects its ranking position by mean performance, with some exceptions: for example, while students in Estonia and Finland scored 36 points on average, Finland has relatively more top performers than Estonia (39% of students compared to 34%); in Israel, despite scoring around the OECD average overall, the proportion of top performers is above the OECD average; and in the United Arab Emirates, the proportion of top-performing students is relatively large (24%) compared to other countries with a similar mean performance (20% of students in Qatar or 15% in Uruguay).
It might be expected that students who were top performers in creative thinking were also top performers in other PISA domains. However, given only around 28% of the total variation in creative thinking performance is uniquely associated with students’ performance in mathematics (Figure III.2.3), it may not necessarily be the same students that performed at the highest level in creative thinking that performed at the highest level in other subject areas.
Figure III.2.8 shows the percentage of top performers in mathematics and reading, respectively, amongst top performers in creative thinking. In all but 12 countries and economies (in descending order, Chinese Taipei, Macao (China), Singapore, Hong Kong (China)*, the Netherlands*, Korea, Slovenia, Czechia, Estonia, Belgium, Hungary and Germany), less than 25% of top performers in creative thinking also performed at Level 5 or above in mathematics. In all but four countries and economies (in descending order, Singapore, Chinese Taipei, New Zealand* and Israel), 25% or less top performers in creative thinking are also top performers in reading.
Figure III 2.8. Top performers in creative thinking and mathematics/reading
Percentage of top performers in mathematics or reading among top performers in creative thinking
** Caution is required when comparing estimates with other countries/economies as a strong linkage to the international PISA creative thinking scale could not be established (see Reader's Guide and Annex A4). Notes: Only countries and economies with available data are shown. Countries and economies are ranked in descending order of the percentage of students who spent less than 30 seconds on an item and who achieved no credit for their response to that item.
Source: OECD, PISA 2022 Database, Table III.A8.1. The StatLink URL of this figure is available at the end of the chapter.
While around 9% of students on average across the OECD performed at the very highest level of proficiency in creative thinking (Level 6), more than double this proportion performed at Level 6 in Australia*, Canada*, Korea and Singapore (Figure III.2.7). In both Israel and the United Arab Emirates, despite mean performance being around or below the OECD average, a relatively large share of students performed at Level 6 in creative thinking.
Box III 2.4. Student engagement with the Creative Thinking assessment
In general, creative work requires task engagement (OECD, 2022[9]). The PISA 2022 Creative Thinking assessment was no exception, with most items in the test requiring students to elaborate a written or visual output. Students needed to invest time in reading the task prompt, understanding the stimulus material, and actively constructing a response in the format required.
Recent research has focused on identifying characteristics of items that are more engaging for test takers (Avvisati et al., 2023[10]) and on measuring engagement in the context of PISA assessments (Buchholz, Cignetti and Piacentini, 2022[11]), often by constructing indicators of disengagement. To examine the relationship between task engagement and creative thinking performance, several indicators of disengaged student behaviour have been constructed. These include identifying students who rapidly moved through the test items leaving insufficient time to provide a valid response (“rapid responders”), as well as students who provided no answers to items (i.e. missing responses) (see Annex A8 for a full description and comparison of different indicators).
Figure III.2.9 shows the percentage of so-called rapid responders for each country/economy.13 In 4 countries and economies (in descending order, Albania**, Baku (Azerbaijan), Bulgaria and North Macedonia) students exhibited rapid responding behaviours on around 10% or more of all items, compared to the OECD average of just under 4% of all items. In all cases, these countries performed well below the OECD average in creative thinking. An alternative indicator of disengagement – the percentage of missing responses for items seen by students – was also large in some countries and economies: for example, over 15% of all items seen by students in 6 countries/economies were not responded to (OECD average around 6%), and in Albania** and Baku (Azerbaijan), this proportion rose to over 20% of all items (Table III.A8.1 in Annex A8).
Figure III 2.9. Engagement with the creative thinking items
Percentage of students who spent less than 30 seconds and did not achieve any credit on an item
Note: Only countries and economies with available data are shown.
Countries and economies are ranked in descending order of the percentage of students who spent less than 30 seconds on an item and who achieved no credit for their response to that item.
Source: OECD, PISA 2022 Database, Table III.A8.1. The StatLink URL of this figure is available at the end of the chapter.
While no single indicator of (dis)engagement is able to capture the full spectrum of disengaged behaviours – especially as some disengaged behaviours may manifest very differently – it should be expected that valid measures of disengagement focusing on similar behaviours correlate highly. For the three measures computed in Annex A8, strong correlations are observed.14
A snapshot of system success in creative thinking
When considering the overall success of education systems in developing creative thinkers, some systems performed better than others. Figure III.2.10 summarises the performance of countries and economies according to three indicators of system success examined in this chapter. The first two indicators describe successful systems in absolute terms: i) those with high mean performance (i.e. countries/economies where the average student score was statistically significantly greater than the OECD average); and ii) systems with a large share of students that reached a baseline level of creative thinking proficiency (at least 75% of students who performed at Proficiency Level 3 or above). The third indicator describes successful systems in relative terms: those where students demonstrated an overall relative strength in creative thinking performance (i.e. they scored around 3 points higher than expected in creative thinking both after accounting for performance in mathematics and after accounting for performance in reading, respectively).
Figure III 2.10. Indicators of system success in creative thinking proficiency
Notes: High-performing countries are countries with a performance in creative thinking statistically higher than the OECD average (see Annex A3).
Countries with 75% of students at baseline proficiency are those where at least 75% of students performed at or above Proficiency Level 3 in creative thinking.
Relative high-performing countries are countries with a relative performance in creative thinking after accounting for both mathematics and reading performance, respectively, that is statistically significant and higher than 2.75 score-points.
Source: OECD, PISA 2022 Database, Tables III.B1.2.1, III.B1.2.2 and III.B1.2.4. The StatLink URL of this figure is available at the end of the chapter.
When considered together, these three indicators show that, overall, Australia*, Canada*, Finland and New Zealand* are the most successful systems in developing students’ capacity to engage in creative thinking. These systems combine high levels of absolute and relative performance in creative thinking. Many education systems performed well in absolute terms, successfully ensuring the majority of their students have reached a baseline level of creative thinking proficiency; and students in Belgium, Denmark, Estonia, Korea, Latvia*, Poland, Portugal and Singapore also performed at a high level in creative thinking compared to the OECD average. However, students in these countries did not show a strong relative performance in creative thinking after accounting for both their performance in mathematics and in reading, respectively.
A cluster of Latin American and Caribbean countries, including Chile, Colombia, Costa Rica, El Salvador, Jamaica*, Mexico, Panama* and Uruguay, demonstrated a relative strength in creative thinking after accounting for students’ mathematics performance. While these results largely reflect a weakness in mathematics in the region, students in Chile, Costa Rica, El Salvador, Mexico and Uruguay also demonstrated a moderate relative strength in creative thinking after accounting for their reading performance. Nonetheless, performance in absolute terms in most of these countries remained low – in fact, in El Salvador and Panama*, less than 50% of students reached Proficiency Level 3 in creative thinking (Figure III.2.7).
Table III 2.3. Student performance in creative thinking: Chapter 2 figures and tables
References
[10] Avvisati, F. et al. (2023), Item characteristics and test-taker disengagement in PISA, https://one.oecd.org/document/EDU/PISA/GB(2023)5/en/pdf.
[11] Buchholz, J., M. Cignetti and M. Piacentini (2022), Developing measures of engagement in PISA, OECD Publishing, Paris, https://doi.org/10.1787/2d9a73ca-en.
[4] Government of Denmark (2018), Aftale om styrket praksisfaglighed i folkeskolen [Agreement on strengthening practical expertise in primary school], https://www.regeringen.dk/media/5650/180612-aftale-om-styrket-praksisfaglighed-i-folkeskolen-ny.pdf (accessed on 29 March 2024).
[2] Lucas, B. (2022), Creative thinking in schools across the world: A snapshot of progress in 2022.
[3] Ministry of Education and Research (2019), Skaperglede, engasjement og utforskertrang: Praktisk og estetisk innhold i barnehage, skole og lærerutdanning [Creative joy, engagement and exploration: Practical and aesthetic content in kindergarten, school and teacher training], https://www.regjeringen.no/contentassets/c8bbb637891443fea7971ba8e936bca4/skaperglede-engasjement--og-utforskertrang.pdf (accessed on 29 March 2024).
[6] Ministry of Education, Ontario (2010), Growing Success: Assessment, Evaluation and Reporting in Ontario Schools. First Edition, Covering Grades 1 to 12, https://www.edu.gov.on.ca/eng/policyfunding/growSuccess.pdf.
[8] OECD (2023), PISA 2022 Creative Thinking Framework, OECD Publishing, Paris, https://doi.org/10.1787/dfe0bf9c-en.
[7] OECD (2023), Supporting Students to Think Creatively: What Education Policy Can Do, OECD Publishing, Paris, https://issuu.com/oecd.publishing/docs/supporting_students_to_think_creatively_web_1_?fr=sMGE0ZjYxMjMxNTE.
[9] OECD (2022), Thinking Outside the Box: The PISA 2022 Creative Thinking Assessment, https://issuu.com/oecd.publishing/docs/thinking-outside-the-box (accessed on 4 March 2023).
[1] Robinson, K. (2011), Out of Our Minds: Learning to be Creative, Capstone Publishing Ltd., Chichester, https://doi.org/10.1002/9780857086549.
[5] So, K., Y. Hu and J. Park (2017), “Making our schools more creative: Korea’s efforts and challenges”, The International Education Journal: Comparative Perspectives, Vol. 16/4, pp. 77-88, https://openjournals.library.sydney.edu.au/index.php/IEJ.
Notes
← 1. In PISA 2022, students in Portugal performed around the OECD average (i.e. not statistically significantly different) in mathematics, reading and science. Students in Belgium and Latvia* also performed around the OECD average in reading but performed statistically significantly above the OECD average in mathematics and science.
← 2. The correlation coefficient between mean performance and standard deviation (at the system-level) is -0.26.
← 3. The correlation between student performance in creative thinking with mathematics, reading and science, respectively, when computed at the PISA average level (all participating countries), is very similar to the OECD average: 0.68 for mathematics, 0.68 for reading, and 0.67 for science.
← 4. In previous years, correlations between performance in the PISA innovative domain assessments and the PISA core domains have also tended to be stronger than those between the creative thinking assessment and the PISA core domains. For reference, the correlations between performance in global competence and the other PISA domains in 2018, among the 27 countries who took the global competence assessment, were as follows: mathematics (0.73), reading (0.84), and science (0.79). In 2015, the correlation between performance in the collaborative problem-solving assessment and performance in the PISA core domains (OECD average) was as follows: mathematics (0.70), reading (0.74), and science (0.79). In 2012, the correlation between performance in the problem-solving assessment and performance in the PISA core domains (OECD average) was as follows: mathematics (0.81), reading (0.75), and science (0.78).
← 5. The analysis in this chapter primarily focuses on the variation in creative thinking associated with mathematics performance for two reasons. First, mathematics was the major focus of the PISA 2022 cycle, meaning around 86% of students who sat the creative thinking assessment also completed one hour of mathematics items (with the remaining 14% of students who sat the creative thinking assessment equally split between reading (around 7%) and science (around 7%) for the other hour of testing time). Second, there are significant correlations between student performance in mathematics, reading and science.
← 6. Very similar proportions of students (OECD average) within each quintile on the reading scale scored within the equivalent quintiles on the creative thinking scale as was observed in mathematics. For example: around 58% of all students who performed in the bottom quintile in mathematics, and around 57% of all students who performed in the bottom quintile in reading, also performed in the bottom quintile in creative thinking; and around 51% of all students who performed in the top quintile in mathematics and in reading, respectively, also performed in the top quintile in creative thinking. Amongst students in the third quintile in creative thinking, 23% performed in second quintile in reading, 27% performed in the third quintile in reading, and 24% performed in the fourth quintile, compared to 23% (second quintile), 28% (third quintile) and 24% (fourth quintile) in mathematics. In both mathematics and reading, 14% of all students within the third quintile performed within the upper quintile in creative thinking.
← 7. Relative performance in creative thinking is estimated by the residual obtained from a cubic polynomial regression of the student’s performance in creative thinking over his or her performance in mathematics (or reading). The regression is performed at the international level, pooling data from all countries and economies that participated in the creative thinking assessment. Students who scored higher than expected in creative thinking are those with positive relative scores.
← 8. The Australian Curriculum, Assessment and Reporting Authority (ACARA) “Critical and Creative Thinking learning continuum” can be accessed at the following link: https://www.australiancurriculum.edu.au/media/1072/general-capabilities-creative-and-critical-thinking-learning-continuum.pdf. The adaptation of the ACARA learning continuum and standards by the Victorian Curriculum and Assessment Authority (VCAA) in the state of Victoria (Australia) can be accessed here: https://victoriancurriculum.vcaa.vic.edu.au/critical-and-creative-thinking/curriculum/f-10.
← 9. The LearnAlberta platform provides simple competency progressions for cross-cutting competencies, including creativity and innovation, for different age ranges.
← 10. More information on the Education State Targets and the CCT assessment can be found online at the following link: https://www.vcaa.vic.edu.au/assessment/f-10assessment/edstateap/Pages/index.aspx. Selected examples from the psychometrically-validated tasks can also be accessed via the website.
← 11. For a detailed description of student performance at each proficiency level in creative thinking, please refer to Table III.1.1 in Chapter 1 of this volume.
← 12. In descending order – from largest share (84%) of students to smallest share of students (just above 50%) that did not reach Level 3 in creative thinking – these countries and economies are: Albania**, Uzbekistan, the Dominican Republic**, the Philippines, Morocco, the Palestinian Authority, Indonesia, North Macedonia, Jordan, Thailand, Bulgaria, Baku (Azerbaijan), El Salvador, Brazil, Saudi Arabia, Peru, Panama*, Kazakhstan, Brunei Darussalam and Moldova.
← 13. Three items from the creative thinking item pool were excluded from the engagement analysis due to their different item characteristics. Two items (T400Q02 and T420Q02) enabled students to select an idea from the previous question via a multiple-choice response format, while for the third item (T300Q01), students could reasonably submit a one-word answer and achieve full credit. It was therefore not appropriate to include these items in measures of disengagement focusing on time-on-task (rapid responders and relative rapid responders). For consistency, these items were also dropped from the measure of leaving responses blank. For more information on the construction of indicators and for results by country, see Annex A8.
← 14. The OECD average correlation between the two “rapid responder” measures (rapid responders and relative rapid responders) is 0.71. The OECD average correlation between rapid responders and students who left responses blank is 0.77. The OECD average correlation between relative rapid responders and students who left responses blank is 0.67. Correlations are computed at the country level and are the average correlations across OECD countries.