This chapter describes strengths and weaknesses in creative thinking performance across countries and economies. It examines performance differences across items in the three ideation processes (generate diverse ideas, generate creative ideas, and evaluate and improve ideas) and the four domain contexts (written expression, visual expression, social problem solving and scientific problem solving) of creative thinking. It also explores performance differences across domains and ideation processes with respect to student characteristics, such as gender and socio-economic and cultural status.
PISA 2022 Results (Volume III)
4. Strengths and weaknesses in creative thinking performance
Abstract
For Australia*, Canada*, Denmark*, Hong Kong (China)*, Jamaica*, Latvia*, the Netherlands*, New Zealand* and Panama* caution is advised when interpreting estimates because one or more PISA sampling standards were not met (see Reader’s Guide, Annexes A2 and A4).
For Albania** and the Dominican Republic**, caution is required when comparing estimates with other countries/economies as a strong linkage to the international PISA creative thinking scale could not be established (see Reader's Guide and Annex A4).
Thinking outside of the box, improving ideas and processes, applying new techniques and methods, and making connections across disciplines are important skills for success in school, the workplace and society. To what extent are students capable of applying creative thinking processes across different activities and subject areas? Which students are better prepared to think of original solutions and which are better at thinking of different ways to address the same problem? While student performance in creative thinking is measured by scores on the PISA scale, single scores do not provide any information about the relative strengths and weaknesses in performance across and within countries. This chapter further examines performance differences in creative thinking by different types of task groupings, including by ideation process and by domain context. It also analyses performance differences across task types by student characteristics including gender and socio-economic status.
What the data tell us
Students in Singapore were the most successful (i.e. achieved full credit) in several types of tasks, particularly social problem-solving tasks. Students in Korea were the most successful in scientific problem-solving contexts and in tasks requiring students to evaluate and improve ideas, and students in Portugal performed the most successfully in visual expression tasks. In total, 7 of the 12 countries that performed significantly above the OECD mean in creative thinking (Australia*, Canada*, Denmark*, Estonia, Korea, New Zealand*, Poland) performed significantly above the OECD average in each task subset.
Within each country/economy, students’ success in a subset of tasks compared to their performance on all other tasks can identify relative strengths and weaknesses in performance. After accounting for the international difficulty of items, students in Lithuania, Serbia and Moldova showed the largest relative strength in “generate creative ideas” tasks compared to other countries, whereas students in Korea and Portugal showed the largest relative weakness in these tasks (while still performing highly in absolute terms). Students in the Ukrainian regions (18 of 27) had the strongest relative performance by far in “evaluate and improve ideas” items, whereas students in Albania** and Hungary demonstrated the weakest relative performance in those items. Students in many countries demonstrated relative weakness in generating diverse ideas, particularly in Germany, Slovenia and the Ukrainian regions (18 of 27).
In general, and after accounting for the difficulty of items, students demonstrated relative strengths in creative expression tasks (written and visual) and relative weaknesses in creative problem-solving tasks, compared to their performance across all other tasks. Countries with the weakest relative performance in written expression tasks compared to other tasks were Malaysia (by some margin, around 8 percentage points) and Thailand (5 percentage points). Students in Italy, Mexico, Chile, Iceland and Czechia showed the largest relative weakness in social problem-solving tasks, compared to their performance on other tasks, whereas students in in Slovenia, Latvia* and Panama* demonstrated the largest weakest relative performance in scientific problem-solving tasks compared to their performance on other tasks.
Gender differences persist across all task subsets. Girls were more successful than boys in all three ideation processes, and considerably more successful than boys in creative expression tasks on average (between 6 and 8 percentage points, OECD average). Girls’ performance advantage was weakest (though still positive) in scientific problem-solving tasks, on average girls outperformed boys in only 13 countries and economies (and performed significantly worse than boys in Mexico). Differences in performance across tasks generally remain significant when comparing boys and girls with similar performance in reading and in mathematics.
The performance gap between advantaged and disadvantaged students across all task groupings is vast, although this association is significantly moderated after accounting for students’ reading and mathematics performance. Nonetheless, it remains significant across all task groupings.
A snapshot of strengths and weaknesses in creative thinking
Items in the PISA 2022 Creative Thinking assessment measure three ideation processes involved in creative thinking: generating diverse ideas, generating original ideas, and evaluating and improving ideas. Items were also contextualised within four distinct domains: written expression, visual expression, social problem solving and scientific problem solving (see Chapter 1 and Annex A1 for more information on the ideation processes and domain contexts included in the PISA 2022 Creative Thinking assessment). By analysing student performance on subsets of test items across countries/economies, it is possible to identify systematic differences in success when tackling different types of tasks (see Box III.4.1).
Box III 4.1. How success at the item level is analysed and reported
The PISA creative thinking scale reports student performance according to a single score. While this approach to scaling and reporting data has many advantages, it can hide interesting differences in patterns of performance at lower levels of aggregation, i.e. on single items or on subsets of items. To explore these patterns, the responses of students who answered each item must be analysed.
The analyses throughout this chapter are based on the average percentages of correct responses for different subsets of items at the country/economy level. For each item, the percentage of correct responses is simply the number of correct answers divided by the number of students who encountered the question (non-reached questions are counted as missing). This percentage is then averaged across groups of items. Table III.4.1 shows the distribution of items in the final item pool across the different task groupings.
When computing the average percentage correct on a subset of items, “correct” responses might include both partial and full credit responses or full credit responses only. For most analyses presented in this chapter, the average percentage correct is computed using full credit responses only. This is because full credit responses reflect ideas that are either sufficiently diverse from each other or sufficiently original compared to other responses, whereas partial credit responses reflect only conventional or similar ideas. In other words, full credit responses reflect skill in creative idea generation as opposed to simply appropriate idea generation.
Measuring the percentage of correct responses on average across countries is also a proxy for the difficulty of items (at the international level). By comparing the percentage of correct responses across two distinct sets of items, it is possible to identify the relative difficulty of each set. By further comparing the percentage of correct responses across two sets of items and across countries, it is possible to identify where the relative strengths and weaknesses of each country lie.
Figure III.4.1. provides an initial snapshot of student success across different types of tasks, on average across OECD countries. In general, students achieved full credit in tasks that required them to think of their own original or diverse ideas more often than those that required them to build on others’ ideas. In terms of task context, students achieved full credit in written expression tasks more than in any of the other domains: students could suggest original or diverse ideas for close to half of all written expression tasks they encountered, on average across OECD countries. In contrast, visual expression tasks were the hardest in which to achieve full credit, with students having done so in only around one-third of the items of this type they encountered, on average. It may be that specific drivers of difficulty were associated with tasks in the visual expression domain (see Box III.4.2).
Figure III.4.1 also shows the highest-performing country/economy for each task subset. Students in Singapore performed the highest in creative thinking overall and were the most successful in several types of tasks, especially social problem-solving tasks. In Korea, students performed the best out of all countries/economies in tasks in the scientific problem-solving domain and in evaluate and improve ideas tasks. Students in Portugal were the most successful in tasks in the visual expression domain out of all countries and economies.
In total, 7 of the 12 countries that performed significantly above the OECD mean in creative thinking (Australia*, Canada*, Denmark*, Estonia, Korea, New Zealand*, Poland) performed significantly above the OECD average in each task subset (Table III.B1.4.1 and Table III.B1.4.2). In Portugal, despite students having performed significantly above the OECD average overall, they performed significantly below the OECD average in generate creative ideas tasks.
Box III 4.2. Success in visual expression tasks: Drivers of difficulty
The relative difficulty (internationally) of achieving full credit for items in the visual expression domain may be explained by several factors. Firstly, students across many countries and economies may have little prior experience engaging with tasks asking them to create visual outputs in a formal assessment context. Secondly, the visual expression tasks required students to use a design tool; despite the tool’s simplicity, some students may not have been familiar with using such graphic design tools. However, when considering student success in the visual domain including partial credit responses, students are the most successful in these kinds of tasks, which implies that their relatively weaker performance at the full credit level reflects a greater difficulty in producing original and diverse designs, specifically, rather than any designs at all.
Why is there such a difference in success across the partial credit and full credit only measures? In all visual expression tasks, students were asked to create a simple design: the graphic tool enabled students to add lines and shapes, modify their colour and fill, and add stickers from the tool library. For every task, the stickers in the tool library were relevant to the task scenario, meaning that a simple design with only one or a few stickers might be considered appropriate (albeit unoriginal). Therefore, students who struggled to generate appropriate ideas for tasks in other domains requiring more active idea generation skill could achieve partial credit in visual expression tasks with only a minimum level of engagement. For example, Figure III.4.2 shows three example appropriate responses for Item 1 of the Science Fair Poster unit (shown in Figure III.1.9 of Chapter 1). Each example response has been composed of a combination of from the tool library, but the students have not combined these elements in any imaginative or meaningful way to connect to the theme of “Life in Deep Space”.
In contrast, full credit responses required students to integrate stickers within a broader and more complex design, or to combine lines, shapes and colours to create other relevant characters, objects or symbols with significance to the theme of “Life in Deep Space” (see Figure III.4.3 for three examples of full credit responses). This clearly required a greater investment in conceptualising and implementing a design with respect to partial credit responses, as well as potentially drawing on domain readiness (e.g. some graphic design skill).
Unsurprisingly, patterns in success across ideation processes (with respect to the OECD average) are more mixed amongst countries whose overall performance lies around the OECD mean (Table III.4.2). For example, students in Czechia and Germany were less successful in generating diverse ideas than the OECD average but more successful in generating creative ideas and evaluating and improving ideas. In Spain, students were less successful in generating creative ideas than in other OECD countries. This remains true even when including partial credit responses in the percentage correct measure, pointing toward a general difficulty for students in Spain to generate appropriate ideas for such tasks (Table III.B1.4.1). The same general difficulty for generating diverse ideas was also observed for students in Germany (see Box III.4.3).
Box III 4.3. Interpreting differences in success using both percentage correct measures (partial credit and full credit only)
In a few cases, patterns in the success of students (compared to the OECD average) are inverted when considering only full credit responses or when also considering partial credit responses in the measure. In Portugal (generate creative ideas), France (generate creative ideas, evaluate and improve ideas), and Hungary (evaluate and improve ideas), students achieved partial credit more often in a given task subset than on average across OECD countries, but achieved full credit in fewer such tasks than the OECD average (Table III.B1.4.1). This means that students in these countries/economies demonstrated better skills in idea generation (i.e. they suggested appropriate ideas) than most students but not in generating diverse or original ideas. In general, students who invested a minimum level of effort in a task and who suggested an idea implicitly or explicitly connected to the task stimulus were awarded partial credit.
Contrasting results across percentage correct measures were also observed in some domain contexts (Table III.B1.4.2). For example, in Uruguay, students achieved partial credit in nearly 3 percentage point more items than the OECD average in visual expression tasks but achieved full credit in fewer such tasks than the OECD average. The same pattern was also observed in France in social problem-solving tasks. These patterns can be interpreted in the same way as for the ideation processes: students in these countries/economies demonstrated more success in generating appropriate ideas in these domains but not in generating diverse or original ideas.
Some countries and economies that performed around or below the OECD mean in creative thinking were more successful in certain domains than the OECD average (Table III.4.2). Students in Czechia and Lithuania performed significantly better in written expression tasks than students across OECD countries in general, whereas other countries/economies with similar scores overall (e.g. France, Germany, Israel, the Netherlands* and Spain) performed significantly worse in tasks in the written expression domain. Students in Czechia and Lithuania, as well as Germany, also performed significantly better in visual expression tasks than students in OECD countries, on average – as did students in Mexico despite their overall performance in creative thinking being significantly below the OECD mean. Students in Israel were more successful at applying creative thinking in the social problem-solving domain, along with students in Chinese Taipei, whereas students in France and Czechia were less successful in this domain than on average across OECD countries and economies. Students in Macao (China), Israel, Italy, Spain and Chinese Taipei were also more successful in the scientific problem-solving domain than on average across OECD, unlike other countries who performed around or below the OECD mean.
Relative strengths and weaknesses within countries and economies
In the PISA 2022 Creative Thinking test, some subsets of items were more difficult than others across countries and economies. How did performance patterns across countries and economies change after accounting for the international difficulty of the items? Within each country/economy, were students more successful in particular tasks compared to their performance on all other tasks? In the two sections that follow on relative strengths and weakness in performance, “relative performance” refers to students’ success within a country/economy in a particular subset of tasks compared to their performance on all other tasks in that country/economy. These analyses are useful for making within-country comparisons to identify relative strengths and weaknesses in students’ performance. However, when interpreting relative performance results within a country/economy, it is important to keep in mind that country/economy’s overall performance in creative thinking.1
Relative performance in creative thinking by ideation processes
After accounting for the international difficulty of items across task groupings (i.e. that students were less successful in general in some items than others), students in Lithuania, Serbia and Moldova, in descending order, showed a particular relative strength in “generate creative ideas” tasks, achieving full credit in over 9 percentage point more of these tasks than all other tasks (Figure III.4.4). Students in Czechia, Slovenia and Germany also demonstrated a relative strength in generating creative ideas compared to other countries and economies. Countries with the weakest relative performance in generating creative ideas were Portugal and Korea, achieving full credit in around 8 and 9 percentage point less items targeting this ideation process, respectively, than others. However, both Portugal and Korea performed above the OECD average in creative thinking overall.
For “evaluate and improve ideas” items, students in the Ukrainian regions (18 of 27) demonstrated the strongest relative performance by far – achieving full credit in over 11 percentage point more of these items compared to other types of tasks (and after accounting for the difficulty of the items) (Figure III.4.5). Students in Qatar, Finland, Baku (Azerbaijan) and Hong Kong (China)* also achieved full credit in over 6 percentage point more “evaluate and improve” items than those corresponding to other ideation processes. Students in Albania** and Hungary demonstrated the weakest relative performance in items asking them to iterate on an idea to achieve a creative outcome (achieving full credit in around 5 percentage point less items).
Students in Korea, Portugal, the Slovak Republic and Singapore and Peru (in descending order) demonstrated a moderate to small relative strength in “generate diverse ideas” tasks (achieving full credit in between 3 and just over 5 percentage point more of these items), but many more countries demonstrated a relative weakness in this type of task after accounting for the international difficulty of the items (Figure III.4.6). In Germany, Slovenia and Ukrainian regions (18 of 27) this relative weakness was particularly large, with students achieving full credit on over 11 percentage point less items asking them to generate diverse ideas in different contexts.
Relative performance in creative thinking by domain contexts
Relative strengths and weaknesses can also be examined across tasks in different contexts. Situating test items in different domain contexts acknowledges that creative thinking is to some extent facilitated by domain readiness (see Box III.4.4). In some countries and economies, are students relatively more successful in demonstrating creative thinking in problem-solving contexts, for example, than those that require them to express their imagination?
Box III 4.4. Is creativity domain-general or domain-specific? Implications for education
Researchers in the field have long debated whether individuals are creative in everything they do or only in certain domains (i.e. a specific area of knowledge or practice). Early theories and tests of creativity focused on general and enduring attributes believed to influence creative endeavours of all kinds, reflecting the notion that an individual’s capacity to be creative in one domain would readily transfer to another (Torrance, 1988[1]). However, researchers now recognise that, to some extent, the internal resources needed to engage in creative work differ by domain (Baer, 2011[2]; Baer and Kaufman, 2005[3]). As such, the PISA 2022 Creative Thinking framework recognises domain readiness as an internal resource that can influence creative thinking performance in the PISA test (OECD, 2023[4]). While defining the boundaries of distinct “domains of creativity” remains an open research question, researchers tend to agree that the capacity to engage creatively in the arts and in maths/scientific domains, respectively, draws upon a different set of internal resources (e.g. knowledge, skills, and attributes) (Runco and Bahleda, 1986[5]; Kaufman and Baer, 2004[6]; Kaufman, 2006[7]; Kaufman, 2012[8]; Kaufman et al., 2010[9]; Kaufman et al., 2015[10]; Chen et al., 2006[11]; Julmi and Scherm, 2016[12])
Embedding creative thinking into the curriculum: Approaches across countries and economies
Creative thought and work draw upon domain-specific knowledge and skills that students may benefit from practicing in the context of specific curricular areas (Vincent-Lancrin et al., 2019[13]). In this sense, if the job of teachers is to design opportunities to integrate creative thinking into the discipline(s) they teach, the job of curriculum designers consists of clearly mapping the opportunities to embed creative thinking across curricular areas – a process that Lucas and Spencer as “split-screen thinking” (2017[14]).
Most jurisdictions acknowledge creative thinking in the curriculum as an interdisciplinary competence…
Virtually all system-level curricula or learning standards reference creative thinking as an educational goal – usually as a cross-cutting theme or competency (70% of PISA-participating systems) or as part of a larger set of transversal competencies, such as critical thinking or social-emotional skills (45% of participating systems) (OECD, 2023[15]). Examples of curriculum reform identifying creative thinking as a desired learning outcome abound: in Australia, Brazil, Canada and Iceland, amongst others, creative thinking (sometimes with critical thinking or entrepreneurship) is seen as one of several competencies that intersect with subjects or learning areas in the curriculum, and in Singapore, a “creative and inquiring mind” is one of the desired outcomes of the competence-based curriculum.
… but its practical integration tends to be limited to certain subject areas…
Despite most systems acknowledging developing creative thinkers as a key goal of education, only about half (53%) report that creative thinking is referenced in specific disciplinary contexts within the curriculum ( (OECD, 2023[15]). Most often, creativity and creative thinking are targeted within the broader arts subjects (e.g. visual arts, performance arts) (see Figure III.1.1, Chapter 1). In some countries, national Arts Councils (or equivalent bodies) have been tasked with promoting the development of creative thinkers through arts education. For example, in Sweden, the Swedish Arts Council has worked with the government since 2008 to run the Creative Schools initiative aiming to strengthen the artistic and cultural offerings of schools: since 2013, the initiative has been rolled out to cover all years of compulsory schooling and both state and independent schools. In other Nordic countries, including Norway and Denmark, reforms have focused on strengthening opportunities for students to engage in practical creative work (see Box II.2.2 in Chapter 2). One concrete way this has been implemented in Denmark is by introducing a two-year practical/musical subject as a compulsory element of the curriculum that must be completed with an exam.
Other jurisdictions have focused on strengthening the link between STEM disciplines (science, technology, engineering and mathematics) and arts education – the so-called STEAM approach. In Ireland, various initiatives targeting design, creativity and innovation skills are underway: for instance, the Sc!ence Blast initiative encourages students to think, create, design, explore and learn new skills by partnering organisations with primary school classrooms and asking them to investigate a question they are curious about, ending with presenting their findings to a STEM judge (móltóir) who provides students with constructive feedback.2 In Korea, STEAM education is led by the Korea Foundation for the Advancement of Science and Creativity, a quasi-governmental organisation in charge of science and ICT that aims to foster creative technical talents.
… or to subject-independent modules or initiatives
Some countries integrated opportunities to develop creative thinking in their curriculum by introducing dedicated interdisciplinary or “subject-free” modules. The national curriculum introduced in Finland in 2014 has required Finnish schools to teach at least one inter-disciplinary module a year. This simple strategy both requires and enables teachers to work in ways likely to foster creativity, as creative thinking invites teachers and their students to look across disciplines (West, 2016[16]). This strategy can optimise instruction time and content coverage by emphasising connections across knowledge domains, because in interdisciplinary learning, the content and skills that learners practice are defined by the questions or themes that they work on rather than a strict separation of disciplines or subjects. Greater interdisciplinarity allows for the articulation of pedagogy around more contextualised and authentic problems, providing opportunities for less fragmented and more meaningful learning experiences for students as well as increasing educators’ professional learning and accountability through collaborative planning and teaching (OECD, 2013[17]).
In Singapore, an elective course in secondary education, the Applied Learning Programme, emphasises interdisciplinary learning and provides opportunities for students to develop creative thinking by generating relatively novel and appropriate ideas or products in authentic society and industry settings. The programme can also be applied to more specific thematic areas (e.g. STEM, language). An alternative course, the Learning for Life Programme, provides opportunities for students to creatively design activities and programmes (among other outputs) for the benefit of their communities.3
Finally in Ireland, to combat the potentially negative pressures of standardised assessment regimes on creative thinking in schools, students can opt to undertake a “transition year” that bridges the country’s junior and senior education cycles. The one-year programme allows students to focus on their personal, social and vocational development in the extended absence of examination pressures. Approximately 75% of schools in the country offered the programme in 2022.4 As part of their transition year, students can participate in different initiatives, such as The B!G iDEA, which aims to put creative thinking at the centre of Ireland’s secondary education system through experiential learning and mentorship. Supported by the broader Creative Ireland programme and industry partners, the B!G iDEA is a 15-week, practically-oriented programme developed to empower students to use their creativity to tackle society’s biggest challenges.
Source: OECD (2023[15]), Supporting Students to Think Creatively: What Education Policy Can Do and Bill Lucas (2022[18]), Creative thinking in schools across the world: A snapshot of progress in 2022.
Prior to accounting for the difficulty of tasks, students in nearly all countries and economies demonstrated a considerable relative strength in written expression tasks (Table III.B1.4.4) – largely since students, on average, were most successful, in written expression tasks. Even after accounting for the difficultly of the items, the relative success of students in this domain remains positive and significant in over half of all countries/economies (Figure III.4.7). In descending order, students in Czechia and Iceland (12 percentage points difference), Italy (around 10 percentage points) and Lithuania (over 8 percentage points) demonstrated the greatest relative performance in written expression items compared to tasks in other domains. After accounting for the (lack of) difficulty of tasks in this domain, the countries with the weakest relative performance in written expression by some margin were Malaysia (more than 8 percentage points difference) and Thailand (5 percentage points difference).
Students performed the strongest in the visual domain relative to their performance in other tasks on average across the OECD (after accounting for the difficulty of items). In 13 countries and economies this relative strength is large (a difference in success of 10 percentage points or more), and in Mexico, the Dominican Republic** and Romania, this relative performance difference was extremely large (a difference of between 17 and 20 percentage points) (Figure III.4.8). Countries with a relative strength in visual expression tasks include high-performing systems (e.g. Portugal) and very low-performing systems (e.g. the Dominican Republic**, Indonesia, the Philippines), as well as those that perform around or just below the OECD average (e.g. Chile, Germany, Malta). Similarly, students in both high-performing countries (e.g. Singapore) and low-performing countries (e.g. Albania**, the Palestinian Authority, Saudi Arabia) demonstrated the greatest overall weaknesses in visual expression tasks relative to their performance in tasks in the other domain contexts and compared to other countries/economies.
On average across OECD countries, students showed a moderate relative weakness in social problem solving (a difference in success of around -4 percentage points), with several countries/economies demonstrating larger relative weaknesses in this domain after accounting for the difficulty of the items (Figure III.4.9). Students in Italy, Mexico, Chile, Iceland and Czechia (in descending order) were all less successful in applying creative thinking to social problem-solving tasks than tasks across the other domains by around -10 to -11 percentage points. Only students in Malaysia were successful in 5 percentage points more items in social problem-solving than items across all the other domains, on average.
In Korea, Albania**, Spain, the Netherlands* and Italy (in descending order), students demonstrated a moderate relative strength in scientific problem-solving contexts after accounting for the relative difficulty of the tasks, achieving full credit in between 5 and 7 percentage point more items in this domain context compared to tasks across all others (Figure III.4.10). However, in general, like relative performance in social problem-solving tasks, students across OECD countries and economies exhibited a relative weakness in applying creative thinking to scientific problem-solving contexts. Students in Slovenia, Latvia* and Panama* demonstrated the weakest relative performance in scientific problem-solving tasks across countries and economies (a difference of between -7 and -8 percentage points).
In general, within-country strengths and weaknesses tend to converge around the more expressive and imaginative tasks, on the one hand, and the more functional problem-solving tasks on the other hand. On average across OECD countries, students performed relatively better in tasks in the written and visual expression domains, and relatively weaker in the two problem-solving domains. This might be expected, given that appropriate, diverse, and original ideas in problem-solving contexts are somewhat more constrained by practical considerations.
Strengths and weaknesses across task types by gender
In Chapter 3, large differences in creative thinking performance were observed between boys and girls in most countries and economies. Are similar gender differences observed across different subsets of items? Table III.4.3 shows that, in nearly all countries and economies, girls performed equally to or outperformed boys, on average, in tasks in all domains and in all ideation processes. In other words, gender differences in performance in favour of girls persist across different groupings of tasks. The only exception is in Mexico, where boys were more successful in scientific problem-solving tasks than girls. In Finland – the country with the largest gender gap in performance overall – girls performed much better than boys in nearly every subset of items. Here, “much better” is defined as a difference of at least 10 percentage points in the success of boys and girls in a given task subset.
Gender differences in success by ideation process
While boys were less successful than girls across all task types, girls had the greatest success compared to boys in tasks requiring them to build on others’ ideas to reach a creative solution or outcome (“evaluate and improve ideas” items). In all but eight countries and economies, girls outperformed boys in these tasks (Figure III.4.11). In only a few countries/economies were gender differences in success smaller for “evaluate and improve ideas” items than for the other two ideation processes. Girls also had a bigger advantage over boys in “generate diverse ideas” items, in general, compared to their advantage in “generate creative ideas” items. In Finland and Iceland, girls were much better at generating diverse ideas than boys. Overall, across countries and economies, girls were the most successful compared to boys in the two ideation processes in which students were least successful at the international level (“evaluate and improve ideas” items and “generate diverse ideas” items).
Gender differences in performance remain significant across tasks in all the three ideation processes after accounting for the performance of boys and girls in the core PISA domains. On average across OECD countries, the difference in the success of boys and girls after accounting for mathematics and reading performance lies between 4 and 5.5 percentage points in the three task groupings. Girls are considerably more successful in “generate creative ideas” tasks in Finland (a difference of 12 percentage points) as well as in Hong Kong (China)*, Latvia* and Macao (China) (around 8 percentage points) after accounting for mathematics and reading performance (Table III.B1.4.7). Similarly, girls had a large performance advantage after accounting for mathematics and reading in “evaluate and improve ideas” tasks in the Ukrainian regions (18 of 27) (around 14 percentage points) and Finland and Jamaica* (over 10 percentage points). Gender differences in “generate diverse ideas” tasks also remain significant on average across OECD countries (around 5.5 percentage points), but in 19 countries/economies, there are no significant differences between boys and girls in these tasks after accounting for mathematics and reading performance.
Gender differences in success by domain context
Performance patterns by gender across domain contexts are more nuanced than by ideation process – in part driven by gender differences in engagement with the test (see Box III.4.5). Girls had the biggest performance advantage in the written expression domain: in only four countries and economies – Chile, Costa Rica, Mexico and Peru – are there no significant differences between the performance of boys and girls in these tasks. Girls in Finland, Iceland, Jordan and Qatar (in descending order) performed much better than boys, achieving full credit in over 10 percentage point more tasks in this domain (rising to a difference of nearly 17 percentage points in Finland) (Figure III.4.12). Even when comparing students with similar mathematics and reading scores, gender differences remain quite large in written expression tasks, on average across OECD countries (over 6 percentage points), and girls in Finland and Iceland still achieve full credit in over 10 percentage point more tasks than boys in this domain (Table III.B1.4.8).
In the visual expression domain, girls also outperformed boys in most countries and economies, and performed much better than boys (achieving full credit in over 10 percentage point more items) in Israel, Latvia*, Estonia, Hong Kong (China)*, Iceland and Romania (in descending order) (Figure III.4.12). In Chile, girls were successful in nearly 9 percentage point more tasks in the visual expression domain despite gender differences being insignificant in all domains groupings. In Croatia, the Palestinian Authority and the Philippines, the opposite was true: there were no significant differences in the success of boys and girls in the visual domain despite girls outperforming boys on all other task context groupings. On average across OECD countries, gender differences in visual expression tasks remain significant and large after accounting for mathematics and reading performance (7 percentage points) (Table III.B1.4.8).
Significant gender differences were observed less frequently in the problem-solving domains than the creative expression domains, although girls still significantly outperformed boys in social problem-solving tasks in just under two-thirds of all countries and economies (Figure III.4.12). Girls were considerably more successful in social problem-solving tasks in Finland (a difference of 13 percentage points), Denmark* and Macao (China) (around 9 percentage points), and Jordan, Hong Kong (China)* Iceland and Saudi Arabia (8 percentage points). In Peru, social problem solving was the only domain context in which girls significantly outperformed boys in creative thinking (by 3 percentage points). In some countries/economies, in particular in Finland and Macao (China), gender differences in success are above 10 percentage points even after accounting for mathematics and reading performance (Table III.B1.4.8). In contrast to the other three domain contexts, girls significantly outperformed boys in scientific problem-solving contexts in only 13 countries and economies – and performed significantly worse than boys in Mexico in these tasks. After accounting for mathematics and reading performance, girls outperformed boys in only 9 countries/economies and achieved full credit in only 2 percentage point more items than boys (Table III.B1.4.8).
Box III 4.5. Differences between boys and girls in engagement with tasks across domain contexts
Gender differences in engagement across tasks in the four domain contexts of creative thinking mirror gender differences in engagement with the test overall – that is, boys show more disengaged behaviours in all types of tasks. Figure III.4.13 shows that the difference between boys and girls in relatively rapid responses is greater in the written domain (around 6 percentage points) and similar in the other domains (between 3 and 4 percentage points). Gender differences in this indicator of disengagement are the largest (over 10 percentage points) in Albania**, Jordan, the Palestinian Authority, Qatar and the United Arab Emirates.
During adolescence, gender differences in attitudes towards school and learning become more evident (OECD, 2015[19]). Research suggests that these attitudes are related to how girls and boys have absorbed society’s notions of “masculine” and “feminine” behaviour. Social pressures, together with possible gender differences in general personality traits like conscientiousness (Costa, Terracciano and McCrae, 2001[20]), might drive differences in the effort girls and boys exert on the PISA test as a whole and across different types of tasks.
Expectancy value theory (Eccles and Wigfield, 2002[21]) might provide another hypothesis: the theory posits that students are more likely to pursue an activity if they expect to do well and if they value the activity. Task value can be broken into four components: attainment value (i.e. importance of doing well), intrinsic value (i.e. personal enjoyment), utility value (i.e. perceived usefulness for future goals), and cost (i.e. competition with other goals). It is possible that societal notions or pressures influence boys and girls to perceive the potential value of certain activities differently and thus influence their levels of engagement in different domains. Differences in performance across student groups are likely to reflect all these drivers of engagement, that are in turn influenced by cultural values and peer pressures.
Strengths and weaknesses across task types by socio-economic background
In the same way that performance differences across subsets of items can be examined by gender, differences can also be observed between students from advantaged and disadvantaged backgrounds (i.e. students in the top quarter and the bottom quarter of the PISA index of ESCS). Chapter 3 established that advantaged students across all countries and economies significantly outperformed disadvantaged students in the creative thinking test. Does this association hold true across all subsets of items? In general, advantaged students maintain a large performance advantage over their disadvantaged peers across all task groupings (Table III.B1.4.9 and Table III.B1.4.10). Given the strength of the relationship between socio-economic background and creative thinking performance overall, this result is not surprising – however, interesting differences in patterns of success by task grouping can be observed across countries and economies.
Differences in success of advantaged and disadvantaged students by ideation process
Advantaged students had the largest difference in success compared to disadvantaged students in “generate diverse ideas” items (19 percentage points on average across OECD countries) and the smallest in “evaluate and improve ideas” tasks (close to 15 percentage points difference) (Figure III.4.14). Advantaged students performed significantly better in every country/economy in “generate diverse ideas” items, and considerably so in Israel, the Slovak Republic, Hungary and Romania (in descending order, achieving full credit in around 25 percentage points or more items) compared to other countries/economies. While performance differences between advantaged and disadvantaged students were significant and, in most cases, large across countries and economies in “generate creative ideas” items, disadvantaged students performed relatively closer to their advantaged peers in Denmark, Jamaica* and Spain in these tasks than in other types of tasks (unlike students in other countries). In Croatia, Hong Kong (China)*, Malta and Uzbekistan, differences in the performance of advantaged and disadvantaged students were non-significant in “evaluate and improve ideas” tasks (and “generate creative ideas” tasks in Uzbekistan). It may be that the additional constraints inherent to “evaluate and improve ideas” items (where students must build on an already-provided idea) weaken some of the performance differences otherwise observed between advantaged and disadvantaged students in task types that are more open and, perhaps, influenced by students’ prior knowledge and experiences (see Box III.4.6).
Differences in task success between students with different socio-economic backgrounds generally remain significant after accounting for mathematics and reading scores – albeit the magnitude of difference is largely reduced. On average across OECD countries, advantaged students achieved full credit in around 6 percentage point more “generate diverse ideas” tasks than their disadvantaged peers, after accounting for mathematics and reading performance, and around 4 percentage point more “generate creative ideas” and “evaluate and improve ideas” tasks respectively (Table III.B1.4.11). However, in Peru and Panama*, disadvantaged students were still less successful in two of the three ideation processes by over 10 percentage points compared to their advantaged peers after accounting for mathematics and reading performance. Disadvantaged students were less successful by a similar margin in Israel and New Zealand* in “generate diverse ideas” tasks, in El Salvador, Lithuania, Macao (China), Qatar, Romania and the Ukrainian regions (18 of 27) in “generate creative ideas” tasks) and in Germany in “evaluate and improve ideas” tasks.
Differences in success of advantaged and disadvantaged students by domain context
What about differences in success between advantaged and disadvantaged students across domain contexts? The association between socio-economic advantage and task success is strongest in written expression tasks, in general, and weakest in visual expression tasks (Table III.B1.4.10). In over one-third of all countries and economies, the performance gap between advantaged and disadvantaged students in written expression tasks is very large at over 20 percentage points, and in Romania, the Ukrainian regions (18 of 27), Brunei Darussalam, Moldova and Hungary (in descending order), this difference is around 25 percentage points or more (Figure III.4.15). These considerable differences especially in the written domain may be influenced, in part, by the likely greater cultural wealth of advantaged students (e.g. more books at home) as well as their overall stronger proficiency in basic literacies. After accounting for the mathematics and reading performance of students, differences in success between advantaged and disadvantaged students remain significant but much smaller across all domains (between 4 and 5 percentage points, on average across OECD countries) (Table III.B1.4.12).
In visual expression tasks – where writing skills are less likely to influence students’ capacity to design visual outputs – there are no significant differences observed in the performance of advantaged and disadvantaged students in six countries and economies (Croatia, Jordan, Macao [China], Malta, the Palestinian Authority and Uzbekistan). In over half of all participating countries and economies, differences in success between advantaged and disadvantaged students are the smallest in the visual expression domain compared to the other three domain contexts, and after accounting for students’ mathematics and reading performance, performance differences between advantaged and disadvantaged students become insignificant in around two-thirds of all countries and economies – supporting the notion that such differences are largely derived from students’ overall basic literacies in task comprehension rather than mathematics/reading skills further inhibiting students’ capacity to express their ideas in visual tasks (Table III.B1.4.12) (see Box III.4.6).
Advantaged students significantly outperformed disadvantaged students in all but one country in social problem-solving contexts (Uzbekistan) and all but four countries/economies in scientific problem-solving contexts (Baku [Azerbaijan], Hong Kong [China]*, Jamaica* and Uzbekistan). Large performance differences (around 20 percentage points or more) between advantaged and disadvantaged students in social problem-solving tasks were observed in Israel, the Slovak Republic, Singapore, Germany and Lithuania (in descending order) and in scientific problem-solving tasks in New Zealand*, Romania, and Israel. When comparing students with similar mathematics and reading scores, performance differences remain significant in around half of all countries/economies in social problem-solving contexts (7.5 percentage points, OECD average) and in around one-third of all countries/economies in scientific problem-solving contexts (around 3 percentage points, OECD average). After accounting for mathematics and reading, advantaged students are still considerably more successful in social problem-solving tasks in Panama*, Macao (China), Lithuania (differences of between 10 and 18 percentage points) and in scientific problem-solving tasks in New Zealand* and Malaysia (over 10 percentage points) (Table III.B1.4.12).
Box III 4.6. Socio-economic background, writing proficiency and success in creative thinking tasks
Many renowned artists, scientists and inventors, like David Bowie, members of The Beatles and Queen, Vincent van Gogh, Henri Matisse, Marie Curie, and Jan Koum (the creator of WhatsApp), came from humble beginnings. Their successes across different fields highlight that creative potential is not limited by economic background: every child, regardless of their family's financial status, has the potential to excel in creative thinking. Encouraging all students to think creatively is crucial for countries to unlock their full talent pool and foster social mobility.
Unfortunately, unequal access to foundational literacies can hinder the development of creative potential; as shown in this and the previous chapter, disadvantaged students face greater challenges in generating, evaluating and improving original ideas and diverse ideas. Analysing how advantaged and disadvantaged students perform on specific items reveals further insights as to what these challenges might be. Disadvantaged students performed notably worse than advantaged students in tasks requiring extended written responses. For example, in the unit Robot Story – where students must describe two different ideas for a movie (see Figure III.1.5 in Chapter 1) – the difference in success between advantaged and disadvantaged students is nearly 23 percentage points. The equivalent gap in item success is much lower (only 11 percentage points) in the Space Comic unit, another written expression task in which students must create a short fictional dialogue between the Earth and the Sun (see Figure III.1.6). In this item, students were able to respond effectively using only a few words. These comparisons suggest that limited writing skills, and possibly less familiarity with typing on computers, may contribute to the socio-economic gaps observed in performance.
Similar patterns can be observed in other domain contexts that require written responses. In the unit Save the River (scientific problem solving), advantaged students were 19 percentage points more successful in the first item (Figure III.1.19) that asked students to elaborate two different reasons as to why there might be a problem with frogs in a particular part of the river; this gap narrowed to 8 percentage points in the second item of the unit in which students must suggest how to change an experiment (possibly using only a few words) (see Figures III.1.20-21 and Box III.1.8 in Chapter 1). Similarly, in the unit Library Accessibility (social problem solving), the performance gap between advantaged and disadvantaged students was much greater in the first item that required three different ideas (18 percentage points) than in the second item where only a short response was sufficient (5 percentage points) (see Figures III.1.13-16 and Box III.1.7 in Chapter 1).
In contrast, differences in success between advantaged and disadvantaged students in visual expression tasks were relatively small and more akin to those observed in other short, written constructed response items. For example, in the two tasks included in the Science Fair Poster unit (see Figures III.1.9-12 and Box III.1.6 in Chapter 1), advantaged students were 7 and 13 percentage points more successful, respectively, than disadvantaged students. These item-level patterns in success suggest that disadvantaged students may struggle to fully express their creative potential when tasks demand more than simple written responses (e.g. a few words). Addressing students’ basic writing skills could therefore help close some of the observed gap in creative thinking performance between advantaged and disadvantaged students across domain contexts.
Examples of initiatives to promote creative skills amongst disadvantaged students
In a few countries, initiatives to promote creative thinking have been explicitly targeted towards engaging students from disadvantaged backgrounds. For example, in Sweden, the Berattarministeriet (or “Ministry of Storytelling”) is a non-profit organisation that has supported teachers to nurture creativity in children in primary schools since 2011, targeting disadvantaged communities and focusing on the practice of storytelling.5 The organisation provides free school programmes, implemented in the classroom and based on the curriculum, that entice students, regardless of their level of knowledge, to “conquer the written word” in a fun and creative way. The teachers are at the same time given tools and support in their professional learning, hence theory and practice are interlaced (Berättarministeriet, Forthcoming[22]).
In England (United Kingdom), the Creative Partnerships programme (funded primarily by the Department for Culture, Media and Sport (DCMS), and managed by Arts Council England) was rolled out from 2002 to 2009 in 36 of the most disadvantaged areas in England. The programme paired schools with creative professionals from all fields, including architects, artists and scientists, to develop joint solutions to an identified issue or priority improvement area in the school (House of Commons, 2007[23]). The programme resulted in more than 8 000 projects in over 2 400 schools, reaching around 1 million children and more than 90 000 teachers.
Table III 4.4. Student performance in creative thinking: Chapter 4 figures and tables
Table III.4.1 |
Distribution of items in the PISA 2022 Creative Thinking test |
Table III.4.2 |
Comparing countries’ and economies’ performance in creative thinking success by ideation processes and domain contexts |
Table III.4.3 |
Gender differences in performance by ideation processes and domain contexts |
Figure III.4.1 |
Performance in creative thinking, across domain contexts and ideation processes |
Figure III.4.4 |
Relative performance in “generate creative ideas” tasks |
Figure III.4.5 |
Relative performance in “evaluate and improve ideas” tasks |
Figure III.4.6 |
Relative performance in “generate diverse ideas” tasks |
Figure III.4.7 |
Relative performance in written expression tasks |
Figure III.4.8 |
Relative performance in visual expression tasks |
Figure III.4.9 |
Relative performance in social problem-solving tasks |
Figure III.4.10 |
Relative performance in scientific problem-solving tasks |
Figure III.4.11 |
Gender differences in success across ideation processes |
Figure III.4.12 |
Gender differences in success across domain contexts |
Figure III.4.13 |
Gender differences in task disengagement across domain contexts |
Figure III.4.14 |
Differences in success across ideation processes related to students' socio-economic status |
Figure III.4.15 |
Differences in success across written expression and social problem-solving tasks related to socio-economic status |
References
[2] Baer, J. (2011), “Domains of Creativity”, in Encyclopedia of Creativity, Elsevier, https://doi.org/10.1016/b978-0-12-375038-9.00079-0.
[3] Baer, J. and J. Kaufman (2005), “Bridging generality and specificity: The amusement park theoretical (APT) model of creativity”, Roeper Review, Vol. 27/3, pp. 158-163, https://doi.org/10.1080/02783190509554310.
[22] Berättarministeriet (Forthcoming), Fostering Creativity in Teaching Enhances Students’ Learning, Swedish Study on Creativity in Education, Berättarministeriet.
[11] Chen, C. et al. (2006), “Boundless Creativity: Evidence for the Domain Generality of Individual Differences in Creativity”, The Journal of Creative Behavior, Vol. 40/3, pp. 179-199, https://doi.org/10.1002/j.2162-6057.2006.tb01272.x.
[20] Costa, P., A. Terracciano and R. McCrae (2001), “Gender differences in personality traits across cultures: Robust and surprising findings.”, Journal of Personality and Social Psychology, Vol. 81/2, pp. 322-331, https://doi.org/10.1037/0022-3514.81.2.322.
[21] Eccles, J. and A. Wigfield (2002), “Motivational Beliefs, Values, and Goals”, Annual Review of Psychology, Vol. 53/1, pp. 109-132, https://doi.org/10.1146/annurev.psych.53.100901.135153.
[23] House of Commons (2007), Creative Partnerships and the Curriculum, House of Commons London: The Stationary Office Limited, https://publications.parliament.uk/pa/cm200607/cmselect/cmeduski/1034/1034.pdf.
[12] Julmi, C. and E. Scherm (2016), “Measuring the domain-specificity of creativity”, Fakultät für Wirtschaftswissenschaft der FernUniversität in Hagen No. 502, https://www.fernuni-hagen.de/imperia/md/images/fakultaetwirtschaftswissenschaft/db-502.pdf.
[8] Kaufman, J. (2012), “Counting the muses: Development of the Kaufman Domains of Creativity Scale (K-DOCS).”, Psychology of Aesthetics, Creativity, and the Arts, Vol. 6/4, pp. 298-308, https://doi.org/10.1037/a0029751.
[7] Kaufman, J. (2006), “Self-reported differences in creativity by ethnicity and gender”, Applied Cognitive Psychology, Vol. 20/8, pp. 1065-1082, https://doi.org/10.1002/acp.1255.
[6] Kaufman, J. and J. Baer (2004), ““Sure, I’m creative -- but not in mathematics!”: Self-reported creativity in diverse domains”, Empirical Studies of the Arts, Vol. Vol. 22/2, pp. pp. 143-155, http://journals.sagepub.com/doi/pdf/10.2190/26HQ-VHE8-GTLN-BJJM.
[9] Kaufman, J. et al. (2010), “Personality and Self-Perceptions of Creativity across Domains”, Imagination, Cognition and Personality, Vol. 29/3, pp. 193-209, https://doi.org/10.2190/ic.29.3.c.
[10] Kaufman, S. et al. (2015), “Openness to Experience and Intellect Differentially Predict Creative Achievement in the Arts and Sciences”, Journal of Personality, Vol. 84/2, pp. 248-258, https://doi.org/10.1111/jopy.12156.
[18] Lucas, B. (2022), Creative thinking in schools across the world: A snapshot of progress in 2022.
[14] Lucas, B. and E. Spencer (2017), Teaching Creative Thinking: Developing Learners Who Generate Ideas and Can Think Critically, Crown House Publishing, https://bookshop.canterbury.ac.uk/Teaching-Creative-Thinking-Developing-learners-who-generate-ideas-and-can-think-critically_9781785832369.
[4] OECD (2023), PISA 2022 Creative Thinking Framework, OECD Publishing, Paris, https://doi.org/10.1787/dfe0bf9c-en.
[15] OECD (2023), Supporting Students to Think Creatively: What Education Policy Can Do, OECD Publishing, Paris, https://issuu.com/oecd.publishing/docs/supporting_students_to_think_creatively_web_1_?fr=sMGE0ZjYxMjMxNTE.
[19] OECD (2015), The ABC of Gender Equality in Education: Aptitude, Behaviour, Confidence, PISA, OECD Publishing, Paris, https://doi.org/10.1787/9789264229945-en.
[17] OECD (2013), Innovative Learning Environments, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/9789264203488-en.
[5] Runco, M. and M. Bahleda (1986), “Implicit Theories of Artistic, Scientific, and Everyday Creativity*”, The Journal of Creative Behavior, Vol. 20/2, pp. 93-98, https://doi.org/10.1002/j.2162-6057.1986.tb00423.x.
[1] Sternberg, R. (ed.) (1988), The nature of creativity as manifest in its testing”, Cambridge University Press, New York, NY, http://psycnet.apa.org/record/1988-98009-002.
[13] Vincent-Lancrin, S. et al. (2019), Fostering Students’ Creativity and Critical Thinking: What it Means in School, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/62212c37-en.
[16] West, R. (2016), “Breaking Down Walls to Creativity through Interdisciplinary Design”, Educational Technology, Vol. 56(6-), pp. 47–52.
Notes
← 1. Within-country relative strengths and weaknesses should be interpreted with caution when used for comparisons across countries, given the way in which relative performance is derived (i.e. average percentage correct in a subset of tasks compared to the average percentage correct on all other tasks). A large positive relative performance result in certain process(es)/context(s) will necessarily be accompanied by a large negative relative performance in the other process(es)/context(s). However, large positive or negative relative performance results are not necessarily reflective of strong or weak overall success in certain processes or domain contexts. When interpreting relative performance results within a country/economy, that country/economy’s overall performance in creative thinking should be kept in mind.
← 2. For more information about the Sc!ence Blast initiative in Ireland, see: https://www.esbscienceblast.com/about/ (accessed 24 May 2024).
← 3. For more information about the Applied Learning Programme and the Learning for Life Programme offered to secondary school students in Singapore, see: https://www.moe.gov.sg/secondary/courses/express/electives/ (accessed 24 May 2024).
← 4. For more information about Ireland’s “Transition Year” option, see: https://ncca.ie/en/senior-cycle/programmes-and-key-skills/transition-year/ (accessed 24 May 2024). Further information about “The B!G iDEA” project can also be found at the above link, or at the following dedicated website: https://thebigidea.ie/ (accessed 24 May 2024).
← 5. For more information about the “Ministry of Storytelling” in Sweden, see: https://www.berattarministeriet.se/om-oss-berattarministeriet-2/. (accessed 24 May 2024).