PISA 2022 Results (Volume III)

Annex C. Technical information for the released items from the PISA 2022 Creative Thinking assessment

Several creative thinking units and items were released from the main survey of the PISA 2022 assessment. Chapter 1 presents selected released items from 9 of the 18 units developed for the PISA 2022 Creative Thinking test, and in some cases, also presents example student responses at different credit levels (full, partial or none).

In this Annex, like in Chapter 1, screenshots of the interface used in PISA 2022 are shown to give readers an understanding of how students interacted with the assessment and its items. This Annex also summarises the technical information (item ID, ideation process, domain context, item format, item-specific coding criteria and proficiency level) for each of the released items described in Chapter 1. Interactive versions of all of these units are also available at www.oecd.org/pisa.

Unit T300: Illustration Titles

Illustration Titles: Released item #1 (DT300Q02)

 

 

Unit Name – Item #	Illustration Titles – DT300Q02
Domain context	Written expression
Ideation process	Generate diverse ideas
Item format	Constructed response - human scored
Coding criteria	In written expression tasks, differences in ideas are based on elements of the form requested, rather than on pre-defined categories. In this case, students are asked to provide their responses in the form of a title for a given illustration, and elements of this form are described in the coding guide. Appropriate titles (ideas that are on task and on topic) can be different in two ways: based on underlying focus – each title makes clearly different associations to the stimulus, so the subjects of each are clearly different: OR based on method of implementation – the titles have a similar underlying focus but implement linguistic or literary devices to change the representation of ideas. Linguistic or literary devices may include (but are not limited to): some titles consist of a literal description of the image or its components, and other titles consist of abstract associations or figurative expressions; each title reflects a different perspective or interpretation of the illustration as a whole, or of a component in the illustration; the titles use punctuation, capitalisation, spelling or other grammatical elements to create distinct meanings.
Proficiency levels	4 (full credit) 1 (partial credit)

Unit T570: Robot Story

Robot Story: Released item #1 (DT570Q01)

 

 

Unit Name – Item #	Robot Story – DT570Q01
Domain context	Written expression
Ideation process	Generate diverse ideas
Item format	Constructed response - human scored
Coding criteria	In written expression tasks, differences in ideas are based on elements of the form requested, rather than on pre-defined categories. In this case, students are asked to provide their responses in the form of a story idea for a film, and elements of this form are described in the coding guide. Appropriate story ideas (ideas that are on task and on topic) can be different in two ways: based on underlying focus – each story idea makes clearly different associations to the stimulus, so the subject of each plot is clearly different; OR based on method of implementation – the story ideas convey similar plots but implement story elements to change the representation of ideas. Story elements may include (but are not limited to): each story is conveyed from a different perspective, affecting how the plot is represented; each story has a different setting that changes how the characters interact or the significance of objects or events; each story conveys different relationships between characters, affecting how they interact or changing the significance of events; the actions and/or choices of the characters in each story are different, causing the plots to unfold differently; the attributes of the characters in each story are different, changing their motivations or the role they play in the story (e.g. background, abilities, personality).
Proficiency level	4 (full credit)

Unit T240: Space Comic

Space comic: Released item #1 (DT240Q01)

 

 

Unit Name – Item #	Space Comic – DT240Q01
Domain context	Written expression
Ideation process	Generate creative ideas
Item format	Constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “generate creative ideas” ideation process. There are two conventional themes for this item: Conventional Theme 1: Dialogue focusing on heat/temperature, weather, or seasons, but excluding discussions about environmental degradation or global warming; Conventional Theme 2: Dialogue focusing on environmental degradation or global warming. Appropriate responses (on task and on topic) corresponding to a conventional theme were awarded partial credit unless combined with an innovative approach or implementation. Appropriate responses corresponding to an original theme (i.e. not one of the conventional themes) were awarded full credit.
Proficiency levels	5 (full credit) 2 (partial credit)

Unit T370: 2983

2983: Released item #1 (DT370Q01)

 

 

Unit Name – Item #	2983 – DT370Q01
Domain context	Written expression
Ideation process	Generate creative ideas
Item format	Constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “generate creative ideas” ideation process. There are two conventional themes for this item: Conventional Theme 1: A positive or neutral account of what life is like for humans in the future (i.e. the year 2983); Conventional Theme 2: The number as a designation or identification for a person, a place, or an object (such as an address, a serial number, model number, or other identification number). Appropriate responses (on task and on topic) corresponding to a conventional theme were awarded partial credit unless combined with an innovative approach or implementation. Appropriate responses corresponding to an original theme (i.e. not one of the conventional themes) were awarded full credit.
Proficiency levels	4 (full credit) 3 (partial credit)

Unit T200: Science Fair Poster

Science Fair Poster: Released item #1 (DT200Q01)

 

 

Unit Name – Item #	Science Fair Poster – DT200Q01
Domain context	Visual expression
Ideation process	Generate creative ideas
Item format	Constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “generate creative ideas” ideation process. There are two conventional themes for this item: Conventional Theme 1: The Earth is the most dominant component used to represent life in space; Conventional Theme 2: Components that convey exploration of space (such as astronaut(s), spacecraft, vehicles, or constructed satellites) are the most dominant representation of life in space. Appropriate responses (on task and on topic) corresponding to a conventional theme were awarded partial credit unless combined with an innovative approach or implementation. Appropriate responses corresponding to an original theme (i.e. not one of the conventional themes) were awarded full credit.
Proficiency levels	6 (full credit) 1 (partial credit)

Science Fair Poster: Released item #2 (DT200Q02)

 

 

Unit Name – Item #	Science Fair Poster – DT200Q02
Domain context	Visual expression
Ideation process	Evaluate and improve ideas
Item format	Constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “evaluate and improve ideas” ideation process. There are three conventional themes for this item: Conventional Theme 1: The Earth is added to the poster to represent life in deep space; Conventional Theme 2: Plants or flora are added to the poster to represent life in deep space; Conventional Theme 3: Components that convey exploration of space (such as astronaut(s), spacecraft, vehicles, or constructed satellites) are added to the poster to represent life in space. Appropriate responses (on task and on topic) corresponding to a conventional theme were awarded partial credit unless combined with an innovative approach or implementation. Appropriate responses corresponding to an original theme (i.e. not one of the conventional themes) were awarded full credit.
Proficiency levels	6 (full credit) 1 (partial credit)

Unit T200: Library Accessibility

Library Accessibility: Released item #1 (DT500Q01)

 

 

Unit Name – Item #	Library Accessibility – DT500Q01
Domain context	Social problem solving
Ideation process	Generate diverse ideas
Item format	Constructed response - human scored
Coding criteria	In social problem-solving tasks, ideas can be different in two ways: based on underlying focus – solutions in a response belong to different categories or to different sub-categories (such as, but not limited to, the list below); OR based on method of implementation – the response introduces specifically different modes of implementing the same or similar solutions (such as by including distinct tools, strategies, people involved, etc.). Example categories and sub-categories for this item include the following: Category 1 – Physical modifications to the library Sub-category 1-1: Adding ramps; Sub-category 1-2: Modifying the staircase; Sub-category 1-3: Adding an elevator; Sub-category 1-4: Having only one floor; Sub-category 1-5: Having lower shelves; Sub-category 1-6: Having a special section for wheelchair users. Category 2 – Providing human assistance to wheelchair users (e.g. staff or volunteers deliver library materials or bring customers to the materials) Sub-category 2-1: Hiring staff or adding volunteers to retrieve library materials for customers, and/or to deliver materials to customers (this excludes solutions unrelated to increasing access to the building, such as buying books online from retailers); Sub-category 2-2: Hiring staff or adding volunteers to aid with taking customers to the materials. Category 3 – Providing technological assistance mechanisms (e.g. to aid with retrieving materials, guiding customers, or requesting deliveries) Sub-category 3-1: To aid with retrieving out-of-reach materials for customers; Sub-category 3-2: To aid with taking customers to the materials; Sub-category 3-3: To request delivery of or access to library materials (this excludes solutions unrelated to increasing access to the library’s building or the use of general solutions, such as “use e-readers”).
Proficiency levels	4 (full credit) 2 (partial credit)

Library Accessibility – Released item #2 (DT500Q02)

 

 

Unit Name – Item #	Library Accessibility – DT500Q02
Domain context	Social problem solving
Ideation process	Evaluate and improve ideas
Item format	Constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “evaluate and improve ideas” ideation process. There are two conventional themes for this item: Conventional Theme 1: Automate the floor of the ramp to move customers in wheelchairs using a conveyor belt; Conventional Theme 2: Automate the ramp in ways to move customers in wheelchairs (push and/or pull devices, ramps that move to different locations, etc.). Appropriate responses (on task and on topic) corresponding to a conventional theme were awarded partial credit unless combined with an innovative approach or implementation. Appropriate responses corresponding to an original theme (i.e. not one of the conventional themes) were awarded full credit.
Proficiency levels	6 (full credit) 5 (partial credit)

Unit T400: Save the Bees

Save the Bees: Released item #1 (DT400Q02)

 

 

Unit Name – Item #	Save the Bees – DT400Q02
Domain context	Social problem solving
Ideation process	Generate creative ideas
Item format	Multiple choice based on previously constructed response, or constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “generate creative ideas” ideation process. There are three conventional themes for this item: Conventional Theme 1: Methods or content that club members may use to verbally communicate the importance of bees (this theme applies to solutions in which someone explains, tells, shares, etc.); Conventional Theme 2: Create and/or present informative visuals (e.g. videos, posters or flyers); Conventional Theme 3: Enable interaction with or the observation of live bees. Appropriate responses (on task and on topic) corresponding to a conventional theme were awarded partial credit unless combined with an innovative approach or implementation. Appropriate responses corresponding to an original theme (i.e. not one of the conventional themes) were awarded full credit.
Proficiency levels	5 (full credit) 4 (partial credit)

Unit T630: Carpooling

Carpooling: Released item #1 (DT630Q01)

 

 

Unit Name – Item #	Carpooling – DT630Q01
Domain context	Social problem solving
Ideation process	Evaluate and improve ideas
Item format	Constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “Evaluate and Improve Ideas” ideation process. There is only one conventional theme for this item: Conventional Theme 1: Establish additional financial incentives (e.g. rewards in the form of currency, OR discounts for services or items other than discounts on fuel and tolls); Appropriate responses (on task and on topic) corresponding to the conventional theme were awarded partial credit unless combined with an innovative approach or implementation. Appropriate responses corresponding to an original theme (i.e. not one of the conventional themes) were awarded full credit.
Proficiency levels	5 (full credit) 4 (partial credit)

Unit T690: Save the River

Save the River: Released item #1 (DT690Q01)

 

 

Unit Name – Item #	Save the River – DT690Q01
Domain context	Scientific problem solving
Ideation process	Generate diverse ideas
Item format	Constructed response - human scored
Coding criteria	In scientific problem solving, appropriate hypotheses can be different in two ways: based on underlying focus – hypotheses in a response belong to different categories (or sub-categories) of reasons why the frog population has decreased (such as, but not limited to, the list in the table below); OR based on method of implementation – hypotheses suggest specifically different effects to explain how the frog population has decreased due to the same underlying cause (for instance, the reason why there are fewer frogs could be the same, but the decrease may have happened because of either increased mortality or increased migration). Example categories and sub-categories for this item include the following: Category 1: Changes to the water habitat Sub-category 1-1: Changes in water level; Sub-category 1-2: Changes in water temperature. Category 2: Changes in surrounding fauna (non-human) Sub-category 2-1: New/additional predators; Sub-category 2-2: New/additional competitors for food. Category 3: Changes to the local flora (e.g. a new invasive plant species, or absence of important flora) Sub-category 3-1: Changes to food availability; Sub-category 3-2: Changes in climate affecting local flora; Sub-category 3-3: Human interventions affecting local flora. Category 4: Changes to the frogs themselves (e.g. infection, disease or mutation) Sub-category 4-1: Infection or disease; Sub-category 4-2: Mutation. Category 5: Changes to the behaviour or activities of humans in the area (e.g. noise, ground vibrations, or humans capturing frogs) Sub-category 5-1: Change in noise; Sub-category 5-2: Excess ground vibrations; Sub-category 5-3: Capturing or removing frogs from the river.
Proficiency level	5 (full credit)

Save the River: Released item #2 (DT690Q02)

 

 

Unit Name – Item #	Save the River – DT690Q02
Domain context	Scientific problem solving
Ideation process	Evaluate and improve ideas
Item format	Constructed response - human scored
Coding criteria	The coding guide defines conventional themes for all items corresponding to the “evaluate and improve ideas” ideation process. There are three conventional themes for this item: Conventional Theme 1: Test the water using a specific method to determine the presence of chemicals or pollution; Conventional Theme 2: Test the frogs for the presence of chemicals in or on their bodies; Conventional Theme 3: Introduce a control to the experiment so that results from affected samples can be compared to results where the pollution from factories and farms are not a variable (e.g. a control group of frogs unaffected by pollution from the farms and factories, or a control sample of unpolluted water unaffected by the farms and factories).
Proficiency levels	6 (full credit) 4 (partial credit)