Four new mathematics units were released from the main survey of the PISA 2022 assessment; the ten items from these four units are presented in this annex.
Screenshots of the interface used in PISA 2022 are shown to give readers an understanding of how students interacted with the assessment and its items. Interactive versions of all of these units are also available at www.oecd.org/pisa.
This is the first item in the unit Solar System. There is no introduction screen for this unit. For this task, students need to determine which three planets have the average distances, in Astronomical Units (au), between them that are shown in the model. To do this, students need to use the table in the stimulus that gives each planet’s average distance from the Sun, in au. The correct answer, from left to right, is Jupiter, Saturn, Uranus.
To respond to the question, students have to drag-and-drop the planets into the model (see below for an image of the planets placed in the model). There is no introduction or practice screen before this item but instructions for how to respond and change a response are given explicitly in the question stem. A full-credit response was given for correctly placing all three planets, and partial credit was given for correctly placing any two planets. This is a moderately difficult item with both full and partial credit being at Level 3 on the proficiency scale.
Below is an image of what the question stem and response area look like after the student has dragged-and-dropped the planets into their respective locations in the model.
For the second item in this unit, students have to determine approximately how many million kilometres the planet Neptune is from the Sun, a process that requires converting Astronomical Units to millions of kilometres. From the stimulus, students are given the conversion that 1 au is approximately 150 million kilometres, and they can read from the table that Neptune has an average distance from the sun of 30.05 au. To determine Neptune’s approximate distance in million kilometres, students need to multiply 30.05 times 150. This gives a result of 4,507.5, which rounds to 4500 (million km). This is a Level 2 (easier) item for students that only requires employing a process for converting units based on the given information.
This is the first item in the unit Triangular Pattern and there is no introduction screen. For this unit, students are presented with a series of items related to a drawing a person has made of rows using alternating red and blue triangles. The stimulus shows the first four rows of the pattern, and this same image is repeated in the stimulus of all three items in the unit.
For the first item, students are asked to compute the percentage of blue triangles shown in the first four rows of the pattern. There are six blue triangles and 16 total triangles, so the percentage of blue triangles is 37.5% (6 ÷ 16 = 0.375). This is an easy item (Level 1a) and is intended to get students thinking about the pattern by employing a simple algorithm with all information shown.
The second item in the unit builds off the first item by again asking students to compute the percentage of blue triangles, but this time it is based on five rows of the pattern. Since the fifth row is not shown, students have to extend the pattern by one row to determine new values for the number of blue triangles and the total number of triangles. With five rows, the percentage of blue triangles is 40.0% (10 blue triangles ÷ 25 total triangles).
This item is intended to be easy and to get students thinking about extending the pattern beyond what is shown, but not extending the pattern so that it requires generalising. This is a Level 2 item, so it is slightly more difficult than the first item in the unit, possibly because it requires working with a part of the pattern that is not shown but is still an overall easy item for students.
This is the final item in this unit, and it builds off the previous two items to now generalise with the pattern. The task for the students is to evaluate a claim that the percentage of blue triangles in the pattern will always be less than 50% as more rows are added. Students have to select either “Yes” or “No” to indicate if the claim is or is not true, but then they also have to provide an explanation to support their selection. This is a reasoning item that requires students to analyse the pattern to recognise a relationship between the number of red and the number of blue triangles in each row, and then use that relationship to support their selection.
The correct selection is “Yes,” that the claim is true, and an acceptable explanation recognises that the number of red triangles in each row will always be greater than the number of blue triangles in each row. Note that students can phrase their response in terms of either the number of blue triangles being fewer or the number of red triangles being greater, as long as there is some language indicating that this relationship is true for every row. Partial-credit responses to this item generally either focus on just the first row, which contains only a red triangle, or do not clearly communicate that the relationship between the number of each color triangle applies to every row.
This is a human-coded item (the coding rubric is shown below) that is difficult (Level 5) for students to provide a full-credit response. There is partial credit available, but that is still moderately difficult (Level 4) for students. Note that the coding rubric does not contain an exhaustive list of responses at any credit level. However, the sample responses in the rubric are representative of how students typically respond to this item.
Full Credit
Code 2: Selects Yes and provides an acceptable explanation for why there will always be more red (or fewer blue) triangles. [An acceptable explanation must state “in each row” (or use similar wording for that concept).]
He is correct because there is always one more red triangle than blue triangle in each row. [Selection of “Yes” is implied here.]
[Yes] There will always be one less blue triangle in each row.
[Yes] There is one more red triangle than blue in each row. [Benefit of the doubt given for not specifying “always” in the response since it is already stated in the question stem.]
[Yes] Because red triangles are on the outside of each row and inside it alternates red and blue triangles. [Acceptable explanation that establishes there are more red than blue in each row.]
Partial Credit
Code 1: Selects Yes and explanation is partially correct but incomplete.
[Yes] Because the first row has only a red triangle.
[Yes] There are no blue triangles in the first row.
[Yes] There is one more red triangle than blue triangle. [Response does not specify “in each row”. Compare to Code 2, dot point 3.]
[Yes] Because red triangles are on the outside of each row and the blue triangles stay inside. [Explanation is incomplete because the red triangles in the interior are not addressed. Compare to Code 2, dot point 4.]
No Credit
Code 0: Other responses, including selecting Yes but giving an incorrect explanation or without giving an explanation OR selecting No with or without an explanation.
[Yes] red = 62.5% and blue = 37.5%. [Percentage of each colour triangle in the first four rows.]
[Yes].
Code 9: Missing
This is the unit Points and it is another single-item unit with no introduction screen. For this item, students are presented with a newspaper headline about a local basketball team, which notes that the team won every game this season, and that they averaged a 19-point margin of victory this season. The definition of margin of victory is also given in the stimulus in cases students are not familiar with the term. The question asks is if it is possible that the team never actually won a game by 19 points given that the average margin of victory for the season is 19 points. This is an abstract reasoning item that requires students to evaluate a conjecture based on their conceptual understanding of an average (i.e., an arithmetic mean). They have to select either “Yes” or “No” and provide an explanation to support their selection.
The correct selection is “Yes” that it is possible that the team never actually won a game by 19 points, even though 19 is the average margin of victory. Students can respond by recognising that the mean does not have to be a member of the data set, or they can provide an example data set that has a mean of 19 but which does not contain 19 in the data set. Note that for this latter approach, students can also provide a counterexample based on a value other than 19 because it still represents an appropriate line of reasoning in this context. For example, the arithmetic mean of the data set 6, 9, and 15 is 10, even though 10 is not a member of the data set. Partial-credit responses address the idea that some values in the data set must be greater and some values in the data set must be less than the mean, but do not explicitly mention that the mean does not have to be a member of the data set.
This is also a human-coded item (the coding rubric is shown below) that is very difficult for students to provide a full-credit response to (Level 6 on the proficiency scale). There is partial credit available, but that is also difficult (Level 5 on the scale). The abstract nature of this task may have contributed to the difficulty. That is, students do not have numerical values they can manipulate to know what really happened, so they are forced to reason based on their understanding of a concept in order to devise a way to explain this with respect to the context. Note that the coding rubric does not contain an exhaustive list of responses at any credit level. However, the sample responses in the rubric are representative of how students typically respond to this item.
Full Credit
Code 2: Selects Yes and explanation states or shows that the average does not have to be a member of the data set.
It is possible because the average does not actually have to be one of the values in your data set. [Selection of “Yes” is implied here.]
[Yes] If the margins of victory create an average of 19, there doesn't necessarily have to be a 19-point margin of victory in any of them. [Full credit for, “…there doesn't necessarily have to be a 19-point margin of victory in any of them”.]
[Yes] If one difference was 16 points and another was 22 points, then the average difference would be 19 points, but 19 was not one of the differences.
[Yes] The mean of the numbers 2, 4, and 9 is 5 but 5 is not one of the numbers.
Partial Credit
Code 1: Selects Yes and explanation is partially correct but incomplete.
[Yes] It is an average difference, so some games were won by more than 19 points and some games were won by fewer than 19 points. [Incomplete; does not explicitly state that 19 does not need to be one of the values. For a response like this to receive partial credit, winning by both more and by less than 19 points must be explicitly stated in the response.]
No Credit
Code 0: Other responses, including selecting Yes but giving an incorrect explanation or without giving an explanation OR selecting No with or without an explanation.
[No] They need to have won at least one game by 19 points.
[Yes].
[Yes] Because the average is all of their margins of victory for the season added together then divided by the number of games they played that season. [Unacceptable explanation that only describes how to compute a mean.]
[Yes] Because it is just an average. [No reason given for why an average means it is possible they never actually won a game by 19 points.]
[Yes] It is an average difference, so some games were won by more than 19 points. [Unacceptable because winning by less than 19 points was not also explicitly stated in the response.]
Code 9: Missing
This is the introduction to the unit Forested Area, which provides students with some background information about the context of the unit – that the amount of forested area in a country can change over time – and lets them know that they will be using a spreadsheet tool to assist with answering the questions.
After the introduction screen, students come to a practice screen where they must perform several actions to familiarise themselves with the functionality of the spreadsheet. The actions include sorting any column, performing a calculation (adding, subtracting, multiplying, or dividing) with the data in any two columns, and generating the mean of any column. Each action comes with instructions for how to use the tool to complete that action, and each action must be completed before the next action is shown (for convenience, they are all shown in this image). The arrow to advance to the next screen only becomes active once all three actions have been completed. Note that the data that students are using in the practice screen is the same data that is used in the unit.
If students get confused about what to do on this screen and are inactive for a certain amount of time, a pop-up message appears to remind them of the action that they need to perform. If another period of inactivity elapses after the pop-up message appears, then an animation shows how to perform each action. Once all of the animations have run, the students can advance to the next screen.
After the practice screen, students come to an instruction screen, which is just to let them know that instructions for using the spreadsheet are available in each item and can always be accessed by clicking on the bar with the text, “How to Use the Spreadsheet”. Clicking on this bar opens the list of instructions, as shown above. Clicking on the bar again closes the list of instructions.
As with the practice screen, students are not allowed to advance past this screen until they have performed the action (i.e., opened the instructions). Again, if there is a period of inactivity, then a pop-up message reminds students of the action they need to perform. If they still do not perform the action, then after another short period of time, an animation plays. After the animation plays, students can advance to the first item in the unit.
The data used for all items in this unit are the amount of forested area as a percentage of the total land area for 15 countries for the years 2005, 2010, and 2015, and those data are always in columns B, C, and D, respectively. Columns E, F, and G are always empty when the students first navigate to each item, and the default ordering of the countries is alphabetical, based on how the country names are translated in each language. Note that in the image above, the data has been manipulated already to correspond to the description of the solution that follows.
The first item in the unit asks the students to identify, in terms of percentage points, the three countries that between 2005 and 2015 had: the greatest gain in its percentage of forested area, no overall change in its percentage of forested area, and the greatest loss in its percentage of forested area. Responses are entered in each row of the table via drop-down menus that contain the name of all 15 countries.
One possible solution method, which is reflected in the image above, is to use the spreadsheet to perform the following calculation: “Column D subtract Column B,” which subtracts the percentage of forested area in 2005 from the percentage of forested area in 2015 for each country. The results of that operation are shown in column E. Next, a student may choose to sort the data in Column E to make it easier to identify each country.
The country with the greatest gain is the country with the largest positive result, which is Greece at 2.34 percentage points; the country with no overall change is the country with a difference of 0.00, which is Armenia; and the country with the greatest loss is the country with the smallest negative result, which is Panama at -2.22 percentage points.
A full-credit response is correctly identifying all three countries and scaled at Level 5, meaning it was a difficult task for students. Partial credit was given for correctly identifying any two countries, and that was still a moderately difficult task that scaled at Level 4, which is not surprising given that partial credit still requires doing the same work as a full-credit response. That is, to identify any two or three countries correctly, students need to determine what calculation(s) to perform, how to use the spreadsheet to perform them, and lastly interpret the results with respect to the context.
Also, depending on the order that the student performs the calculation, identifying the countries could be more difficult. For example, if the student calculates “Column B subtract Column D” (instead of “Column D subtract Column B”), then the sign of each result that appears in column E will be reversed (e.g., Greece = -2.34 and Panama = +2.22). However, based on these data, the percentage of forested area for Greece actually increased for each year shown, and the percentage of forested area for Panama actually decreased for each year shown.
In the second item in this unit, students are told to consider the data in terms of two time periods, 2005 to 2010 and 2010 to 2015, and then asked to identify the statement that correctly describes the mean change in the percentage of forested area for each time period.
One possible solution method is to have the spreadsheet compute the mean of Columns B, C, and D and just notice that it decreased from 2005 to 2010 (from 33.33 down to 33.18) and that it also decreased from 2010 to 2015 (from 33.18 down to 33.05). Since the mean change decreased in each time period, the correct answer is that “The mean change was negative for both time periods.”
Students may also choose to perform a sequence of operations, such as:
“Column C subtract Column B” (the results of that operation are shown in Column E), which represents the change in the percentage of forested area for the time period 2005 to 2010.
“Column D subtract Column C” (the results of that operation are shown in Column F), which represents the change in the percentage of forested area for the time period 2010 to 2015.
Compute the mean of Columns E and F.
This is a difficult item that scaled at Level 5 on the proficiency scale. Students again have to devise a strategy for using the spreadsheet but this time there is more flexibility in how the spreadsheet can be used before having to interpret the results. Possibly contributing to the difficulty of this item is having to correctly interpret “change” in the context of the problem, when the results can be either positive or negative depending on what operations the student performs, and the order in which they perform them.
In the third item in this unit, students are again told to consider the data in terms of the two time periods, 2005 to 2010 and 2010 to 2015, but this time they are asked to identify the two countries that had biggest change in their percentage of forested area from one time period to the other time period. Answers are given by selecting the country name from a drop-down menu. The order that the countries are given in the response does not matter.
One possible solution method, which is reflected in the image above, is to perform the following sequence of operations using the spreadsheet (Note that these two calculations are the same two calculations that could also be performed in the second item in the unit):
“Column C subtract Column B” (the results of that operation are shown in Column E), which represents the change in the percentage of forested area for the time period 2005 to 2010.
“Column D subtract Column C” (the results of that operation are shown in Column F), which represents the change in the percentage of forested area for the time period 2010 to 2015.
Once the students have calculated the change in the percent of forested area for each time period, they need to compute the change between the two time periods by performing a calculation such as “Column E subtract Column F” (the results of that operation are shown in Column G). Students may also find it helpful to sort the results in Column G.
The two countries with the biggest change between time periods are India (0.40 percentage points) and Colombia (-1.29 percentage points). Full credit is given for correctly identifying both countries, and partial credit is given for correctly identifying one country.
This is a very difficult item that scaled at Level 6 on the proficiency scale. Partial credit was also difficult at Level 5, and similar to the first item in the unit, requires doing the same work that is needed for a full-credit response. Students again have to devise a strategy for using the spreadsheet, which this time requires performing multiple operations, before being able to evaluate the results with respect to the context. Possibly contributing to the difficulty of this item is recognising that “biggest change” in this context does not just mean an increase, and in fact, one of the correct answers is the country with the biggest decrease in its percentage of forested area between time periods. However, unlike previous items in this unit, the correct countries can still be identified even if the signs of the results are reversed (due to the order that operations are performed) because students are looking for change in term of the absolute value, and not interpreting the results specifically as increases or decreases.
This the final item in this unit. Students are presented with a claim that South Korea has more forested area than the other 15 countries on the list for the years shown, and they have to determine if the claim is supported by the data in the spreadsheet. As with some other human-coded items, students need to select either “Yes” or “No,” and then provide an explanation to support their selection. Unlike the previous items in the unit, this item does not actually require manipulating the data in the spreadsheet to answer; however, all the functionality of the spreadsheet is still available.
Even though South Korea is the country in this list with the highest percentage of forested area for each of the three years, the correct answer is “No,” the claim is not supported by the data in the spreadsheet. It is not possible to conclude anything about the actual amount of forested area in these countries from the data shown because the data shown are only the percentage of forested area. The total land area of each country is not also given in the spreadsheet, and this “missing” information is necessary for determining the actual amount of forested area in each country. That is, because the data shown are percentages of different quantities (i.e., different land areas, which are not included in the spreadsheet) they do not support the claim.
This is a reasoning item that requires students to evaluate a claim by understanding the limits of what can be inferred from the available data. That is, students do not have to determine if the specific claim about South Korea is actually true or not; they have to determine if the claim is supported by the available data. It is a very difficult item that scaled at Level 6. There is no partial credit response for this item. The coding rubric is shown below. Note that the coding rubric does not contain an exhaustive list of responses. However, the sample responses in the rubric are representative of how students typically respond to this item.
Full Credit
Code 1: Selects No and explains that the spreadsheet only shows the percentage of forested area OR that the spreadsheet does not show the total land area for each country OR that the areas of the countries are different.
[No] This is not true because the spreadsheet only shows the values as a percentage.
Her claim is not supported by the data in the spreadsheet because we do not know the total area for each of the countries listed. [Selection of “No” is implied here.]
[No] Because the total area of each country is different.
[No] Each country does not have the same area.
No Credit
Code 0: Other responses, including selecting No but giving an incorrect explanation or without giving an explanation OR selecting Yes with or without an explanation.
[No].
[No] Because it is different.
[Yes] South Korea has the greatest amount for each year shown.
Code 9: Missing