PISA 2018 Results (Volume II)

Annex A1. The construction of proficiency scales and of indices from the student context questionnaire

Explanation of the indices

This section explains the indices derived from the PISA 2018 parent, student, school, teacher and educational career questionnaires used in this volume.

Several PISA measures reflect indices that summarise responses from students, their parents, teachers or school representatives (typically principals) to a series of related questions. The questions were selected from a larger pool on the basis of theoretical considerations and previous research. The PISA 2018 Assessment and Analytical Framework (OECD, 2019[1]) provides an in-depth description of this conceptual framework. Item response theory modelling was used to confirm the theoretically expected behaviour of the indices and to validate their comparability across countries. For this purpose a joint model across all countries was estimated. Item fit (RMSD) was evaluated separately for each item and each group (county by language). This procedure is in line with the PISA 2015 scaling approach. For a detailed description of other PISA indices and details on the methods, see the PISA 2015 Technical Report (OECD; 2017) and the PISA 2018 Technical Report (OECD, forthcoming[2])

There are three types of indices: simple indices, new scale indices and trend scale indices.

Simple indices are the variables that are constructed through the arithmetic transformation or recoding of one or more items in exactly the same way across assessments. Here, item responses are used to calculate meaningful variables, such as the recoding of the four-digit ISCO-08 codes into “Highest parents’ socio-economic index (HISEI)” or teacher-student ratio based on information from the school questionnaire.

Scale indices are the variables constructed through the scaling of multiple items. Unless otherwise indicated, the index was scaled using a two-parameter item-response model (a generalised partial credit model was used in the case of items with more than two categories) and values of the index correspond to Warm likelihood estimates (WLE) (Warm, 1989[3]) For details on how each scale index was constructed, see the PISA 2018 Technical Report (OECD, forthcoming[2]). In general, the scaling was done in two stages:

1. The item parameters were estimated based on all students from equally-weighted countries and economies; only cases with a minimum number of three valid responses to items that are part of the index were included. In the case of trend indices, a common calibration linking procedure was used: countries/economies that participated in both PISA 2009 and PISA 2018 contributed both samples to the calibration of item parameters; each cycle and, within each cycle, each country/economy contributed equally to the estimation.1

2. For new scale indices, the Warm likelihood estimates were then standardised so that the mean of the index value for the OECD student population was zero and the standard deviation was one (countries were given equal weight in the standardisation process).

Sequential codes were assigned to the different response categories of the questions in the sequence in which the latter appeared in the student, school or parent questionnaires. Where indicated in this section, these codes were inverted for the purpose of constructing indices or scales. Negative values for an index do not necessarily imply that students responded negatively to the underlying questions. A negative value merely indicates that the respondents answered less positively than all respondents did on average across OECD countries. Likewise, a positive value on an index indicates that the respondents answered more favourably, or more positively, on average, than respondents in OECD countries did. Terms enclosed in brackets < > in the following descriptions were replaced in the national versions of the student, school and parent questionnaires by the appropriate national equivalent. For example, the term <qualification at ISCED level 5A> was translated in the United States into “Bachelor’s degree, post-graduate certificate program, Master’s degree program or first professional degree program”. Similarly the term <classes in the language of assessment> in Luxembourg was translated into “German classes” or “French classes”, depending on whether students received the German or French version of the assessment instruments.

In addition to simple and scaled indices described in this annex, there are a number of variables from the questionnaires that were used in this volume and correspond to single items not used to construct indices. These non-recoded variables have prefix of “ST” for the questionnaire items in the student questionnaire and “SC” for the items in the school questionnaire. All the context questionnaires, and the PISA international database, including all variables, are available through www.oecd.org/pisa.

Student-level simple indices

Parents’ level of education

Students’ responses to questions ST005, ST006, ST007 and ST008 regarding their parents’ education were classified using ISCED 1997 (OECD, 1999[4]). Indices on parental education were constructed by recoding educational qualifications into the following categories: (0) None, (1) <ISCED level 1> (primary education), (2) <ISCED level 2> (lower secondary), (3) <ISCED level 3B or 3C> (vocational/pre-vocational upper secondary), (4) <ISCED level 3A> (general upper secondary) and/or <ISCED level 4> (non-tertiary post-secondary), (5) <ISCED level 5B> (vocational tertiary) and (6) <ISCED level 5A> and/or <ISCED level 6> (theoretically oriented tertiary and post-graduate). Indices with these categories were provided for a student’s mother (MISCED) and father (FISCED). In addition, the index of highest education level of parents (HISCED) corresponded to the higher ISCED level of either parent. The index of highest education level of parents was also recoded into estimated number of years of schooling (PARED). In PISA 2018, to avoid issues related to the misreporting of parental education by students, students’ answers about post-secondary qualifications were considered only for those students who reported their parents’ highest level of schooling to be at least lower secondary education. The conversion from ISCED levels to year of education is common to all countries. This international conversion was determined by using the modal years of education across countries for each ISCED level. The correspondence is available in the PISA 2018 Technical Report (OECD, forthcoming[5]).

Parents’ highest occupational status

Occupational data for both the student’s father and the student’s mother were obtained from responses to open-ended questions. The responses were coded to four-digit ISCO codes (ILO, 2007) and then mapped to the international socio-economic index of occupational status (ISEI) (Ganzeboom and Treiman, 2003[6]). In PISA 2018, as in PISA 2015, the new ISCO and ISEI in their 2008 version were used rather than the 1988 versions that had been applied in the previous four cycles (Ganzeboom, 2010[7]). Three indices were calculated based on this information: father’s occupational status (BFMJ2); mother’s occupational status (BMMJ1); and the highest occupational status of parents (HISEI), which corresponds to the higher ISEI score of either parent or to the only available parent’s ISEI score. For all three indices, higher ISEI scores indicate higher levels of occupational status. In PISA 2018, in order to reduce missing values, an ISEI value of 17 (equivalent to the ISEI value for ISCO code 9000, corresponding to the major group “Elementary Occupations”) was attributed to pseudo-ISCO codes 9701, 9702 and 9703 (“Doing housework, bringing up children”, “Learning, studying”, “Retired, pensioner, on unemployment benefits”).

Immigrant background

Information on the country of birth of the students and their parents was also collected. Included in the database are three country-specific variables relating to the country of birth of the student, mother and father (ST019). The variables are binary and indicate whether the student, mother and father were born in the country of assessment or elsewhere. The index on immigrant background (IMMIG) is calculated from these variables, and has the following categories: (1) native students (those students who had at least one parent born in the country); (2) second-generation students (those born in the country of assessment but whose parent[s] were born in another country); and (3) first-generation students (those students born outside the country of assessment and whose parents were also born in another country). Students with missing responses for either the student or for both parents were given missing values for this variable.

Language spoken at home

Students also indicated what language they usually spoke at home, and the database includes a variable (LANGN) containing country-specific code for each language. In addition, an internationally comparable variable (ST022Q01TA) was derived from this information and has the following categories: (1) language at home is same as the language of assessment for that student; (2) language at home is another language.2

Doing homework

In a subset of 32 countries and economies that participated in PISA 2018, students were asked how long they studied in the morning before going to school (EC158) and after school (EC159) on the most recent day prior to the PISA test (response choices included “I did not study” or “I do not remember”). Students’ answers were averaged to measure the percentage of students who responded that they “did not study at all at home on the most recent day prior to the PISA test”, “studied at home but less than one hour”, and “studied at home more than one hour”.

Time spent reading for enjoyment

PISA 2018 asked students (ST175): “about how much time do you usually spend reading for enjoyment?”. The answers (“more than 30 minutes to less than 60 minutes a day”; “more than 30 minutes to less than 60 minutes a day”; “1 to 2 hours a day”; “more than 2 hours a day”) were aggregated, against “I do not read for enjoyment” and “30 minutes or less a day” to create an index that corresponds to more than 30 minutes of reading a day.

Career expectations

In PISA 2018, students were asked to answer a question (ST114) about “what kind of job [they] expect to have when [they] are about 30 years old”. Answers to this open-ended question were coded to four-digit ISCO codes (ILO, 2007), in variable OCOD3.

This variable was used to derive several indices related to career expectations.

The proportion of students who had no clear idea about their future job was computed excluding students who did not answer the question or gave an invalid answer, such as a smiley face (9998). It corresponds to students who reported that “they do not know” (9704) or gave a vague answer such as “a good job”, “a quiet job”, “a well-paid job”, “an office job” (9705).

The definition of high-skilled, medium and low-skilled career expectations is based on the one-digit ISCO-08 classification of occupations. High-skilled occupations correspond to ISCO codes 1 to 3 (managers; professionals; technicians and associate professionals), medium-skilled to codes 4 to 8 (clerical support workers; service and sales workers; skilled agricultural, forestry and fishery workers; craft and related trades workers; plant and machine operators and assemblers) and low-skilled to code 9 (elementary occupations).

Science-related career expectations are defined as those career expectations whose realisation requires further engagement with the study of science beyond compulsory education, typically in formal tertiary education settings. The classification of careers into science-related and non-science-related is based on the four-digit ISCO-08 classification of occupations.

Only professionals (major ISCO group 2) and technicians/associate professionals (major ISCO group 3) were considered to fit the definition of science-related career expectations. In a broad sense, several managerial occupations (major ISCO group 1) are clearly science-related; these include research and development managers, hospital managers, construction managers, and other occupations classified under production and specialised services managers (submajor group 13). However, when science-related experience and training is an important requirement of a managerial occupation, these were not considered to be entry-level jobs, and 15-year-old students with science-related career aspirations would not expect to be in such a position by age 30.

Several skilled agriculture, forestry and fishery workers (major ISCO group 6) could also be considered to work in science-related occupations. The United States (O*NET OnLine, 2019[8]) classification of science, technology, engineering and mathematics (STEM) occupations indeed include these occupations. These, however, do not typically require formal science-related training or study after compulsory education. Thus, only major occupation groups that require ISCO skill levels 3 and 4 were included amongst science-related occupational expectations.

Amongst professionals and technicians/associate professionals, the boundary between science-related and non-science-related occupations is sometimes blurred, and different classifications draw different lines.

The classification used in this report includes four groups of jobs:

1. Science and engineering professionals: All science and engineering professionals (sub-major group 21), except product and garment designers (2163), graphic and multimedia designers (2166).

2. Health professionals: All health professionals in sub-major group 22 (e.g. doctors, nurses, veterinarians), with the exception of traditional and complementary medicine professionals (minor group 223).

3. ICT professionals: All information and communications technology professionals (sub-major group 25).

4. Science technicians and associate professionals, including:

• physical and engineering science technicians (minor group 311)

• life science technicians and related associate professionals (minor group 314)

• air traffic safety electronic technicians (3155)

• medical and pharmaceutical technicians (minor group 321), except medical and dental prosthetic technicians (3214)

• telecommunications engineering technicians (3522).

Education expectations

Students’ responses to question ST225 regarding the level of education they expect to complete were used for identifying those students who expected to complete tertiary education, defined using International Standardised Classification of Education 1997 <ISCED level 5A> and/or <ISCED level 6> (theoretically oriented tertiary and post-graduate).

This indicator was used to measure the proportion of students with ambitious and realistic expectations, defined as the proportion of students who achieved Level 2 in the three core subjects and Level 4 in at least one of them (defined as high performers) and who expect to complete tertiary education. It was also used to estimate the proportion of high performers who do not expect to complete tertiary education while

Further education and career

Learning about future study and career

In a subset of 32 countries and economies that participated in PISA 2018, students were asked in an optional Education Career Questionnaire whether they had done any of the following to find out about future study or types of work (EC150): did an internship; attended job shadowing or work-site visits; visited a job fair; spoke to a career advisor at school; spoke to an advisor outside of school; completed a questionnaire to find out about [his/her] interests and abilities; researched the Internet for information about careers; went to an organised tour in a higher education institution; or researched the Internet about higher education programmes.

Skills related to future study and career

In a subset of 32 countries and economies that participated in PISA 2018, students were asked in an optional Education Career Questionnaire whether they had acquired at school, outside of school or not acquired the following skills (EC151): finding information about jobs they are interested in; searching for a job; writing a résumé or a summary of their qualifications; preparing for a job interview; or finding information about financing higher education (e.g. student loans or grants). As some students may have acquired these skills both at and outside of school, the sum of the proportion of students who had not acquired one of these skills and the proportions of students who had acquired some of these skills at school and outside of school may be higher than 100%.

Factors that influence students’ career expectations

In a subset of 32 countries and economies that participated in PISA 2018, students were asked in an optional Education Career Questionnaire how important (“not important”, “somewhat important”, “important”, “very important”) are some factors in the decisions they make about their future occupation (EC153). Answers to this question were used to measure the proportion of students who considered that their school grades, the school subjects they are good at, financial support for education or training, education or training options for the occupation they want to pursue, and employment opportunities for the occupation they want to pursue are important or very important in the decisions they make about their future occupation.

Student-level scale indices

Attitudes towards competition

The index of attitudes towards competition (COMPETE) was constructed using students’ responses to a new question (ST181) over the extent they “strongly agreed”, “agreed”, “disagreed” or “strongly disagreed” with the following statements: “I enjoy working in situations involving competition with others”; “It is important for me to perform better than other people on a task”; and “I try harder when I’m in competition with other people”. Positive values on this scale mean that students expressed more favourable attitudes towards competition than did the average student across OECD countries.

Fear of failure

Students in PISA 2018 were asked to report the extent to which they agree (“strongly disagree”, “disagree”, “agree”, “strongly agree”) with the following statements (ST183): “When I am failing, I worry about what others think of me”; “When I am failing, I am afraid that I might not have enough talent”; and “When I am failing, this makes me doubt my plans for the future”. These statements were combined to create the index of fear of failure (GFOFAIL). Positive values in this index mean that the student expressed a greater fear of failure than did the average student across OECD countries.

Learning goals

Students in PISA 2018 were asked (ST208) to respond how true (“not at all true of me”, “slightly true of me”, “moderately true of me”, “very true of me”, “extremely true of me”) the following statements are for them: “My goal is to learn as much as possible”; “My goal is to completely master the material presented in my classes”; and “My goal is to understand the content of my classes as thoroughly as possible”. These statements were combined to construct the index of learning goals (MASTGOAL). Positive values in the index indicate more ambitious learning goals than the average student across OECD countries.

Motivation to master tasks

PISA 2018 asked students (ST182) to report the extent to which they agree or disagree (“strongly disagree”, “disagree”, “agree”, “strongly agree”) with the following statements: “I find satisfaction in working as hard as I can”; “Once I start a task, I persist until it is finished”; “Part of the enjoyment I get from doing things is when I improve on my past performance”; and “If I am not good at something, I would rather keep struggling to master it than move on to something I may be good at”. The first three statements were combined to create the index of motivation to master tasks (WORKMAST). Positive values in the index indicate greater motivation than the average student across OECD countries.

Meaning in life

PISA 2018 asked students (ST185) to report the extent to which they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “My life has clear meaning or purpose”; “I have discovered a satisfactory meaning in life”; and “I have a clear sense of what gives meaning to my life”. These three statements were combined to form the index of meaning in life (EUDMO). Positive values in the index indicate greater meaning in life than the average student across OECD countries.

Perception of competence in reading and perceived difficulty in reading

PISA 2018 included a question (ST161) with six items asking students about their competence in reading and whether they encountered difficulties in learning how to read. The four response categories were “not at all”, “very little”, “to some extent”, and “a lot”. The index of perception of competence in reading (SCREADCOMP) was derived from the following three statements: “I am a good reader”; “I am able to understand difficult texts”; and “I read fluently”. The index of perceived difficulty in reading (SCREADDIFF) was derived from the next three statements: “I have always had difficulty with reading”; “I have to read a text several times before completely understanding it”; and “I find it difficult to answer questions about a text”. Positive values in these indices mean that the student indicated greater perception of competence/difficulty than the OECD average.

Positive feelings

PISA 2018 asked students (ST186) to report how frequently (“never”, “rarely”, “sometimes”, “always”) they feel happy, lively, proud, joyful, cheerful, scared, miserable, afraid and sad. Three of these positive feelings – happy, joyful and cheerful – were combined to create an index of positive feelings (SWBP). Positive values in this index mean that the student reported more positive feelings than the average student across OECD countries. An index of negative feelings was not created because of the low internal consistency of the index across PISA-participating countries.

Self-efficacy

PISA 2018 asked (ST188) students to report the extent to which they agree (“strongly disagree”, “disagree, “agree”, “strongly agree”) with the following statements about themselves: “I usually manage one way or another”; “I feel proud that I have accomplished things”; “I feel that I can handle many things at a time”; “My belief in myself gets me through hard times”; and “When I’m in a difficult situation, I can usually find my way out of it”. These statements were combined to create the index of self-efficacy (RESILIENCE). Positive values in this index mean that the student reported higher self-efficacy than did the average student across OECD countries.

Student competition

PISA 2018 asked (ST205) students how true (“not at all true”, “slightly true”, “very true”, “extremely true”) the following statements about their school are: “Students seem to value competition”; “It seems that students are competing with each other”; “Students seem to share the feeling that competing with each other is important”; and “Students feel that they are being compared with others”. The first three statements were combined to create the index of student competition (PERCOMP). Positive values in this index mean that students perceived their peers to compete with each other to a greater extent than did the average student across OECD countries.

Student co-operation

PISA 2018 asked (ST206) students how true (“not at all true”, “slightly true”, “very true”, “extremely true”) the following statements about their school are: “Students seem to value co-operation”; “It seems that students are co-operating with each other”; “Students seem to share the feeling that co-operating with each other is important”; and “Students feel that they are encouraged to cooperate with others”. The first three statements were combined to create the index of student co-operation (PERCOOP). Positive values in this index mean that students perceived their peers to co-operate to a greater extent than did the average student across OECD countries.

Teacher enthusiasm

PISA 2018 asked (ST213) students whether they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements about the two language-of-instruction lessons they attended prior to sitting the PISA test: “It was clear to me that the teacher liked teaching us”; “The enthusiasm of the teacher inspired me”; “It was clear that the teacher likes to deal with the topic of the lesson”; and “The teacher showed enjoyment in teaching”. These statements were combined to create the index of teacher enthusiasm (TEACHINT). Positive values in this index mean that students perceived their language-of-instruction teachers to be more enthusiastic than did the average student across OECD countries.

ICT use outside of school for leisure

In PISA 2018 an optional ICT familiarity questionnaire was distributed in 52 countries and economies that participated. It included questions about how teenagers use digital devices (IC008). Specifically, 15-year-old students were asked to report how often (“never or hardly ever”, “once or twice a month”, “once or twice a week”, “every day”) they use digital devices for the following activities outside of school: playing one-player games; playing collaborative online games; using e-mail; chatting on line; participating in social networks (e.g. <Facebook>, <MySpace>); playing online games via social networks; browsing the Internet for fun (such as watching videos, e.g.<YouTube™>); reading news on the Internet (e.g. current affairs); obtaining practical information from the Internet (e.g. locations, dates of events); downloading music, films, games or software from the Internet; uploading [your] own created content for sharing (e.g. music, poetry, videos, computer programs); or downloading new apps on a mobile device. Students’ answers to these questions were summarised in an index measuring the frequency of ICT use outside of school for leisure (ENTUSE). The index was standardised to have a mean of 0 and a standard deviation of 1 across OECD countries.

Indices included in earlier assessments

Disciplinary climate

The index of disciplinary climate (DISCLIMA) was constructed using students’ responses to a trend question about how often (“every lesson”, “most lessons”, “some lessons”, “never or hardly ever”) the following happened in their language-of-instruction lessons (ST097): ”Students don’t listen to what the teacher says”; “There is noise and disorder”; “The teacher has to wait a long time for students to quiet down”; “Students cannot work well”; and “Students don’t start working for a long time after the lesson begins”. Positive values on this scale mean that the student enjoyed a better disciplinary climate in language-of-instruction lessons than the average student across OECD countries.

Enjoyment of reading

The index of enjoyment of reading (JOYREAD) was constructed based on a trend question (ST160) from PISA 2009 (ID in 2009: ST24) asking students whether they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “I read only if I have to”; “Reading is one of my favourite hobbies”; “I like talking about books with other people”; “For me, reading is a waste of time”; and “I read only to get information that I need”. Positive values on this scale mean that the student enjoyed reading to a greater extent than the average student across OECD countries.

Parents’ emotional support

The index of parents’ emotional support (EMOSUPS) was constructed based on a trend question (ST123) asking students whether they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “My parents support my educational efforts and achievements”; “My parents support me when I am facing difficulties at school”; and “My parents encourage me to be confident”. Positive values on this scale mean that students perceived greater levels of emotional support from their parents than did the average student across OECD countries.

Sense of belonging

The index of sense of belonging (BELONG) was constructed using students’ responses to a trend question about their sense of belonging to school. Students were asked whether they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements (ST034): “I feel like an outsider (or left out of things) at school”; “I make friends easily at school”; “I feel like I belong at school”; “I feel awkward and out of place in my school”; “Other students seem to like me”; and “I feel lonely at school”. Three of these items were reversed-coded so that positive values on this scale mean that students reported a greater sense of belonging at school than did the average student across OECD countries.

Teacher-directed instruction

The index of teacher-directed instruction (DIRINS) was constructed from students’ reports on how often (“never or almost never”, “some lessons”, “many lessons”, “every lesson or almost every lesson”) the following happened in their language-of-instruction lessons (ST102): “The teacher sets clear goals for our learning”; “The teacher asks questions to check whether we have understood what was taught”; “At the beginning of a lesson, the teacher presents a short summary of the previous lesson”; and “The teacher tells us what we have to learn”. Positive values on this scale mean that students perceived their teachers to use teacher-directed practices more frequently than did the average student across OECD countries.

Teacher feedback

The index of teacher feedback (PERFEED) was constructed using students’ responses to a trend question (ST104) about how often (“never or hardly ever”, “some lessons”, “most lessons”, “every lesson”) the following things happen in their language-of-instruction lessons: “The teacher gives me feedback on my strengths in this subject”; “The teacher tells me in which areas I can still improve”; and “The teacher tells me how I can improve my performance”. Positive values on this scale mean that students perceived their teachers to provide feedback more frequently than did the average student across OECD countries.

Teachers’ stimulation of reading engagement

The index of teachers’ stimulation of reading engagement (STIMREAD) was constructed based on a trend question (ST152) from PISA 2009 (ID in 2009: ST37) asking students how often (“never or hardly ever”, “some lessons”, “most lessons”, “every lesson”) the following occur in their language-of-instruction lessons: “The teacher encourages students to express their opinion about a text”; “The teacher helps students relate the stories they read to their lives”; “The teacher shows students how the information in texts builds on what they already know”; and “The teacher poses questions that motivate students to participate actively”. Positive values on this scale mean that the students perceived their teacher to provide greater stimulation than did the average student across OECD countries.

Teacher support

The index of teacher support (TEACHSUP) was constructed using students’ responses to a trend question (ST100) about how often (“every lesson”, “most lessons”, “some lessons”, “never or hardly ever”) the following things happen in their language-of-instruction lessons: “The teacher shows an interest in every student’s learning”; “The teacher gives extra help when students need it”; “The teacher helps students with their learning”; and “The teacher continues teaching until the students understand”. Positive values on this scale mean that students perceived their teacher to be more supportive than did the average student across OECD countries.

Value of school

The index of value of school (ATTLNACT) was constructed based on a trend question (ST036) asking students whether they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “Trying hard at school will help me get a good job“; “Trying hard at school will help me get into a good <college>”; and “Trying hard at school is important”. Positive values on this scale mean that the student valued schooling to a greater extent than the average student across OECD countries.

Scaling of indices related to the PISA index of economic social and cultural status

The PISA index of economic, social and cultural status (ESCS) was derived, as in previous cycles, from three variables related to family background: parents’ highest level of education (PARED), parents’ highest occupational status (HISEI), and home possessions (HOMEPOS), including books in the home. PARED and HISEI are simple indices, described above. HOMEPOS is a proxy measure for family wealth.

Household possessions

In PISA 2018, students reported the availability of 16 household items at home (ST011), including three country-specific household items that were seen as appropriate measures of family wealth within the country’s context. In addition, students reported the amount of possessions and books at home (ST012, ST013). HOMEPOS is a summary index of all household and possession items (ST011, ST012 and ST013).

Computation of ESCS

For the purpose of computing the PISA index of economic, social and cultural status (ESCS), values for students with missing PARED, HISEI or HOMEPOS were imputed with predicted values plus a random component based on a regression on the other two variables. If there were missing data on more than one of the three variables, ESCS was not computed and a missing value was assigned for ESCS.

In previous cycles, the PISA index of economic, social and cultural status was derived from a principal component analysis of standardised variables (each variable has an OECD mean of zero and a standard deviation of one), taking the factor scores for the first principal component as measures of the PISA index of economic, social and cultural status. In PISA 2018, ESCS is computed by attributing equal weight to the three standardised components. As in PISA 2015, the three components were standardised across all countries and economies (both OECD and partner countries/economies), with each country/economy contributing equally (in cycles prior to 2015, the standardisation and principal component analysis was based on OECD countries only). As in every previous cycle, the final ESCS variable was transformed, with 0 the score of an average OECD student and 1 the standard deviation across equally weighted OECD countries.

School-level simple indices

School size

The PISA 2009 index of school size (SCHLSIZE) contains the total enrolment at school based on the enrolment data provided by the school principal, summing the number of girls and boys at a school.

School type

Schools are classified as either public or private according to whether a private entity or a public agency has the ultimate power to make decisions concerning its affairs. As in previous PISA surveys, the index of school type (SCHLTYPE) has three categories: (1) public schools managed directly or indirectly by a public education authority, government agency or governing board appointed by government or elected by public franchise; (2) government-dependent private schools, managed directly or indirectly by a non-government organisation (e.g. a church, trade union, business or other private institution), which receive more than 50% of their total funding in a typical school year from government agencies (including departments, local, regional, state and national agencies); and (3) government-independent private schools, controlled by a non-government organisation, which receive less than 50% of their core funding from government agencies.

Socio-economic profile of the schools

Advantaged and disadvantaged schools are defined in terms of the socio-economic profile of schools. All schools in each PISA-participating education system are ranked according to their average PISA index of economic, social and cultural status (ESCS) and then divided into four groups with approximately an equal number of students (quarters). Schools in the bottom quarter are referred to as “socio-economically disadvantaged schools”; and schools in the top quarter are referred to as “socio-economically advantaged schools”.

Quantity and qualifications of teaching staff at school

Principals were asked to report the number of full-time and part-time teachers at school (question SC018). Principals were also asked the number of full-time and part-time teachers who are fully certified by the appropriate authority, of those who have an ISCED Level 5A master’s degree qualification and of those who have an ISCED Level 6 qualification (those levels correspond to the International Standard Classification of Education 1997).

The number of part-time teachers was weighted by 0.5 and the number of full-time teachers was weighted by 1.0. The number of teachers who have at least a master’s degree was computed as the sum of the numbers of teachers with ISCED Level 5A or Level 6.

Principals were also asked to report the percentage of teaching staff in their school who has attended a programme of professional development in the previous three months (SC025), defined as a formal programme designed to enhance teaching skills or pedagogical practices. It may or may not lead to a recognised qualification. The programme must have lasted for at least one day and was focused on teaching and education.

School enrolment practices

As in previous surveys, school principals were asked about admittance policies at their school (SC012). Amongst these policies, principals were asked how much consideration was given to the following factors when students are admitted to the school, based on a scale with the categories “never”, “sometimes”, and “always”: students’ academic record (including placement tests) and residence in a particular area.

Career guidance at school

PISA 2018 asked school principals who, at their school, is responsible for career guidance for students in the national modal grade for 15-year-olds (question SC161). This indicator was used to measure the proportion of students in schools where career guidance is provided by a specialised counsellor when the principal reported that there is “one or more specific career guidance counsellors employed at school” or that there is “one or more specific career guidance counsellor[s] who regularly visit the school”. The indicator was also used to measure the proportion of students in schools that do not provide career guidance by a specialised counsellor, but where either “all teachers share the responsibility for career guidance” or “specific teachers have the main responsibility for career guidance”. It also measures the proportion of students in schools where career guidance is not available.

In schools where some career guidance is provided, principals were asked whether the career guidance is sought voluntarily by students or is formally scheduled into students’ time at school (question SC162).

School-level scale indices

Indices included in earlier assessments

School resources: Shortage of educational material and staff

As in PISA 2015 and 2012, PISA 2018 included an eight-item question about school resources, measuring school principals’ perceptions of potential factors hindering instruction at school (“Is your school’s capacity to provide instruction hindered by any of the following issues?”). The four response categories were “not at all”, “very little”, “to some extent”, and “a lot”. A similar question was used in previous cycles, but items were reduced and reworded for 2012 focusing on two derived variables. The index of staff shortage (STAFFSHORT) was derived from the four items: a lack of teaching staff; inadequate or poorly qualified teaching staff; a lack of assisting staff; inadequate or poorly qualified assisting staff. The index of shortage of educational material (EDUSHORT) was scaled using the following four items: a lack of educational material (e.g. textbooks, IT equipment, library or laboratory material); inadequate or poor quality educational material (e.g. textbooks, IT equipment, library or laboratory material); a lack of physical infrastructure (e.g. building, grounds, heating/cooling, lighting and acoustic systems); inadequate or poor quality physical infrastructure (e.g. building, grounds, heating/cooling, lighting and acoustic systems). Positive values in these indices mean that principals viewed the amount and/or quality of resources in their schools as an obstacle to providing instruction to a greater extent than the OECD average.

Teacher behaviour hindering learning

The index of teacher behaviour hindering learning (TEACHBEHA) was constructed using school principals’ responses to a trend question (SC061) about the extent to which (“not at all”, “very little”, “to some extent”, “a lot”) they think that student learning in their schools is hindered by such factors as “Teachers not meeting individual students’ needs”; “Teacher absenteeism”; “School staff resisting change”; “Teachers being too strict with students”; and “Teachers not being well-prepared for classes”. Positive values reflect principals’ perceptions that these teacher-related behaviours hinder learning to a greater extent; negative values indicate that principals believed that these teacher-related behaviours hinder learning to a lesser extent, compared to the OECD average. Answers to this question were also used to measure the proportion of students in schools where instruction is hindered at least to some extent by teacher absenteeism, according to principals’ reports.

Parent-level scale indices

Indices included in earlier assessments

Parents’ perception of school quality

The index of parents’ perceived school quality (PQSCHOOL) was constructed using parents’ responses to the trend question (PA007) about the extent to which they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “Most of my child’s school teachers seem competent and dedicated”; “Standards of achievement are high in my child’s school”; “I am happy with the content taught and the instructional methods used in my child’s school”; “I am satisfied with the disciplinary atmosphere in my child’s school”; “My child’s progress is carefully monitored by the school”; “My child’s school provides regular and useful information on my child’s progress”; and “My child’s school does a good job in educating students”. Positive values reflect that parents perceived their child’s school to be of higher quality, negative values indicate that parents perceived their child’s school to be of lower quality than the OECD average parents’ perceptions.

School policies for parental involvement

The index of school policies for parental involvement (PASCHPOL) was constructed using parents’ responses to the trend question (PA007) about the extent to which they agree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “My child’s school provides an inviting atmosphere for parents to get involved”; “My child’s school provides effective communication between the school and families”; “My child’s school involves parents in the school’s decision-making process”; “My child’s school offers parent education”; “My child’s school informs families about how to help students with homework and other school-related activities”; and “My child’s school co-operates with <community services> to strengthen school programmes and student development”. Positive values reflect parents’ perceptions that these school policies for parental involvement exist to a greater extent, negative values indicate that these school policies for parental involvement exist to a lesser extent, than the OECD average.

Teacher-level simple indices

Novice teachers

In the 19 countries and economies that distributed an optional questionnaire for teachers, teachers were asked to report how many years of work experience they have worked as a teacher, respectively in the school where they worked at the date of the survey (TC007Q01NA) and in total (TC007Q02NA). Answers to this last question was used to measure the proportion of novice teachers, defined as those who have worked at most 5 years in total as a teacher.

Originally trained teachers

In the 19 countries and economies that distributed an optional questionnaire for teachers, teachers were asked whether they completed a teacher education or training programme (TC014) and whether they received their initial teaching qualification (TC015) by attending a standard teacher education or training programme an education institute, an in-service teacher education or training programme, an work-based teacher education or training programme or training in another pedagogical profession. Answers to these two questions were combined to create the variable OTT2 for “original trained teacher (strict definition)” that is used in this report.

Participation to professional development activities

In the 19 countries and economies that distributed an optional questionnaire for teachers, teachers were asked (TC193) whether they participated during the last 12 months in one of the following professional development activities: “Courses/workshops (e.g. on subject matter or methods and/or other education-related topics)”; “Education conferences or seminars (where teachers and/or researchers present their research results and discuss educational issues)”; “Observation visits to other schools”; “Observation visits to business premises, public organisations, non-governmental organisations; and In-service training courses in business premises, public organisations, non-governmental organisations”. Answers to this question were used to measure the proportion of teachers who have participated to professional development activities (any of these five items) during the last 12 months.

Teacher-level scale indices

Teachers’ satisfaction with the teaching profession

In the optional teacher questionnaire, PISA 2018 asked (TC198) teachers how they feel about their job, specifically the degree to which they agree or disagree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “The advantages of being a teacher clearly outweigh the disadvantages”; “If I could decide again, I would still choose to work as a teacher”; “I regret that I decided to become a teacher”; and “I wonder whether it would have been better to choose another profession”. Teachers’ responses to these items were used to create an index of satisfaction with the teaching profession (SATTEACH). The index was standardised to have a mean of 0 and a standard deviation of 1 across OECD countries that distributed the optional teacher questionnaire. Higher values in the indices correspond to greater satisfaction.

Teachers’ satisfaction with their current job environment

In the optional teacher questionnaire, PISA 2018 asked teachers (TC198) how they feel about their job, in general, and specifically the degree to which they agree or disagree (“strongly agree”, “agree”, “disagree”, “strongly disagree”) with the following statements: “I enjoy working at this school”; “I would recommend my school as a good place to work”; “I am satisfied with my performance in this school”; and “All in all, I am satisfied with my job”. Teachers’ responses to these items were used to create an index of satisfaction with the current job (SATJOB). The index was standardised to have a mean of 0 and a standard deviation of 1 across OECD countries that distributed the optional teacher questionnaire. Higher values in the indices correspond to greater satisfaction.

Teachers’ self-efficacy in maintaining positive relations with students

In the optional teacher questionnaire, PISA 2018 asked teachers (TC199) to what extent (“not at all”, “to some extent”, “quite a bit”, “a lot”) they: “Get students to believe they can do well in school work”; “Help [my] students value learning”; and “Motivate students who show low interest in school work”. Teachers’ responses to these items were used to create an index of self-efficacy in maintaining positive relations with students (SEFFREL). The index was standardised to have a mean of 0 and a standard deviation of 1 across OECD countries that distributed the optional teacher questionnaire. Higher values in the indices correspond to greater self-efficacy.

Teachers’ self-efficacy in instructional settings

In the optional teacher questionnaire, PISA 2018 asked teachers (TC199) to what extent (“not at all”, “to some extent”, “quite a bit”, “a lot”) they: “Craft good questions for [my] students”; “Use a variety of assessment strategies”; “Provide an alternative explanation for example when students are confused”; and “Implement alternative instructional strategies in [my] classroom”. Teachers’ responses to these items were used to create an index of self-efficacy in instructional settings (SEFFINS). The index was standardised to have a mean of 0 and a standard deviation of 1 across OECD countries that distributed the optional teacher questionnaire. Higher values in the indices correspond to greater self-efficacy.

Teachers’ self-efficacy in classroom management

In the optional teacher questionnaire, PISA 2018 asked teachers (TC199) to what extent (“not at all”, “to some extent”, “quite a bit”, “a lot”) they: “Control disruptive behaviour in the classroom”; “Get students to follow classroom rules”; and “Calm a student who is disruptive or noisy”. Teachers’ responses to these items were used to create an index of self-efficacy in classroom management (SEFFCM). The index was standardised to have a mean of 0 and a standard deviation of 1 across OECD countries that distributed the optional teacher questionnaire. Higher values in the indices correspond to greater self-efficacy.

References

[7] Ganzeboom, H. (2010), A new international socio-economic index (ISEI) of occupational status for the international standard classification of occupation 2008 (ISCO-08) constructed with data from the ISSP 2002-2007, http://www.harryganzeboom.nl/pdf/2010-ganzeboom-isei08-issp-lisbon-(paper).pdf.

[6] Ganzeboom, H. and D. Treiman (2003), “Three Internationally Standardised Measures for Comparative Research on Occupational Status”, in Advances in Cross-National Comparison, Springer US, Boston, MA, http://dx.doi.org/10.1007/978-1-4419-9186-7_9.

[8] O*NET OnLine (2019), All STEM disciplines, https://www.onetonline.org/find/quick?s=all+STEM+disciplines (accessed on 2 October 2019).

[1] OECD (2019), PISA 2018 Assessment and Analytical Framework, OECD Publishing, Paris, https://dx.doi.org/10.1787/b25efab8-en.

[4] OECD (1999), Classifying educational programmes: Manual for ISCED-97 implementation in OECD Countries, OECD Publishing, Paris, http://www.oecd.org/education/1841854.pdf.

[5] OECD (n.d.), PISA 2018 Technical Report.

[2] OECD (forthcoming), PISA 2018 Technical Report, OECD Publishing, Paris.

[3] Warm, T. (1989), “Weighted likelihood estimation of ability in item response theory”, Psychometrika, Vol. 54/3, pp. 427-450, http://dx.doi.org/10.1007/BF02294627.

Notes

← 1. PISA expert groups identified a few indices that should be scaled to make index values directly comparable between PISA 2009 and PISA 2018. These indices include DISCLIMA, JOYREAD and JOYREADP. For these trend indices, a common calibration linking procedure was used. Countries/Economies that participated in both PISA 2009 and PISA 2018 contributed both samples to the calibration of item parameters. Each country/economy contributed equally to the estimation in each cycle. Trend indices were equated so that the mean and standard deviation of rescaled PISA 2009 estimates and of the original estimates included in the PISA 2009 database, across OECD countries, matched. Trend indices are therefore reported on the same scale as used in PISA 2009, so that values can be directly compared to those included in the PISA 2009 database.

← 2. The mappings of options provided in national versions of the student questionnaire (and recorded in variable LANGN) for the two possible values for the “International Language at Home” variable (ST022Q01TA) are the responsibility of national PISA centres. For students in the Flemish Community of Belgium, “Flemish dialect” was considered (together with “Dutch”) as equivalent to the “Language of test”; for students in the French Community and German-speaking Community (respectively), Walloon (a French dialect) and a German dialect were considered to be equivalent to “Other language”.