This chapter examines differences between countries and economies in the disciplinary climate during language-of-instruction lessons, and how the disciplinary climate is associated with student and school characteristics, and reading performance. It also looks at the disciplinary climate in schools with different proportions of girls and boys, and examines who may benefit the most from a positive disciplinary climate.
PISA 2018 Results (Volume III)
What School Life Means for Students’ Lives
PISA
Chapter 3. Disciplinary climate
Abstract
People have different views on what constitutes a positive classroom environment, yet most people recognise an environment that is conducive to learning. One of the key components of such a learning environment is the disciplinary climate, or the degree to which noise and disorder are kept at bay, students listen to what their teachers (and other students) say, and students can concentrate on academic tasks (Moos, 1979[1]). Cheema and Kitsantas (2014[2]) conceptualise disciplinary climate as the perceptions that students hold on the consistency of classroom rules and how teachers address behavioural problems during class. PISA adopts a more pragmatic definition according to which the disciplinary climate is measured by the extent to which students miss learning opportunities due to disruptive behaviour in the classroom.
Teachers have the main responsibility of ensuring that the classroom environment is conducive to learning (Matsumura, Slater and Crosson, 2008[3]). Students may feel that the school climate is negative, and may double down on deviant exploits, if they perceive that their teachers are unfair or biased in their interpretations of students’ behaviour (Pena-Shaff et al., 2019[4]). However, previous studies have shown that the disciplinary climate also varies according to school characteristics that are largely out of teachers’ control. For instance, socio-economically advantaged schools typically have a more positive disciplinary climate than disadvantaged schools (Ma and Willms, 2004[5]; OECD, 2016[6]).
Classrooms with a better disciplinary climate offer greater teaching and learning opportunities for students. In a structured classroom environment with fewer disruptions, teachers have more time to cover the curriculum and use diverse teaching strategies, and students can concentrate on their work more easily (Mostafa, Echazarra and Guillou, 2018[7]). Previous PISA results have consistently shown that there is a positive association between students’ perceptions of the classroom disciplinary climate and students’ academic performance, even after accounting for socio-economic status (OECD, 2016[6]) and other student and school characteristics (Ning et al., 2015[8]). Blank and Shavit (2016[9]) further reveal that disruptive behaviours in the classroom – but not the disciplinary policies at the school – are negatively correlated with student achievement. Other studies indicate that some students, such as ethnic minorities and disadvantaged students, may benefit more than others from an orderly classroom environment (Cheema and Kitsantas, 2014[2]). A positive disciplinary climate may also have benefits for other student outcomes, such as students’ sense of belonging at school (OECD, 2017[10]).
This chapter examines the disciplinary climate in language-of-instruction lessons. PISA asked students how frequently ( “never or hardly ever”, “some lessons”, “most lessons”, “every lesson”) the following things happen in their language-of-instruction lessons: “Students don’t listen to what the teacher says”; “There is noise and disorder”; “The teacher has to wait a long time for students to quiet down”; “Students cannot work well”; and “Students don’t start working for a long time after the lesson begins”. These statements were combined to create the index of disciplinary climate whose average is 0 and standard deviation is 1 across OECD countries. Positive values on this scale mean that the student enjoys a better disciplinary climate in language-of-instruction lessons than the average student in OECD countries.
What the data tell us
On average across OECD countries, almost one in three students reported that, in every or most lessons, students do not listen to the teacher or there is noise and disorder.
Student reports of disciplinary climate generally improved between 2009 and 2018, especially in Albania, Korea and the United Arab Emirates.
In all countries and economies, students with higher reading scores tended to report a more positive disciplinary climate, after accounting for socio-economic status. Even occasional disciplinary problems were negatively associated with reading performance.
Student reports of disciplinary climate were more positive in schools where more than 60 % of students were girls and in gender-balanced schools than in schools where more than 60 % of students were boys, on average across OECD countries.
On average across OECD countries, the positive relationship between disciplinary climate and reading performance was relatively stable across students’ gender, socio-economic status and immigrant background.
How the disciplinary climate varies across countries, schools and students
On average across OECD countries, the most common disciplinary problems in language-of-instruction lessons (amongst those included in the student questionnaire) were that students do not listen to what the teacher says and that there is noise and disorder in the classroom (Table III.B1.3.1). For example, almost one in three students reported that, in every or most lessons, students do not listen to the teacher or there is noise and disorder. About one in four students reported that, in every or most lessons, they start working a long time after the lesson begins or the teacher has to wait a long time for students to quiet down. Interestingly, fewer than one in five students reported that students cannot work well in every or most language-of-instruction lessons, which suggests that, at least from the students’ perspective, these disciplinary problems do not always interfere with their learning.
There are wide variations across countries and economies in the disciplinary climate in language-of-instruction lessons. Albania, Beijing, Shanghai, Jiangsu and Zhejiang (China), Belarus, Japan, Kazakhstan, Korea and Viet Nam show the most positive disciplinary climate, while Argentina, Brazil, France, Greece and Spain show the least positive climate (Figure III.3.1). For instance, in Japan only 3 % of students reported that there is noise and disorder in every lesson, compared to 23 % of students in France who so reported (Table III.B1.3.1). Perhaps more important, in some countries a significant share of students could not work well during language-of-instruction lessons, according to students’ reports. For instance, in Argentina, Brazil, France, Greece, Israel, Morocco and Turkey, at least 25 % of students reported that they, and their peers, cannot work well in every or most language-of-instruction lessons.
In many of these countries the disciplinary problems are highly concentrated in some schools (Table III.B1.3.3). In Argentina, for instance, about 11 % of students attend schools where at least 75 % of their schoolmates reported that, in every or most lessons, there is noise and disorder in their language-of-instruction lessons. The typical student in Argentina is enrolled in a school where about 55 % of their schoolmates reported so.
Differences across schools are also large. As much as 11 % of the variation in the index of disciplinary climate lies between schools, on average across OECD countries, which is a larger proportion than for the other indices analysed in this report (Table III.B1.3.5). According to students’ reports, in a majority of countries and economies the disciplinary climate in language-of-instruction lessons was more positive in socio-economically advantaged than in disadvantaged schools (Figure III.3.1). This was observed in 45 education systems, while in only 5 systems, namely Macao (China), the Republic of Moldova (hereafter “Moldova”), Morocco, Panama and Peru, the disciplinary climate was better in disadvantaged schools. On average across OECD countries, and in 18 other education systems, the disciplinary climate was better in private schools than in public schools, while the opposite was true only in Japan and Chinese Taipei. Moreover, the disciplinary climate was similar in rural and urban schools across OECD countries. However, in 14 school systems the disciplinary climate in rural schools was more positive than that in city schools, and especially so in Belarus, Jordan, Mexico, the Russian Federation and Ukraine.
As for student characteristics, girls reported a better disciplinary climate in language-of-instruction lessons than boys did, on average across OECD countries and in a majority of countries and economies (Table III.B1.3.4). Only in Denmark and Finland did boys report a better disciplinary climate than girls. Boys and girls may perceive the same learning environment differently, but this gender gap may also be explained by the fact that the typical boy and girl often attend schools and classes with very different proportions of boys and girls, particularly in countries with single-sex schools, a widespread use of ability grouping and with large proportions of 15-year-old students enrolled in vocational schools (see Box III.3.1). On average across OECD countries, and in almost a third of the school systems with available data, students without an immigrant background reported a more positive disciplinary climate than students with an immigrant background. The largest differences, in favour of students without an immigrant background, were observed in Colombia, Georgia, Indonesia and the Philippines. However, in 12 countries and economies, many of them English-speaking, immigrant students reported a more positive disciplinary climate than did students without an immigrant background.
Trends in disciplinary climate during language-of-instruction lessons
PISA 2009 and PISA 2018 asked students the same question about the disciplinary climate in language-of-instruction lessons, with only slight changes.1 A comparison of both cycles reveals that the disciplinary climate generally improved during this period (Table III.B1.3.2), which mirrors the trend, reported in the OECD Teaching and Learning International Survey (TALIS), of teachers highlighting improvements in disciplinary climate over the preceding five years (OECD, 2019[11]). For instance, on average across OECD countries, the percentage of students who reported that their classmates in their language-of-instruction lessons always, or almost always, listen to what the teacher says or can work well increased by about four percentage points between 2009 and 2018.2 The improvement in disciplinary climate was remarkable in several school systems, such as Albania, the Czech Republic, Denmark, Korea, Moldova, Montenegro, Norway, Serbia and the United Arab Emirates.
For instance, in 2018, 41 % of students in Denmark reported that the teacher never, or hardly ever, has to wait a long time for students to quiet down (Table III.B1.3.1), compared to 30 % of students who so reported nine years earlier (Table III.B1.3.2). In Montenegro, 52 % of students in 2018 reported that there is never, or hardly ever, noise and disorder during language-of-instruction lessons, compared to 40 % of students who so reported in 2009. In a few school systems, particularly Panama and Spain, the disciplinary climate deteriorated between 2009 and 2018. In these countries, 15-year-old students in 2018 were less likely than their counterparts in 2009 to report that the five types of disruption never happened during their lessons.
Figure III.3.1. Index of disciplinary climate, by school characteristics
Based on students’ reports
Note: Higher values in the index indicate a more positive disciplinary climate.
Countries and economies are ranked in descending order of the index of disciplinary climate.
Source: OECD, PISA 2018 Database, Tables III.B1.3.1 and III.B1.3.5.
How the disciplinary climate in language-of-instruction lessons is related to reading performance
In all countries and economies, students who reported a better disciplinary climate in their language-of-instruction lessons performed better in reading, after accounting for the socio-economic profile of students and schools (measured by the PISA index of economic, social and cultural status) (Figure III.3.2). On average across OECD countries, every unit increase in the index of disciplinary climate (equivalent to one standard deviation across OECD countries) was associated with an increase of 11 score points in reading performance. In Brunei Darussalam and Malaysia, the increase amounted to more than 25 score points.
Amongst the five items that make up the index of disciplinary climate, the one that shows the strongest association with reading performance is the frequency of situations in which “students cannot work well” (Table III.B1.3.6). On average across OECD countries, students who reported that students cannot work well in every or most language-of-instruction lessons scored 25 points lower in reading than students who reported that this never happened or happened only in some lessons, after accounting for socio-economic status. At the school level, the analyses paint a similar picture (Table III.B1.3.7). In the schools where students were least likely to report that students cannot work well (that is, the schools in the bottom quarter of the indicator in their country/economy) the average reading score was 512 points; in the schools where students were most likely to report so (the schools in the top quarter of the indicator in their country/economy) the average reading score was 456 points, a significant difference of 56 points.
Figure III.3.2. Disciplinary climate and reading performance
1. The socio-economic profile is measured by the PISA index of economic, social and cultural status (ESCS).
Notes: Higher values in the index indicate a more positive disciplinary climate.
Statistically significant values are shown in darker tones. All differences after accounting for students’ and schools’ socio-economic profile are statistically significant (see Annex A3).
Countries and economies are ranked in ascending order of the score-point difference associated with a one-unit increase in the index of disciplinary climate, after accounting for students’ and schools’ socio-economic profile.
Source: OECD, PISA 2018 Database, Table III.B1.3.6.
The analysis of the frequency of disciplinary problems in language-of-instruction lessons and students’ reading performance shows that even occasional disciplinary problems are negatively associated with reading performance (Figure III.3.3). On average across OECD countries and after accounting for the socio-economic profile of students and schools, students who reported that disciplinary problems occur in some language-of-instruction lessons scored between 5 and 9 points lower in reading than students who reported that the problems never, or hardly ever, occur. Students scored between 12 and 21 points lower in reading when they reported that the disciplinary problems occur in most lessons. However, less frequent disciplinary problems were not always negatively associated with reading performance. For instance, in 8 countries and economies, students who reported that there is noise and disorder in some lessons scored higher than students who reported that these problems never happen; and in another 38 school systems there was no significant association between sporadic noise and disorder, and reading performance (Table III.B1.3.8).
Figure III.3.3. Prevalence of disciplinary problems and reading performance
OECD average
Notes: All values are statistically significant (see Annex A3).
All regression models account for students’ and schools’ socio-economic profile. The socio-economic profile is measured by the PISA index of economic, social and cultural status (ESCS).
Source: OECD, PISA 2018 Database, Table III.B1.3.8.
The relationship appeared to be much stronger when students reported that disciplinary problems occur repeatedly in their language-of-instruction lessons. For instance, compared to students who reported that disciplinary problems never or hardly ever occur, students scored 50 points lower in reading when they reported that students cannot work well in every lesson, and 43 points lower when they reported that, in every lesson, the teacher has to wait a long time for students to quiet down. In every school system, students who reported that any of five disciplinary problems happen in every lesson scored lower than students who reported that these problems never happen (the only exceptions were Finland and the Philippines, where the negative association between repeated noise and disorder and reading performance was not significant).
Relationship between disciplinary climate and reading performance by gender, socio-economic status and immigrant background
Researchers have widely documented the benefits of an orderly classroom environment, but only a handful of studies has addressed the question of who benefits more from a positive disciplinary climate. Using PISA 2003 data for the United States, Cheema and Kitsantas (2014[2]) showed that the achievement gap in mathematics between white and minority ethnic group students tended to be considerably narrower in schools with better disciplinary climate in mathematics lessons. Using PISA 2012 data for the Nordic countries, Sortkaer and Reimer (2018[12]) found that the association between disciplinary climate and mathematics achievement was significantly stronger for boys than for girls. Do PISA 2018 data show any differences in the association between the disciplinary climate in language-of-instruction lessons and reading performance across different groups of students?
On average across OECD countries, the relationship between disciplinary climate and reading performance was relatively stable across students’ gender, socio-economic status and immigrant background (Figure III.3.4). If anything, the strength of the relationship seemed somewhat stronger for boys than for girls. The only countries where this was the case were Finland, Israel and Qatar, while the only school systems where the association was stronger for girls were Baku (Azerbaijan) and Peru (Table III.B1.3.9). Further, in a number of countries and economies, such as Brunei Darussalam, the Dominican Republic, Hungary, Mexico, the Philippines, Qatar and Thailand, the association between the disciplinary climate in language-of-instruction lessons and reading performance was stronger for socio-economically advantaged students than for disadvantaged students; only in Ireland and Moldova was the opposite true. In the Dominican Republic, Israel, Malaysia, Mexico, Qatar, the Slovak Republic, Sweden and Thailand, the association between disciplinary climate and reading performance was stronger amongst students with an immigrant background than amongst students without an immigrant background. In short, despite the results observed in a limited number of countries, the positive relationship between disciplinary climate and reading performance was relatively stable across students’ gender, socio-economic status and immigrant background.
Figure III.3.4. Disciplinary climate and reading performance, by student characteristics
OECD average
Note: All values are statistically significant (see Annex A3).
Source: OECD, PISA 2018 Database, Table III.B1.3.9.
Comparing the disciplinary climate in gender-balanced and single-sex schools
Co-educational schools are today the norm across OECD countries, but single-sex schools still exist in some PISA-participating countries and economies. In addition, in a large number of countries, mostly those where the share of 15-year-old students enrolled in pre-vocational or vocational schools is comparatively high, there is a significant number of schools where either boys or girls represent a clear majority (see Box III.3.1). Advocates of single-sex schools argue that those schools have, amongst other advantages, a better disciplinary climate where students can concentrate more on their learning tasks. This view implies that gender-balanced schools have a less-positive disciplinary climate, and that students enrolled in those schools should be at a disadvantage compared to students enrolled in gender-unbalanced schools, notably in single-sex schools. But others argue that it is the number of boys enrolled in a school that affects the disciplinary climate, rather than whether the school is mixed or single-sex. In this second scenario, all-boys schools should show the least positive disciplinary climate, and the students enrolled in this type of school should be at a disadvantage compared to every other student.
PISA asked school principals about the number of boys and girls in their schools. Based on their answers, there were only a limited number of PISA-participating countries with a sufficiently large number of sampled students who attended single-sex schools; but a comparison of gender-balanced schools and those where either boys or girls are a clear majority (more than 60 % of the student body, including single-sex schools) provide valuable insights. On average across OECD countries and in more than half of the PISA-participating education systems, the disciplinary climate was more positive in schools where more than 60 % of students were girls than in schools where more than 60 % of students were boys (Figure III.3.5). Perhaps more important, in almost half of the school systems with available data, the disciplinary climate was also more positive in gender-balanced schools (those schools where boys and girls represented between 40 % and 60 % of students) than in schools where boys represented more than 60 % of the student body (Table III.B1.3.10). The only two countries where the disciplinary climate was significantly worse in gender-balanced schools than in schools with a clear majority of boys were Australia and the Dominican Republic. A previous study in Australia comparing co-educational and single-sex schools also found that misbehaviour was somewhat more frequent in co-educational schools than in both types of single-sex schools (Cohen and Barrington Thomas, 1984[13]).
Figure III.3.5. Disciplinary climate, by gender composition at school
Notes: Higher values in the index indicate a more positive disciplinary climate.
Statistically significant differences between predominantly girls’ and predominantly boys’ schools are shown in darker tones (see Annex A3).
Countries and economies are ranked in descending order of the differences in the index of disciplinary climate between predominantly girls’ and predominantly boys’ schools.
Source: OECD, PISA 2018 Database, Table III.B1.3.10.
Similar findings emerge from the comparison of single-sex and gender-balanced schools in the countries and economies with a sufficiently large number of students in these schools (Table III.B1.3.10). In about half of these school systems, the disciplinary climate was better in gender-balanced schools than in all-boys schools, and in a clear majority of these schools systems, students enjoyed a better disciplinary climate in all-girls schools than in all-boys schools; in no education system did all-boys schools show a better disciplinary climate than that in other types of schools. In Israel, Malta, Qatar and Singapore, in particular, boys enrolled in all-boys schools seemed to be at a great disadvantage regarding the disciplinary climate in language-of-instruction lessons compared to all other students, including boys in gender-balanced schools and girls in any type of school. These comparisons, however, should be interpreted with caution as they are based on comparisons of a maximum of 17 education systems.
Box III.3.1. Why are there schools with different proportions of boys and girls?
On average across OECD countries in 2018, some 22 % of students attended a school where more than 60 % of students were either boys or girls, according to PISA data (Table III.B1.3.11). This may be partly explained by the larger number of boys born every year, or by a gender gap in school dropout rates. However, the main reasons are probably the presence of single-sex schools and educational tracking (sorting students into different programmes and schools).
On average across OECD countries in 2018, some 5 % of students were enrolled in a single-sex school; in 15 PISA-participating countries and economies, at least 10 % of students were enrolled in such schools (Table III.B1.3.11). In addition, across OECD countries and in many education systems, girls were more likely than boys to attend single-sex schools, which is likely to further contribute to an unbalanced gender composition amongst the co-educational schools in these education systems.
In countries and economies where there are few or no single-sex schools, the main reason why schools have an unbalanced gender composition is, most probably, the prevalence of tracking. Figure III.3.6 clearly shows that the share of students who attended gender-balanced schools was lower in education systems with larger proportions of students enrolled in a pre-vocational or vocational programme. In Croatia, Serbia and Slovenia, for instance, more than half of students were enrolled in a pre-vocational or vocational programme, which probably explains why less than 40 % of students attended a gender-balanced school. By contrast, more than 95 % of students attended gender-balanced schools in more comprehensive education systems, including Canada, Estonia, Finland, Iceland, Norway, the Philippines and Spain.
Figure III.3.6. Programme orientation and gender composition at school
Note: Gender-balanced schools are those with between 40 % and 60 % of girls or boys.
Source: OECD, PISA 2018 Database, Table III.B1.3.11.
References
[9] Blank, C. and Y. Shavit (2016), “The association between student reports of classmates’ disruptive behavior and student achievement”, AERA Open, Vol. 2/3, http://dx.doi.org/10.1177/2332858416653921.
[2] Cheema, J. and A. Kitsantas (2014), “Influences of disciplinary classroom climate on high school student self-efficacy and mathematics achievement: a look at gender and racial-ethnic differences”, International Journal of Science and Mathematics Education, Vol. 12/5, pp. 1261-1279, http://dx.doi.org/10.1007/s10763-013-9454-4.
[13] Cohen, B. and E. Barrington Thomas (1984), “The disciplinary climate of schools”, International Journal of Educational Management, Vol. 22/2, pp. 113-133, http://dx.doi.org/10.1108/eb009888.
[3] Matsumura, L., S. Slater and A. Crosson (2008), “Classroom climate, rigorous instruction and curriculum, and students’ interactions in urban middle schools”, The Elementary School Journal, Vol. 108/4, pp. 293-312, http://dx.doi.org/10.1086/528973.
[5] Ma, X. and J. Willms (2004), “School disciplinary climate: Characteristics and effects on eight grade achievement”, Alberta Journal of Educational Research, Vol. 50/2, pp. 169-188.
[1] Moos, R. (1979), Evaluating Educational Environments, Jossey-Bass, San Francisco, CA.
[7] Mostafa, T., A. Echazarra and H. Guillou (2018), “The science of teaching science: An exploration of science teaching practices in PISA 2015”, OECD Education Working Papers, No. 188, OECD Publishing, Paris, http://dx.doi.org/10.1787/f5bd9e57-en.
[8] Ning, B. et al. (2015), “The influence of classroom disciplinary climate of schools on reading achievement: a cross-country comparative study”, School Effectiveness and School Improvement, Vol. 26/4, pp. 586-611, http://dx.doi.org/10.1080/09243453.2015.1025796.
[11] OECD (2019), TALIS 2018 Results (Volume I): Teachers and School Leaders as Lifelong Learners, TALIS, OECD Publishing, Paris, https://dx.doi.org/10.1787/1d0bc92a-en.
[10] OECD (2017), PISA 2015 Results (Volume III): Students’ Well-Being, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264273856-en.
[6] OECD (2016), PISA 2015 Results (Volume II): Policies and Practices for Successful Schools, PISA, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264267510-en.
[4] Pena-Shaff, J. et al. (2019), “Racial and ethnic differences in high school students’ perceptions of school climate and disciplinary practices”, Race Ethnicity and Education, Vol. 22/2, pp. 269-284, http://dx.doi.org/10.1080/13613324.2018.1468747.
[12] Sortkær, B. and D. Reimer (2018), “Classroom disciplinary climate of schools and gender – evidence from the Nordic countries”, School Effectiveness and School Improvement, Vol. 29/4, pp. 511-528, http://dx.doi.org/10.1080/09243453.2018.1460382.
Notes
← 1. The wording of the response options changed slightly across both cycles ( “in all lessons”, “in most lessons” and “in some lessons” changed to “every lesson”, “most lessons” and “some lessons” in PISA 2018), and so did the order in which the response options appeared in the questionnaire.
← 2. For convenience, in the description of the results of negatively framed items, such as “students don’t listen to what the teacher says”, “never or hardly ever” has often been replaced by “always or almost always”. For instance, “Never or hardly ever don’t listen” has been replaced by “always or almost always listen”.