The two previous chapters have examined the experience of schooling (during school closures in March to June 2020) and characteristics of the home situation of school age children. In this chapter, the available evidence regarding children’s psychological well-being and academic progress during this period is examined. How well did school-age children cope with the period of lockdown and school closures? Was the academic progress of school children slowed during this period and, if so, to what extent?
Schooling During a Pandemic
The Experience and Outcomes of Schoolchildren During the First Round of COVID-19 Lockdowns
4. School children’s psychological well-being and academic progress
Abstract
Introduction
The two previous chapters have examined the experience of schooling (during school closures in March to June 2020) and characteristics of the home situation of school age children. In this chapter, the available evidence regarding children’s psychological well-being and academic progress during this period is examined. How well did school-age children cope with the period of lockdown and school closures? Was the academic progress of school children slowed during this period and, if so, to what extent? The academic progress of children will be examined from two perspectives: the perceptions of parents and to a lesser extent teachers and students themselves, and the more “objective” perspectives of standardised tests when available.
The psychological well-being of children
Lockdowns and the associated closure of schools represented a dramatic disruption to the lives of school age children to which they had to adapt more or less overnight. How did school age children cope with the consequences for their lives of lockdowns, school closures and the presence of the COVID-19 virus? Relatively few representative studies on the mental health of children during lockdowns have been published.1 The available information is sketchy and approaches the question from a range of different perspectives.
The concerns and feelings of children
Several studies provide an insight into the concerns and feelings of children during the period of lockdown. These include studies using self-reports of children and the reports of parents/guardians regarding their children (and, sometimes, both).
From the perspective of parents
Parents in England reported that their children (aged 5-16 years) were worried about missing school (40%) and that friends might catch COVID (37%) but were less worried about catching COVID themselves (22%) or infecting others (16%) (NHS Digital, 2020[1], Table 3.2). Just over half (52%) of the parents of secondary school students in France reported that their children were worried about the future (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2]). Only a small proportion (13%) of parents of children born in 2011 in France (i.e. aged 8-9 years at the date of the survey) reported that their child experienced social-emotional difficulties such as isolation, anxiety, difficulties in concentrating and impulsiveness during the period of confinement (Thierry et al., 2021[3]). In the United States, the experience of lockdown by children did not seem to be overly negative, at least early on in the period. Nearly 90% parents reported that their children exerienced enjoyment (89%) and happiness (88%) for “a lot of the day” on the day before they were interviewed in March 2020 and the proportions of parents reporting that their children experienced “negative” emotions such as “worry”, “stress”, “anger”, “sadness” and “loneliness” were in the range of 20%‑26%. The only exception was “boredom” which was reported as being experienced by 65% of children (Jones, 2020[4]).
From the perspective of children
School age children reported a generally negative view of the impact of lockdowns on their life, although positive aspects were also noted. A majority of French school children surveyed at the start of the 2021 school year (September 2020) regarding the experience of confinement over April-June 2020 reported that the period of confinement had been too long (63% of children in grades K-1, 70% in year 5 and 54% in year 9) and that they had been affected by the absence of contact with their friends (75% of children in grades K-1, 82% in year 5 and 80% in year 9). Between 25% and 39% (depending on grade) had experienced a fear of Coronavirus and between 38-51% had experienced boredom. At the same time, a majority appreciated the ease of studying at home (57-61%) and between 36-56% expressed satisfaction at being able to remain at home all the time (Direction de l’évaluation, de la prospective et de la performance (DEPP), 2021[5], Tables 1-3).
In the Netherlands, almost all (90%) children (aged 8-18 years) reported that the COVID-19 lockdown had a negative impact on their daily life. The issues most often mentioned were: 1) missing contact with friends, 2) not being allowed to go to school, 3) missing freedom, 4) not being allowed to participate in sports, 5) missing joyful activities (e.g., birthdays, holidays, parties, shopping), 6) difficulties with homeschooling 7) missing extended family, and 8) boredom (Luijten et al., 2021[6], Table 5). Broadly similar results were found in Germany (Ravens-Sieberer et al., 2021[7]). Nearly two-thirds (71%) of German children (aged 11‑17 years) stated that they felt burdened by the COVID-19 pandemic. Four out of five (83%) reported fewer social contacts during the pandemic, 64% found schooling and learning to be more difficult than before the pandemic and 39% reported that their relationships with their friends had been impaired. Children and adolescents also reported depressive symptoms: 62% had trouble concentrating, 58% had little interest or joy in activities, and 34% felt sad.
In contrast, a study in England found a more mixed appreciation of the period of lockdown with 43% of 11‑16 year-olds reporting that it had made their life worse, 30% reporting that it had made no change and the remaining 27% reporting that it had made their life better (NHS Digital, 2020, p. 45[8]). Some 55% of children reported that they were hardly ever or never lonely with only 5% stating that they were often or always lonely during lockdown (NHS Digital, 2020[1], Table 3.6).
Change in psychological well-being
In addition to understanding how children felt about and reacted to the situation of lockdown and school closures, a key question for evaluating the impact of lockdowns/school closures on the pyschological health and well-being of children is whether it was associated with change in their pyschological state.
From the perspective of parents
In the studies reviewed, a sizable minority of parents reported a worsening of their child(ren)’s psychological welbeing during lockdown. In a Canadian study, 25% of parents indicated that their children’s mental health had worsened since the onset of the COVID-19 pandemic with the majority of parents (60%) reporting their children’s mental health had stayed the same (Gadermann et al., 2021[9]). Parents in Germany reported that their children (aged 7-17 years) suffered from more mental health problems during than prior to the pandemic. The prevalence of noticeable mental health problems was 10% before the pandemic and increased to 18% during the pandemic. This increase was greatest among 7-10 year-olds (from 7 to 27%). At the same time, declines in the incidence of emotional symptoms, conduct problems, hyperactivity and peer problems were reported (Ravens-Sieberer et al., 2021[7]). In a survey conducted in Israel in the first week of April 2020, 28% of parents stated that their children’s emotional state had deteriorated as a result of lockdown (Central Bureau of Statistics, 2020[10]) and, in the United Kingdom, 43% of parents home schooling their children agreed that remote schooling was negatively affecting their children’s well-being (Office for National Statistics (ONS), 2020[11], Table 1).
From the perspective of children
Studies based on the reports of children themselves provide contrasting results. A national mental health cohort study in England (Figure 4.1) found that the proportion of 5-16 year-olds who were unlikely to have a mental disorder had remained unchanged between 2017 and July 2020 at around 74-75% (NHS Digital, 2020[1], Table 1.1). The main change between 2017 and 2020 was that the estimated proportion of children with a “probable” disorder increased by 5 percentage points and the proportion with a “possible” disorder declined by the same margin. In Germany, children and adolescents (11-17 years of age) were found to have lower health related quality of life (HRQoL)2 during than prior to the pandemic. Before the pandemic, 15% of children and adolescents reported low HRQoL, increasing to 40% during lockdown, with the increase being greater for younger than older children (Ravens-Sieberer et al., 2021[7]). In addition, 11-17 year-olds, experienced higher levels of generalised anxiety during the COVID-19 pandemic (24%) compared with before the pandemic (15%). However, the prevalence of depressive symptoms did not change. In the Netherlands, worse average scores on scales measuring Anger, Peer Relationships, Global health, Sleep-related Impairment, Anxiety, and Depressive Symptoms were recorded among 8-18 year-olds during the period of lockdown (data collected between 10 April and 5 May 2020) than before among children and adolescents of similar ages surveyed in 2018. However the proportions of children manifesting severe symptoms were relatively low and, with the exception of severe Anxiety (17% during lockdown compared to 9% before) and severe Sleep-Related Impairment (12% compared to 6%) were unchanged or slighly lower during lockdown than before (Luijten et al., 2021[6]) (Figure 4.2 below).
Figure 4.1. Likelihood of a mental disorder, 5-16 year-olds, England, 2017 and July 2020 (%)
Source: (NHS Digital, 2020[1], Table 1.1).
Figure 4.2. Incidence of mental and social health problems in children (8-18 years) and adolescents before and during the COVID-19 lockdown: the Netherlands
Percentage of respondents with poor functioning or severe symptoms
Source: (Luijten et al., 2021[6]).
The picture that emerges from the above is that the lockdowns and school closures had a range of negative aspects for many, though by no means all children. It was a period of inconvenience, difficulties and stresses for children, many of which would not have experienced (or would have experienced to a lesser degree) in normal conditions. The proportion of school age children experiencing serious or severe symptoms of mental or psychological disorders appears to have risen during the period of lockdown. However, the proportion of children concerned was relatively small. Most, both before and during the period of lockdowns, did not display such symptoms. An important (but unanswered question) is whether a similar pattern of change in psychological state as that observed among adults occurred among children: that of a sharp decline in psychological well-being associated with the introduction of lockdowns followed by a subsequent improvement as with the reduction of restrictions and habituation with the pandemic situation and its consequences.
Home learning: Parents’, teachers and students’ perceptions
Did the arrangements put in place to support home learning during school closures allow children to maintain their link with schools and teachers and to continue to learn effectively? Two main types of information relevant to this question exist. First, there is the perception of the actors involved’, principally parents and to a lesser extent students and teachers. Second, there are a small number of studies that have compared results on standardised tests for students in the cohorts affected by the pandemic with results for students in the same tests in previous years. Information on the perceptions of the actors involved is presented before presenting the results of testing programmes.
The views of parents
How satisfied were parents with the home schooling experience and the support offered by schools and how did they assess the impact of the period of home schooling on children’s learning and social development? Table 4.1 summarises the views of parents in France, Germany, Ireland, the United Kingdom and the United States. Overall, parents/guardians had mixed views. Satisfaction with the efforts made by schools and teachers during the period of school closures was balanced by concerns regarding their children’s educational progress and, in some cases, their broader social development.
Table 4.1. Parents’ views regarding their children’s schooling and educational progress during lockdowns
Sources: France: (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figure 9-3); Germany: (D21 Initiative/TUM/Kantar, 2020[12]; Wößmann et al., 2020[13]); Ireland: (Central Statistics Office (CSO), 2020[14]); United Kingdom: (Office for National Statistics (ONS), 2020[11]); United States: (1) (Horowitz, 2020[15]); (2) (University of Southern California (USC), 2020[16]); (3) (Brenan, 2020[17]).
In terms of an overall appreciation of the work of teachers and schools during school closures, a large majority of parents in France, the United Kingdom and the United States expressed satisfaction. High proportions of parents of secondary school students in France agreed that the activities offered by teachers during the period of school closures had been beneficial to their children (75% to 81% depending on their level of schooling). The amount of work that gave to their children was seen as appropriate by nearly two out of three parents of secondary school students with between 17% and 23% of parents seeing it as being too much and between 12% and 20% as too little (depending on the educational level) (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figure 2-11). Four out of five US parents (83%) reported being satisfied with the way their children’s school had been handling instruction during school closures and 64% were satisfied with how much their children were learning. In addition, high proportions of US parents expressed satisfaction with the communication with their child(ren)’s school (Jones, 2020[18]; University of Southern California (USC), 2020[16]).3
The levels of satisfaction of German parents were lower than in the other three countries. However, the majority were satisfied. Some 54% of German parents were moderately or very satisfied with the school lessons of their children during the pandemic, feeling that schools had done “all that was in their power” (59%) and acknowledging that teachers transformed their teaching on their own initiative (54%). At the same time, 33% were moderately or very unsatisfied overall, with 42% finding that teachers were overwhelmed with the digital transformation of their teaching and 24% that schools went at a slower pace (D21 Initiative/TUM/Kantar, 2020[12]). Another study found similar results: 56% of parents reported being very or rather satisfied with school activities during school closures and 38% reported being very or rather dissatisfied (Wößmann et al., 2020[13]).
Figure 4.3. Opinions of parents regarding the instructional activities offered to their children by teachers, by child's level of schooling: France (%)
Source: (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figures 4-5 and 4-6).
Figure 4.4. Level of satisfaction of parents of K-12 children with the way children’s school has been handling instruction during the school closure: United States, April 2020
Source: (Horowitz, 2020[15])
Parents were, however, generally less satisfied with their children’s learning and academic progress. Only 66% of French parents of secondary school students strongly or somewhat agreed that their children’s learning had been maintained and far fewer were in agreement that it had progressed (39%) or that there had been improvement in their children’s level in certain subjects (35%). At the same time, French parents noted positive effects of the period of closures such as the increased independence of their children (57%) and the discovery of new methods of learning (56%) (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figure 5-7). In Germany, Ireland and the United Kingdom, similar results were observed. Almost two-thirds of German parents (64%) agreed that their child had learned much less than usual (Wößmann et al., 2020[13]). In Ireland, just under half of parents/guardians had a negative perception of the impact of enforced school closures when surveyed in August 2020. Closures were seen as having a major or moderate negative impact on students’ learning by 41% of parents of primary and 46% of parents of secondary students as well as on students’ social development (42% of primary and 43% of secondary parents). Few parents/guardians of either primary or secondary students (close to 15% in both cases) viewed the impact of school closures as neutral or positive on either their children’s learning or social development (Figure 4.5). One in three (36%) parents of secondary school students were worried about their child returning to school because he/she had fallen behind due to lockdown (Office for National Statistics (ONS), 2020[11]). In the United Kingdom, 70% of parents reported that the children/child within their household were continuing to learn whilst being home schooled and 42% reported that their oldest or only child struggled to continue education remotely.
Figure 4.5. Parents’ views of the impact of enforced school closures on children's learning by level of schooling: Ireland, August 2020
Source: (Central Statistics Office (CSO), 2020[14], Tables 2.1 and 2.4).
Figure 4.6. Parents’ views regarding the likelihood that the COVID-19 pandemic will lead to their child not making as much progress academically (% by category), United States, May 2020
Evidence regarding US parents’ assessment of the likely impact of school closures on their children’s educational progress is mixed with between one-third to two-thirds of parents expressing concerns. In late March 2020, less than half (42%) of parents of K-12 students were “very” or “moderately” concerned that the pandemic would have a negative impact on their child’s education. A poll conducted in early April 2020, reported less positive opinions: 64% of parents were concerned about their children falling behind as a result of the Coronavirus outbreak. However, in May in another survey (University of Southern California (USC), 2020[16]), US parents were more positive: only 34% of parents thought it somewhat or very likely that their child would not make as much progress academically due to school closures (Figure 4.6) and 74% agreed that their child(ren) would be well prepared for school in the next school year.
The evidence regarding the relationship between concerns regarding the negative impact of lockdowns and school closures on children’s academic progress is mixed (Table 4.2). No relationship between views of the impact of closures on student progress and parental education or income is seen in the Irish and UK data. In France, the satisfaction of parents was only weakly related to their socio-economic status. Views regarding the benefits of the activities provided to children were unrelated to social background, but advantaged parents were slightly more satisfied with their children’s progress than their less advantaged peers. In the United States, the existence and direction of an association between parental views regarding the impacts of closures on their children’s academic progress and parental education, income or race varies between studies.
Table 4.2. The relationship between socio-economic background and parental concerns regarding the academic progress of children
France, Ireland, the United Kingdom and the United States
Sources: France: (Direction de l’évaluation, de la prospective et de la performance (DEPP), 2020[19]); Ireland: (Central Statistics Office (CSO), 2020[14]); United Kingdom: (Office for National Statistics (ONS), 2020[11]); United States: (Brenan, 2020[17]; Horowitz, 2020[15]; University of Southern California (USC), 2020[16]).
The views of students and teachers
Unfortunately, in most countries, little information is available regarding the views of pupils or teachers regarding the utility and effectiveness of home-based schooling during the period of school closures. An exception is France where information is available on the views of students, teachers and school principals.
Just under two-thirds of French secondary school pupils (63-64% depending on the type of school) felt that the quantity of school work that they were asked to do was appropriate with around a third considering that it was too great and 4-7% too little (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figure 2-12). In a poll conducted in the United States in late July/early August 2020, 46% of school students in grades 3-12 reported that they believed that they would have to catch up in the coming school year because of the time spent learning from home in the spring of 2020 (Marken and Clayton, 2020[20]).
A large majority of school teachers in France were of the view that their students had learnt in a satisfactory or highly satisfactory manner during the period of school closures. This was true of 77% of primary school teachers and 68% of secondary teachers (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figures 1-1 and 1-2). In addition, most secondary teachers considered the arrangements put in place during school closures to have had (to a large or to some extent) a positive influence on the acquisition of digital skills (80%) and on the autonomy of pupils (78%). In contrast, few considered that these arrangements had a positive impact on either the reduction of inequalities (10%) or the motivation of pupils regarding their schooling (38%) (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figure 9-2).
School principals in France offered a broadly similar evaluation to teachers with 74% of primary principals and between 67% and 84% (according to the type of school) of the senior management in secondary schools considering that their students had learnt in a satisfactory or highly satisfactory manner during the period of school closures (Direction de l'évaluation de la prospective et de la performance (DEPP), 2020[2], Figure 1‑1). Reflecting this, only a quarter of French primary school principals felt that the level of proficiency in reading (26%) and in mathematics (calculation and number) (24%) was lower for most or all pupils enrolled in year 2 at the start of the 2020-21 school than was the case for pupils in the same grade at the start of the previous school year (Direction de l’évaluation, de la prospective et de la performance (DEPP), 2021[21], Figure 2).
Evidence from standardised achievement tests
To what extent are the concerns of parents that their children’s academic progress was negatively affected during the period of school closures supported by actual evidence regarding student performance? Potential sources of empirical information regarding the academic progress of pupils affected by closures can come from: (1) comparisons of the academic performance of the same students before and after (or during) the period of school closures; (2) comparisons of students in given years who experienced disruption to their education with cohorts of in the same years of schooling in previous years. Unfortunately, due to the pandemic, most countries suspended national testing programmes in the 2019-20 school year. Few data are available, therefore, that permit an evaluation of the immediate effect of the school closures of March‑June 2020 on the achievement of the pupils concerned (i.e. data collected during or within 6 months following immediately the period of closures).
Description of the available data
Relevant data are available, nevertheless, in a number of countries and several comparisons of the performance of students experiencing school closures in the first half of 2020 with students in the same year of schooling in 2019 and earlier years have been published. These include results from national or provincial level testing programmes (Baden-Württemberg, France, Italy and the Netherlands) and system-specific testing programmes (Catholic schools in Flanders). Data from online tests used in schools (England, Switzerland and the United States) has also been analysed as have results from smaller scale studies designed to examine the impact of school closures on student performance (Australia, England).
In assessing the strength of the available evidence regarding the impact of school closures during the first half of 2020 on the performance of school children, it is important to note the considerable differences between studies. They differ in terms of their design (especially in the nature of samples), the year groups covered, and rates of participation by schools and by pupils in tests conducted in 2020/21. Table 4.3 provides a summary of the characteristics of the data and analysis used in the different studies.4 Of the studies reviewed, those from Australia, Flanders, France, Germany (Baden-Württemberg) and Italy are based on censuses or representative samples (with the Flemish tests having very low rates of participation of schools in the 2020 testing round). The remainder are based on non-representative or convenience samples of various kinds involving a degree of self-selection by participating schools5 and/or pupils – e.g. schools that use a particular (proprietary) test, volunteer schools, etc.
The type of information on performance also varies. Some studies assess growth in performance in the 2020 school year (i.e. change in performance between different measurement points in the 2020 school year) compared to the change observed among similar pupils in previous years. Others compare performance levels, i.e. the performance of a grade cohort (e.g. year 6 students) at the end of the 2020 school year or in the course of the 2021 school year with the performance of the same year cohort in previous years. The statistics used to report performance also vary. Where possible, differences in scaled scores have been standardised as a proportion of a standard deviation (SD) to help comparison if this has not already been done by the authors of the studies concerned.
In addition, the dates at which the tests were administered vary and with this, the interval of time that has elapsed between the first wave of school closures and the conduct of the assessment. This needs to be taken into account when interpreting (and comparing) the results. The results of tests taken during or soon after the March-June 2020 period of closures, in addition to cumulated learning to that point, will reflect primarily the impact of these closures. However, the results of testing undertaken at later dates will reflect a range of additional influences. These include: disruptions to schooling in the 2020-21 school year (including any further school closures or lockdowns), the effects of measures taken by schools to consolidate the instruction which pupils did not fully cover in the 2019-20 school year as well as any action taken by students and their families to make up for any missed instruction (e.g. additional after-school tuition). For the most part, the results presented in Table 4.3 come from tests administered either during or within six months of the end of the first wave of school closures. The exception is the Italian assessments which were administered in May 2021 and which, therefore, are far more affected by experience of pupils following the first wave of school closures than the others. Finally, the assessments may also vary in terms of their objectives and purposes.
In terms of coverage of students in the grade levels tested, the best data concerning the academic progress of the cohort of students affected by school closures come from France, the German Länder of Baden‑Württemberg, Italy and Ohio in the United States. Both the French and Italian assessments are national assessments covering students in public and private schools. The Baden-Württemberg assessments cover students in public schools and the Ohio assessment covers pupils in all primary schools in the state. In all, the participation rates among the target populations are high.
Findings concerning academic achievement and progress
The results reported in the available studies vary considerably. Improvement, as well as stability and decline in the performance of the “COVID cohorts” relative to their peers tested in previous years is observed (Table 4.4). There are no clear patterns within or between countries or by learning area (e.g. reading/language compared to mathematics). In the studies presented, performance in mathematics declines as opposed to remaining unchanged or improving more frequently than does performance in tests of language skills (e.g. reading in the national language). However, even in the case of mathematics, cases of no change and improvement are observed.
An important question is how to evaluate the scale of the observed differences in the academic achievement of the COVID cohorts with cohorts in previous years. What is a “large” and what is a “small” difference? Expressing the differences in terms of the normal performance gain over a given time period is one way of doing this. This is undertaken in a number of the studies presented. However, variation in the basis of the calculations, particularly assumptions regarding normal achievement gain mean that the comparisons should be made with caution. The big challenge is estimating what range of growth in performance would be expected over a year in “normal” conditions. A range of benchmarks have been proposed, from 0.25 of a standard deviation in test scores (Avvisati and Givord, 2021[22], based on the Programme for International Student Assessment [PISA]) to 0.65 (Schult et al., 2021[23]).6 These depend, to some extent, on the particular studies, the country and the grade group concerned.
Whatever the benchmark used, the falls in performance reported in Flanders (particularly in Dutch language) and, to a lesser degree, in Italy at secondary level (final year of high school) in Italian and mathematics seem very large. For example, the reported fall in the performance of year 6 students in 2020 relative to students in previous years in Dutch language (-0.3 SD) in Flanders represents the equivalent of between 50%-75% of the estimated average achievement gain in a normal school year depending on the benchmark used. The decline in scores in Italian among Year 13 students represents between 40% and 60% of “normal” annual gain.
In particular, the scale of the declines found in Flanders seem implausible. They imply that the substitution of remote schooling for school-based instruction for a period of seven weeks7 (around 20% of yearly instruction time) meant that the improvement in achievement of Year 5 students in Dutch was between 25%‑50% of what it would have been in a “normal” year. At the same time, the estimates of a performance decline equivalent to around two to three months of usual annual learning gain found in the Dutch and English studies suggest that, at best, students maintained the level of performance they had achieved when their schools closed. This again seems surprising given that most students continued some form of school learning from home during school closures.
The results that suggest no impact of the disruption to schooling on performance also raise questions. In particular, they stand in contrast with the evidence that even if most pupils continued with their education, they spent less time, on average, in learning activities than they would have done in a normal year.
An important issue that is not addressed in any detail by the studies reviewed is that of the variation observed in the results of testing programmes over time prior to 2020. Placing the results of COVID cohorts in the context of longer run trends (where they exist) is instructive (see Figures 4.7 and 4.8). As can be seen, the magnitude of the changes in performance observed between 2019 and 2020 in Baden-Württemberg and in France is within the range of what has been observed in the recent past. Importantly, falls as well as increases in performance are also observed in previous years.8 The main point to be drawn from this is that care should be exercised in making causal inferences regarding the effect of school closures on academic performance on the basis of observed changes in performance between 2020 and 2019 alone.
Figure 4.7. Mean scores in reading and mathematics: Year 5 pupils, Baden-Württemberg, 2015-2020
Source: (Schult et al., 2021[23], Table 2).
Figure 4.8. Proportion of Year 6 pupils with satisfactory or better mastery of French and Mathematics: France, 2017-2020
Source: (Direction de l’évaluation, de la prospective et de la performance (DEPP), 2020[24], Figures 3.3.1 and 3.3.2).
Evidence regarding the differential impact of the disruptions to education caused by school closures by socio‑economic background is also mixed. Several studies find little evidence of change in the performance gaps related to social background. Others find increases9 and in a few cases declines (for some year groups in some subjects). In countries in which results are available for pupils in several year levels in different subjects, the extent and direction of change in performance gaps related to socio-economic background can vary by year and subject (e.g. France and Italy).
The variation in the results found between studies is intriguing and its explanation is beyond the scope of this report. The experience of lockdowns and school closures may have varied considerably between countries and regions depending on the severity of restrictions and the nature of teaching and support for learning provided to schoolchildren. Apart from issues of sampling, the timing of testing and missing data mentioned above, the conditions in which the tests were administered is also relevant. Assessments administered during March-June 2020 were conducted in conditions of considerable disruption to normal schooling arrangements. This was less true of tests administered in September at the start of the 2021-22 school year (especially in Europe). The extent to which tests conducted later in the 2021-22 school year took place in relatively “normal” conditions depends on the country concerned. In France, for example, schools remained open over the 2021-22 school year whereas in other countries (e.g. Italy) further episodes of closures occurred or schools remained closed for much of the year (e.g. many States in the United States). In addition, the extent to which the tests evaluate knowledge directly related to the content of the curriculum may differ. For example, the German and French assessments are primarily diagnostic in focus rather than intended to evaluate what had been learnt in the previous year. It is possible that performance on tests that focus on more “generic” content and procedural knowledge are less affected than those focusing on specific curriculum content by the changes to the mode and content of instruction during school closures. Closures, for example, may have meant that pupils covered some of the content tested in assessments designed to evaluate the mastery of curricular knowledge and skills either incompletely or not at all. The risk of this is far less in assessments designed to assess more general or generic skills and knowledge.
Table 4.3. Methodological features of comparisons of the academic performance of COVID cohorts with pre-COVID cohorts
Table 4.4. Academic performance of COVID cohorts compared to peers in previous years
Note: (ns) not statistically significant.
* The French language and maths tests cover several domains that are reported separately. The threshold level is that of “satisfactory performance” or higher (“maîtrise satisfaisante” or “très bonne maîtrise”).
Sources: Australia: (Gore et al., 2021[25]); England: (Rose et al., 2021[26]), (Renaissance Learning, Education Policy Institute, 2021[27]); Flanders: (Maldonado and De Witte, 2021[28]); France: (Direction de l’évaluation, de la prospective et de la performance (DEPP), 2020[29]; 2020[24]; 2021[30]); Germany: (Schult et al., 2021[23]); Italy: (INVALSI, 2021[31]); the Netherlands: (Engzell, Frey and Verhagen, 2020[32]); United States: (Johnson and Kuhfeld, 2020[33]; Kuhfeld et al., 2020[34]; Kogan and Lavertu, 2021[35]).
In the final analysis, time will be needed before it is possible to gain a comprehensive understanding of the short- and long-run consequences of the period of school closures during the first wave of the pandemic on the achievement and broader development of students. Placing the results for 2020 in the context of longer run trends is essential for their interpretation and the next waves of testing programmes will provide vital information. For the moment, considerable caution should be exercised in attributing a causal relationship between the disruption to children’s education due to lockdowns and school closures and differences in the performance in standardised assessments of students in a given grade tested in 2020 and 2021 compared to that off students in the same grade(s) tested in previous years. Many factors can lead to variations in performance between different cohorts at the same point in their schooling: different educational experiences, variation in their demographic characteristics and social composition, measurement errors (including variation in test content and test administration between years), and in the case of sample studies, sampling errors. Quite large variations in the performance of different cohorts in the same jurisdiction are often observed in “normal” conditions in standardised testing programmes10 without any obvious explanation. Adjustments can be made to account for some of these factors in analysis, but not for others.
Summary
Overall, the period of school closures and wider lockdowns appears to have had some negative effects on the psychological well-being of school students. It was the source of inconveniences, constraints, difficulties and stresses additional to those experienced by children in the normal course of life. The majority of children reported a negative appreciation of the period of lockdown and school closures, particularly the lack of social contacts with friends. The share of school age children experiencing serious or severe symptoms of mental or psychological disorders appears to have risen during the period of lockdown. However, the proportion of children concerned was relatively small. Parents had an overall more positive view of the effects of the lockdowns on their children.
Drawing conclusions regarding the effect of the period of school closures and remote schooling on learning and academic progress is relatively difficult at this point. Among parents, satisfaction with the efforts made by schools and teachers during the period of school closures was balanced by concerns regarding their children’s educational progress and, in some cases, on their broader social development. The evidence from achievement tests is mixed. Improvement, as well as stability and decline in the performance of the “COVID cohorts” relative to their peers tested in previous years is observed. Evidence regarding the differential impact of the disruptions to education caused by school closures by socio-economic background is also mixed, with some studies finding performance gaps increased and others finding that gaps remained stable or, in a few cases, were reduced. In some countries, the scale and direction of changes in the associations between test performance and social origin varied by year level and subject.
References
[22] Avvisati, F. and P. Givord (2021), “The learning gain over one school year among 15-year-olds: An analysis of PISA data for Austria and Scotland (United Kingdom)”, OECD Education Working Papers, No. 249, OECD Publishing, Paris, https://dx.doi.org/10.1787/d99e8c0a-en.
[17] Brenan, M. (2020), 42% of Parents Worry COVID-19 Will Affect Child’s Education, Gallup Panel, 24-29 March 2020, https://news.gallup.com/poll/305819/parents-worry-covid-affect-child-education.aspx.
[10] Central Bureau of Statistics (2020), Civilian Resilience in Israel and the COVID-19 Pandemic: Analysis of a CBS Survey, INSS Insight No. 1318, https://www.inss.org.il/publication/coronavirus-survey/.
[14] Central Statistics Office (CSO) (2020), Social Impact of COVID-19 Survey: The Reopening of Schools, CSO statistical publication, 27 August 2020, Ireland, https://www.cso.ie/en/releasesandpublications/ep/p-sic19ros/socialimpactofcovid-19surveyaugust2020thereopeningofschools/.
[39] Curriculum Associates (2020), Understanding Student Needs: Early Results from Fall Assessments, Research Report No. 2020-49, Curriculum Associates, North Billerica, MA, https://www.curriculumassociates.com/-/media/mainsite/files/i-ready/iready-diagnostic-results-understanding-student-needs-paper-2020.pdf.
[12] D21 Initiative/TUM/Kantar (2020), Erfolgreiches Homeschooling während Corona, https://www.kantar.com/de/inspiration/d21/erfolgreiches-homeschooling-waehrend-corona.
[5] Direction de l’évaluation, de la prospective et de la performance (DEPP) (2021), Dispositif d’évaluation des conséquences de la crise sanitaire : comment les élèves ont-ils vécu le confinement de mars-avril 2020 ?, Note d’information n°21.19 – Avril, Ministère de l’Education nationale, de la Jeunesse et des Sports, https://www.education.gouv.fr/dispositif-d-evaluation-des-consequences-de-la-crise-sanitaire-comment-les-eleves-ont-ils-vecu-le-322830.
[21] Direction de l’évaluation, de la prospective et de la performance (DEPP) (2021), Dispositif d’évaluation des conséquences de la crise sanitaire : le point de vue des directeurs et directrices d’école, Note d’Information n°21.04, Ministère de l’Education nationale, de la Jeunesse et des Sports, https://www.education.gouv.fr/dispositif-d-evaluation-des-consequences-de-la-crise-sanitaire-le-point-de-vue-des-directeurs-et-309164.
[30] Direction de l’évaluation, de la prospective et de la performance (DEPP) (2021), “Évaluations 2021 Point d’étape CP : Premiers résultats”, Série Études, No. 2021-E03, mars 2021, Ministère de l’Education nationale, de la Jeunesse et des Sports, Paris, https://www.education.gouv.fr/evaluations-2021-point-d-etape-cp-premiers-resultats-322673.
[19] Direction de l’évaluation, de la prospective et de la performance (DEPP) (2020), Confinement : un investissement scolaire important des élèves du second degré, essentiellement différencié selon leur niveau scolaire, Note d’information n°20.42, Ministère de l’Éducation nationale, de la Jeunesse et des Sports, https://www.education.gouv.fr/confinement-un-investissement-scolaire-important-des-eleves-du-second-degre-essentiellement-307441.
[29] Direction de l’évaluation, de la prospective et de la performance (DEPP) (2020), “Evaluations 2020 Repères CP, CE1 : Premiers résultats”, Série Études, No. 2020-E04, novembre 2020, Ministère de l’Education nationale, de la Jeunesse et des Sports, Paris, https://www.education.gouv.fr/evaluations-2020-reperes-cp-ce1-premiers-resultats-307122.
[24] Direction de l’évaluation, de la prospective et de la performance (DEPP) (2020), “Évaluations de début de sixième 2020 : Premiers résultats”, Série Études, No. 2020-E05, novembre 2020, Ministère de l’Education nationale, de la Jeunesse et des Sports, Paris, https://www.education.gouv.fr/evaluations-de-debut-de-sixieme-2020-premiers-resultats-307125.
[2] Direction de l’évaluation de la prospective et de la performance (DEPP) (2020), Continuité pédagogique - période de mars à mai 2020 - enquêtes de la DEPP auprès des familles et des personnels de l’Éducation nationale – premiers résultats, Document de travail n°2020-E03, Ministère de l’Education nationale, de la Jeunesse et des Sports, https://www.education.gouv.fr/continuite-pedagogique-periode-de-mars-mai-2020-enquetes-de-la-depp-aupres-des-familles-et-des-305262.
[38] Education Endowment Foundation (2021), Best evidence on impact of Covid-19 on pupil attainment: Research examining the potential impact of school closures on the attainment gap, https://educationendowmentfoundation.org.uk/eef-support-for-schools/covid-19-resources/best-evidence-on-impact-of-school-closures-on-the-attainment-gap/.
[32] Engzell, P., A. Frey and M. Verhagen (2020), Learning Loss Due to School Closures During the COVID-19 Pandemic, Center for Open Science, http://dx.doi.org/10.31235/osf.io/ve4z7.
[9] Gadermann, A., K. Thomson, C. Richardson, M. Gagné, C. McAuliffe, S. Hirani and E. Jenkins (2021), “Examining the impacts of the COVID-19 pandemic on family mental health in Canada: findings from a national cross-sectional study”, BMJ Open, Vol. 11/1, p. e042871, http://dx.doi.org/10.1136/bmjopen-2020-042871.
[25] Gore, J., L. Fray, A. Miller, J. Harris and W. Taggart (2021), “The impact of COVID-19 on student learning in New South Wales primary schools: an empirical study”, The Australian Educational Researcher, Vol. 48/4, pp. 605-637, http://dx.doi.org/10.1007/s13384-021-00436-w.
[15] Horowitz, J. (2020), Lower-income parents most concerned about their children falling behind amid COVID-19 school closures, Fact Tank April 15, 2020, Pew Research Centre, https://www.pewresearch.org/fact-tank/2020/04/15/lower-income-parents-most-concerned-about-their-children-falling-behind-amid-covid-19-school-closures/.
[31] INVALSI (2021), I Risultati delle Prove INVALSI 2021, https://www.invalsiopen.it/risultati/risultati-prove-invalsi-2021/.
[33] Johnson, A. and M. Kuhfeld (2020), Fall 2019 to fall 2020 MAP Growth attrition analysis. Technical Brief, NWEA, https://www.nwea.org/content/uploads/2020/11/Technical-brief-Fall-2019-to-fall-2020-MAP-Growth-attrition-analysis-NOV2020.pdf.
[18] Jones, J. (2020), Social Factors Most Challenging in COVID-19 Distance Learning, Gallup Panel, 11-24 May 2020, https://news.gallup.com/poll/312566/social-factors-challenging-covid-distance-learning.aspx.
[4] Jones, J. (2020), Amid School Closures, Children Feeling Happiness, Boredom, Gallup Panel, 25 May-8 June 2020, https://news.gallup.com/poll/306140/amid-school-closures-children-feeling-happiness-boredom.aspx.
[35] Kogan, V. and S. Lavertu (2021), The COVID-19 Pandemic and Student Achievement on Ohio’s Third-Grade English Language Arts Assessment, http://glenn.osu.edu/educational-governance/reports/reports-attributes/ODE_ThirdGradeELA_KL_1-27-2021.pdf.
[34] Kuhfeld, M., E. Ruzek, A. Johnson, B. Tarasawa and K. Lewis (2020), Technical appendix for: Learning during COVID-19: Initial findings on students’ reading and math achievement and growth, NWEA, https://www.nwea.org/content/uploads/2020/11/Technical-brief-Technical-appendix-for-Learning-during-COVID-19-Initial-findings-on-students-reading-and-math-achievement-and-growth-NOV2020.pdf.
[6] Luijten, M., M. van Muilekom, L. Teela, T. Polderman, C. Terwee, J. Zijlmans, L. Klaufus, A. Popma, K. Oostrom, H. van Oers and L. Haverman (2021), “The impact of lockdown during the COVID-19 pandemic on mental and social health of children and adolescents”, Quality of Life Research, http://dx.doi.org/10.1007/s11136-021-02861-x.
[28] Maldonado, J. and K. De Witte (2021), “The effect of school closures on standardised student test outcomes”, British Educational Research Journal, http://dx.doi.org/10.1002/berj.3754.
[20] Marken, S. and T. Clayton (2020), “Parents’ and Students’ Thoughts on Support Needed This Fall”, Gallup Blog August 27, 2020, https://news.gallup.com/opinion/gallup/317957/parents-students-thoughts-support-needed-fall.aspx.
[8] NHS Digital (2020), Mental Health of Children and Young People in England, 2020 - Wave 1 follow up to the 2017 survey, Main Report, https://files.digital.nhs.uk/AF/AECD6B/mhcyp_2020_rep_v2.pdf.
[1] NHS Digital (2020), Mental Health of Children and Young People in England, 2020: Wave 1 follow up to the 2017 survey, https://digital.nhs.uk/data-and-information/publications/statistical/mental-health-of-children-and-young-people-in-england/2020-wave-1-follow-up/data-sets.
[11] Office for National Statistics (ONS) (2020), Coronavirus and homeschooling in Great Britain: April to June 2020. Analysis of homeschooling in Great Britain during the coronavirus (COVID-19) pandemic from the Opinions and Lifestyle Survey, https://www.ons.gov.uk/peoplepopulationandcommunity/educationandchildcare/articles/coronavirusandhomeschoolingingreatbritain/apriltojune2020.
[7] Ravens-Sieberer, U., A. Kaman, M. Erhart, J. Devine, R. Schlack and C. Otto (2021), “Impact of the COVID-19 pandemic on quality of life and mental health in children and adolescents in Germany”, European Child & Adolescent Psychiatry, http://dx.doi.org/10.1007/s00787-021-01726-5.
[27] Renaissance Learning, Education Policy Institute (2021), Understanding Progress in the 2020/21 Academic Year, Complete findings from the Autumn term: June 2021, Department for Education, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/994350/Understanding_Progress_in_the_2020_21_Academic_Year_Report_2.pdf.
[26] Rose, S., L. Twist, P. Lord, S. Rutt, K. Badr, C. Hope and B. Styles (2021), Impact of school closures and subsequent support strategies on attainment and socio-emotional well-being in Key Stage 1: Interim Paper 1, National Foundation for Educational Research (NFER), Slough, https://educationendowmentfoundation.org.uk/public/files/Publications/Covid-19_Resources/Impact_of_school_closures_KS1_interim_findings_paper_-_Jan_2021.pdf.
[23] Schult, J., N. Mahler, B. Fauth and M.A. Lindner (2021), Did Students Learn Less During the COVID-19 Pandemic? Reading and Mathematics Competencies Before and After the First Pandemic Wave, Center for Open Science, http://dx.doi.org/10.31234/osf.io/pqtgf.
[3] Thierry, X., B. Geay, A. Pailhé, N. Berthomier, J. Camus, N. Cauchi-Duval, J-L. Lanoë, S. Octobre, J. Pagis, L. Panico, T. Siméon, A. Solaz et l’équipe SAPRIS (2021), Les enfants à l’épreuve du premier confinement, Institut national d’études démographiques (Ined), https://www.ined.fr/fichier/s_rubrique/31037/585.enfants.confinement.population.societes.janvier.2021.fr.pdf.
[37] Tomasik, M., L. Helbling and U. Moser (2020), “Educational gains of in‐person vs. distance learning in primary and secondary schools: A natural experiment during the COVID ‐19 pandemic school closures in Switzerland”, International Journal of Psychology, Vol. 56/4, pp. 566-576, http://dx.doi.org/10.1002/ijop.12728.
[16] University of Southern California (USC) (2020), Understanding Coronavirus in America Tracking Survey – Methodology and Select Crosstab Results UAS 242, Wave 4 April 29-May 26, 2020, University of Southern California, Dornsife Center for Economic and Social Research, https://uasdata.usc.edu/index.php.
[36] Viner, R., S. Russell, R. Saulle, H. Croker, C. Stansfeld, J. Packer, D. Nicholls, A-L. Goddings, C. Bonell, L. Hudson, S. Hope, N. Schwalbe, A. Morgan, S. Minozzi (2021), Impacts of school closures on physical and mental health of children and young people: a systematic review, Cold Spring Harbor Laboratory, http://dx.doi.org/10.1101/2021.02.10.21251526.
[13] Wößmann, L., V. Freundl, E. Grewenig, P. Lergetporer, K. Werner and L. Zierow (2020), “Bildung in der Coronakrise: Wie haben die Schulkinder die Zeit der Schulschließungen verbracht, und welche Bildungsmaßnahmen befürworten die Deutschen?”, ifo Schnelldienst, Vol. 73/9, pp. 25-39, https://www.ifo.de/publikationen/2020/aufsatz-zeitschrift/bildung-der-coronakrise-wie-haben-die-schulkinder-die-zeit.
Notes
← 1. See Viner et al. (2021[36]) for a review of studies on this topic. Most of the studies reviewed covering the incidence of mental health symptoms among children during lockdowns were based on convenience samples. Only one “high quality” study based on a representative sample is cited. However, even this study provides no information on its sampling strategy. For this reason, its findings are not reported in this report.
← 2. An individual’s perception and subjective evaluation of their health and well-being.
← 3. Three quarters (76%) of parents were satisfied with the communication to support learning from their child(ren)’s school(s) (University of Southern California (USC), 2020[16]). High proportions of parents rated their child(ren)’s school as doing an excellent or good job in terms of teachers availability to answer questions (77%), communication about the distance education programme from the superintendent and/or principal (71%), provision of materials and equipment needed for the child to do schoolwork (75%) and communication about specific assignments from teachers (72%) as doing an excellent or good job in terms of teachers availability to answer questions (77%), communication about the distance education programme from the superintendent and/or principal (71%), provision of materials and equipment needed for the child to do schoolwork (75%) and communication about specific assignments from teachers (72%) (Jones, 2020[18]).
← 4. Other studies have been published [see Tomasik, Helbling and Moser (2020[37]), Curriculum Associates (2020[39]), and the list provided in Education Endowment Foundation (2021[38])]. Many of these studies do not provide detailed information on study samples and methods and their results have not been reported included for this reason.
← 5. The effects of selection biases and non-response on the representativeness of the results are argued to be negligible by the authors of all the studies concerned.
← 6. See, also, the discussion in Engzell, Frey and Verhagen (2020[32]). For Kogan and Lavertu (2021[35]), average year-to-year student achievement gains in reading between second and third grade are approximately 0.6 standard deviations.
← 7. Nine weeks of the normal school year including two weeks of holidays over Easter.
← 8. Falls are also observed in the Dutch data set. For example, a fall between mid- and end-year performance in reading language and spelling was observed in 2017 (Engzell, Frey and Verhagen, 2020[32], Table A6).
← 9. The evidence of the NFER study in England (Rose et al., 2021, p. 10[26]) is particularly unconvincing. The 2017 comparison sample “does not provide data on the performance of disadvantaged and non‑disadvantaged pupils”. The authors, instead, compare the standardised achievement gap observed among the 2020 sample with that derived from another assessment carried out in 2019 to estimate whether the gap has grown.
← 10. See, for example, the results reported for Baden-Württemberg by Schult et al. (2021[23]). See also results for studies such as PISA (Programme for International Student Assessment), TIMMS (Trends in International Mathematics and Science Study) and PIRLS (Progress in International Reading Literacy Study).