Empowering Young Children in the Digital Age

Annex A. Technical annex

Data informing the Empowering Young Children in the Digital Age report and its supplementary outputs (online tables, country notes and case study compendium) were derived from two main sources, developed specifically for the Early Childhood Education and Care in a Digital World project carried out throughout 2021 and 2022:

1. A policy survey prepared by the OECD Secretariat and distributed to the OECD’s Early Childhood Education and Care (ECEC) Network and some non-member countries.

2. A collection of case studies building on a template prepared by the OECD Secretariat and distributed to the OECD’s ECEC Network.

The ECEC in a Digital World policy survey

Design and administration of the questionnaire

In the early stages of the project, the OECD’s ECEC Network was consulted about the structure of the survey questionnaire and a draft instrument was circulated for review in December 2021. Building on the feedback received from the ECEC Network, the Secretariat prepared a final version of the questionnaire, including both multiple-choice and open-ended questions, which was then administered online to members of the OECD’s ECEC Network and some non-member countries between February and April 2022. Thirty-seven responses were received from 26 countries for the reference year 2022 (Table A A.1).

Guided by the project’s analytical framework, the substantive sections of the survey covered the following areas:

• Policy challenges, aiming to identify policy challenges in relation to the impact of digitalisation on young children generally, and ECEC more specifically.

• Governance and standards, exploring roles and responsibilities in decision making about digital technologies in ECEC and standards to protect and guide young children in their interactions with digital technology.

• Curriculum and pedagogy frameworks, investigating how ECEC curriculum and pedagogy frameworks respond to or reflect digitalisation trends.

• Workforce development, examining whether and how professional development programmes for ECEC staff address digital competencies, as well as opportunities for using digital technologies to support their professional growth.

• Engagement with families and communities, exploring whether and how digital technologies are used in ECEC settings to engage with families and communities.

• Monitoring and data, investigating how goals and practices related to digital technology are integrated into ECEC quality monitoring, as well as the availability and uses of ECEC data systems.

Table A A.1. Countries and subnational jurisdictions that responded to the policy survey

Australia1	Denmark	Morocco
South Australia	Finland	Norway
Tasmania	France	Portugal
Victoria	Germany1	Slovak Republic
Belgium2	Bavaria	Slovenia
Flemish Community	Hungary	South Africa
Canada1	Iceland	Spain
Alberta	Ireland	Sweden
British Columbia	Israel	Switzerland
Manitoba	Italy	United Arab Emirates2
New Brunswick	Japan	Dubai
Quebec	Korea
Czech Republic	Luxembourg

1. Countries that responded to the questionnaire at both national/federal level and jurisdictional level.

2. Countries that responded to the questionnaire only at jurisdictional level.

Note: Not all countries and jurisdictions responded to all questions and for all types of settings.

Following the online administration of the survey, the OECD Secretariat reviewed responses from participating countries and jurisdictions in order to clean and harmonise the data. A file with harmonised responses was circulated for validation by participating countries and jurisdictions. The final responses to the survey used for the analysis presented in this report are available in table format (see Annex B).

Scope of the policy survey

Given the goal of providing internationally comparative indicators, the Starting Strong VII policy review focused on collecting national data from all participating countries. However, for some federal countries where subnational authorities are responsible for ECEC, information was also collected from some subnational jurisdictions.

Generally, countries and jurisdictions were asked to report on their policies based on the structure of their ECEC systems, including whether the system is integrated for children aged 0-5, or split for children under age 3 and children aged 3-5/primary school entry. Regarding policy challenges, governance and standards, and monitoring and data, the questionnaire requested information with reference to country-wide or jurisdiction-wide policies. To address questions around curriculum frameworks and family and community engagement, countries and jurisdictions were asked to report with reference to specific age groups of children, as discussed below. Regarding curriculum and pedagogy frameworks, countries and jurisdictions were given the opportunity to report information with reference to both official curriculum frameworks and other documents and frameworks such as directives, statements or strategies on digitalisation with potential application to ECEC. Countries and jurisdictions could provide sets of responses for up to three relevant frameworks. Regarding engagement with families and communities, countries and jurisdictions were asked to report about settings belonging to their regulated ECEC systems, regardless of type, funding, opening hours or programme content. Regarding questions on workforce development, countries and jurisdictions were asked to report on three main categories of ECEC staff: teachers, assistants and leaders, with some questions focusing exclusively on teachers.

Further details on the scope of the policy review and the questionnaire are available in the Reader’s guide.

Application of standardised age groups to curriculum frameworks and types of settings

Building on the experience of the Starting Strong VI policy review, ECEC curriculum frameworks and types of settings were categorised by standardised age groups to facilitate the use of information, enable comparisons across age groups within and across countries and jurisdictions, and ensure consistency with the development of ECEC indicators included in other OECD databases such as Education at a Glance.

When responding to the questionnaire, countries and jurisdictions were asked to select among three standardised age groups prior to providing information about ECEC curriculum frameworks and types of settings. Standardised age groups were defined according to the following rules:

• Age 0-2: if the majority of years of a setting or curriculum target or cover ages 0-2. This includes settings or curricula which start after 0 years (e.g. 12 weeks, 3 months, etc.) and end by age 3.

• Age 3-5/primary school entry: if the majority of years of a setting or curriculum target or cover ages 3-5. This includes settings or curricula which start earlier than age 3 (e.g. 2.5 years) or later than age 3 (e.g. 4 years).

• Age 0-5/primary school entry: if a setting or curriculum targets or covers ages below and above the cut-off point of 3 years to a similar extent (e.g. 0-12 years).

These age groups were standardised by the OECD Secretariat and may not correspond exactly to the organisation of ECEC systems within specific countries and jurisdictions.

Calculation of indicators presented in the report

Weighting of countries and subnational jurisdictions

For the calculation of percentages in tables and charts, responses were weighted so that each country is equally represented; that is, by setting the total weight of each country equal to one. This aims to ensure that countries with responses for subnational jurisdictions are not over-represented in calculated statistics. When multiple responses were received from the same country, each response was given an equal weight. For example, the same weighting of 0.143 was applied to each of the seven responses from Canada in Figure 4.1, which include two Canada-level responses and five responses from provinces and territories. Thus, the total weight for all Canadian responses is the same as the total weight for other countries having submitted only one response.

Weighting of ECEC curriculum frameworks

The ECEC in a Digital World policy survey collected some data on ECEC curriculum frameworks as classified by the standardised age groups described above. In these cases, weights and percentages were calculated for each age group separately so that each country’s total weight equals one in the responses for each age category.

In some countries and jurisdictions, information was provided for more than one curriculum framework. For example, Germany provided data on two curriculum frameworks for the age group “0-5/primary school entry”. The total weight for Germany was therefore split equally between these curricula so that each had a weight of 0.5 for the given age category in the results presented in Figure 4.4.

Weighting for questions with multiple items

Several questions in the ECEC in a Digital World policy survey included multiple items, asking respondents to choose from a selection of response categories. Figure 4.1 shows an example of the results from one such question, where question items included “Preparing young children for social and political participation in the digital age” and “Promoting young children’s agency and empowerment as users of digital technologies”. For each item, respondents could report “do not know” or assign a level of importance from “low” to “very high”.

If a country or jurisdiction did not select any response category for any question item (that is, all data were missing for a question), they were excluded from the calculation of weights. For example, Australia (Victoria) was excluded from Figure 4.1 because its response to each item on policy challenges regarding digitalisation and young children and ECEC was missing (see Tables B.1 and B.2). However, countries and jurisdictions were included in the calculation of weights if they included at least one non-missing answer to a relevant item. For example, Australia (South Australia) was included in the calculation of weights for Australia in Figure 4.1.

The same weight was used for each country and jurisdiction across every item relating to the same question. Thus, Australia and Australia (Tasmania) were given a weight of 0.33 across all items, even though Australia (South Australia) had a missing response for some items.

Weighting for questions with multiple response options

In some questions of the ECEC in a Digital World policy survey, countries and jurisdictions could select multiple response categories in response to a single question item. For example, in the Canada-level response for school-based programmes, it was reported that digital devices are provided to ECEC settings by both regional and local authorities (see Table B.10).

In these cases too, the total weight assigned to responses for each country was set to one. Within each country, this total weight was equally divided between the number of jurisdictions that provided at least one answer (in any response category) to one of the relevant question items. For example, all countries’ and jurisdictions’ responses were included in the calculation of weights for Table 5.3 because they all gave at least one non-missing answer to at least one of the four relevant question items. Here, similarly, the same weight was used for each country or jurisdiction across every item relating to the same question.

Treatment of “not applicable” and “not known”, and missing responses

Information reported by countries or jurisdictions as “not applicable” or “not known” was checked against explanatory notes provided by countries and jurisdictions and sometimes recoded to enhance the comparability of information. In cases where questionnaires presented blank items (missing responses), comments provided by countries and jurisdictions were considered for the interpretation of the data.

Weighted percentages were calculated using the weights assigned to each country or jurisdiction as described above. Generally, “not applicable” and “not known” answers were included in the calculation of weighted percentages.

However, countries or jurisdictions with missing data for a question item were excluded from the calculation of weighted percentages for that item. For example, Australia (South Australia) was excluded from the total N in the calculation of weighted percentages of countries and jurisdictions identifying of “Preparing young children for social and political participation in the digital age” as a policy challenge in Figure 4.1.

Significance tests

Where appropriate, tests of statistical significance were conducted to understand whether observed differences in sampled data are likely to represent actual differences within the population. In this report, differences are labelled as statistically significant when a difference would be observed less than 5% of the time if there was actually no difference in corresponding population values (statistical significance at the 95% level). In other words, the risk of reporting a difference as significant when such difference, in fact, does not exist, is contained at 5%.

Calculation of indicators in Chapter 1

Tables 1.2 and 1.3 list countries and jurisdictions that have been identified as active on a particular policy lever and as addressing a particular policy challenge, respectively. The selection of countries and jurisdictions was informed by responses to the policy survey and case studies submitted by countries and jurisdictions, as well as by desk research by the OECD Secretariat and qualitative analysis presented throughout the report. Concerning selection on the basis of the policy survey, respondents’ answers to a selection of relevant items were computed in weighted averages to identify countries and jurisdictions scoring above a predefined threshold.

Case studies

Countries and jurisdictions participating in the policy review were invited to submit case studies about recent or ongoing policy initiatives regarding digitalisation and ECEC, using a template provided by the OECD Secretariat to enhance the consistency and comparability of the information. In total, 20 case studies were received from 16 countries and jurisdictions. The OECD Secretariat reviewed the submissions and light copy editing was performed where necessary. References to case studies in the report are based on qualitative analysis of templates completed by countries and jurisdictions.

For more information, see Annex C.

Country notes

Country notes were produced for six countries that engaged in the policy review in greater depth: Canada, Finland, Japan, Korea, Norway and Sweden. These country notes follow a standardised format and address a common set of issues but vary also in focus with the goal to explore questions deemed of particular relevance to these countries. The notes were prepared by the OECD Secretariat and reviewed by the countries and jurisdictions. The preparation of the notes followed the same methodological procedures implemented for the main report.

The format of the figures in the country notes varies slightly from the format of those in the main report, whereby the responses for the country in question are displayed in the left-hand section of the chart, for reference. In the right-hand section, all non-missing responses were included in the calculation of the share of all countries’ responses, including the country featured in the note. The calculation of shares followed the same methodology of weighted percentages as in the main report.

Additional data sources

OECD Teaching and Learning International Survey (TALIS) Starting Strong 2018

Figures 2.6, 5.7 and 6.2 well as Tables B.24-30 rely on data from TALIS Starting Strong 2018, the first large-scale international survey that focuses on the ECEC workforce. Questionnaires were administered to staff and leaders to collect data on their characteristics, practices at work and views on the ECEC sector, with an emphasis on those aspects that promote conditions for children’s learning, development and well‑being.

Nine countries participated in TALIS Starting Strong 2018: Chile, Denmark, Germany, Iceland, Israel, Japan, Korea, Norway and the Republic of Türkiye. All these countries collected data from staff and leaders in pre-primary education (ISCED level 02) settings. In addition, four of the nine countries (Denmark, Germany, Israel and Norway) collected data from staff and leaders in settings serving children under age 3. For each level of ECEC in which these countries participated, the study aimed to survey a representative sample ECEC staff and centre leaders.

For more information, see the TALIS Starting Strong 2018 Technical Report (OECD, 2019[1]).

OECD International Early Learning and Child Well-being Study (IELS)

Figure 7.3 relies on data from IELS, an international survey that assessed the skills of children at age 5 attending early childhood education centres or schools in Estonia, the United Kingdom (England) and the United States in 2018. The study aimed to identify key factors that drive or hinder the development of early learning.

IELS data in Figure 7.3 are disaggregated by various measures of children’s social and economic backgrounds. Information on parental/guardian’s education comes from the parent questionnaire, with levels of parental education classified following ISCED. Figure 7.3 reports on the highest educational level of either parent. ISCED levels 0-5 (short-cycle tertiary education and below) were categorised as “no higher education” and ISCED levels 6-8 (bachelor’s level tertiary education or higher) were categorised as “higher education”. The measure of socio-economic status was derived nationally, based on three indices: 1) highest parental occupational status of parents; 2) highest educational level of parents (in years of education according to ISCED); and 3) household income. The number of books in the home refers to the number of children’s books that parents reported as present in the home environment.

For more information, see the IELS Technical Report (OECD, 2021[2]).

OECD Programme for International Student Assessment (PISA) 2018

Figures 2.2, 2.3 and 7.2 rely on data from PISA 2018. PISA is a triennial survey of 15-year-old students around the world that assesses the extent to which they have acquired key knowledge and skills essential for full participation in social and economic life. In addition, PISA uses student questionnaires to collect information from students on various aspects of their home, family and school background, and school questionnaires to collect information from schools about various aspects of organisation and educational provision in schools. There are also optional questionnaire modules for students, including about familiarity with information and communications technology (ICT). PISA 2018 was conducted in 37 OECD countries and 42 partner countries/economies.

Results in Figure 7.2 refer to students’ socio-economic background as “advantaged” and “disadvantaged”. Measures of socio-economic background are based on the PISA index of economic, social and cultural status (ESCS). This index is based on three variables related to family background: 1) parents’ highest level of education; 2) parents’ highest occupational status; and 3) home possessions, including books in the home. A socio-economically disadvantaged (or advantaged) student is a student in the bottom (or top) quarter of the ESCS index in his or her own country.

For more information, see the PISA 2018 Technical Report (OECD, 2020[3]).

Glossary of key terms

The OECD Secretariat provided a glossary to facilitate the completion of the policy questionnaire.

Data system, also known as an information system, refers to a technology that facilitates the collection, storage and use of data. In the ECEC and education sectors, data systems typically maintain and link a range of centre/school-level or individual-level data elements collected at different points in time.

Digital competencies broadly refer to the set of knowledge, skills and values that ECEC professionals need to be able to seize the potential of digital technologies in the context of their work.

Digital content refers to any content published in computer-readable format. For the purposes of this questionnaire, digital educational material refers to digital content designed and intended to be used for educational and pedagogical purposes.

Digital service provider refers to any natural or legal person that provides products and services, electronically and at a distance. Examples of providers include online search engines, online marketplaces, Internet service providers, news providers, entertainment providers (e.g. music, movies) or social media.

Early digital literacy refers to the set of knowledge, skills and values that enable young children to confidently and autonomously play, learn, socialise, prepare for work and participate in civic action in digital environments in a way that is appropriate for their age, local language and local culture.

Digital technologies broadly comprise any product or service that can be used to create, view, distribute, modify, store, and transmit and receive information electronically in a digital form. Generally, digital technologies include computer networks (e.g. the Internet) and online services supported by these (e.g. websites, social networks, online libraries, etc.); software (e.g. programmes, apps, virtual environments, games); hardware, devices or “connected” objects (e.g. computers, mobile devices, digital whiteboards; programming or robotics kits; “smart” objects or toys with sensors); and digital content.

Early generally refers to the age group 0-6, corresponding to ages when children may be enrolled in ECEC.

ECEC teachers refers to individuals with the most responsibility for a group of children at the class- or playroom-level. They may also be called core practitioners, pedagogues, educators, pedagogical staff, pre-school, pre-primary, kindergarten or early childhood teachers. In small settings, teachers may also be head of the setting while still working with children.

ECEC assistants refers to individuals working alongside teachers/core practitioners with a group of children or class on a daily basis. Assistants usually have to meet lower qualification requirements than teachers/core practitioners, which may range from no formal requirements to, for instance, vocational education and training. This role does not exist in every country.

ECEC centre leader refers to the person in an ECEC centre with the most responsibility for administrative, managerial and/or pedagogical leadership. They may also be called the head of the ECEC centre. Centre leaders may be responsible for the monitoring of children; the supervision of other staff; contact with parents and guardians; and/or the planning, preparation and carrying out of the pedagogical work in the centre. Leaders may also spend part of their time working with children.

Young children refers to infants, toddlers and pre-schoolers aged birth to six years (or primary school entry age, if different from six).

References

[2] OECD (2021), International Early Learning and Child Well-being Study: Technical Report, OECD, Paris, https://www.oecd.org/education/school/early-learning-and-child-well-being-study/Technical%20Report_with%20covers_v1.pdf.

[3] OECD (2020), PISA 2018 Technical Report, OECD, Paris, https://www.oecd.org/pisa/data/pisa2018technicalreport.

[1] OECD (2019), TALIS Starting Strong 2018 Technical Report, OECD, Paris, https://www.oecd.org/education/talis/TALIS-Starting-Strong-2018-Technical-Report.pdf.