Benchmarking Higher Education System Performance

Reader’s Guide

Statistical coverage

The statistics reported in this publication cover the entire respective national higher education system, including higher education research and development, within the national or jurisdictional territory and regardless of ownership, sponsorship and mode of delivery, except when differently specified. All higher education students, graduates, staff and programmes are included, following internationally agreed definitions (UNESCO Institute for Statistics, OECD and Eurostat, 2018[1]; OECD, 2018[2]; OECD, 2015[3]). Deviations from this general rule are reported in the text or notes within this publication.

Country and jurisdiction coverage

The indicators in this publication cover all OECD countries for which data is available, and in some cases subnational units when data are specifically available at that level (for example, England (United Kingdom) or the French Community of Belgium). The policy analysis carried out in this publication focuses primarily on the four jurisdictions that participated in the 2017-2018 Benchmarking Higher Education System Performance exercise. These four jurisdictions are Estonia, the Flemish Community of Belgium, the Netherlands and Norway, and are referred to as the “participating jurisdictions” throughout the report. Policies from other jurisdictions are discussed throughout the report when relevant.

As the Flemish Community of Belgium is a participating jurisdiction in the benchmarking exercise, data have been included for the jurisdiction wherever possible. Data sources for the Flemish Community of Belgium include OECD Regional Statistics, and a special data collection conducted for the benchmarking exercise in collaboration with the Flemish Ministry for Education and Training. The Flemish Community of Belgium is referred to throughout as “The Flemish Community”. In some cases, data are reported for the region of Flanders; this is specified within the text.

Use of the term “higher education” in this report

The term “higher education” in this publication is equivalent to the term “tertiary education”, as defined in the ISCED 2011 classification (UNESCO Institute for Statistics, 2012[4]): “Tertiary education builds on secondary education, providing learning activities in specialised fields of education. It aims at learning at a high level of complexity and specialisation. Tertiary education includes what is commonly understood as academic education but also includes advanced vocational or professional education”. This comprises the short-cycle, bachelor’s, master’s or doctoral levels of education (Table 1). The term “higher education” is used throughout this report rather than “tertiary education” due to its wider use in academic and policy literature.

Table 1. Higher education levels in the ISCED 2011 classification

Label (as used in the publication)	Complete name and description
Short-cycle programmes	Short-cycle tertiary education (ISCED level 5): Programmes at ISCED level 5 aim to provide professional knowledge, skills and competencies. Typically, they are practically based, occupationally specific and prepare students to enter the labour market, but may also provide a pathway to other higher education programmes. Academic higher education programmes below the bachelor’s level are also classified as ISCED level 5. Programmes classified at ISCED level 5 may be referred to as (higher) technical education, community college education, technician or advanced/higher vocational training, an associate degree, or the bac+2.
Bachelor's programmes	Bachelor’s or equivalent level (ISCED level 6): Programmes at ISCED level 6 aim to provide intermediate academic and/or professional knowledge, skills and competencies, leading to a first degree or equivalent qualification. Programmes are typically theoretically based, but may include practical components and are informed by research and/or best professional practice. Programmes at this level do not necessarily involve the completion of a research project or thesis, but if they do, it is less advanced, less independent or is undertaken with more guidance than those at ISCED level 7 or 8. Programmes classified at ISCED level 6 may be referred to as a bachelor’s programme, a licence, or the first university cycle.
Master's programmes	Master’s or equivalent level (ISCED level 7): Programmes at ISCED level 7 are designed to provide advanced academic and/or professional knowledge, skills and competencies, leading to a second degree or equivalent qualification. Typically, programmes at this level are theoretically based, but may include practical components and are informed by state-of-the-art research and/or best professional practice. Programmes at this level may involve the completion of a research project or thesis that is more advanced than those expected at ISCED level 6 and less advanced than those expected at ISCED level 8. Master’s programmes can be also entirely coursework-based in some countries, or there may be a differentiation between a coursework programme and a research programme. Programmes classified at ISCED level 7 may be referred to in many ways, for example: master’s programmes, magister, or MPhil.
Doctoral programmes	Doctoral or equivalent level (ISCED level 8): Programmes at ISCED level 8 lead to an advanced research qualification. Programmes at this ISCED level are devoted to advanced study and original research, and are typically offered only by research-oriented higher education institutions, such as universities. Doctoral programmes exist in both academic and professional fields, and usually conclude with the submission and defence of a thesis, dissertation or equivalent written work of publishable quality, representing a significant contribution to knowledge in the respective field of study. In some education systems, ISCED level 8 programmes contain very limited course work, or none at all, and individuals working towards a doctoral degree engage in research mostly independently or in small groups with varying degrees of supervision. Other countries require the completion of coursework before the doctoral candidates can progress to the thesis component of the programme (see Chapter 6). Programmes classified at ISCED level 8 may be referred to in many ways, for example: PhD, DPhil, D.Lit, D.Sc, LL.D, Doctorate or similar terms.

Note: Descriptions are taken from the UNESCO Institute for Statistics (2012[4]). Short-cycle programmes at the ISCED 5 level are not recognised as part of the higher education system in Norway and are offered through vocational colleges. Norway offers a two-year programme at ISCED 6 level (høgskolekandidatgrad) and students who successfully complete the two-year programme can enter into the third year of a bachelor’s programme in the same field.

Calculation of the averages

Unless otherwise specified in the text, the averages presented in the charts and tables of this publication are the unweighted arithmetic averages across the OECD jurisdictions with available data, following the rules outlined in Table 2.

Table 2. Rules used for the calculation of averages

Jurisdictions used for the calculation	All jurisdictions with available data on all of the series presented in a chart are used to calculate the average. There are some exceptions to this general rule, reported within this table.
Calculation of averages of indicators by level of higher education	When indicators are broken down by higher education level, the average for the bachelor’s, master’s and doctoral levels includes all jurisdictions with available data for all of the series presented in the chart, except for the series related to the short-cycle level. The average for the short-cycle level is calculated separately, for all jurisdictions with available data for this level of education. This choice has been made because short-cycle programmes do not exist in a number of OECD jurisdictions.
Exclusion of Flemish data	Whenever data are available for both Belgium and the Flemish Community (or the Region of Flanders), the latter is excluded from the calculation of the average.
Non-applicable data	In some instances, data are “not applicable” for a jurisdiction. For example: if short-cycle programmes do not exist in a jurisdiction, enrolment at the short-cycle level is not applicable; if a public student loan scheme does not exist in a jurisdiction, then the amount of money spent on loans is not applicable. In the calculation of indicators, non-applicable data is treated as zero (e.g. zero students enrolled in short-cycle programmes and zero dollars spent on loans). When data are not applicable both at the numerator and the denominator of an indicator (e.g. proportion of international students at the short-cycle level over total enrolment at the short-cycle level), then the data are treated as missing in the calculation of the average.

Data sources

The majority of the indicators in this publication come from OECD data collections, for example the joint UNESCO-OECD-Eurostat (UOE) data collection, the OECD Indicators of Education Systems (INES) data collection, the Survey of Adult Skills, or the OECD Career of Doctorate Holders Survey. When possible, OECD data have been extracted from the OECD Education Statistics (OECD, 2018[5]) or from the OECD Science, Technology and R&D Statistics (OECD, 2018[6]) databases. In the other cases, the data collection is indicated as the data source.

Other data sources, from outside the OECD, have been used for selected indicators within the publication. For example, some indicators on financial and human resources are based on the European Register for Tertiary Education (ETER) dataset; and data from the World Economic Forum and the European Community Innovation Survey have been used to present indicators on higher education engagement.

In addition, a survey was issued to the four participating jurisdictions to collect data on a variety of topics, including a number of statistics broken down by subsector (universities and professional higher education institutions). The survey results are published in a number of tables within the publication. In these cases the source is stated as “adapted from data/information provided by the participating jurisdictions”.

Data updates

This publication makes use of the most recent available data at the time of its preparation. Data released after 31 December 2018 have not been included in the analysis, except for the data on human resources in Chapter 6, which were released in early 2019, in order to standardise as much as possible the reference years used in Chapter 6.

A note on the statistical collaboration with LinkedIn

Box 5.10 was produced in collaboration with LinkedIn, a platform for professional networking. These data cover self-reported information on professional and educational experiences; and information on individual skills, either self-reported or reported by other individuals on the professional platform.

LinkedIn staff extracted the data on request of the OECD. The data provided by LinkedIn cover around 2 710 000 members who indicated that they earned their first master’s degree between 2010 and 2013 in eight jurisdictions (Australia, Canada, Estonia, the Flemish Community, France, the Netherlands, Norway and the United States). By comparison, the OECD estimated the number of first-time master’s graduates covering the same period and jurisdictions to be around 5 000 000 (based on data returned by jurisdictions in UNESCO-OECD-Eurostat (UOE) data collection). Graduates who reported over seven educational and professional experiences in the five years after graduating (1.5% of the total) were excluded from the analysis.

To check the robustness of the results, the same data extraction and calculations have been performed for both first-time bachelor’s and master’s graduates. In addition, the extraction of data on interpersonal skills has been performed based on two different skill lists: LinkedIn’s own list; and a list of skills closely matching (as agreed by the OECD and LinkedIn) the list of keywords on intrapersonal, interpersonal and problem-solving skills provided by (Binkley et al., 2005[7]). The conclusions discussed in Box 5.10 hold for all variations of the analysis carried out.

Sources of qualitative information

A substantial amount of qualitative information has been collected to prepare this publication. The main sources of this information are:

• documents sent by the participating jurisdictions (one per jurisdiction) describing their higher education systems and policies

• discussions between the OECD and the participating jurisdictions’ project coordinators held during six workshops between February 2017 and November 2018

• other meetings and webinars with the participating jurisdictions’ project coordinators and national experts on higher education policies or statistics.

Throughout the publication, the information gathered from these sources is referred to as “adapted from information provided by the participating jurisdictions”.

The publication also makes use of structured qualitative data on university autonomy in Europe from the European University Association (EUA) (Bennetot Pruvot and Estermann, 2017[8]); and on higher education academic staff categories from Eurydice (European Commission, EACEA and Eurydice, 2017[9]). Both organisations (EUA and Eurydice) gave permission to the OECD to use their qualitative data collection for additional data collection or validation. For example, qualitative data on the autonomy of professional HEIs and independent private institutions were collected by the OECD through interviews of representatives of these institutions or government officials, based on the EUA tool.

Symbols for missing data and abbreviations

The following symbols and abbreviations are used to convey statistical information in the linked files to the figures presented (statlinks) throughout this publication:

b There is a break in the time series, implying that comparisons across time should be made with caution

c There are too few observations to provide reliable estimates

d Difference in methodology

e Estimated value

m Data are not available (missing)

p Provisional value

q Data have been withdrawn at the request of the country concerned.

r Values are below a certain reliability threshold and should be interpreted with caution

w The indicator is overestimated because it includes data from another category

x Data are included in another category or column within the table

z Data are not applicable because the category does not apply

References

[8] Bennetot Pruvot, E. and T. Estermann (2017), University Autonomy in Europe III The Scorecard 2017, European University Association, Brussels, http://www.eua.be/Libraries/publications/University-Autonomy-in-Europe-2017.

[7] Binkley, M. et al. (2005), “Moving Towards Measurement: the Overarching Conceptual Framework for the ALL Study”, in Murray, T., Y. Clermont and M. Binkley (eds.), Measuring Adult Literacy and Life Skills: New Frameworks for Assessment, Statistics Canada, Ottawa.

[9] European Commission, EACEA and Eurydice (2017), Modernisation of Higher Education in Europe - Academic Staff 2017, Publications Office of the European Union, Luxembourg, https://publications.europa.eu/en/publication-detail/-/publication/40f84414-683f-11e7-b2f2-01aa75ed71a1/language-en (accessed on 18 May 2018).

[5] OECD (2018), OECD Education Statistics, http://dx.doi.org/10.1787/edu-data-en (accessed on 20 December 2018).

[2] OECD (2018), OECD Handbook for Internationally Comparative Education Statistics 2018: Concepts, Standards, Definitions and Classifications, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264304444-en.

[6] OECD (2018), OECD Science, Technology and R&D Statistics, https://doi.org/10.1787/strd-data-en (accessed on 20 December 2018).

[3] OECD (2015), Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264239012-en.

[4] UNESCO Institute for Statistics (2012), International Standard Classification of Education ISCED 2011, UNESCO Institute for Statistics, Montreal, http://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf.

[1] UNESCO Institute for Statistics, OECD and Eurostat (2018), UOE Data Collection on Formal Education - Manual on Concepts, Definitions and Classifications, UNESCO-UIS, Montreal; OECD, Paris; , Luxembourg, http://uis.unesco.org/en/files/uoe-data-collection-manual-2018-en-pdf-0.