Ensuring Quality Digital Higher Education in Hungary

Executive summary

While accredited distance learning programmes represent only a very small share of the total number of higher education programmes on offer in Hungary today (0.004% in 2021), the COVID-19 pandemic prompted many higher education institutions (HEIs) to rapidly develop their digital course offerings. This has happened outside of existing regulation on study formats and programme accreditation, with public authorities granting exceptional approval to authorise their initiatives. As part of wider efforts to support a modernisation of teaching and learning in general, the Hungarian government is committed to supporting a further expansion of digital higher education in Hungary and introduce measures to assure its quality.

Based on an analysis of existing policies and practices for the quality assurance of digital higher education in Hungary, and drawing on international best practice, this report presents nine recommendations – and within those, a range of policy options – across three areas for the Hungarian Ministry of Culture and Innovation (KIM) and the Hungarian Accreditation Committee (MAB), in consultation with HEIs, to consider.

• Area 1: Modernisation of regulation and external quality assurance to increase flexibility, innovation and digitalisation. The first area includes recommendations and policy options for the adoption of new quality standards as a basis for government policymaking, as well as a revision of the existing regulation on study formats. They seek to give institutions greater flexibility to develop innovative (and digital) study programmes (including micro-credentials) that permit students to more flexibly choose when, where, and how to study, and for academic instructors to make fuller use of the potential of digital technology to enhance the quality of teaching and assessment.

• Area 2: Reorientation of accreditation processes to strengthen institutional responsibility for quality. The second area includes recommendations and policy options for a reorientation of the existing accreditation processes for higher education. They seek to support Hungary to move from an ex ante (or input-oriented) to an ex post (or process and output-oriented) accreditation system that places enhanced responsibility and accountability with HEIs for assuring the quality of their (digital) education offerings.

• Area 3: Strengthening institutional supports for the quality enhancement of digital teaching and learning. The third area includes recommendations and policy options on how the Hungarian government and other key higher education stakeholder organisations can provide institutions, instructors, and support staff with additional supports and incentives to take up their enhanced responsibilities for quality and fully capitalise on the opportunities offered by the revised regulatory framework for higher education to expand study flexibility and digital delivery.

Co-ordinated and continued action across all three areas will be needed in the years ahead to support a deep modernisation of teaching and learning in Hungarian higher education. The implementation of the recommendations and policy options will need to be carefully sequenced, piloted, and accompanied by proper incentives and supports to drive individual behaviour and institutional action. Institutions, instructors, and support staff need to be supported to meet their enhanced responsibilities for quality and equipped with the (digital) skills and resources to offer students a high-quality learning experience, appropriately supported by digital technology. Any student, regardless of their background, the discipline or mode within which they study, should have access to high-quality (digital) teaching, learning and assessment.

Modernisation of regulation and external quality assurance to increase flexibility, innovation and digitalisation

The OECD review team identified the existing regulation on study formats in Hungarian higher education as one of the main barriers to the further development of digital higher education in Hungary. This distinguishes between full-time, part-time and distance learning programmes, including strict requirements on the minimum/maximum number of contact hours per semester (study intensity) as well as when (i.e. evening/daytime, weekdays/weekend) and how (i.e. online/in-person) instruction is to be delivered (study mode). This categorisation does not reflect an up-to-date understanding of how teaching and learning takes place in today’s digital world, and is unable to meet the demands from digitally savvy secondary school graduates (who have lived through remote instruction during the COVID-19 pandemic) and adult learners (in search of flexible, and often online, upskilling and reskilling opportunities) for greater flexibility to decide on what, how, where, and when to study.

The second key barrier is the near absence of specific digital considerations in the minimum operating requirements of HEIs as well as the standards and indicators employed by the Hungarian Accreditation Committee (MAB) for the external quality assurance of higher education providers and their programmes. Specific standards for digital education can only be found in MAB’s procedures for the ex ante accreditation of distance learning programmes. Institutions that wish to offer distance learning programmes are required to meet ten criteria (or, “special provisions”) in addition to those that apply to regular programmes.

Table 1 summarises the recommendations and policy options for area 1, which were developed in close consultation with higher education stakeholders and draw on international best practice across the OECD.

Table 1. Recommendations and policy options for Hungary to support a modernisation of regulation and external quality assurance for digital higher education

Recommendations	Policy Options
Recommendation 1: Consider allowing institutions to offer programmes in three study modes, with some limits on study intensity	The report recommends Hungary to revise its existing categorisation of study formats by introducing a clear distinction between three modes of study (i.e. online, hybrid and in-person/blended) and two types of study intensity (i.e. full-time, part-time). Institutions should have full autonomy to decide whether to offer courses or programmes in the online, hybrid or in-person/blended study mode, whether to offer them on a full-time or part-time basis, and whether to introduce additional requirements or supports for fully online or hybrid study to mitigate the risk of study delays or drop-out. The following definitions of digital education are proposed: Online education refers to a study mode where instruction is delivered off campus, either synchronously or asynchronously, or a combination of both. Students complete their course or programme of study at a distance, without the need for on-campus instruction. Hybrid education refers to a study mode where instruction involves a mix of on-campus and off-campus instruction. Learners have some flexibility regarding the location in which they complete their study. For example, learners might complete laboratory segments of an engineering course on campus, while participating in lecture-based course segments through live web streaming. Blended education refers to a study mode where courses are intentionally designed to harness the capacities of digital technology, using it to enrich rather than substitute in-person instruction. For example, a language or mathematics course delivered on campus might use learning analytics to adapt problem sets to learner abilities. Importantly, most instruction continues to take place on a physical campus.
Recommendation 2: Develop specific indicators for digital education and embed them in existing accreditation frameworks	The report recommends that Hungary co-ordinate the development and integration of specific indicators for digital education across all its accreditation frameworks, drawing on existing quality frameworks for digital higher education developed by the European Network for Quality Assurance in Higher Education (ENQA), the European Commission, and Hungarian HEIs. To support Hungary with this task, based on an analysis of specific indicators for digital education included in international quality frameworks across the OECD and European Higher Education Area (EHEA), the report provides a list of: Potential minimum requirement for providers of digital higher education, related to HEIs’ capacity for digital delivery, pedagogical innovation and study flexibility. Potential indicators for institutional accreditation, including 24 additional indicators for digital education, as well as small revisions to the wording of existing indicators across all parts of the template.

Reorientation of accreditation processes to strengthen institutional responsibility for quality

The COVID-19 pandemic has prompted Hungarian HEIs to adapt their internal quality assurance systems to the specific challenges of digital education. The expansion of fully online and hybrid education has now become an explicit priority in many Hungarian HEIs’ institutional development strategies, with many scaling up investments in digital technology – although a renewed emphasis on place-based education is present in several institutions. There is also an emergence of staff professional development for digital education, and an increased focus on supporting students with digital learning. Incentivising staff to engage in the professional development of their pedagogical practices and providing greater mental health support to students will be key challenges going forward. Processes to monitor student performance and collect feedback on the quality of digital education are developing at a slower pace in Hungarian HEIs. Institutions are primarily embedding questions related to digital education in end-of-course or end-of-year staff and student feedback surveys, and are only slowly starting to make use of the opportunities offered by digital technology – such as learning analytics – to diversity their methods of data collection and analysis to get a more in-depth and real-time picture of quality.

Despite an emergence of inspiring practice across Hungarian HEIs for the QA of digital higher education, higher education stakeholders interviewed by the OECD review team underlined that institutional quality cultures in general are still developing in Hungary. Stakeholders explained that in many institutions, QA is still seen as a “box-ticking exercise” purely to satisfy external expectations. In this context, HEIs mentioned the two-stage ex ante programme accreditation process as an example of a highly burdensome administrative procedure, which diverts institutions’ – and MAB’s – attention from quality enhancement. In recent years, however, MAB has introduced several changes to its accreditation procedures, increasing its compliance with international quality standards and practices. Higher education stakeholders mentioned that the introduction of cyclical accreditation for institutions and doctoral schools based on the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) – and the recommendations emerging from the institutional self-assessment and site visit underpinning its process – were highly relevant for institutional quality enhancement. Stakeholders welcomed a further evolution towards institutional and outcomes-oriented approaches to QA, including at programme level.

Table 2 summarises the recommendations and policy options for area 2, which were developed in close consultation with higher education stakeholders and draw on international best practice across the OECD.

Table 2. Recommendations and policy options for Hungary to support a reorientation of MAB’s accreditation processes

Recommendations	Policy Options
Recommendation 3: Grant self-accreditation status to institutions with demonstrated capacity to manage study programmes at a high level of quality	The report recommends that Hungary considers granting self-accreditation status to HEIs with demonstrated capacity to manage study programmes at a high level of quality, in line with the ESG. Based on international examples of best practice across the OECD and EHEA, the report presents a potential model for the introduction of a performance-based self-accreditation system in Hungary: Unlimited self-accreditation status could be granted to those HEIs demonstrating the capacity to manage all their study programmes at a high level of quality, including in different disciplines, study modes (fully online, hybrid, blended), intensities (full-time, part-time) and levels (bachelor’s, master’s, PhD). Limited self-accreditation status could be granted to those HEIs demonstrating the capacity to manage some (types of) study programmes at a high level of quality, for example programmes in certain disciplines, certain study modes (fully online, hybrid, blended), certain intensities (full-time, part-time) or at certain levels (bachelor’s, master’s, PhD).
Recommendation 4: Introduce a performance and outcomes-based programme monitoring system, coupled with a targeted cyclical programme review procedure	The report recommends that Hungary considers introducing a performance and outcomes-based programme monitoring system for all HEIs, based on a limited number of national key performance indicators (KPIs), complemented by a cyclical programme review procedure (in disciplinary clusters) for HEIs without self-accreditation status. Based on international examples of best practice across the OECD and EHEA, the report presents a potential model for the introduction of a performance and outcomes-based programme review system in Hungary: Ongoing monitoring of programme performance against numerical thresholds for a limited set of national KPIs (e.g. drop-out rates, completion rates, graduate employment rates), developed in close consultation with HEIs and based on available national data on sectoral trends. The ongoing monitoring of programme quality could be used as a mechanism to identify “potential concerns with quality”, and form the basis for more in-depth and ad hoc reviews of specific courses or programmes, to understand the reasons and propose solutions for potential poor performance against national KPIs. Cyclical programme review (in disciplinary clusters) could be introduced for HEIs without self-accreditation status, and build on the approach followed by MAB for the accreditation of medical training programmes. This process consists of the preparation of a self-assessment report by the institution, based on the standards of the World Federation of Medical Education (WFME), followed by an institutional site visit and accreditation report, which are conducted and prepared by an external review team co-ordinated by MAB.
Recommendation 5: Increase institutional autonomy for the establishment of new programmes, depending on accreditation status	The report recommends that Hungary gives institutions and instructors more autonomy and flexibility to launch new study programmes in line with key societal challenges and emerging skills needs nationally and internationally, rather than the rarely updated education and learning outcome requirements included in the National Qualifications Register. A simplification of the two-stage programme accreditation process would free up MAB’s capacity to conduct cyclical quality reviews of programmes, and to play a bigger role in expanding its quality enhancement services for HEIs. Based on international examples of best practice across the OECD and EHEA, the report presents a potential model for a simplification of the ex ante programme accreditation procedures in Hungary, with progressive responsibility for institutions depending on their accreditation status: Institutions with self-accreditation status could be allowed to establish new programmes directly with the Educational Authority (OH), providing basic information such as the relevance and need for the new programme, and the institution’s own account of the programme’s proposed educational content and learning outcomes. Accredited institutions without self-accreditation status could also be allowed to establish new programmes directly with the OH, except in the case of programmes launched in certain study fields, modes or levels within which the institution is not yet offering degree programmes. Only the new programme proposals of non-accredited institutions would be required to undergo a full quality review by MAB prior to the programme being registered with the OH.

Strengthening institutional supports for the quality enhancement of digital teaching and learning

While responsibility for the formal quality assurance of higher education in Hungary is shared between MAB, the OH and KIM, a wide range of organisations can (and do) play a role in the quality enhancement of (digital) higher education in Hungary. This includes Tempus Public Foundation, the Hungarian Rectors’ Conference (MRK), the National Union of Students (HÖOK), the Association of Hungarian PhD and DLA Candidates (DOSZ), the academies of science as well as the Digital Government Development and Project Management Ltd. (DKFKT). Several of these organisations have started to more actively support HEIs with the quality enhancement of their digital teaching and learning practices. For example, by launching national surveys on the quality of digital learning, developing guidance materials and self-assessment tools for HEIs, organising conferences on digital higher education and managing online platforms to facilitate peer learning, or by increasing funding for the development of institutions’ digital education infrastructure.

However, stakeholder interviews carried out by the OECD review team reveal that the current institutional support landscape in Hungary is insufficiently coordinated and focused on the key challenges facing HEIs for the quality enhancement of their digital teaching and learning practices. The three key challenges mentioned by higher education stakeholders were: developing, maintaining, upgrading, and supporting the effective use of digital technology; supporting and incentivising the professional development and assessment of staff for digital teaching and learning; and developing effective processes for the collection, monitoring and use of data on the performance of digital higher education.

Table 3 summarises the recommendations and policy options for area 3, which were developed in close consultation with higher education stakeholders and draw on international best practice across the OECD.

Table 3. Recommendations and policy options for Hungary to strengthen institutional support for the quality enhancement of digital teaching and learning

Recommendations	Policy Options
Recommendation 6: Support the development of shared national standards and guidance for the purchase, maintenance, upgrading and effective use of digital technology	Based on international examples of best practice across the OECD and EHEA, the report presents the following policy options to support Hungarian HEIs with the purchase, maintenance, upgrading and effective use of digital technology: Steering and targeted funding: Through national steering as well as targeted and competitive funding, the government (KIM) and its responsible bodies (e.g. Digital Hungary Academy) could support and incentivise HEIs to invest in digital technologies that have demonstrated potential to enhance the quality of digital higher education, without prescribing which providers to choose. IT maintenance and support: Hungary’s National Research and Education Network (NREN), KIFÜ, could strengthen its role in supporting institutions with central network management and hosting services, to free up the capacity of institutional IT support staff to help instructors with the effective use of digital technology. Guidance and training: The NREN (KIFÜ), MAB or a sectoral (stakeholder) organisation could be tasked with coordinating the development of shared sectoral guidance and training, to support HEIs with the purchase, maintenance and effective use of digital technologies.
Recommendation 7: Introduce national regulation and support for the quality enhancement of staff professional development	Based on international examples of best practice across the OECD and EHEA, the report presents the following policy options to strengthen the professional development of academic staff for digital teaching and learning in Hungarian HEIs: National regulation on staff professional development: Hungary could introduce a requirement for HEIs to organise staff professional development and performance assessments of instructors’ pedagogical skills, including their skills for online course design, delivery and assessment. National standards for staff professional development programmes: Hungary could coordinate the development of national standards for the QA of HEIs’ staff professional development programmes. Guidance and training for the (self-) assessment of instructors’ digital skills: Hungary could coordinate the provision of national training, guidance materials and capacity building activities for the performance assessment of staff’s (digital) skills and competencies by HEIs. National centre for teaching and learning: Hungary could fund the establishment of a national centre for teaching and learning in higher education, with dedicated responsibility for developing quality enhancement activities for teaching and learning, including online course design, delivery and assessment. National digital content sharing platform: Building on already existing online platforms, Hungary could fund the development of a national digital education content sharing platform for higher education instructors.
Recommendation 8: Embed digitalisation in existing national data collection and monitoring instruments for higher education	Based on international examples of best practice across the OECD and EHEA, the report presents the following policy options for Hungary to strengthen its system-level evidence base on the quality of digital higher education, to inform institutional decision-making, inter-institutional benchmarking and best practice sharing: National administrative data collection and information on digital higher education: Hungary could add a “digitalisation lens” to its Graduate Career Tracking Survey (DPR), Higher Education Database and Information System (FIR) and Felvi.hu student admission and application website, to collect and publish up-to-date information on the performance of digital study programmes. National survey on digital teaching and learning: Building on existing national surveys of digital learning, Hungary could introduce a regular national survey of students’ and/or staff’s experience with digital teaching and learning, informed by a careful methodological analysis of the existing survey instruments. Thematic reviews of digital higher education: Through competitive funding calls, Hungary could fund thematic reviews of key challenges and best practices in digital higher education across institutions, focused on specific areas of focus or priority (e.g. student online mental health and wellbeing, online assessment). Thematic reviews of institutional quality assurance: As part of its accreditation reviews, MAB could ask experts to collect best practices identified as part of their analysis of institutional self-assessment reports and site visits, for dissemination through MAB’s communication channels with the sector.
Recommendation 9: Support and coordinate the development of an institutional self-assessment or benchmarking tool for digital higher education	Based on international examples of best practice across the OECD and EHEA, the report recommends Hungary to support the development of a Working Group (WG), consisting of national and international digital education experts and practitioners, to develop a self-assessment or benchmarking toolkit for digital higher education, adapted to the specific needs and challenges of the Hungarian higher education sector. The WG could focus on the following three questions: Plan and adjust: How can digital teaching and learning be embedded in the institutional strategy and quality culture, and supported through investments in digital technology? Implement: Which quality assurance processes and supports should be developed to enhance the quality of digital teaching and learning across the institution? Monitor: Which processes can be implemented to collect feedback on and monitor the performance of digital teaching and learning quality, and how is this data best used?