Health System Performance Assessment Framework for the Czech Republic

Annex A. Concepts behind HSPA framework development and international practices

HSPA has been developed and used across countries and regions in the previous decades and has brought stakeholders in the health sector together to share common health system objectives and supported them to work together towards attaining higher health system goals. Initial initiatives to develop HSPA frameworks were commenced by the WHO, especially WHO/EUR (Fekri, Macarayan and and Klazinga, 2018[3]). More recently the development of HSPA frameworks has been systematically supported by the European Union. Countries such as England (NHS), Malta, and the Netherlands have developed national HSPA frameworks, joined more recently by Ireland, Croatia, Latvia, Portugal, and Slovenia. In 2021, the Czech Republic and Estonia have initiated new initiatives do develop and implement HSPA frameworks.

In the aftermath of the COVID‑19 pandemic, HSPA gained in importance, particularly as a mechanism to develop and monitor more resilient health systems. The COVID‑19 crisis led to new inequalities and competing priorities between health stakeholders within health systems. Health systems need support in collectively making the system more resilient to similar or other shocks that may occur in the future. HSPA’s system-wide collaboration can bring all stakeholders together to shape a resilient health system.

Several steps need to be undertaken to make an operational HSPA framework. HSPA needs (1) to set its objectives, (2) to set a scope of Health, System, Performance analysis unit and tools, and Assessment functions, and (3) to develop a framework. Then, (4) HSPA needs to be substantiated by regularly updated data and reported in a timely manner.

Setting objectives

First, clear objectives for HSPA need to be set. These objectives are set and shared among stakeholders involved – particularly those whose performance will be assessed through HSPA. For each HSPA, relevant stakeholders may vary depending on the scope and purpose of the HSPA. Meetings represented by relevant stakeholders are usually held to set and share objectives and provide input. In an increasing number of countries, citizens and patient groups/representatives have also been involved in setting HSPA objectives as efforts are made to transform health systems more people‑centred (i.e. HSPA Ireland in 2021).

According to the HSPA developed so far, objectives usually include the following:

• Public reporting on the performance of the healthcare system,

• National and international benchmarking,

• Identification of strategic priorities,

• Monitoring of policy reforms, and/or

• Increasing accountability of various parts/services that constitute the healthcare system.

Setting scope of HSPA

Second, the scope of HSPA needs to be set and it needs to clarify each of the following areas:

• Health (i.e. whether to refer to poor health outcomes, disease prevalence, and mortality, or will include disabilities and well-being)

• Systems (i.e. whether to refer to health system, healthcare systems and/or social care)

• Performance of overarching system, specific services and/or delivery systems

• Assessment functions such as management, policy-making, accountability, and/or improvement.

Like objective setting, the scope of HSPA needs to be identified collectively by relevant stakeholders. A multi-stakeholder consultation process is particularly important for the level of performance assessment and assessment functions because buy-ins and engagement from relevant stakeholders bring successful operationalisation of HSPA and subsequently lead to health system strengthening.

The rest of this section describes various elements to consider when deciding the scope of each of these areas.

Health

HSPA usually includes assessment outcomes, such as deaths and disease prevalence and severity, to assess health system performance. Mortality and life expectancy are classical parameters used to measure health system performance from a public health perspective. To use these measures, a well-functioning death registry is needed. Prevalence and incidence of diseases are another set of classical parameters of health system outcomes used to assess morbidity of diseases in a country. Related outcome measures of quality of life (e.g. Quality-Adjusted Life Years (QALYs)) aim to capture the reduction in morbidity and the outcomes due to specific diseases. Medical/clinical perspective is the dominant way of operationalising these outcome measures, and these measures are dependent on the availability and quality of clinical registries (such as on cancer and diabetes). These measures are often linked to costs (value) at the system level to assess the burden of diseases and to specific services and interventions to evaluate cost-effectiveness.

HSPA can also assess disabilities as part of health systems outcomes. This is because many chronic diseases cause long-term disabilities, and HSPA could address the way a health system deals with disabilities. At the system level, DALY (Disability Adjusted Life Expectancy) is the most well-known measure of disabilities, and at health services level, various instruments are available to assess disabilities (e.g. inter RAI initiative). These outcomes mainly use administrative databases and surveys as their data sources.

Recently, HSPA includes well-being as part of health systems outcomes (OECD, 2019[4]). So far, health outcomes experienced by citizens/patients (PROMS, Patient-Reported Outcome Measures) are mainly tested for clinical procedures and treatments, and they are still under development for chronic conditions. Instead, EQ5D,1 a more generic measure not related strictly to a specific clinical procedure and treatment, is used as HSPA’s health systems outcomes. PREMs (Patient-Reported Experience Measures) are also used (Fujisawa and Klazinga, 2017[5]), although these instruments (e.g. CAHPS,2 Picker) have limited international validation.

Countries are increasingly using Electronic Health Records for more detailed assessment of health system outcomes such as safe healthcare. Data linkage capabilities based on unique patient identifier (UPI) are increasingly enhanced in countries to assess more complex health systems outcomes such as care integration within health systems and between health and social care systems.

System

HSPA can assess different systems and the boundary of system assessed in HSPA needs to be clarified. The health system as a whole is often assessed in HSPA. However, the performance of healthcare systems such as public health, primary care, hospital care, mental health care, community care, long-term care can be also evaluated. Due to a growing number of populations in need of both health and social care, the performance of health systems including social care is sometimes assessed in an HSPA. The scope of systems used for HSPA needs to align with the boundary used for health policy making and also with health system priorities within countries and regions.

Performance

The performance of health system(s) can be assessed at various levels such as overarching system level, specific services or delivery systems levels, or specific provider or professional levels, and the unit of analysis needs to align with the objectives set for a specific HSPA. Health system performance can be assessed at the overarching regional or national health system level if the objective of HSPA includes national and international benchmarking. HSPA often uses the analysis unit of healthcare services such as public health, primary care, hospital care, mental health care, community care, long-term care for national and international benchmarking, to identify strategic priorities for the health system, and/or to monitor health policy reforms undertaken in these subsystems. The OECD analyses as shown in its flagship publication, Health at a Glance, include both system-level assessment and analyses of priority healthcare services.

The unit of analysis can be set as specific as provider and professional levels if public reporting commitment requires such details, for example, for population’s provider choice or to increase accountability of healthcare delivery at provider and/or professional levels.

Since patients often seek care in various settings, such as primary care, acute care, and long-term care, and they need a smooth transition of care and continuity of care, HSPA can assess the performance of integrated delivery systems. In order to make a crosscutting assessment on integrated care delivery systems covering various care settings, data from different providers and systems need to be linked. This requires UPI and appropriate privacy protection regulations so that the data are used adequately while protecting citizen’s privacy (OECD, 2015[6]; OECD, 2013[7]).

Assessment

HSPA needs to link measurements and reporting to health system management and policy-making. In order to do so, clear assessment functions need to be set based on the objectives identified. HSPA usually has an accountability function towards citizens, financiers, policy makers and a management function for the entity responsible for healthcare delivery such as Ministries of Health. HSPA can also have an improvement function for relevant stakeholders including citizens, healthcare providers, professionals, and industries to learn and improve health systems collectively through performance of each stakeholder. Assessment functions may need to be reviewed. For example, the United Kingdom has a history of developing separate outcome frameworks for public health, healthcare services and social care that have been used for a mix of assessment functions to govern the various NHS services over time. The balance between formative (learning) and summative (accountability) functions has been changing over time.

International practices in developing HSPA framework

HSPA framework needs to reflect objectives and scope set and needs to lay out key domains for health systems assessment.

HSPAs usually have a framework like the one shown in Figure A A.1. Many HSPAs use structures/inputs, processes, outputs, and outcomes as key dimensions, and in each dimension, key domains that reflect health policy priorities are identified. Given the COVID‑19 pandemic, recent HSPAs have started to include telemedicine as an important sub-domain under the health technologies domain. Given an increasing number of patients with chronic conditions needing care in different settings, care co‑ordination, service integration and continuity of care are becoming common domains in the processes dimension. As part of output dimension, access and costs are classical domains included in an HSPA framework, but quality of care is also included recently, as health data infrastructures are strengthened and more complex data are becoming available to assess quality of healthcare (OECD, 2015[6]; OECD, 2013[7]). Even when health systems mature, people‑centred care delivery is still considered challenging and hence lately people‑centredness is increasingly included as an HSPA domain.

Figure A A.1. HSPAs have a largely similar framework: example from Ireland

Source: Kringos et al. (2021[8]).

Across all domains, a health system aims to tackle and improve crosscutting themes such as equity, efficiency and/or sustainability. As the COVID‑19 pandemic brought enormous shocks to health systems, and in order to manage the current crisis and prepare for future shocks, nowadays resilience is also considered an important crosscutting theme to be included in an HSPA framework.

International practices in substantiating and operationalising HSPA

The assessment framework needs to be substantiated with measurements, which are updated regularly to capture changes in health system performance in a timely manner. Measurements can be drawn from existing health statistics and/or newly developed for instance by administering surveys or by allowing secondary use of health data and linking data sources which may require changes in legislations (OECD, 2015[6]; OECD, 2013[7]). For national and international benchmarking, use of international indicators such as those developed at the OECD covering all spectrum of healthcare can be used (OECD, 2021[9]). Depending on the objectives and scope of HSPA, newly developed indicators including people‑centredness, patient safety culture, patient-reported experience measures for specific conditions and on patient safety, integrated care, end-of-life care can be considered as measurements for HSPA.

Different types of measurements can be used to communicate health system performance. HSPA can have a number of indicators, dashboards with key indicators or compound index, which summarises health system performance. A decision on the type of measurements depends on the objectives as well as functions of the HSPA. If an HSPA is meant to signal overall performance to policy makers or for health system management, a summarised tool such as dashboards or compound indices may communicate the message better, but if an HSPA is meant to serve many stakeholders, for example, to change practices, more detailed data catered to each stakeholder may be useful. A mix of measurement types can be used since HSPAs often have a mix of objectives and functions.

Notes

← 1.  EQ‑5D is a standardised measure of health-related quality of life developed by the EuroQol Group to provide a simple, generic questionnaire for use in clinical and economic appraisal and population health surveys.

← 2.  The Consumer Assessment of Healthcare Providers and Systems (CAHPS) is standardised survey of patients’ experiences with ambulatory and facility-level care in commercial and Medicaid plans, developed by the Agency for Healthcare Research and Quality (AHRQ) in the United States.