Rethinking Health System Performance Assessment

1. Revising the OECD’s Health System Performance Assessment Framework

For decades, health policy makers have strived to achieve high-performing health systems that deliver good health outcomes to populations, prevent ill-health and provide high-quality care available to all at affordable cost for payers and patients.

These issues have become even more pressing today. Health systems are under enhanced pressure to perform well under new circumstances. The COVID‑19 crisis has rocked societies and economies. It highlighted the necessity for health systems to be adequately equipped to handle and promptly recover from high-impact shocks. In this context, it has become essential for health systems to develop ways of assessing their resilience to disruption. This notion of resilient health systems is critical no matter whether the shock is caused by an external cause, such as a pathogen with pandemic potential, other biological threats, war, or by more enduring challenges such as population ageing and environmental threats.

Too often, health systems have been focused on what the system can deliver, rather than on people’s needs and preferences. People demand care that is better centred around what matters most to them, and health systems must be able to measure and assess their ability to meet patients’ needs and expectations. Health systems also need to improve the outcomes and experience of care, including as measured and reported by patients themselves. Furthermore, with the ageing of populations, evolving health needs also call for more continuity of care, integration of services and data, and health literate populations. These are all key components of high-performing and people‑centred health systems.

Climate change is an enduring challenge putting further pressure on health systems. It calls for immediate action to respond adequately to the new and incremental health needs caused by heat waves and other climate‑related threats. It also requires reducing the environmental impact and carbon footprint of health systems themselves. Yet today we still do not adequately assess the ability of health systems to respond to the stress posed by global warming on the health and well-being of people, nor their ability to contribute to environmental sustainability and reduce their carbon footprint.

Digitalisation and artificial intelligence provide new opportunities to rethink how health systems can deliver high-quality care that is accessible to all. Telemedicine offers opportunities to bring services closer to people. Artificial Intelligence has great potential to improve clinical care, research, and system governance. These could improve health system performance, but ethical and governance challenges need to be tackled – and health systems must be able to assess their impact more effectively.

Finally, the need for health systems to provide high-quality and safe care in a context of growing pressure on budgets means that they are increasingly being asked to deliver, reward and measure value rather than sheer volume of care. Efforts to improve value for money needs to be accompanied by improved ability to assess the implications for different population groups, irrespective of age, income, or gender, with special attention given to the most vulnerable.

This evolving policy context call for a new vision of health system performance assessment that integrates new dimensions of performance such as resilience, people‑centredness, and environmental sustainability.

Health system performance assessment (HSPA) has become a crucial element in ensuring that health systems provide quality healthcare and meet people’s needs. By consistently and systematically evaluating health systems, HSPA serves as a valuable instrument for policy makers to identify areas that require improvement, support a more efficient allocation of resources, and assess the achievement of key policy objectives.

Over the past three decades, assessing the performance of health systems has formed a key part of the OECD’s efforts to support a high quality of care amid rising expenditures and changing demographic trends. Measures of health system performance have evolved as new evidence from health sector policy and experiences emerged since the OECD first published comparative reviews of health system reforms in 1992 (see Annex A for a more detailed description of the history of health system performance assessment). The renewed Framework developed by the Secretariat reflects the changing landscape of health systems today and combines several components of other frameworks currently in use.

1.1. Recent OECD frameworks for health

Conceptual frameworks provide a structured way to organise and understand complex public policies ideas and relationships. Their intended purpose is to help policy makers, policy analysts, practitioners and other stakeholders make sense of a particular policy area, having practical applications across various domains, including guiding indicator development and policy analysis through the selection of pertinent measures, and laying a foundation for research. They further facilitate knowledge integration by categorizing information from different sources, supporting the creation of a comprehensive understanding of a subject.

These frameworks enable international benchmarking and policy evaluation using sets of agreed indicators and visual tools. They also promote effective communication, collaboration, and knowledge exchange by providing a common language and shared understanding. Moreover, they contribute to decision-making by offering a systematic evaluation approach that helps assess potential impacts and guide well-informed decisions.

While countries have their own national frameworks, internationally comparable statistics of health systems and analysis across OECD countries are essential for comparative analysis and to facilitate international learning.

For decades, the OECD has helped countries to identify the key principles of high-performing health systems and to assess health system performance based on internationally comparable health indicators (see Annex A). This work has contributed to, and been guided by, the development of conceptual frameworks for health system performance developed by the OECD overtime (Hurst and Jee-Hughes, 2001[1]; Kelley and Hurst, 2006[2]; Carinci et al., 2015[3]) as well as more recent and focused developments that zoom in on two specific angles: people‑centredness (OECD, 2021[4]) and resilience (OECD, 2023[5]).

1.1.1. The 2015 Health System Performance Assessment Framework

While the 2015 HSPA Framework (Figure 1.1) “zooms in” on the quality dimension (highlighted in yellow), it recognises other elements such as access, financing and health system inputs, and in fact has been used as the conceptual framework underpinning wider analytical and indicator work by the OECD.

This Framework can trace its roots to the year 2000 (Hurst and Jee-Hughes, 2001[1]), and it was further developed later as part of the activities of OECD’s Health Care Quality Indicators expert group (the predecessor of the current Health Care Quality and Outcomes Working Party) between 2004‑06 (Arah et al., 2006[6]; Kelley and Hurst, 2006[2]) (see Annex A for more details). Since its adoption, this Framework has been used as the foundation for the development of indicators to assess health system performance across OECD countries and to evaluate gaps in indicator portfolios.

Figure 1.1. The OECD 2015 Health System Performance Assessment Framework

Source: Adapted from Carinci et al. (2015[3]), Towards actionable international comparisons of health system performance: Expert revision of the OECD framework and quality indicators, http://dx.doi.org/10.1093/intqhc/mzv004.

1.1.2. The People‑Centred Health Systems Framework

The People‑Centred Health Systems (PCHS) Framework was a response to the 2017 OECD meeting of Health Ministers. At the time, OECD Ministers agreed that health systems needed to maximise their effectiveness and efficiency, to deliver seamless care across services and providers; and to deliver improvements that matter to people and their changing care needs. They agreed about the need to equip health professionals with the right skills and adapt training and work models to deliver care that maximise patients’ outcomes. Ministers further asked the OECD to develop the Patient Reported Indicators Survey (PaRIS) to provide cross-country comparative measures of patients’ own experience of medical care and healthcare outcomes. They also gave the OECD a strong mandate to set up a framework and assessment tool to help them identify suitable actions to adapt their system towards people‑centred care.

The development of the PCHS Framework built on the 2015 Framework itself, zooming into the characteristics that are directly related to people‑centredness. It also built on the multiple efforts conducted by other organisations to identify and categorise the key components of people‑centredness (Nolte, Merkur and Anell, 2020[7]; World Health Organization Regional Office for Europe, 2016[8]).

The OECD PCHS Framework was completed in 2021 and covers five dimensions for assessing people‑centredness of health systems: Voice, Choice, Co-production, Integration, and Respectfulness (OECD, 2021[4]). This Framework helps to methodically analyse people‑centredness of health systems along these dimensions. To help countries to assess the progress they are making towards people‑centred care and identify whether there are certain dimensions or policy areas that could benefit from particular attention, a scorecard was developed that identifies policy benchmarks across the five dimensions of the OECD People‑Centred Health Systems Framework.

The indicators included in the scorecard draw on a range of data sources, including OECD Health Statistics, the European Quality of Life Surveys, and the Commonwealth Fund International Health Policy Surveys. The full data collection of the of the PaRIS survey of People Living with Chronic Conditions, which includes 20 countries and is underway in 2023, will drastically improve the availability of indicators of people‑centredness for participating countries, allowing for a new, more completed edition of the PCHS scorecard to be published.

1.1.3. The Resilience Shock Cycle Framework

Resilience is understood as the ability of systems to prepare for, absorb, recover from, and adapt to major shocks (OECD, 2020[9]). The COVID‑19 pandemic revealed weaknesses in health systems and in how they respond to shocks, highlighting the need to improve their resilience.

Rather than relying solely on planning for, avoiding and absorbing shocks, a resilience approach acknowledges that some shocks will be of a size and scale that will disrupt an entire health system. In this scenario, it is important that the health system is capable of recovering and adapting for the future. In this context, building on the lessons of the pandemic and considering further shocks, the OECD Resilience Shock Cycle Framework served as the basis for the OECD analysis of health system resilience (OECD, 2023[5]).

The disruption cycle in the Framework comprises four stages:

• Prepare includes the steps taken by the health system and related institutions to plan and prepare critical functions and features to avoid and mitigate a universe of potential shocks.

• Absorb comprises the capability of the health system to maintain core functions and absorb the consequences of an acute shock or extended stress without collapse. It involves limiting the extent of the disruption and minimising the morbidity and mortality impact.

• Recover involves the health system attempting to regain lost functions as quickly and efficiently as possible. It also refers to the time and resources needed for the system to recover its functionality after the shock.

• Adapt relates to the capacity of the health system to “learn” and improve its capacity to absorb and recover from shocks based on past experience, reducing the impact of similar threats in the future. It informs planning and preparation for the next cycle.

These stages are presented in sequence but are dynamic and integrated in practice. This is illustrated by the COVID‑19 context. For example, a new variant of SARS‑CoV‑2 that escapes immune protection from vaccination could result in a shift backwards from the recover to the absorb stage. Furthermore, decisions made in one stage of the disruption cycle may have an impact on the subsequent stages – for example, stopping elective surgery during the absorb stage of the pandemic may affect the recover stage.

1.2. Towards an integrated and updated OECD HSPA Framework

1.2.1. The need to adjust the Framework to the world we live in

The OECD Health System Performance Assessment Framework was last updated in 2015, with a focus on quality and outcomes (Carinci et al., 2015[3]). Since then, health systems have been called to become more centred around people needs and expectations, while many disruptive events, including the greatest health emergency in a century, have occurred, and enduring challenges are pressuring health systems on a daily basis. All of this has fostered a major shift in the broader health policy context, and expectations of what a health system performance assessment framework should measure have evolved substantially.

As the world changes, so must the approach to how health system performance is assessed. The COVID‑19 pandemic revealed that health systems were underprepared and therefore not as resilient as they should have been or were thought to be. As the legacy of these crises continues to reverberate, an interrelated set of enduring and emerging crises – cost of living, climate change, war – calls for urgent action to ensure that health systems are more resilient to future shocks. At the same time, megatrends such as ageing populations and increasing digitalisation are likely to have a significant impact on health and healthcare in the coming years.

These seismic shifts present not only an immense challenge, but also an opportunity to collectively chart a path towards building greener, more sustainable and more resilient health systems that yield dividends for populations, societies and economies. The conceptual framework underpinning health system performance assessment and the OECD’s work on health should reflect the policy challenges health systems face today or those that are certainly expected for the future.

1.2.2. Using a renewed HSPA Framework

A fit-for-purpose framework is a tool that consolidates a collective understanding of high-level policy objective of health systems, define the scope of OECD work on health, and help agree on a common language and terminology. It is not intended to replace national-level health system performance assessment frameworks, but to enable international or regional-level benchmarking and mutual learning.

The renewed OECD HSPA Framework also helps steering indicator development, knowledge integration, international collaboration and decision making. It provides a shared vision of the main elements of health systems that deserve policy attention and will serve the needs of different strands of OECD work on health. Box 1.1 details the main practical applications of such Framework.

Box 1.1. The practical use of the OECD Health System Performance Framework

• Providing a foundation for indicator development, data collection and policy analysis;

• Knowledge integration by organising and integrating knowledge from different sources and different streams of work;

• Facilitating international benchmarking and learning by sets of agreed indicators, dashboards and data visualisations;

• Facilitating international collaboration by providing a common language, definitions and shared understanding among country officials, policy makers, stakeholders and organisations;

• Help guiding decision makers towards informed and well-grounded decisions by providing the basis for health system performance assessments;

• Stimulating innovative analyses such as stress testing of health systems and assessing health system’s impact on the environment.