Section 3.1 first presents the Estonian HSPA framework and outlines its areas and domains. The indicator selection process is then reviewed in Section 3.2 alongside criteria applied to select indicators for the HSPA framework. Section 3.3 provides information on the distribution of selected indicators in domains and subdomains. The Estonian HSPA framework has been developed through an extensive consultation process with key stakeholders, the project’s Principal Working Group and High-Level Advisory Board. The development process of the framework is described in detail in Annex A. Further detail on indicators in each area and domain, including potential disaggregation, primary use, data source and custodian, is provided in Annex E.
Health System Performance Assessment Framework for Estonia
3. The Estonian HSPA framework
3.1. The areas and domains of the Estonian HSPA framework
Five indicator areas of the framework are divided into 18 domains (Figure 3.1). The final framework places health status at the centre. This is the key outcome indicated in the National Health Plan 2020‑30 and the main aim for health systems. Health status, at the centre, is encircled by three main health system areas: outcomes, processes, and structures. The fifth area, cross-cutting domains, surrounds the circle, illustrating the cross-cutting impact these domains have on the whole health system. Each indicator area is further divided into domains and subdomains. There are 64 subdomains in the 18 domains in the framework.
Health status gives an overview of the state of health of the population and is assessed using subjective and objective indicators – healthy life expectancy, life expectancy and self-reported health status. The domains included in the Outcomes area include health outcomes, healthy choices, health supportive environment, access and patient safety and quality. This Outcomes area thus focuses on indicators reflecting the health, well-being, and safety of a person or population more widely. Access and patient safety and quality components measure these from the perspective of the healthcare system.
Processes are detailed by domains related to health literacy, person-centredness, integrated care, and governance. The domains concentrate on how people are supported by the system and how the system is governed. Health literacy brings in a theme of how people understand and use information on health topics and interact with health service providers.
The area of Structures includes domains of health infrastructure, workforce, financing, digitalisation, and innovation. The components describe the capacity of the healthcare system and evaluate possible gaps in service provision. These also aim to assess the use of innovative solutions in service provision and the development and use of digital health technologies, programs, and software in the health system.
There are three cross-cutting domains encircling the main HSPA circle: equity, efficiency, and resilience (see Figure 3.1). These components look at socio‑economic factors, use of resources, and capacity and adaptability of different layers of the health system to understand and ensure continuity and quality of service delivery. The HSPA framework visualisation embeds the cross-cutting themes in all other HSPA domains, while highlighting the main policy direction of the Estonian health system to become more person-centred by placing the population health status in the central part to other domains.
Further explanations on the development of the definition of Estonian HSPA framework components is provided in Annex A. The domains illustrated in the high-level HSPA framework diagram are further divided into subdomains (see Section 3.3).
Figure 3.1. The Estonian HSPA framework
Source: The Estonia HSPA framework project, April 2023.
3.2. Indicator selection process and criteria
The selection of indicators to populate domains relied on two main concepts 1) the relevance of the indicator for performance assessment and 2) quality of the underlying data and feasibility of measurement. A balance must be reached between the conceptual relevance of indicators and the practical possibilities for measuring and reporting on the selected indicators (Figure 3.2). Thus, current feasibility to report an indicator was not a necessary prerequisite for inclusion in the framework and indicators which were considered highly relevant could still be included as placeholders. Consideration was given to the level of commitment and capacity to further develop and operationalise these indicators as part of the HSPA implementation plan.
Figure 3.2. Trade-off considered in HSPA indicator selection
Source: Dutch HSPA development process (van den Berg et al., 2014[5]), The Dutch health care performance report: seven years of health care performance assessment in the Netherlands, https://doi.org/10.1186/1478-4505-12-1.
Potential indicators were discussed and identified via a series of seven focus groups held over the course of Autumn 2022 (see Annex C: Focus groups to inform indicator selection and HSPA capacity building study guide). A long list of 468 selected indicators was then compiled using input from representatives from key stakeholder organisations. The list included indicators reported and published by key stakeholders within the Estonian health system; indicators reported by Estonia to international databases such as Eurostat and OECD; indicators featuring in the strategic documents for the Estonian health system; and indicators marked as placeholders from the many technical discussions during this project, which need further development.
To reduce the total number of indicators towards the initial target of 100 most relevant indicators for the Estonian HSPA, all key stakeholder institutions and quality managers of public hospitals were asked to assess each of the indicators. Stakeholders were sent the long list of indicators for evaluation, though they had an option to focus on assessing only sections relevant to their institution. Altogether, approximately 45 people across the Estonian health system contributed to the evaluation exercise.
Four main selection criteria were used in the exercise: 1) the availability of the data, 2) the significance for national policy priorities, 3) the suitability and relevance for HSPA objectives, and 4) the possibility to benchmark the indicator internationally or nationally. The evaluators assigned scores on each of the criteria for each indicator, with answers marked by zero for “no”, one for “maybe”, and two for “yes” (Table 3.1). Then, the mean evaluation score across criteria was calculated for each suggested indicator, and a traffic light system was used to present the results. To organise the results of the exercise, a mean score of 0‑0.9 was presented as red, 1‑1.7 was shown yellow, and 1.8‑2 as green. While indicators marked in red were excluded from populating the framework, all green ones stayed in. A significant number of indicators were also allocated in the yellow category, meaning their inclusion remained undecided in the initial scoring exercise. These indicators were subject to further stakeholder discussion before a consensual decision was taken on inclusion some of them to populate the framework. This exercise allowed the initial long list of possible HSPA indicators to be reduced from 468 to 289 indicators, which entered the next selection round.
To decide on the inclusion of indicators marked in yellow, further discussions were held. Over two half-day in-person meetings, key stakeholders discussed each of these indicators individually. Additionally, separate meetings were held to discuss indicators relevant to mental health and digitalisation and innovation. Special attention was given to these areas due to their relevance to specific national policy priorities within the Estonian health system (see Annex B). Mental health indicators were separately discussed by the experts from MoSA’s Department of Mental Health, NIHD, and EHIF. The digitalisation and innovation subdomains were worked through by the New Generation Health Information System (UpTIS) team, HWISC and MoSA’s experts. After all rounds of discussion, a final set of 212 indicators was identified to serve as input for the first HSPA cycle.
Table 3.1. Application of the indicator selection criteria in the evaluation of the indicator set
3.3. Overview of indicators selected to populate the HSPA framework
Distribution of indicators selected to populate the Estonian HSPA framework into areas, domains, and subdomains is provided in Table 3.2. Six core indicators are considered as part of the area of Health Status. Outcomes remains the largest indicator area with 92 indicators, while Processes include 29. The area of Structures contains 48 indicators while 37 indicators were left in the Cross-cutting area. A full list of selected indicators is provided in Annex E.
As data availability was among the key criteria for indicator selection, many of the indicators included in the Estonian HSPA are already regularly calculated by various stakeholders, and thus do not represent an additional workload for inclusion within the HSPA reporting. In addition to the existent indicators, 81 indicators remain in the framework as placeholder indicators, which require yet additional development. These include indicators that were deemed important for a comprehensive assessment of the Estonian health system but were impossible to calculate at the time of selection due to absence of data, gaps in methodology, irregularity of unavailability of the data.
The final list of indicators included in the HSPA was designed with flexibility in mind. The governance and implementation plans (Sections 4.1 and 5.1) for the HSPA define a working structure for further development of the framework and its indicators, allowing for changes in-between regular HSPA assessments. This will ensure the relevance of the HSPA to changing priorities while maintaining the comprehensive and robust focus to address strategic priorities.
Table 3.2. Overview of indicators in domains and subdomains of the Estonian HSPA
Note: For more information on the indicators please see Annex E.
For additional information:
Annex C: Focus groups to inform indicator selection and HSPA capacity building study guide
Annex D: Indicator technical sheet example
Annex F: Example of analysis of indicators for HSPA reporting
Annex G: The HSPA framework and description of domains and subdomains in the Estonian language
OECD (2022[2]), Situational Analysis Report: The Development of the Estonian Health System Performance Assessment Framework, https://www.oecd.org/health/Development-of-Estonian-Health-System-Performance-Assessment-Framework.pdf.