The first draft of the scope and purpose of the Czech HSPA, defined in early 2022, provided guidance on selecting the domains and indicators to populate the HSPA framework. Common HSPA functions often relate to accountability and performance monitoring too. Other common purposes relate to evaluation of policies and strategy development, assessment of specific sectors or programs, and providing platform for accountability to the government or general public. Together, these goals typically balance reporting and assessment with learning and improvement functions. A number of common HSPA domains have been used frequently in existing HSPA frameworks in Europe (Figure A C.1).
Health System Performance Assessment Framework for the Czech Republic
Annex C. Process of determining the Czech HSPA framework domains
Figure A C.1. Examples of common domains used in HSPA frameworks in Europe
HSPA framework development workshops
During HSPA framework development workshops, held on 7 April and 8 April 2022, in Prague, more than 35 participants representing various government institutions as well as important health sector stakeholders came together to establish the framework structure and begin to identify indicators that would be used in measuring key domains of the HSPA. Meeting participants included representatives of the Ministry of Health, Ministry of Finance, Institute for Health Information and Statistics, the Czech Statistical Office, National Institute of Public Health, representatives of the statutory health insurance funds (CPZP, VZP, VoZP, OZP, ZPS), Health Insurance Bureau, Association of Ambulatory Specialists, representatives of teaching hospitals (Bulovka), patient representatives from the Patient Committee of the Minister of Health, and academia (Charles University). The OECD team was also in person in Prague.
Following a summary of international approaches to HSPA domain definition, participants were asked to write 4‑8 potential themes to be addressed in the HSPA on colored sticky notes. Workshop participants then added their sticky notes to posters identifying broad areas of potential domains. In total, 135 proposals for HSPA domains/themes were identified in this exercise (see Figure A C.2).
Workshop participants were assigned into 4 groups based on the perspective and role their institution plays in the health system. The distribution was done via colored sticky notes, which allowed for analysis of preferences based on the generated themes. For instance, while the patient group stressed the most the themes of quality and safety, the insurers put focus on costs and efficiency, but both groups placed strong emphasis on equity and access.
Figure A C.2. The 135 potential themes were identified via the HSPA consensus building process
Source: The Czech HSPA project, April 2022 working group meeting.
Following the exercise to generate themes, the workshop participants participated in a moderated discussion to summarise the flipcharts and emerging domains, clarifying with participants where notes were not clear. The consolidation process aligned the 135 proposed topics into 22 themes, that were further prioritised via a voting exercise to identify the most important ones for representation in the HSPA. Participants gave votes to 19 of the 22 themes, with the themes related to experienced health status, cost-effectiveness, quality, data structure, waiting times, and access to care receiving the highest score (see Figure A C.3).
Figure A C.3. The voting process identified experienced health status, cost-effectiveness, quality, data structure, and waiting times as the top five themes
Source: The Czech HSPA project, April 2022 working group meeting.
Following the voting exercise, the domains were organised by the management team of the Ministry of Health and the OECD, grouping related themes as subdomains into common domains. The draft framework was then presented to workshop participants for validation and feedback. Following incorporation of comments from the working group participants, the draft framework was revised to the version represented in Figure A C.4.
Figure A C.4. The Czech HSPA draft framework developed during the April 2022 development workshops
Source: The Czech HSPA project, April 2022 working group meeting.
Technical focus groups
The initial HSPA draft framework was used to determine the themes of the technical focus groups. The April workshops participants agreed to hold separate technical discussions on the topics of Access, Quality, Workforce, and Health information infrastructure and health technologies. These were held virtually in the months of May, June, September, and October 2022 and their particular focus linked to the HSPA draft framework is depicted in Figure A C.5.
Figure A C.5. Linking technical focus groups to the first HSPA framework draft
Source: The Czech HSPA project, HLAB meeting in January 2023.
The technical focus groups served the purpose of detailed technical discussions among selected working group members, OECD experts with expertise in each domain, and other invited the Czech and international experts. The meetings took place virtually and the number of participants was held low, to 10 to 14 people, to allow for thorough discussion and review of existing international practice and indicator methodology and options for the Czech HSPA localisation. The below list provides a summary of the technical focus groups and their discussions.
Focus Group #1: Person-centredness, within the Quality domain (18 May 2022)
Participants included MoH, CZSO, UZIS, patient representative, national manager of the PaRIS project.
During the focus group, OECD experts shared their expertise regarding the state of the OECD PREMs items data collection and patient reported safety indicators, and the patient reported indicator survey projects (PaRIS), with a special focus on patient reported experience measures (PREMs) – the PaRIS International survey of people living with chronic conditions. During the meeting, it was suggested to also use a summary indicator from the patient satisfaction survey, which is run by the Ministry of Health for inpatient facilities. The hospitals participate in the survey on a voluntary basis, but over the years the number of participating hospitals has grown substantially.
Focus Group #2: Digitalisation and Health technologies domains (1 June 2022)
Participants included various departments of the MoH, 3 health insurance funds, and patient representative.
During the focus group, OECD experts shared with the participants the current status of the adherence to OECD council health data governance recommendation, the OECD and other country’s examples of indicators used in monitoring electronic health record (EHR) systems adoption and maturity, and indicators used in monitoring telemedicine and remote care. Participants agreed that developing a continuous indicator for eHealth adoption measurement would be more suitable for the HSPA purposes than using the Y/N questions.
Focus Group #3: Workforce monitoring domain (12 September 2022)
Participants included various departments of the MoH (including the nursing and non-medical health professions department), CZSO, UZIS, VoZP, VZP, labour union (OSZSP), National Center for Nursing and Non-Medical Health Professions.
During the focus group, OECD experts shared with the participants the OECD and international experience with indicators used for workforce monitoring. The Czech data submission on health workforce data collected through OECD/ Eurostat/WHO-Europe Joint Questionnaire is generally very good. However, many more health workforce data are required for health workforce planning at national level to guide policy decision-making, including supply-side and demand-side data, past/current/future (based on different scenarios). The Netherlands provides one of the best examples of good national health workforce planning. The discussion then focused on the possible indicators that would allow for monitoring of workforce capacities, shortages, and future needs, including the remuneration. The number of health workforce indicators will need to be limited to the most important/relevant ones.
Focus Group #4: Accessibility
Participants included MoH, CZSO, various departments of UZIS, health insurance funds (VZP, VoZP), university hospitals (Olomouc, VFN).
During the focus group, OECD experts shared their knowledge on measuring the financial accessibility in OECD countries and internationally, using indicators for monitoring of financial protection, catastrophic spending, and coverage measurement by public finance spending by type of healthcare. The second item focused on measuring time accessibility and waiting times measurement based on OECD experience were shared with group participants. The good practice country example of Slovenia was also presented, focusing on its methodology of waiting lists and waiting time measurement.
Study visit to Belgium
The study visit to Belgium was organised by the OECD team to support knowledge sharing, mutual networking, and bilateral relationship building between the Czech HSPA main authorities and their Belgian counterparts. The decision to visit Belgium was based on the interest of the Czech authorities in Belgian HSPA and the fact that HSPA in Belgium has been well developed and maintained over many years, there are experiences with HSPA implications for health policy proposals and implementation, and there is an in-built process for continuous HSPA refinement and an on-going process of health objectives setting in Belgium.
The study visit occurred over two and half days aiming to explore functioning, maintaining and governance of the Belgian HSPA. It included meetings with the Institut National Assurance Maladie Invalidité (INAMI), The Belgian Health Care Knowledge Centre (KCE), Sciensano, Belgian federal MoH Santé Publique and the European Commission (DG Reform). The Czech delegation included seven senior-level representatives from the Ministry of Health, health insurers (VZP, VoZP), and the National Public Health Institute.
The Czech study visit group learned about the history of the Belgian HSPA, its initial intentions and how it evolved over time. The Belgian counterparts also shared information on the latest evolution in their HSPA framework development and details on how HSPA is embedded in the policy making, how key stakeholders are engaged, and what are the buy-in practices. One of the currently ongoing activities involves the multiannual budgeting process as the output of reflexion on Belgian HSPA, and another one focuses on the process of setting health objectives and methodology of implementing those objectives into Belgian healthcare insurance. Discussions were also held with the KCE and Sciensano on technical development of the HSPA report and the thematic reports, and the indexes review cycles.