This section provides information about the technical focus groups conducted to discuss data available for populating the HSPA framework with indicators. Data issues, policy priorities for measurement, and methodologies for calculation of individual indicators were also discussed during the focus groups. This annex is divided into two parts, the first of which provides an overview of the focus group meetings, including key conclusions drawn from the discussions. The second part provides a study guide for further HSPA-related capacity building in Estonia by using the recordings and notes from the focus group discussions. The study guide also includes proposed questions for reflection for students, which shall be reflected ahead of viewing the recordings.
Health System Performance Assessment Framework for Estonia
Annex C. Focus groups to inform indicator selection and HSPA capacity building study guide
Focus groups conducted to inform the selection of indicators
From September to December 2022, seven focus groups identifying themes and possible indicators for populating the Estonian HSPA framework were held. During the focus groups, OECD experts introduced indicators that are regularly collected in international databases, including the OECD Health Statistics, and discussed collection caveats and data comparability limitations, especially for data and indicators that Estonia is not collecting or submitting. This was followed by the discussions with Estonian experts and moderated by the OECD team. These discussions suggested themes and some possible indicators relevant to the Estonian context which could be included in the populated HSPA framework. The focus groups by theme and date were conducted as follows:
Focus Group #1: Workforce Capacity, Composition and Skills-Mix on 13 September 2022
Focus Group #2: Data and Digitalisation/Innovation on 13 September 2022
Focus Group #3: Patient-Experience and Health Literacy on 13 October 2022
Focus Group #4: Health Supportive Environment on 13 October 2022
Focus Group #5: Mental Health on 10 November 2022
Focus Group #6: Integrated Care and Access on 10 November 2022
Focus Group #7: Resilience on 7 December 2022
During the discussions, it was indicated that the Estonian health system generates large quantities of data that is already available. However, data use is fragmented because the collection is executed by different stakeholders. The use of data for secondary purposes and policy making is currently considered insufficient. Therefore, there is a lot of potential to build on the already existing data. However, it should not limit indicator selection – the need for information and understanding must be the basis of this process and the Estonian context must be considered. The main conclusions from each focus group discussion are highlighted in Figure A C.1.
Figure A C.1. Conclusions from focus groups 1‑4
Figure A C.2. Conclusions from focus groups 5‑7
Focus group discussion guides for future HSPA-related capacity building
This section develops the use of focus groups to guide educational activities related to capacity building. It can be used as a study guide accompanying the recordings of the focus groups shared with students learning about the HSPA in university courses, or as background material for HSPA-themed seminars (Section 5.3). A brief overview is provided of each of the Focus Groups held from September to December 2022, followed by questions to guide the use of the recordings for independent research, further reflection, and group discussions.
Each workshop started with an overview of the HSPA project and updates on the current state of the Estonian HSPA development process, followed by one or more thematic presentations. A group discussion followed the presentations in each of the focus groups to populate corresponding themes in the HSPA Framework.
Focus group 1: Workforce capacity, composition, and skills mix.
In focus group 1, the OECD presented on how shortages of healthcare workers are measured and what indicators are used for international benchmarking.
Questions to think about:
Why is it not so easy to measure shortages in healthcare workers?
By measuring what could you estimate shortages in healthcare workers?
What does the available data show about shortages in Estonian healthcare workers?
What kind of indicators does the OECD use for health employment and education?
What are the themes regarding the workforce capacity, composition, and skills mix that each key stakeholder speaks of that should be integrated to the Estonian HSPA?
Based on the discussion, could you name a few indicators that could be integrated to the Estonian HSPA?
Focus group 2: Data and digitalisation/innovation.
Focus group 2 included two presentations from the OECD on (1) indicators used monitoring electronic health record (EHR) adoption and maturity and (2) indicators used in monitoring telemedicine and remote care.
Questions to think about:
What are the groups of key indicators proposed by the OECD to monitor the adoption of EHR? Give a few examples of indicators in each indicator group.
What are the indicators on the impact of telemedicine?
What has changed between 2019 and 2020 in in-person doctor consultations and doctor teleconsultations in Estonia?
Name the three areas of OECD indicator development on digitalisation.
What are the themes regarding the data and digitalisation/innovation each key stakeholder speaks of that should be integrated to the Estonian HSPA?
Based on the discussion, could you name a few indicators that could be integrated to the Estonian HSPA?
Focus group 3: Patient experience and health literacy.
Focus group 3 included three presentations on (1) patient-reported experience measures, patient experience in care integration and patient-reported safety, (2) the Patient-Reported Experience Measures for Patients living with chronic health conditions (PaRIS) survey and (3) a short introduction to health literacy.
Questions to think about:
What are the seven principles for establishing national systems of patient experience measurement?
Give a few examples of OECD patient-reported experience measures that has been used in Estonia.
What is PaRIS?
What are the four sections in the PaRIS patient questionnaire?
Do PREMs only measure patient-centredness?
What is health literacy?
What are the themes regarding the patient experience and health literacy domains each key stakeholder speaks of that should be integrated to the Estonian HSPA?
Based on the discussion, could you name a few indicators that could be integrated to the Estonian HSPA?
Focus group 4: Health supportive environment and healthy choices.
Focus group 4 included a presentation on the indicators OECD uses for public and environmental health follows.
Questions to think about:
What is a health supportive environment?
Give a few examples of indicators on health supportive natural environment that the OECD uses. What are the other potential indicators?
Give a few examples of indicators on health supportive cities that the OECD uses. What are the other potential indicators?
Give a few examples of indicators on health supportive transport that the OECD uses. What are the other potential indicators?
Give a few examples of indicators on health supportive workplaces that the OECD uses.
Give a few examples of indicators on health supportive healthcare that the OECD uses. What does the OECD analysis suggest? What are the other potential indicators?
What are the themes regarding the domains of health supportive environment and healthy choices each key stakeholder speaks of that should be integrated to the Estonian HSPA?
Based on the discussion, could you name a few indicators that could be integrated to the Estonian HSPA?
Focus group 5: Mental Health.
In focus group 5, the OECD gave a summary about the indicators used for mental health, followed by a description of youth mental health and mental health system benchmarking.
Questions to think about:
What are the mental health indicators listed in the National Health Plan 2030?
What clinical indicators of mental health does the Estonian Health Insurance Fund use?
Name a few categories of mental health indicators that the OECD uses with an example of an indicator.
How does Estonia compare to other OECD countries regarding the indicator “death by suicide”?
What principles does the New Benchmark for Mental Health Systems assess and has Estonia given data about those principles?
What are the themes regarding the domain of mental health each key stakeholder speaks of that should be integrated to the Estonian HSPA?
Based on the discussion, could you name a few indicators that could be integrated to the Estonian HSPA?
Focus group 6: Integrated Care and Access (waiting times).
Focus group 6 included a summary of indicators that could be considered related to integrated care and access follows with presentations about the indicators OECD uses for those domains.
Questions to think about:
What are the core integrated care indicators?
What indicators could be considered in the domains related to advancement of integration, use of care services, health outcomes and experiences of care and quality of life?
On what does the OECD focus when gathering data about waiting times?
What are the seven elective surgical procedures that the OECD gathers data on and on which of them Estonia submits data?
How does Estonia compare to other OECD countries regarding hip and knee replacement waiting times?
Which health conditions did the OECD focus on during the 2021‑22 pilot data collection on integrated care?
Which three types of indicators were included to the 2021‑22 pilot data collection on integrated care?
How does Estonia compare to other OECD countries in regard to patient outcomes within the first year after discharge due to stroke and chronic heart failure?
What are the themes regarding the domain of integrated care and access each key stakeholder speaks of that should be integrated to the Estonian HSPA?
Based on the discussion, could you name a few indicators that could be integrated to the Estonian HSPA?
Focus group 7: Resilience.
Focus group 7 included a summary of the indicators for resilience that are used in other HSPAs, and by the OECD.
Questions to think about:
What are the resilience indicators used in other HSPAs?
How is health system resilience defined?
Describe the “shock cycle”.
What are the different approaches of measuring resilience?
What is hard to measure in relation to resilience?
Which themes are mentioned when focusing on resilience as capacity to adapt efficiently?
Which themes are mentioned when focusing on societal resilience of pandemics?
Which themes are mentioned when focusing on response and recovery from a shock?
What are the themes regarding resilience each key stakeholder speaks of that should be integrated to the Estonian HSPA?
Based on the discussion, could you name a few resilience indicators that could be integrated to the Estonian HSPA?
Access to the archives of the focus group discussions
Access to online recordings of the focus groups is available on request for educational capacity building purposes. Please send requests to Eleri Lapp (eleri.lapp@sm.ee).