The COVID‑19 pandemic has provided a great natural experiment in the utilisation of telemedicine services, creating opportunities to explore the impact of remote care services on different aspects of health care system performance, from quality, to access and cost-effectiveness. Yet only ten OECD countries have collected data or conducted studies on the impact of telemedicine on quality indicators since the start of the pandemic. This chapter provides an overview of how telemedicine services affect health care system performance, showing that there is clear evidence that remote care services are good value for money for patients, but that more work is needed to understand the impact on health system efficiency and waste.
The COVID-19 Pandemic and the Future of Telemedicine
OECD Health Policy Studies
3. The impact of telemedicine on health care system performance
Abstract
In the last OECD data collection on telemedicine (Oliveira Hashiguchi, 2020[1]), conducted before the COVID‑19 pandemic, country experts considered overwhelmingly that telemedicine services have a positive impact on several aspects of health system performance, from access to quality and cost-effectiveness (see Figure 3.1 for a depiction of the OECD Framework for Health Care System Performance used in this report). However, experts also stressed that telemedicine is only a tool and that, like any other tool, it can be well used or misused; it can have benefits but also has the potential to cause harm.
Before the start of the pandemic, there was already a substantial body of evidence showing that, when well used, telemedicine services can be effective, safe, patient-centred (responsive) and cost-effective (Eze, Mateus and Cravo Oliveira Hashiguchi, 2020[2]). The body of evidence has only grown since the start of the pandemic, and the findings of a new systematic literature review confirm that telemedicine services can be very beneficial (see Box 3.1). There are certainly limits to what can be done remotely, and not all uses of telemedicine are effective and/or cost-effective. If used inappropriately, telemedicine services may lead to increased workload for health professionals; increase unnecessary care among younger, healthier, and more affluent populations; promote cherry picking, fraud, and abuse; and put additional pressure on already strained government budgets for health. The key is understanding what are high-quality value‑adding uses of telemedicine that should be more widely adopted and made available to patients. Findings from the new literature review indicated that there is no shortage of good uses of remote care services, that are valued by both patients and health care providers, while being also cost-effective.
Figure 3.1. The OECD Framework for Health Care System Performance Measurement
Source: Adapted from OECD (2017[3]), Caring for Quality in Health: Lessons learnt from 15 reviews of health care quality, https://doi.org/10.1787/9789264267787-en.
Box 3.1. Systematic review of the literature on telemedicine during COVID‑19
A systematic review of the literature on telemedicine published between January 2020 and May 2022 was conducted to capture high quality evidence of the use of telemedicine in OECD countries since the onset of the COVID‑19 pandemic, and to clarify the impact of the pandemic on telemedicine use. The following five dimensions were considered most relevant:
User (e.g. patients, health care workers) satisfaction with telemedicine services.
Impact of telemedicine services on patient safety.
Utilisation rates by socio‑economic status and demographics.
Utilisation rates by type of telemedicine (video, remote monitoring, asynchronous).
Cost analysis and full economic evaluations, and pricing of telemedicine services.
A systematic search of the literature was performed on PubMed/Medline, the Health Management and Policy Database from the Health Information Management Consortium (HMIC), and the Cochrane Library, using controlled terms or free text depending on the database functionality (e.g. “Telemedicine (MESH term)” or “Telemedicine” free text). The keywords “Telemedicine”, “Telehealth”, and “Teleconsultation” were each searched to maximise results towards ensuring the search picked up all relevant publications in the set timeframe. The OECD countries were used as keywords in the search. A search of grey literature within the same time parameters was undertaken to identify possible additional studies that met the inclusion criteria.
The searches yielded 3 942 results after duplicates were removed, of which 2 646 were not directly relevant to the outcomes of the review and were excluded. Full texts of 1 296 studies were reviewed to assess eligibility, and 272 were eventually included in the review. The grey literature search yielded 5 additional relevant results. More details are provided in Annex C.
3.1. COVID‑19 has created opportunities for countries to measure the impact of telemedicine on health system performance, but not all countries have done so
The COVID‑19 pandemic has provided a great natural experiment in the utilisation of telemedicine services. The boom in telemedicine use has created opportunities to explore the impact of remote care services on different aspects of health care system performance, but not all countries have taken advantage of these opportunities to collect data and conduct studies (see Figure 3.2).
Almost two in three reporting countries (18 out of 28 countries) stated that their country’s Ministry of Health, a governmental agency, or an academic institution collected data or conducted studies on the impact of telemedicine on access to health care during the COVID‑19 pandemic. In 16 countries, data on patient and/or provider experience and outcome measures are available, but only nine countries have collected other indicators or metrics to assess the quality of telemedicine services (e.g. safety in telemedicine, telemedicine outcomes compared to in-person outcomes, avoidable admissions). Only nine countries (Belgium, Canada, England, Estonia, France, Israel, Mexico, Norway, and the United States) reported that there are data collections on all three aspects depicted in Figure 3.2: access, experience, and other quality indicators.
As discussed in Chapter 1, Section 1.4, administrative data on teleconsultations are very limited, with fewer than half of 31 OECD countries having data on patient characteristics, type of telemedicine service, reasons for telemedicine use, and subsequent care. Without such data, it is difficult to understand the impact of telemedicine on health care system performance. Rather, most data on access, equity and quality during the pandemic have been collected through surveys. Examples include Canada Health Infoway’s survey Canadians’ Health Care Experiences During COVID‑19 from January 2022 and the Patient and Clinician Survey conducted in Ireland in October 2020. Some surveys span both the periods before and after the start of the COVID‑19 pandemic including, for example, the General Practice Patient Survey in England, the National District Health Board Survey in New Zealand, and the National Electronic Health Records Survey in the United States.
Figure 3.2. Only nine OECD countries have collected data or conducted studies on the impact of telemedicine on quality indicators such as safety, health outcomes and avoidable admissions
Note: *Australia and Poland collect only data on patient experience and outcomes. Other metrics or indicators of quality include safety in telemedicine, telemedicine outcomes compared to in-person outcomes, avoidable admissions, etc.
Source: OECD Survey on Telemedicine and COVID‑19 (2022).
Some examples of the governmental institutions involved in collecting data and conducting studies on telemedicine are the Australian Institute of Health and Welfare, the Belgian National Health Care Knowledge Center, Canada Health Infoway, the French National Health Authority, and The National Telehealth Steering Committee in Ireland. In the Netherlands, there was an e‑health monitor set up in 2020 by the Ministry of Health, Welfare and Sport, which monitors the transition to digital health care and collects data on the utilisation, quality, and impact of telemedicine (this was an update to the 2013 e‑health monitor). In Canada, Ireland, Mexico, and the United States, there is also research conducted by institutions at a subnational level. While the pandemic certainly increased the level of interest, research on telemedicine was already well established before the pandemic in some OECD countries. For example, in France, the Cour des comptes, or Court of Accounts, published a report on telemedicine in 2017 and the Institute for Research and Information in Health Economics (IRDES in French) published a systematic review on the experiences of patients and providers with telemedicine in 2019. In Norway, an institute solely dedicated to the research on digital health, the Norwegian Centre for E‑health Research, was founded in 2011 and has collected a significant amount of data on telemedicine, even before the start of the COVID‑19 pandemic.
3.2. While crucial in maintaining access to care during the pandemic, telemedicine could not completely offset in-person care reductions for everyone
As detailed in Chapter 1, Section 1.3, reductions in the number of in-person health care services provided in OECD countries were partly offset by an unprecedented scale‑up of telemedicine services, following government policies to promote remote care. Provisional data from the 2022 OECD Health Statistics and ad-hoc data collection on teleconsultations show that the number of total doctor consultations increased in 2020 compared to 2019 in Australia, Denmark, and Norway (see Figure 3.2). The total number of consultations in Australia was actually 16% higher in April 2020 than in the same period in 2019, partly due to an increase in telephone and video consultations after the introduction of new telehealth Medicare Benefits Schedule items from March 2020 (AIHW, 2021[4]). COVID‑19 related in-person consultations dropped by 15.3% to 36.3 million consultations, while telephone consultations increased by 43.1% to 12.7 million consultations, and video conferencing consultations increased by 165.5% from about 615 000 to 1.6 million consultations (ibid). A 10% drop in in-person antenatal care between January and September 2020, compared to 2019, was almost entirely offset by an uptake of 91 000 telehealth services (AIHW, 2021[5]).
In Denmark, the number of in-person doctor consultations went down 0.9% while doctor teleconsultations increased by 8.1% in 2020 compared to 2019, causing the total number of doctor consultations to go up by 3.2%. In Norway, doctor teleconsultations increased 700% making up for a 15.9% drop in in-person doctor consultations and leading to a 2.3% increase in the total number of doctor consultations. In Reykjavik in Iceland, the use of web-based consultations increased by 213% and telephone consultations increased by 93%, while office visits decreased by 41%, leading to an overall increase of 45% in the total number of consultations in primary health care centres and out-of-hours services (Sigurdsson et al., 2020[6]). In Germany, the number of patients receiving psychotherapy services in-person had dropped by 18% by the end of the COVID‑19 lockdown, but this drop was completely offset by remote psychotherapy services resulting in an overall increase in the number of patients treated of 12% (Probst et al., 2021[7]). In the Czech Republic, a reduction of 71% in the number of patients receiving psychotherapy was also completely compensated by an increase in psychotherapy delivered via telemedicine (ibid).
Table 3.1. Teleconsultations partly compensated for reductions in in-person consultations
Doctor consultations per capita (total, in-person, and teleconsultations) in 2019 and 2020, and growth rates in doctor consultations per capita between 2019 and 2020, in OECD countries with available data
Note: All values are provisional; * Norway excludes teleconsultations by medical specialists; values for the Czech Republic are estimates. OECD9 shows unweighted averages. In Estonia, 2015‑19 data include teleconsultations with family physicians only (funded by the Health Insurance Fund), while from 2020, also specialist medical practitioners’ teleconsultations data are available.
Source: OECD Health Statistics (2022); OECD ad-hoc data collection on teleconsultations (2022).
In other countries, the total number of doctor consultations dropped in 2020 compared to 2019, despite the massive growth in the number of doctor teleconsultations. In Lithuania, for example, the number of doctor teleconsultations increased 43 500%; a hike that still could not offset a 34.4% drop in in-person doctor consultations. In England, the number of in-person primary care consultations fell by 2.3 consultations per person per year, only partially offset by an increase in remote consultations of 1.0 consultation per person per year (Watt et al., 2020[8]). In Austria, the number of patients receiving psychotherapy care in-person dropped 81% in the weeks following the introduction of COVID‑19 restrictions, while the number of patients receiving care via telephone increased 979% and via the internet 1 561% (Probst et al., 2021[7]). Still, growth in remote psychotherapy was not sufficient to compensate for the drop in in-person care, leading to an overall reduction of 28% in the number of patients receiving psychotherapy in Austria during the first weeks of the COVID‑19 lockdown. In the Slovak Republic, the number of in-person psychotherapy patients fell by 76%, and while telemedicine use increased, there was still an overall drop of 25% in the number of patients treated three weeks after the first COVID‑19 wave restrictions were relaxed (ibid).
The COVID‑19 pandemic has been a reminder that many health care services cannot currently be delivered remotely and require in-person appointments. There is no question that telemedicine played a vital role in ensuring access to, and continuity of, care during the COVID‑19 pandemic. However, as detailed in Chapter 1, Section 1.1, the impact of the pandemic on the delivery of care was far-reaching, affecting elective surgeries, preventive and emergency care (OECD, 2021[9]). As senior health officials in Canada, have put it, “there are limits to what can be done virtually” and “there are many patients for whom the standard of care cannot be met in a solely virtual care environment” (CMA, CFPC and RCPSC, 2022[10]). While 84% of the Canadian physicians agree that virtual care improves their patients’ access to their care, four in five physicians have experienced challenges with virtually examining patients and over half recognise the inability of some patients and communities to access virtual care (CMA and Canada Health Infoway, 2021[11]). Indeed, telemedicine services cannot currently substitute for many in-person health care services. One study of claims data in the United States estimated that 20% of all Medicare spending could cover virtual services, leaving a significant 80% of spending to be delivered in-person (Bestsennyy et al., 2021[12]).
3.2.1. Access to telemedicine among certain groups of patients remains a key concern
Previous OECD research argued that while older patients, those on lower incomes and those with lower educational attainment would most stand to benefit from the increased access to health services that telemedicine provides, they are also the most likely to lack the health and digital literacy to use telemedicine (Oliveira Hashiguchi, 2020[1]). While during the pandemic countries moved swiftly to promote the adoption and utilisation of telemedicine services, some barriers to telemedicine use are structural and less amenable to short-term regulatory and financing changes. There is a risk that the rapid uptake of remote care services during the pandemic may have exacerbated pre‑pandemic inequalities in access to care. The available evidence from OECD countries suggests that the impact of telemedicine on access to health care services among subgroups of patients after the start of the pandemic has been mixed, and possibly not as clear-cut as before the pandemic. However, access among older, poorer, and rural patients remains concerning, especially in some OECD countries.
The age distribution of telemedicine users varies across countries and, at least in the United States, seems to be changing with the pandemic. In Canada (Ontario), telemedicine use increased across all age groups, with the highest rates of use reported among older adults aged 65 and older (Chu et al., 2021[13]), while in England, patients older than 74 years were up to 28% more likely to have an in-person consultation than those aged 25‑44 years (Edge Health, 2021[14]). Data from the United States suggests that younger patient groups (aged under 55 years, especially those aged 31‑40 years) were most likely to use telehealth in 2020 (FAIR Health, 2022[15]; Cordina et al., 2022[16]), but that the age distribution of telemedicine patients is changing. Utilisation of telemedicine services among people aged 51 years and older went up and these patients now represent a larger share of all users of telemedicine (FAIR Health, 2022[15]). Furthermore, based on data from the Census Bureau’s Household Pulse Survey from April to October 2021, telehealth use rates were similar among most demographic subgroups but were much lower among young adults aged 18 to 24 years (Karimi et al., 2022[17]).
Patients living in rural areas still seem to use telemedicine services less than patients living in urban areas, based on available data, and evidence from the systematic literature review (see Annex C). In the United States, in urban areas, telehealth increased from 0.23% of medical claims in 2019 to 16.1% in 2020, while in rural areas, the increase was from 0.10% in 2019 to 9.8% in 2020 (FAIR Health, 2022[15]). Indeed, the increase in telemedicine use seems to have been greatest among patients in metropolitan areas (Samson et al., 2021[18]). In Canada (Ontario), before the pandemic, telemedicine use was steadily low in 2012‑19 for both rural and urban populations, but slightly higher overall for patients living in rural areas (Chu et al., 2021[13]). With the pandemic, the rate of telemedicine use increased among patients living in both rural and urban areas, but more markedly among the latter (ibid). In the first four months of the pandemic, in Canada (Ontario), residents in rural areas had lower use of telemedicine (Glazier et al., 2021[19]).
Patient income remains an important correlate of telemedicine use, although recent data from the United States suggests that the association between income and telemedicine use may be changing. InCanada (Ontario), patients in the highest income quintile had higher proportions of use of telemedicine during the first wave of the pandemic (Glazier et al., 2021[19]). In the United States, the increase in telemedicine use in 2020 was greatest among patients in counties with low poverty levels (Samson et al., 2021[18]). About two‑thirds of households with total income of USD 25 000 or more have access to providers that offer telemedicine services, compared to 58% of households with total income below USD 25 000 (Samson et al., 2021[18]). Indeed, over nine in ten beneficiaries with a total household income of USD 25 000 or more have access to the Internet compared to two‑thirds of beneficiaries with a total household income of less than USD 25 000 (ibid). However, more recent data from 2021 suggests that rates of telemedicine use were highest among patients earning less than USD 25 000 (Karimi et al., 2022[17]).
More recent analyses from the United States, using responses to the Census Bureau’s Household Pulse Survey from April to October 2021, show that there are significant differences across groups of patients in the use of audio versus video telehealth (Karimi et al., 2022[17]). Young adults, patients earning at least USD 100 000, patients with private insurance and White individuals have higher shares of use of video telehealth. Lower rates of video use were found among patients with a high school diploma, adults ages 65 and older, patients with low incomes, and Latino, Asian and Black individuals. Access to high-speed broadband and a lack of privacy at home or at work, are possible explanations for these differences in the use of audio and video modalities (Karimi et al., 2022[17]).
Language barriers can make it difficult to access health care services, both in-person and remotely, and hinder the quality of the health care delivered (Al Shamsi et al., 2020[20]). In studies from the United States (see Annex C), patients with a primary language other than English and Spanish were less likely to use telemedicine (Haynes et al., 2021[21]; Xiong et al., 2021[22]). However, it has been shown that online translation tools can increase the satisfaction of both medical providers and patients as well as improve the quality of care delivered and patient safety. These tools can likely be used in remote care services as well.
Telemedicine may be the only way to access care for difficult-to-reach patient groups
Some patient groups face significant barriers to accessing health care, be it because of cost, physical access or even privacy. Telemedicine services can help bring down those barriers. Australia, Denmark, France, Germany, Iceland, Italy, the Netherlands, Norway, Spain and Swedenall provide telemedicine services to seafarers on board ships (Oliveira Hashiguchi, 2020[1]). In Australia, the Centre for Antarctic, Remote and Maritime Medicine and the Australian Antarctic Program provide telemedicine services to patients up to 5 500 km away, including monitoring vital signs (ibid), and on-call telemedicine services with a specialist ophthalmologist are available to Aboriginal and Torres Strait Islanders (Nguyen, Baker and Turner, 2020[23]). Indeed, telemedicine services are used to promote better health care among indigenous people in Canada as well, where the Ontario Telemedicine Network provides virtual care to 80% of 133 First Nation Communities, one in four of which are only accessible by air or ice road (Ontario Telemedicine Network, n.d.[24]). Australia, Canada, Greece, Korea, Spain, United Kingdom and the United States all provide health care services to prison populations, reducing the need for potentially costly and risky transfers, and with high patient and provider satisfaction (Cuadrado et al., 2021[25]; Khairat et al., 2021[26]; Tian et al., 2021[27]).
3.3. While some health care workers are still sceptic and worried about quality, patient experiences with telemedicine are positive and satisfaction is very high
3.3.1. Physicians have mixed views of the quality of, and need for, telemedicine services
As stated, there is a substantial body of evidence showing that telemedicine services can be effective, safe, and patient-centred/responsive (Eze, Mateus and Cravo Oliveira Hashiguchi, 2020[2]). For example, in England, the Care Quality Commission – which conducts annual inspection of health care providers – published in 2019 an inspection showing that PushDoctor, a digital primary care provider, delivered safe, effective, and responsive services (CQC, 2019[28]). Yet, physicians’ views of the value and quality of health care services delivered through telemedicine vary significantly across and within countries.
In Australia, among fellows of the Royal Australasian College of Surgeons, 77% felt that satisfactory care could be delivered via telehealth in half or more consultations, but only 38% of respondents felt that the quality of care was equivalent to an in-person consultation (Wiadji et al., 2021[29]). Notably, respondents to a patient survey rated their telehealth consultations more highly than their treating surgeons with 93% being happy with the quality of their telehealth consultation and 75% rating it as equivalent to an in-person consultation (ibid). In Canada, 78% of the physicians agree that virtual care enables them to provide quality care for their patients (CMA, CFPC and RCPSC, 2022[10]). Over two‑thirds of physicians were satisfied with video visits and 71% were satisfied with telephone consultations (ibid). In Austria, despite the increased use of remote psychotherapy services during the pandemic, and positive experiences reported by physicians, health professionals do not believe that remote psychotherapy can substitute for in-person psychotherapy (Probst et al., 2021[7]). In Sweden, approximately four out of ten doctors do not want to work more with digital care visits at all (Vårdanalys, 2020[30]).
In survey data from 1 000 general practitioners in Norway who have provided video consultations during the pandemic, respondents considered that half of the almost 3 500 video consultations provided were as suitable or even more suitable to assess the main reason of contact, compared to in-person appointments (Johnsen et al., 2021[31]). Doctors considered that telemedicine was most suitable when there was already a relationship with the patient (rather than for a first contact), to follow-up on ongoing care (rather than for a new issue), and for specific conditions such as anxiety, depression, administrative matters and for chronic conditions that require multiple follow-up consultations (ibid). In a survey of 719 health care workers in Ireland (it is unclear how representative the sample is), 87% reported that they were extremely likely, very likely or likely to recommend video consulting to a friend or client and 49% reported unanticipated benefits from their use of the virtual platform (Lane and Clarke, 2021[32]). A third (32%) of respondents reported that it reduced non-attendance rates when compared with in-person consultations, and 29% reported that use of video consultations enabled them to increase the number of appointments offered to patients (ibid).
There is concern among health care professionals that scaling-up remote care services so quickly during the pandemic may have led to unnecessary and substandard care
Even before the pandemic, in 2019, the Swedish Medical Ethics Council expressed concern that a lack of regional integration of information systems could create risks for patients consulting remotely out-of-region (Statens medicinsk-etiska råd, 2019[33]). The Council was also concerned that providers of remote care were promoting their services to patients with health issues that were suitable for self-care, a situation that would likely lead to increased unnecessary health spending, and high opportunity costs in foregone care for patients with more serious health issues (ibid).
In Canada, there has been some concern about the appropriateness of care delivered virtually during the pandemic (CMA, CFPC and RCPSC, 2022[10]). There have been complaints to regulatory authorities from patients unable to access in-person care. Emergency physicians and other specialists have also expressed concern that patients referred to them have not been seen in-person or given an appropriate examination by a primary care physician. The College of Family Physicians of Canada has reported that “for-profit services offering access to episodic care by unfamiliar providers have increasingly been used to fill access gaps, threatening system efficiency and continuity of care” (CFPC, 2021[34]). To ensure that appropriate standards of care were maintained during the pandemic, regulatory authorities in several jurisdictions reviewed and updated remote care guidelines and the Canadian Medical Protective Association made available to their members a collection of articles, podcasts and learning activities related to virtual care (CMA, CFPC and RCPSC, 2022[10]).
3.3.2. Patients are overwhelmingly satisfied with telemedicine services
There is much more agreement on the value of telemedicine services among patients, across and within countries, than among physicians. The Health Foundation, in England, analysed 7.5 million patient-initiated requests for primary care made using the askmyGP online consultation system between March 2019 and September 2021 at 146 general practices with 1.35 million patients (Clarke, Dias and Wolters, 2022[35]). They found that, before the pandemic, 30% of patients stated a preference for in-person care, but at the start of the pandemic, fewer than 4% of patients did so. After the start of the pandemic, requests for in-person consultations increased but still represented only 10% of all requests by September 2021. There are significant differences among both patients and providers regarding which communication technology to use for remote care services (see Box 3.2).
Box 3.2. More research is needed to understand how different communication technologies, such as video and telephone, are associated with health care quality and patient satisfaction
There is limited understanding of which communication technology – video, telephone, e‑mail, text messaging – is most appropriate/desirable for a specific patient, health care setting, specialty, health issue, and health care professional. Based on available data, the choice of communication technology is not clear-cut. For instance, in England, general practitioners consider that video consultations provide little added value over telephone consultations (Greenhalgh et al., 2022[36]). In another study in England, 55% of patients stated a preference for telephone consultations after 2020 (compared to 44% pre‑pandemic), a third of patients preferred text messaging or e‑mail, and less than 1% of patients asked for a video consultation, with significant subnational variation in these preferences (Clarke, Dias and Wolters, 2022[35]). In Canada, general practitioners are more satisfied with telephone consultations and specialists are more satisfied with video consultations (CMA and Canada Health Infoway, 2021[11]).
In Sweden, people aged 60 years or older prefer telephone consultations to video, text messaging or e‑mail, while younger patients prefer video, text messaging and e‑mail (Vårdanalys, 2020[30]). As previously mentioned, in the United States, young adults, patients earning at least USD 100 000, patients with private insurance and White individuals have higher shares of use of video telehealth, with lower rates of video use found among patients with a high school diploma, adults ages 65 and older, patients with low incomes, and Latino, Asian and Black individuals. Beyond preferences and potential barriers to access, it is important to understand which communication technology is most cost-effective in which conditions.
In an Australian survey, based on 1 166 consultations, 94% of patients were satisfied with the quality of their surgical telehealth consultation and 75% felt it delivered the same level of care as in-person consultations (Wiadji et al., 2021[29]). Telehealth was considered convenient to use by 96% of respondents. In a survey of 1 000 Austrians, older age groups were more likely to be satisfied with their telephone or video consultation than patients in younger age groups (Kletečka-Pulker et al., 2021[37]). Just 1% of patients in the age group 45‑59 and 2% of patients in the age group above 60 years indicated that they were rather unsatisfied with their teleconsultations. Perhaps unexpectedly, only 60% of patients aged 16‑29 felt they were sufficiently understood during teleconsultations, compared to 92% of patients over 60 years old. Overall, nine in ten respondents were satisfied or very satisfied with their teleconsultations. In a survey of over 5 000 Belgian patients, around three in four respondents (77%) were generally satisfied or very satisfied with teleconsultations, and only 4% were not satisfied at all (Avalosse et al., 2020[38]). As in Austria, older respondents were more likely to be satisfied.
In Canada, a national poll of 1 800 people conducted by Abacus Data between 14 May‑17, 2020, found that 91% of patients who connected with their doctor virtually during COVID‑19 were satisfied, which is 17 percentage points higher than the satisfaction rate for in-person emergency room visits (CMA and Abacus Data, 2020[39]). In another Canadian survey of more than 3 000 people with stroke, heart disease or vascular impairment, conducted in the spring of 2021, found that 80% of respondents had a virtual appointment during the pandemic, with about four in five reporting that they found virtual appointments to be convenient (Heart and Stroke Foundation, 2021[40]). Moreover, 60% or respondents stated that virtual care was as good as in-person appointments. In yet another Canadian survey of over 12 000 people conducted between 14 July and 6 August 2021, nine in ten respondents who had used teleconsultations were satisfied (Canada Health Infoway, 2022[41]). Furthermore, 89% of respondents felt they were involved in the decision making around their care and 88% felt the visit was effective in helping with the health issue they consulted about. For patients using e‑mental health services, 84% were satisfied and 81% felt the visit was effective in helping with the health issue they consulted about (ibid). An astounding 74% of users of e‑mental health services agreed that remote care had helped them deal with a moment of crisis and distress that would have resulted in physical harm or suicide.
In a survey of 696 patients in Ireland (it is unclear how representative the sample is), 95% of patients reported that they were likely, very likely, or extremely likely to recommend remote care to a friend, and 91% reported that they were able to communicate everything they wanted to their health care professional (Lane and Clarke, 2021[32]). The majority (81%) of patients reported that their experience of video appointments was the same (50%) better (15%) or significantly better (16%) than previous in-person appointments. In a survey of almost 14 000 patients in Poland, over nine in ten respondents who had used teleconsultations reported that their problem was resolved, and 96.8% were satisfied, stating that they had received comprehensive information about their condition and medication (National Health Fund, Department of Patient Services (Departament Obsługi Pacjenta, Narodowy Fundusz Zdrowia), 2020[42]). Concerns that older adults would not be able to manage using telemedicine and that remote care services be inaccessible to wide range of patients were not confirmed (ibid). In the United States, among respondents to multiple waves of McKinsey Physician Surveys and McKinsey Consumer Surveys conducted in 2020 and 2021, two‑thirds of physicians and 60% of patients agreed that virtual health is more convenient than in-person care for patients, but only 36% of physicians agreed remote care was more convenient for themselves (Cordina et al., 2022[16]).
3.3.3. Around two in five patients who used remote care services during the pandemic prefer these to in-person consultations, and want to continue using telemedicine
Surveys from across OECD countries indicate a desire from patients who have used telemedicine services during the pandemic to continue using remote care services in the future. In Australia, 41% of patients who participated in telehealth surgical consultations indicated they would prefer telehealth (24% video-link and 17% telephone) over in-person appointments in the future (Wiadji et al., 2021[29]). In Canada, almost half (46%) of respondents who used virtual care after the start of the pandemic stated they would prefer a virtual appointment as a first point of contact with their doctor (CMA and Abacus Data, 2020[39]). In Israel, around 82% of men, 73% of women, around 80% of patients with chronic conditions, and 73% of patients without chronic conditions agreed that they would continue to use telemedicine in the future (Reicher, Sela and Toren, 2021[43]). In Poland, 43% of respondents believe that telehealth should be one of the main ways to contact their primary health care provider in the future (National Health Fund, Department of Patient Services (Departament Obsługi Pacjenta, Narodowy Fundusz Zdrowia), 2020[42]). In the United States, 40% of consumers surveyed in May 2021 said they believe they will continue to use telehealth, while in November 2021, 55% of consumers said they were more satisfied with telehealth visits than with in-person appointments (Cordina et al., 2022[16]). Patients in Belgium consider remote care visits particularly useful for administrative matters, in particular drug prescribing and monitoring of chronic diseases (Avalosse et al., 2020[38]). Around 70% of Belgian respondents indicate that an in-person consultation is the most appropriate and desirable way to see their health care provider and 62% agree that teleconsultation should be the exception rather than the rule (Avalosse et al., 2020[38]).
Compared to patients, physicians have more mixed views of the role of remote care services in a new phase of the pandemic in which most people are vaccinated and in-person services have mostly resumed. Some physicians are interested in continuing to provide remote care. In Australia, 85% of surgeons surveyed by the Australasian College of Surgeons expressed a desire to continue providing access to telehealth (Wiadji et al., 2021[29]). In Canada, nearly 25% of physicians expect to increase their use of virtual care in the future, 20% expect their use to remain stable, and only 4% expect to stop using remote care (CMA and Canada Health Infoway, 2021[11]). In Norway, general practitioners estimate that they will conduct about one in every five consultations by video in the future (Johnsen et al., 2021[31]). In England, in a June 2020 survey of over 2000 general practitioners, 88% felt that greater use of remote consultations should be maintained in the longer term (Morris, 2020[44]). In Sweden, just over half of doctors surveyed state that they want to work more with digital care visits than they do today (Vårdanalys, 2020[30]).
Some physicians expect to reduce their use of telemedicine services or even stop using them altogether following a period of increased use in 2020. In Sweden, approximately four in ten doctors do not want to work more with digital care visits at all (Vårdanalys, 2020[30]). In the United States, 62% of physicians state that they would recommend in-person care over remote care to patients (Cordina et al., 2022[16]). Furthermore, physicians expect teleconsultations to drop by one‑third compared to the number of remote care services they provided during the pandemic. Over half of physicians reported being less likely to provide remote care services if provider payment rates were 15% lower than for equivalent in-person services (ibid). With increasing numbers of patients in the United States demanding telemedicine services, physicians who are reluctant to provide virtual care may lose patients in favour of a growing number of remote care providers that are able to meet patient demand (Cordina et al., 2022[16]).
Evidence from the systematic literature review (see Box 3.1) confirms high user satisfaction, not only among patients but also among providers, although providers do tend to have more mixed views (see Annex C). Several studies, from different OECD countries, indicate that high proportions of patients would be happy to continue using telemedicine services, and would even prefer them over in-person care. Examples include access to early medical abortion care in the United Kingdom (Aiken et al., 2021[45]); follow-up of patients with a stoma in Norway; people receiving mental health care in a panel of OECD countries (Barnett et al., 2021[46]); cancer care in Canada (Berlin et al., 2021[47]); patients with gestational diabetes in a panel of OECD countries (Bertini et al., 2022[48]); older adults with a pacemaker in Norway (Catalan-Matamoros et al., 2020[49]); communications with older adults with hearing loss in Australia (Convery et al., 2020[50]); adults being treated in the general surgery department in Spain (Cremades et al., 2020[51]); community-based gastroenterology in the United States (Dobrusin et al., 2020[52]); populations with chronic neurological dysfunction in multiple OECD countries (Gopal et al., 2022[53]); adults with major depressive disorder in the United States (Guaiana et al., 2021[54]); patients with COVID‑19 in Korea (Jang et al., 2021[55]) and the United States (Legler et al., 2021[56]); anaesthesia preoperative evaluations in the United States (Kamdar et al., 2020[57]); urological patients in Spain (Leibar Tamayo et al., 2020[58]); oncology care in rural areas in a panel of OECD countries (Morris, Rossi and Fuemmeler, 2021[59]); patients with rheumatoid arthritis in France (Pers et al., 2021[60]); child neurology outpatient care in the United States (Rametta et al., 2020[61]); patients with a medical diagnosis of schizophrenia and bipolar disorder in Germany (Stentzel et al., 2021[62]); outpatient sports medicine in the United States (Tenforde et al., 2020[63]); women at risk of developing preeclampsia in the Netherlands; and older adults with cognitive impairment in a panel of OECD countries (Yi et al., 2021[64]). Having the choice to use telemedicine services was considered important (Pogorzelska and Chlabicz, 2022[65]).
Technological barriers and service design flaws can limit the benefits of telemedicine services or even make them unsafe and unusable. One study found that 36% of participants experienced anxiety using mobile health technology (Fraser et al., 2022[66]). In another study in the United States, patients highlighted technical difficulties, and were especially concerned with incomplete health information being used to make care decisions (Gomez-Roas et al., 2022[67]). Other technical difficulties mentioned in peer-reviewed studies from a panel of OECD countries included Internet access (especially in rural locations with low connection speeds), the costs of technologies and hardware, and communication barriers (Peden, Mohan and Pagan, 2020[68]). Moreover, it is important to note that some patients prefer in-person appointments (Hadeler, Gitlow and Nouri, 2021[69]; Martinez et al., 2020[70]; Zischke et al., 2021[71]; Taxonera et al., 2021[72]; Schultz et al., 2021[73]). It is important that patients are offered in-person alternatives, whenever possible, if they are not comfortable receiving care remotely through telemedicine. Furthermore, telemedicine services will not completely substitute for face‑to-face care, so the interface between telemedicine and face‑to-face care needs to be carefully assessed.
3.3.4. Telemedicine in the context of national policies on quality of health care
According to the OECD Survey on Telemedicine and COVID‑19, only 12 countries refer to telemedicine in national legislation or policy on quality of care. Eleven countries report that there is no reference to telemedicine services in legislation on quality of health care while in two countries, Slovenia and Finland, there is no national legislation and/or policy on health care quality in general.1 In France, Ireland, Japan, Mexico, Norway and Türkiye, the Health Ministry or a similar authority is responsible for overseeing quality of telemedicine services. In England, the National Health Service has published policy and regulatory requirements on telemedicine (the terms used in England are online access or remote consultation). Ireland similarly reported that the Health Service Executive, which is the publicly funded health care system in Ireland, has worked on this. The Netherlands reported that, within their framework on quality of health care, telemedicine services should fulfil the same quality criteria as in-person care, while there are some additional criteria given the specifics of remote care at home.
In some countries, several authorities are involved in assessing quality of telemedicine care services. New Zealand, for instance, reports that the Health Quality and Safety Commission, the Health and Disability Commissioner, and the colleges of medical practitioners are all in some way involved in defining and ensuring health care quality in general, including telemedicine. Likewise, in France, besides the French National Health Authority, the National Consultative Ethics Council also works on the quality of telemedicine services. In Norway, the Norwegian Board of Health Supervision does regular information and communication technology audits. In the United States, the National Quality Forum, a non-profit organisation, has referenced the need for measure development for telehealth and worked on how rural telehealth supports health care system readiness and health outcomes during emergencies. In Canada, provinces and territories have primary jurisdiction over the administration and delivery of health care. The Canada Health Act, which governs the delivery and provision of health care in Canada, does not explicitly reference virtual care and/or telemedicine services.
Professional communities or colleges that represent, support, or regulate different medical groups are also typically involved in discussions surrounding the quality of care delivered via telemedicine. In France, for instance, all the colleges of medical practitioners now have a telehealth statement. In Ireland, the National Health care Communication Programme – designed to support health care staff to learn, develop and maintain their communication skills with patients, their families and with colleagues – has published telehealth guides for clinicians. In Israel, a Telehealth Community was established following the start of the COVID‑19 pandemic, which operates ten groups of therapists in various professional fields (e.g. oncology, paediatrics, psychiatry, geriatrics etc.). As part of the activities of the professional groups, there are dialogues between therapists in which patient and/or provider experiences with telemedicine are shared. Belgium, Finland and Poland report that they have plans to make or update guidelines or legislations on health care quality that specifically reference telemedicine.
3.4. Telemedicine services are very good value for money for patients, but more work is needed to understand the impact on health system efficiency and waste
While it is challenging to make broad statements regarding the cost-effectiveness of telemedicine services due to the heterogeneity of applications and differences in how remote care is delivered, there is evidence that telemedicine interventions can be cost-effective (Eze, Mateus and Cravo Oliveira Hashiguchi, 2020[2]). Besides being clinically effective, telemedicine interventions can deliver value for money by reducing the workload of health care workers, reducing waiting and travelling times, reducing unnecessary in-person care, shortening the length of consultations, and having lower unit costs than in-person care services (Oliveira Hashiguchi, 2020[1]). There is plenty of evidence that telemedicine services reduce patient costs and save patients time, however more research is needed to understand the impact of telemedicine on overall health care utilisation and, especially, on wasteful health spending.
3.4.1. Telemedicine services save patients time and money
As discussed in Section 3.3.2, patient satisfaction with telemedicine services is consistently high across OECD countries, with large majorities of patients stating that they would recommend remote care to friends, that their health issues were well addressed, and that they considered teleconsultations as effective as in-person consultations. Many country surveys, as well as peer-reviewed studies (see Annex C), also show that patients generally save time and money by using remote care, making telemedicine services very good value for money. In an Australian survey, based on 1 166 consultations, 60% of patients reported cost savings due to teleconsultations, and 77% felt that their telehealth appointment was good value for money (Wiadji et al., 2021[29]). Most patients (51%) avoided travelling less than 15 km to see their surgeon in-person, while 13% would have had to travel more than 150 km if teleconsultations were not available and they had to see their surgeon in-person. Besides saving money and time with transportation, patients reported avoiding costs from taking time off work, accommodation, and childcare.
An analysis of responses to a survey of patients who used a video consultation platform, totalling 3 million teleconsultations, in England, between 1 April 2020 and 31 March 2021, estimated that remote care saved patients 2.25 million hours of waiting time, with almost 99 000 hours saved in urgent and emergency care settings (Edge Health, 2021[14]). The same study estimated that video consultations saved patients a combined 530 years of travel and waiting time and GBP 40 million in patient travel costs. As a result, more than 14 200 tonnes of greenhouse gas emissions were avoided, and 3 million work hours potentially added to the economy or GBP 108 million in labour productivity gains. Moreover, while 6% of in-person outpatient appointments resulted in a missed attendance by patients, only 4% of remote attendances were missed, an apparently small difference that is however estimated to have led to nearly 55 800 fewer missed attendances due to use of video consultations (ibid). In another survey conducted between April 2020 and March 2021 in England, video consultations contributed to a 10‑fold reduction in the time patients waited immediately before an appointment.
In a survey of 696 patients in Ireland (it is unclear how representative the sample is), there was a 32% reduction in non-attendance rates in teleconsultations compared to in-person appointments (Lane and Clarke, 2021[32]). The majority (86%) of patients reported that they would usually travel to an in-person consultation by private car, on average travelling 33 miles for an in-person appointment. In Israel, the Maccabi Telecare Center, a multi-disciplinary health care service providing remote care to complex chronic patients founded by Maccabi Healthcare Services, entails lower overall average monthly costs for frail patients and is considered to be a cost-effective way to improve care quality and health outcomes in frail older patients (Porath et al., 2017[74]). In Canada, in a survey of over 12 000 people conducted in mid‑2021, patients using teleconsultations instead of in-person care reported saving on average CAD 53.28 by not having to arrange for care for a dependent (Canada Health Infoway, 2022[41]). Respondents also saved on average CAD 55.92 by not having to take time off work and CAD 36.76 by avoiding travel and associated costs (e.g. parking expenses). In one study from the United States, teledermatology resulted in average travel savings of 163 minutes, 145 miles and USD 60 per person (Lee, Dana and Newman, 2020[75]). In a systematic review and meta‑analysis of virtual preoperative anaesthesia assessments in a panel of OECD countries, studies reported savings of between 24 and 137 minutes, and between USD 60 and USD 67 per patient (Zhang et al., 2021[76]).
3.4.2. It is unclear whether remote care substitutes for or complements in-person care, and if telemedicine is a complement whether it adds value or is wasteful
Spending on telemedicine services is wasteful when it does not deliver benefits (or is even harmful), and when it could be replaced with cheaper alternatives with identical or better outcomes (i.e. alternatives that provide better value for money). On the one hand, there is a good deal of data suggesting that telemedicine services reduce subsequent health care utilisation (especially more costly services like emergency care and hospitalisations) and lower the probability of patients missing appointments (a phenomenon that causes disruption to both health care providers and patients who would have attended). On the other hand, teleconsultations can lead to subsequent in-person care (in which case perhaps an in-person appointment would have been more appropriate than the teleconsultation), and under certain provider payment schemes may lead to higher spending at no extra value for health systems and patients. The evidence on health worker productivity is somewhat mixed, with some studies finding that teleconsultations increase the number of patients that can be seen in the same amount of time (Lane and Clarke, 2021[32]) but that health care workers may have higher workloads (BMA, 2020[77]).
Having granular data on why patients use teleconsultations and on subsequent use of health care services is key to understanding the impact of telemedicine on health system efficiency. A key underlying uncertainty is whether telemedicine services substitute for or complement in-person care, and if remote care is a complement, then whether it is good value for money or wasteful. The answers to these questions are not clear. In a 2021 Canadian survey of over 12 000 people, 81% of people using video consultations and 77.1% of e‑mental health patients reported that remote care had avoided them at least one in-person visit to a doctor or emergency room (Canada Health Infoway, 2022[41]). In the same survey, 11% of virtual visits resulted in a patient referral to an in-person appointment with a specialist and 10% in advice to patients to make an in-person appointment with their family doctor. In a national survey of 1 800 people conducted in 2020, 50% believed that remote care will reduce total costs for the health care system, and only 8% expect telemedicine to increase total health spending (CMA and Abacus Data, 2020[39]).
In England, an analysis of what happened to patients who attended outpatient appointments shows that 18% of patients were discharged following a telephone or telemedicine appointment in April 2020, compared to 25% in February 2020, while the proportion of patients discharged after in-person appointments remained consistent at around 22% (Morris, 2020[44]). The same analysis also shows increased prescribing and referrals following a teleconsultation. The reasons behind these trends are unclear (e.g. change in the case mix and/or defensive medicine), and more recent data for 2021 are not yet available.
One challenge, discussed in Chapter 1, Section 1.4.1, is the lack of data collection in OECD countries on the reasons why patients used teleconsultations (available in 17 out of 31 reporting countries) and on subsequent use of health care services following a teleconsultation (available in ten out of 31 reporting countries). Such data represent the first step in understanding whether telemedicine services represent good value for money for health care systems. A survey of over 5 000 Belgian patients (Avalosse et al., 2020[38]) found that the main reason for a teleconsultation was to follow-up on an existing or chronic disease (27% of teleconsultations). Other reasons included asking for a drug prescription (21%), discussing symptoms related to COVID‑19 (17%), discussing a new symptom unrelated to COVID‑19 (16%), obtaining a medical certificate (15%), and requesting information on the results of a test (5%). Following-up on an existing or chronic condition was frequently cited as a reason for teleconsulting with medical specialists (67%), psychiatrists (80%) and psychologists (72%), while obtaining a drug prescription was often the reason to remotely consult with a general practitioner (24%).
Analyses from Sweden show that, before 2018, users of remote care had lower primary care utilisation rates than users of in-person care services, but that in 2018 the opposite became true (Vårdanalys, 2020[30]). While remote care services did replace some in-person care, overall, they led to higher numbers of total consultations. A more in-depth analysis showed that remote care visits led to a higher number of follow-up appointments than in-person consultations, and that 10% of remote care users had at least six visits (either teleconsultations or in-person). The difference in follow-up rates between teleconsultations and in-person appointments was clearest in the first month after the appointment, and in a longer time window the differences disappeared. It is unclear, however, whether increased utilisation was a consequence of underlying health care needs that were previously unmet or whether they represented unnecessary demand. Furthermore, subnational variation in provider payment and regulation of remote care services in Sweden also make it difficult to understand whether telemedicine services are good value for money (see Box 3.3).
Box 3.3. Regional variation in provider payment and regulation of remote care services in Sweden may lead to different care provision arrangements and be conducive of variations in outcome
The introduction and expansion of the remote care services in Sweden has mainly taken place in primary care because of legislation and regulations for out-of-region care, that is care provided by a region other than the patient’s “home region”. Before digital care was accepted as part of out-of-region care, a small number of private companies had agreements on digital platforms with some county councils. With new nationally regulated freedom of choice in outpatient care, including primary care, remote care services established in just a couple of regions became available nationwide. When a person receives out-of-region care from a digital care provider, the patient’s “home region” is responsible for payment and is invoiced for the remote consultation. The patient pays a co-payment fee according to the rules and guidelines that apply to digital visits in the region where the remote care companies have an agreement.
In 2016, companies based in the Jönköping region provided most of the remote care in Sweden. Remote care services expanded significantly since 2016 and then, in 2020, most remote care providers moved to the Sörmland region, a move that is believed to have followed an agreement for patients to be charged SEK 0 in that region. This effectively made remote care visits completely free of charge for the entire Swedish patient population.
In March 2018, the board of the Swedish Association of Local Authorities and Regions (SKR in Swedish) recommended that all regions adopt a minimum patient fee for digital visits of SEK 100. The Sörmland region first chose not to follow this recommendation, but since 2019 the patient fee is SEK 100 in the region. Another six Swedish regions have also chosen a patient fee of SEK 100 for digital visits to primary care, while eight regions have chosen SEK 200 and three regions SEK 300. Naturally, these differences in regional patient fees create complicated incentives for providers, and it remains unclear how they affect patient experience and outcomes.
Source: Vårdanalys (2020[30]), “Three perspectives on digital care visits – the perceptions of the population, patients and healthcare”.
Several peer-reviewed studies suggest that telemedicine services reduce hospital admissions (see Annex C). A systematic review of the effects of telemonitoring on health care use and costs in patients with chronic heart failure in a panel of OECD countries found cost reductions associated with fewer hospitalisations. In a systematic review of eHealth interventions (including telemedicine and remote consultations) to support patients on peritoneal dialysis and their caregivers, five studies reported lower rates of hospitalisations leading to lower costs (Cartwright et al., 2021[78]). In Spain, a study found that mobile health-enabled integrated care reduced overall expenses because of a reduction in unplanned visits and hospital admissions (Colomina et al., 2021[79]). In Australia, a novel smartphone app-based model of care targeting patients after admissions for acute coronary syndrome and heart failure, resulted in a statistically significant reduction in six‑month readmissions (Indraratna et al., 2021[80]). A systematic review of telemonitoring interventions for chronic obstructive pulmonary disease in various OECD countries found that adding telemonitoring to usual care led to reductions in unnecessary emergency department visits (Jang, Kim and Cho, 2021[81]). In a randomised controlled trial of telerehabilitation for late‑stage cancer patients in the United States, the telemedicine service led to total inpatient hospitalisation costs that were significantly lower than those of enhanced usual care (Longacre et al., 2020[82]). Also in the United States, a randomised controlled trial of a telephonic diabetes self-management intervention found reductions in hospital use and costs (Tabaei et al., 2020[83]).
Beyond the question of whether telemedicine services are substitutes or complements to in-person care, estimates from an English study suggest that the use of remote care services during the pandemic may have reduced the number of hospital associated infections, and led to savings of millions of pounds in avoided use of personal protective equipment (Edge Health, 2021[14]).
3.5. Despite considering that the use of remote care is associated with some risks, country experts see most risks as unlikely to materialise
There is concern that the rapid scale‑up in telemedicine use during the COVID‑19 public health emergency may have led to inappropriate remote care and risks to both patients and health care systems. As part of the OECD Survey on Telemedicine and COVID‑19, country experts were asked to assess the likelihood and potential impact of possible risks associated with telemedicine services (see Table 3.2). Experts from 22 OECD countries responded to this section of the questionnaire.
Table 3.2. Country experts tend to rate the likelihood of possible risks of telemedicine materialising as unlikely, but consider that the impact of some risks could be moderate or even high
Agreement among experts (in 22 countries) on the likelihood and expected impact of possible risks of remote care
Note: Answers from England are for general practice.
Source: OECD Survey on Telemedicine and COVID‑19 (2022).
For most potential risks associated with remote care services, most country experts agreed that it was unlikely that these risks would materialise. However, for three of the purported risks, country experts were split on whether the risk was unlikely or likely to materialise. These three risks were related to the potential of telemedicine services to compound inequalities in access to care and/or care outcomes, and the vulnerability of remote care services to data and privacy breaches as well as to ransomware attacks. In the context of health care system performance, it is noteworthy that a prominent majority of country experts considered it unlikely that telemedicine services promote duplication of care and unnecessary procedures/visits, or that they increase the risk of malpractice and medical negligence.
Expert agreement on the impact of purported risks, should they materialise, was mixed. While the possibility that telemedicine services may result in incorrect diagnoses was considered the most unlikely risk to materialise, it was the only risk for which most country experts rated the expected impact as high. The risk that telemedicine services may be vulnerable to ransomware attacks that limit care provision was considered to have a moderate to high impact, should it materialise. Should telemedicine services compound inequalities or be vulnerable to data and privacy breaches, most country experts considered that this could have a moderate impact.
Combining expert assessments of the likelihood and expected impact of possible risks of telemedicine services shows that experts are most concerned about the potential for telemedicine to compound inequalities and to be vulnerable to data breaches and ransomware attacks. Among the 22 countries that provided responses, experts from Mexico, Slovenia and Sweden considered risks were more likely to materialise than experts in other countries, and experts from Latvia, Slovenia and Sweden rated the expected impact as high more often than experts in other countries. Experts from Costa Rica rated all possible risks of remote care as unlikely and low impact, while experts from Norway considered all risks as very unlikely and the impact of all but two risks as low.
The country expert assessments of risks associated with telemedicine services, elicited through the OECD Survey on Telemedicine and COVID‑19, are in broad agreement with evidence from the systematic literature review, which overwhelmingly suggests that telemedicine services are at least as safe as in-person care in a wide range of medical specialties and diseases, and using different remote care technologies (see Annex C). In some cases, telemedicine services – especially remote patient monitoring – have the potential to produce early warnings of deterioration, which can boost patient safety (Breteler et al., 2020[84]).
For the most part, the COVID‑19 pandemic does not seem to have had an impact on the safety of health care delivered through telemedicine (Joshi et al., 2021[85]), with most services being as safe as in-person care both before and after the start of the pandemic. However, as with in-person services, telemedicine services need to be well designed to be safe and effective, and well received by both patients and providers. In a systematic review of randomised controlled trials of eHealth interventions aimed at solid organ transplant recipients in a panel of OECD countries, some trials reported failures with the technology used, anxiety, and privacy concerns (Tang et al., 2020[86]). In a retrospective cohort study from the United States, rapid implementation of telemedicine services suffered from poor audio and video quality, with a significant proportion (40%) of encounters being affected (Rametta et al., 2020[61]). Finally, in a cross-sectional study in Germany, satisfaction with video‑electroencephalography-monitoring systems at epilepsy centres was low, with hardware or software errors affecting diagnoses (Willems et al., 2021[87]).
References
[4] AIHW (2021), Impacts of COVID-19 on Medicare Benefits Scheme and Pharmaceutical Benefits Scheme: quarterly data, Impact on MBS service utilisation, Australian Institute of Health and Welfare, Canberra, https://www.aihw.gov.au/reports/health-care-quality-performance/impacts-of-covid19-mbs-pbs-quarterly-data/contents/impact-on-mbs-service-utilisation (accessed on 5 May 2022).
[5] AIHW (2021), New report looks at uptake of telehealth in antenatal care during COVID-19 lockdowns, Australian Institute of Health and Welfare, Canberra, https://www.aihw.gov.au/news-media/media-releases/2021-1/february/new-report-looks-at-uptake-of-telehealth-in-antena (accessed on 5 May 2022).
[45] Aiken, A. et al. (2021), “Effectiveness, safety and acceptability of no-test medical abortion (termination of pregnancy) provided via telemedicine: a national cohort study.”, BJOG : an international journal of obstetrics and gynaecology, Vol. 128/9, pp. 1464-1474, https://doi.org/10.1111/1471-0528.16668.
[20] Al Shamsi, H. et al. (2020), “Implications of Language Barriers for Healthcare: A Systematic Review”, Oman Medical Journal, Vol. 35/2, p. e122, https://doi.org/10.5001/OMJ.2020.40.
[38] Avalosse, H. et al. (2020), Intermutualist survey teleconsultations, CIN-NIC.
[46] Barnett, P. et al. (2021), “Implementation of Telemental Health Services Before COVID-19: Rapid Umbrella Review of Systematic Reviews.”, Journal of medical Internet research, Vol. 23/7, pp. e26492-e26492, https://doi.org/10.2196/26492.
[47] Berlin, A. et al. (2021), “Implementation and Outcomes of Virtual Care Across a Tertiary Cancer Center During COVID-19.”, JAMA oncology, Vol. 7/4, pp. 597-602, https://doi.org/10.1001/jamaoncol.2020.6982.
[48] Bertini, A. et al. (2022), “Impact of Remote Monitoring Technologies for Assisting Patients With Gestational Diabetes Mellitus: A Systematic Review.”, Frontiers in bioengineering and biotechnology, Vol. 10, p. 819697, https://doi.org/10.3389/fbioe.2022.819697.
[12] Bestsennyy, O. et al. (2021), Telehealth: A quarter-trillion-dollar post-COVID-19 reality?, McKinsey & Company.
[77] BMA (2020), BMA COVID-19 tracker survey 2020 October Wave, British Medical Association.
[84] Breteler, M. et al. (2020), “Are current wireless monitoring systems capable of detecting adverse events in high-risk surgical patients? A descriptive study.”, Injury, Vol. 51 Suppl 2, pp. S97-S105, https://doi.org/10.1016/j.injury.2019.11.018.
[41] Canada Health Infoway (2022), Infoway Insights: Canadian Digital Health Survey, https://insights.infoway-inforoute.ca/digital-health-survey (accessed on 16 May 2022).
[2] Carter, H. (ed.) (2020), “Telemedicine in the OECD: An umbrella review of clinical and cost-effectiveness, patient experience and implementation”, PLOS ONE, Vol. 15/8, p. e0237585, https://doi.org/10.1371/journal.pone.0237585.
[78] Cartwright, E. et al. (2021), “eHealth interventions to support patients in delivering and managing peritoneal dialysis at home: A systematic review.”, Peritoneal dialysis international : journal of the International Society for Peritoneal Dialysis, Vol. 41/1, pp. 32-41, https://doi.org/10.1177/0896860820918135.
[49] Catalan-Matamoros, D. et al. (2020), “Assessing Communication during Remote Follow-Up of Users with Pacemakers in Norway: The NORDLAND Study, a Randomized Trial.”, International journal of environmental research and public health, Vol. 17/20, https://doi.org/10.3390/ijerph17207678.
[34] CFPC (2021), Strengthening Health Care - Access Done Right, The College of Family Physicians of Canada, Mississauga.
[13] Chu, C. et al. (2021), “Rural Telemedicine Use Before and During the COVID-19 Pandemic: Repeated Cross-sectional Study”, Journal of medical Internet research, Vol. 23/4, https://doi.org/10.2196/26960.
[35] Clarke, G., A. Dias and A. Wolters (2022), Access to and delivery of general practice services: a study of patients at practices using digital and online tools, The Health Foundation, http://www.health.org.uk/IAU (accessed on 12 May 2022).
[39] CMA and Abacus Data (2020), What Canadians think about virtual health care: national survey results, Canadian Medical Association.
[11] CMA and Canada Health Infoway (2021), 2021 National Survey of Canadian Physicians.
[10] CMA, CFPC and RCPSC (2022), Virtual care in Canada: Progress and potential - Report of the virtual care task force.
[79] Colomina, J. et al. (2021), “Implementing mHealth-Enabled Integrated Care for Complex Chronic Patients With Osteoarthritis Undergoing Primary Hip or Knee Arthroplasty: prospective, Two-Arm, Parallel Trial”, Journal of medical Internet research, Vol. 23/9, pp. e28320-e28320, https://doi.org/10.2196/28320.
[50] Convery, E. et al. (2020), “A Smartphone App to Facilitate Remote Patient-Provider Communication in Hearing Health Care: Usability and Effect on Hearing Aid Outcomes.”, Telemedicine journal and e-health : the official journal of the American Telemedicine Association, Vol. 26/6, pp. 798-804, https://doi.org/10.1089/tmj.2019.0109.
[16] Cordina, J. et al. (2022), Patients love telehealth - physicians are not so sure, McKinsey & Company, https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/patients-love-telehealth-physicians-are-not-so-sure (accessed on 16 May 2022).
[28] CQC (2019), Push Dr Limited, Care Quality Commission, https://www.cqc.org.uk/location/1-5345986073 (accessed on 17 May 2022).
[51] Cremades, M. et al. (2020), “Telemedicine to follow patients in a general surgery department. A randomized controlled trial.”, American journal of surgery, Vol. 219/6, pp. 882-887, https://doi.org/10.1016/j.amjsurg.2020.03.023.
[25] Cuadrado, A. et al. (2021), “Telemedicine efficiently improves access to hepatitis C management to achieve HCV elimination in the penitentiary setting”, The International journal on drug policy, Vol. 88, p. 103031, https://doi.org/10.1016/j.drugpo.2020.103031.
[52] Dobrusin, A. et al. (2020), “Gastroenterologists and Patients Report High Satisfaction Rates With Telehealth Services During the Novel Coronavirus 2019 Pandemic.”, Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association, Vol. 18/11, pp. 2393-2397.e2, https://doi.org/10.1016/j.cgh.2020.07.014.
[14] Edge Health (2021), Video Consultations in Secondary Care, NHS England, NHS Improvement.
[15] FAIR Health (2022), FH ® Healthcare Indicators and FH ® Medical Price Index 2022, FAIR Health, New York, https://s3.amazonaws.com/media2.fairhealth.org/whitepaper/asset/FH%20Medical%20Price%20Index%20and%20F (accessed on 17 May 2022).
[66] Fraser, M. et al. (2022), “Does Connected Health Technology Improve Health-Related Outcomes in Rural Cardiac Populations? Systematic Review Narrative Synthesis.”, International journal of environmental research and public health, Vol. 19/4, https://doi.org/10.3390/ijerph19042302.
[19] Glazier, R. et al. (2021), “Shifts in office and virtual primary care during the early COVID-19 pandemic in Ontario, Canada”, CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne, Vol. 193/6, pp. E200-E210, https://doi.org/10.1503/CMAJ.202303.
[67] Gomez-Roas, M. et al. (2022), “Postpartum during a pandemic: Challenges of low-income individuals with healthcare interactions during COVID-19.”, PloS one, Vol. 17/5, pp. e0268698-e0268698, https://doi.org/10.1371/journal.pone.0268698.
[53] Gopal, A. et al. (2022), “Remote Assessments of Hand Function in Neurological Disorders: Systematic Review.”, JMIR rehabilitation and assistive technologies, Vol. 9/1, pp. e33157-e33157, https://doi.org/10.2196/33157.
[36] Greenhalgh, T. et al. (2022), “Why do GPs rarely do video consultations? qualitative study in UK general practice”, British Journal of General Practice, Vol. 72/718, pp. e351-e360, https://doi.org/10.3399/BJGP.2021.0658.
[54] Guaiana, G. et al. (2021), “A Systematic Review of the Use of Telepsychiatry in Depression.”, Community mental health journal, Vol. 57/1, pp. 93-100, https://doi.org/10.1007/s10597-020-00724-2.
[69] Hadeler, E., H. Gitlow and K. Nouri (2021), “Definitions, survey methods, and findings of patient satisfaction studies in teledermatology: a systematic review.”, Archives of dermatological research, Vol. 313/4, pp. 205-215, https://doi.org/10.1007/s00403-020-02110-0.
[21] Haynes, S. et al. (2021), “Disparities in Telemedicine Use for Subspecialty Diabetes Care During COVID-19 Shelter-In-Place Orders”, Journal of diabetes science and technology, Vol. 15/5, pp. 986-992, https://doi.org/10.1177/1932296821997851.
[40] Heart and Stroke Foundation (2021), New survey reveals concern for people with heart disease or stroke, Heart and Stroke Foundation, https://www.heartandstroke.ca/what-we-do/media-centre/news-releases/new-survey-reveals-concern-for-people-with-heart-disease-or-stroke (accessed on 16 May 2022).
[80] Indraratna, P. et al. (2021), “Trials and Tribulations: mHealth Clinical Trials in the COVID-19 Pandemic.”, Yearbook of medical informatics, Vol. 30/1, pp. 272-279, https://doi.org/10.1055/s-0041-1726487.
[55] Jang, S. et al. (2021), “Telemedicine and the Use of Korean Medicine for Patients With COVID-19 in South Korea: Observational Study.”, JMIR public health and surveillance, Vol. 7/1, pp. e20236-e20236, https://doi.org/10.2196/20236.
[81] Jang, S., Y. Kim and W. Cho (2021), “A Systematic Review and Meta-Analysis of Telemonitoring Interventions on Severe COPD Exacerbations.”, International journal of environmental research and public health, Vol. 18/13, https://doi.org/10.3390/ijerph18136757.
[31] Johnsen, T. et al. (2021), “Suitability of Video Consultations During the COVID-19 Pandemic Lockdown: Cross-sectional Survey Among Norwegian General Practitioners”, J Med Internet Res 2021;23(2):e26433 https://www.jmir.org/2021/2/e26433, Vol. 23/2, p. e26433, https://doi.org/10.2196/26433.
[85] Joshi, C. et al. (2021), “Risk of Admission to the Emergency Room/Inpatient Service After a Neurology Telemedicine Visit During COVID-19 Pandemic.”, Pediatric neurology, Vol. 122, pp. 15-19, https://doi.org/10.1016/j.pediatrneurol.2021.06.005.
[57] Kamdar, N. et al. (2020), “Development, Implementation, and Evaluation of a Telemedicine Preoperative Evaluation Initiative at a Major Academic Medical Center.”, Anesthesia and analgesia, Vol. 131/6, pp. 1647-1656, https://doi.org/10.1213/ANE.0000000000005208.
[17] Karimi, M. et al. (2022), National Survey Trends in Telehealth Use in 2021: Disparities in Utilization and Audio vs. Video Services, Office of the Assistant Secretary for Planning and Evaluation, U. S. Department of Health and Human Services.
[26] Khairat, S. et al. (2021), “Implementation and Evaluation of a Telemedicine Program for Specialty Care in North Carolina Correctional Facilities”, JAMA network open, Vol. 4/8, pp. e2121102-e2121102, https://doi.org/10.1001/jamanetworkopen.2021.21102.
[37] Kletečka-Pulker, M. et al. (2021), “Telehealth in Times of COVID-19: Spotlight on Austria”, Healthcare 2021, Vol. 9, Page 280, Vol. 9/3, p. 280, https://doi.org/10.3390/HEALTHCARE9030280.
[32] Lane, A. and V. Clarke (2021), Report on the Findings of the First National Evaluation of the use of Video Enabled Health Care in Ireland, eHealth Ireland.
[75] Lee, S., A. Dana and J. Newman (2020), “Teledermatology as a Tool for Preoperative Consultation Before Mohs Micrographic Surgery Within the Veterans Health Administration.”, Dermatologic surgery : official publication for American Society for Dermatologic Surgery [et al.], Vol. 46/4, pp. 508-513, https://doi.org/10.1097/DSS.0000000000002073.
[56] Legler, S. et al. (2021), “Evaluation of an Intrahospital Telemedicine Program for Patients Admitted With COVID-19: Mixed Methods Study.”, Journal of medical Internet research, Vol. 23/4, pp. e25987-e25987, https://doi.org/10.2196/25987.
[58] Leibar Tamayo, A. et al. (2020), “Evaluation of teleconsultation system in the urological patient during the COVID-19 pandemic.”, Actas urologicas espanolas, Vol. 44/9, pp. 617-622, https://doi.org/10.1016/j.acuro.2020.06.002.
[82] Longacre, C. et al. (2020), “Cost-effectiveness of the Collaborative Care to Preserve Performance in Cancer (COPE) trial tele-rehabilitation interventions for patients with advanced cancers”, Cancer medicine, Vol. 9/8, pp. 2723‐2731-2723‐2731, https://doi.org/10.1002/cam4.2837.
[70] Martinez, K. et al. (2020), “The Association Between Physician Race/Ethnicity and Patient Satisfaction: an Exploration in Direct to Consumer Telemedicine.”, Journal of general internal medicine, Vol. 35/9, pp. 2600-2606, https://doi.org/10.1007/s11606-020-06005-8.
[59] Morris, B., B. Rossi and B. Fuemmeler (2021), “The role of digital health technology in rural cancer care delivery: A systematic review.”, The Journal of rural health : official journal of the American Rural Health Association and the National Rural Health Care Association, https://doi.org/10.1111/jrh.12619.
[44] Morris, J. (2020), The remote care revolution in the NHS: understanding impacts and attitudes, The Nuffield Trust, https://www.nuffieldtrust.org.uk/resource/the-remote-care-revolution-in-the-nhs-understanding-impacts-and-attitudes (accessed on 16 May 2022).
[42] National Health Fund, Department of Patient Services (Departament Obsługi Pacjenta, Narodowy Fundusz Zdrowia) (2020), Raport z badania satysfakcji pacjentów korzystających z teleporad u lekarza podstawowej opieki zdrowotnej w okresie epidemii COVID-19, Polish Health Ministry/ Ministerstwo Zdrowia.
[23] Nguyen, A., A. Baker and A. Turner (2020), “On-call telehealth for visiting optometry in regional Western Australia improves patient access to eye care”, Clinical & experimental optometry, Vol. 103/3, pp. 393-394, https://doi.org/10.1111/cxo.12979.
[9] OECD (2021), Health at a Glance 2021: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/ae3016b9-en.
[3] OECD (2017), Caring for Quality in Health: Lessons Learnt from 15 Reviews of Health Care Quality, OECD Reviews of Health Care Quality, OECD Publishing, Paris, https://doi.org/10.1787/9789264267787-en.
[1] Oliveira Hashiguchi, T. (2020), “Bringing health care to the patient: An overview of the use of telemedicine in OECD countries”, OECD Health Working Papers, No. 116, OECD Publishing, Paris, https://doi.org/10.1787/8e56ede7-en.
[24] Ontario Telemedicine Network (n.d.), Virtual Care for Indigenous Communities - Connecting Indigenous Communities to Care, https://otn.ca/providers/indigenous-video/ (accessed on 5 October 2022).
[68] Peden, C., S. Mohan and V. Pagan (2020), “Telemedicine and COVID-19: an Observational Study of Rapid Scale Up in a US Academic Medical System.”, Journal of general internal medicine, Vol. 35/9, pp. 2823-2825, https://doi.org/10.1007/s11606-020-05917-9.
[60] Pers, Y. et al. (2021), “A randomized prospective open-label controlled trial comparing the performance of a connected monitoring interface versus physical routine monitoring in patients with rheumatoid arthritis.”, Rheumatology (Oxford, England), Vol. 60/4, pp. 1659-1668, https://doi.org/10.1093/rheumatology/keaa462.
[65] Pogorzelska, K. and S. Chlabicz (2022), “Patient Satisfaction with Telemedicine during the COVID-19 Pandemic-A Systematic Review.”, International journal of environmental research and public health, Vol. 19/10, https://doi.org/10.3390/ijerph19106113.
[74] Porath, A. et al. (2017), “Maccabi proactive Telecare Center for chronic conditions – the care of frail elderly patients”, Israel Journal of Health Policy Research, Vol. 6/1, https://doi.org/10.1186/S13584-017-0192-X.
[7] Probst, T. et al. (2021), “Psychotherapie auf Distanz in Österreich während COVID‑19. Zusammenfassung der bisher publizierten Ergebnisse von drei Onlinebefragungen”, Psychotherapie Forum 2021 25:1, Vol. 25/1, pp. 30-36, https://doi.org/10.1007/S00729-021-00168-3.
[61] Rametta, S. et al. (2020), “Analyzing 2,589 child neurology telehealth encounters necessitated by the COVID-19 pandemic.”, Neurology, Vol. 95/9, pp. e1257-e1266, https://doi.org/10.1212/WNL.0000000000010010.
[43] Reicher, S., T. Sela and O. Toren (2021), “Using Telemedicine During the COVID-19 Pandemic: Attitudes of Adult Health Care Consumers in Israel”, Frontiers in Public Health, Vol. 9, p. 385, https://doi.org/10.3389/FPUBH.2021.653553/BIBTEX.
[18] Samson, L. et al. (2021), Medicare Beneficiaries’ Use of Telehealth in 2020: Trends by Beneficiary Characteristics and Location, Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet (accessed on 6 May 2022).
[73] Schultz, K. et al. (2021), “Implementation of a virtual ward as a response to the COVID-19 pandemic.”, Australian health review : a publication of the Australian Hospital Association, Vol. 45/4, pp. 433-441, https://doi.org/10.1071/AH20240.
[6] Sigurdsson, E. et al. (2020), “How primary healthcare in Iceland swiftly changed its strategy in response to the COVID-19 pandemic”, BMJ Open, pp. 10:e043151, https://doi.org/10.1136/bmjopen-2020-043151.
[33] Statens medicinsk-etiska råd (2019), Digifysiskt vårdval – Tillgänglig primärvård baserad på behov och kontinuitet (SOU 2019:42), Statens medicinsk-etiska råd, Stockholm.
[62] Stentzel, U. et al. (2021), “Telemedical care and quality of life in patients with schizophrenia and bipolar disorder: results of a randomized controlled trial”, BMC psychiatry, Vol. 21/1, p. 318, https://doi.org/10.1186/s12888-021-03318-8.
[83] Tabaei, B. et al. (2020), “Impact of a Telephonic Intervention to Improve Diabetes Control on Health Care Utilization and Cost for Adults in South Bronx, New York”, Diabetes care, Vol. 43/4, pp. 743‐750-743‐750, https://doi.org/10.2337/dc19-0954.
[86] Tang, J. et al. (2020), “eHealth Interventions for Solid Organ Transplant Recipients: A Systematic Review and Meta-analysis of Randomized Controlled Trials.”, Transplantation, Vol. 104/8, pp. e224-e235, https://doi.org/10.1097/TP.0000000000003294.
[72] Taxonera, C. et al. (2021), “Innovation in IBD Care During the COVID-19 Pandemic: Results of a Cross-Sectional Survey on Patient-Reported Experience Measures.”, Inflammatory bowel diseases, Vol. 27/6, pp. 864-869, https://doi.org/10.1093/ibd/izaa223.
[63] Tenforde, A. et al. (2020), “Telemedicine During COVID-19 for Outpatient Sports and Musculoskeletal Medicine Physicians.”, PM & R : the journal of injury, function, and rehabilitation, Vol. 12/9, pp. 926-932, https://doi.org/10.1002/pmrj.12422.
[27] Tian, E. et al. (2021), “The impacts of and outcomes from telehealth delivered in prisons: A systematic review”, PloS one, Vol. 16/5, pp. e0251840-e0251840, https://doi.org/10.1371/journal.pone.0251840.
[30] Vårdanalys (2020), Three perspectives on digital care visits - the perceptions of the population, patients and healthcare, Swedish Agency for Health and Care Analysis, Stockholm.
[8] Watt, T. et al. (2020), Use of primary care during the COVID-19 pandemic - Patient-level data analysis of the impact of COVID-19 on primary care activity in England, The Health Foundation, https://www.health.org.uk/news-and-comment/charts-and-infographics/use-of-primary-care-during-the-covid-19-pandemic (accessed on 16 May 2022).
[29] Wiadji, E. et al. (2021), “Patient perceptions of surgical telehealth consultations during the COVID 19 pandemic in Australia: Lessons for future implementation”, ANZ Journal of Surgery, Vol. 91/9, pp. 1662-1667, https://doi.org/10.1111/ANS.17020.
[87] Willems, L. et al. (2021), “Satisfaction with and reliability of in-hospital video-EEG monitoring systems in epilepsy diagnosis - A German multicenter experience.”, Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology, Vol. 132/9, pp. 2317-2322, https://doi.org/10.1016/j.clinph.2021.04.020.
[22] Xiong, G. et al. (2021), “Telemedicine Use in Orthopaedic Surgery Varies by Race, Ethnicity, Primary Language, and Insurance Status”, Clinical orthopaedics and related research, Vol. 479/7, pp. 1417-1425, https://doi.org/10.1097/CORR.0000000000001775.
[64] Yi, J. et al. (2021), “Telemedicine and Dementia Care: A Systematic Review of Barriers and Facilitators.”, Journal of the American Medical Directors Association, Vol. 22/7, pp. 1396-1402.e18, https://doi.org/10.1016/j.jamda.2021.03.015.
[76] Zhang, K. et al. (2021), “Virtual preoperative assessment in surgical patients: A systematic review and meta-analysis.”, Journal of clinical anesthesia, Vol. 75, p. 110540, https://doi.org/10.1016/j.jclinane.2021.110540.
[71] Zischke, C. et al. (2021), “The utility of physiotherapy assessments delivered by telehealth: A systematic review.”, Journal of global health, Vol. 11, p. 4072, https://doi.org/10.7189/jogh.11.04072.
Note
← 1. While there is no single act on health care quality and safety in Slovenia, there are resolutions and acts that address the topic. Moreover, a strategy for quality and safety in health care is under preparation and its adoption is expected in early 2023. There is also an independent body within the Ministry of Health responsible for monitoring and managing quality in the health system.