First lessons from government evaluations of COVID-19 responses: A synthesis

Given that government responses to the crisis have required the mobilisation of resources in virtually all policy domains and involved a wide range of actors in different policy areas, co‑operation among government institutions and agencies has been vital to ensure a coherent response. Governments put in place, if needed, structures and mechanisms to ensure a co‑ordinated response (almost half of OECD countries deployed new institutional arrangements to manage the pandemic (OECD, 2021[3])), and developed protocols to clarify the responsibilities of each actor in managing the crisis response. Overall, the structures used by governments to manage the pandemic illustrate the complexity of this shock, in which traditional crisis management authorities, such as the centre of government, must contend with multiple stakeholders with potentially overlapping responsibilities (OECD, 2020[20]).

• Respondent institutions note the flexibility and rapidity shown by many governments in activating and setting up governance arrangements to facilitate inter-agency co‑operation to manage the crisis. These committees, teams and tasks forces were often multi-disciplinary and represented several ministries, but may have focused on one aspect of the crisis response, such as health or economic measures (Australian National Audit Office, 2020[21]) (French mission to control the quality of the management of the health crisis, 2020[22]) (Danish Parliamentary Committee on COVID-19, 2021[8]).

• On top of existing crisis management arrangements, many countries created ad hoc structures that played a co‑ordination role on specific topics. The multiplicity of structures meant that there could be an overlap in staff attending each meeting, which improved communication across the different groups/ committees (Danish Parliamentary Committee on COVID-19, 2021[8]) (Australian National Audit Office, 2020[23]) (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Canadian Office of the Auditor General, 2021[7]). On the other hand, this overlap meant that division of labour and chains of command were not always clear to the major stakeholders involved (Australian National Audit Office, 2020[23]).

• As a result, evaluations underline the importance of clarifying responsibilities and mandates of each of these crisis management groups at the outset. This includes making the decision-making processes of these groups clear, as well as their relative hierarchies (who has the final say and who is responsible for the decisions) and their organisational charts (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]) (Swiss Federal Chancellery, 2020[11]) (US Government Accountability Office, 2021[24]).

• Several evaluations highlight the need to send high-level civil servants and senior staff to these crisis management groups, to ensure their ministries are engaged in decision-making (Australian National Audit Office, 2020[21]) (French mission to control the quality of the management of the health crisis, 2020[22]). Crisis management structures benefit from strong leadership. Having a high level of seniority helps give more weight to the discussions, ensures stronger accountability and facilitates speedier decision-making (Belgian Health Care Knowledge Centre, 2020[12]).

In Australia, the Department of Industry, Science, Energy and Resources and the Department of Health co‑operated to manage an increase in the national medical stockpile. The departments used a flexible taskforce approach to manage the procurement and engage executive management in decision-making. They established a process for managing conflicts of interest in both departments, which was deemed fit-for-purpose by the Australian National Audit Office.

The French government created an Inter-ministerial Crisis Unit in March 2020 to co‑ordinate the actions of various ministries in their collective response to the pandemic. In spite of its inter-ministerial nature, the Unit was perceived as being under the tutelage of the Ministry of the Interior. As a result, many ministries either did not participate in the Unit’s meetings or chose to send ‘junior’ colleagues who were unable to make commitments on behalf of their ministries.

The evaluation by the “Mission on the quality control of the management of the health crisis” recommended that all ministries participate at a proper level of seniority to allow for better inter-agency co‑operation within the Inter-ministerial Crisis Unit.

In Belgium, the Hospital & Transport Surge Capacity (HTSC) committee was set up in March 2020. As an ad hoc working group of the standing Risk Management Group (RMG), its mission was to “monitor the number and type of COVID-19 patients admitted to acute hospitals […] and to propose decisions to the RMG on the organisation of adequate hospital care” (Belgian Health Care Knowledge Centre, 2020[12]). As part of its mission, the HTSC intensively interacted with other bodies and working groups that were involved in the management of the COVID-19 crisis.

According to the evaluation conducted by the Belgian Health Care Knowledge Centre, various factors contributed to the successful functioning of the committee: respect of confidentiality rules by participants, strong leadership, agreement about what was communicated and respecting these agreements.

Effective internal communication is a key component for successful inter-agency co‑operation. This includes communication among actors who may not be used to working together during normal times. During the COVID-19 crisis, government agencies had to work with many different public or quasi-public entities to build a coherent response to the pandemic.

• Communication among national agencies and actors was generally positively assessed for those who had previous experience in working together. Several evaluations reported that this communication sometimes relied more on personal relationships than established communication channels, nevertheless (Swiss Federal Chancellery, 2020[11]) (US Government Accountability Office, 2021[9]).

• However, evaluators reported that conveying relevant information in a timely manner to the local or regional level and to frontline staff was a major challenge encountered by governments in their response to the crisis (Swiss Federal Chancellery, 2020[11]) (French mission to control the quality of the management of the health crisis, 2020[22]) (UK National Audit Office, 2020[13]). Indeed, managing the crisis required national governments to co‑ordinate with a variety of national, regional and local actors that have different mandates and do not usually collaborate. Some countries, such as France (French mission to control the quality of the management of the health crisis, 2020[22]), took a decentralised approach to passing on key health information and instructions to local governments and health actors. This resulted in loss of information and signficant delays. This particular evaluation points to the need to simplify information channels and chains of command in times of crisis by having the national government talk directly to the local level. OECD data suggest that, overall, dynamic and interactive forms of internal information are most effective. Indeed, while it is important to “push information out” through, for example, emails or internal websites, using both traditional and new digital platforms for internal communication can lead to greater buy-in from stakeholders (OECD, 2021[25]).

• Several evaluations underlined the important challenges encountered by national governments in collecting data from multiple health actors involved in the crisis response (national agencies, sub-national governments, hospitals). These challenges included the lack of clear standards and homogeneous processes for reporting data to the national government, the variety of information technology systems used by these different actors, and the fact that surveys did not reflect the realities of the field (Swiss Federal Chancellery, 2020[11]; Belgian Health Care Knowledge Centre, 2020[12]; Canadian Office of the Auditor General, 2021[7]) (US Government Accountability Office, 2021[9]).

• Reports showed the value of building data standards and processes together with local health actors, to make sure that they are understood by all relevant stakeholders and adapted to operational realities. For example, the GAO recommended that the Department of Health and Human Services establish an expert committee made up of health care professionals and public agencies to review and inform data collection and reporting standards for key health indicators (US Government Accountability Office, 2021[9]). Having a clear point of contact in the national administration for questions about data standards (for example to understand the definition of ICU bed availability) was also recommended in several evaluations. These conclusions are in line with the 2021 OECD Recommendation on Enhancing Access to and Sharing of Data, which recommends promoting inclusive representation and engaging relevant stakeholders in the data ecosystem.

• Contributors also emphasised the interoperability of IT systems among health actors, or between emergency crisis response actors at the national and local levels, as an important issue for further investigation (Swiss Federal Chancellery, 2020[11]; French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]). The report by the Swiss Federal Chancellery on “Crisis management during the COVID-19 pandemic” mentioned that such IT systems should be built so that they can be used for internal management and not just for reporting purposes (this is especially important given time constraints during crises).

The crisis has put governments in a challenging situation where they had to ensure clear, trusted and legitimate decision-making processes informed by the best available evidence, while there were many “unknown unknowns” and very little time for dialogue and gathering information. Governments were also faced with the need to review and synthesise information from multiple sources, and to use it to feed into governments’ responses to the COVID-19 crisis. In many cases, putting in place ad hoc structures to gather scientific advice was seen as the best approach given such a high level of uncertainty (OECD, 2020[20]).

• Many respondent countries created ad hoc scientific committees to provide advice to the government on managing the COVID-19 pandemic, sometimes as early as January 2020 (New Zealand Parliament, 2021[26]) (US Government Accountability Office, 2021[9]) (Belgian Health Care Knowledge Centre, 2020[12]; Swiss Federal Chancellery, 2020[11]). The evaluations generally assess this approach as having improved government decision-making on complex issues. In some cases, these committees have also filled an expertise and analysis gap in government institutions (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]). However, several evaluations note that these scientific committees or groups had difficulties accesssing data or information (French mission to control the quality of the management of the health crisis, 2020[22]) (New Zealand Parliament, 2021[26]) (US Government Accountability Office, 2021[9]). Furthermore, prior to the COVID-19 pandemic, countries had already recognised the need to strengthen their domestic capacities for scientific advice in crises, by developing national mechanisms for providing scientific advice to governments in emergencies – in particular for sense-making in complex and novel crises. (OECD, 2018[27])

• Several evaluations also point to the fact that representation from the field or civil society organisations was lacking in these committees (Belgian Health Care Knowledge Centre, 2020[12]; Swiss Federal Chancellery, 2020[11]; French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]). Openness about evidence bases and decision-making processes is crucial when authorities are dealing with health risks characterised by great uncertainty, which includes formalising processes for consulting scientific committees or advice (Danish Parliamentary Committee on COVID-19, 2021[8]).

A clear and consistent communication strategy was quickly recognised by many governments as crucial for responding to the crisis. The general population’s acceptance of decisions has direct consequences for compliance with mitigation measures, particularly in contexts where mis- and dis-information spreads rapidly (OECD, 2020[29]). Governments greatly increased the frequency of their external communication to the public and relied on a variety of channels to do so, from press conferences to social media and online databases. Well-established risk communication practice (OECD, 2016[31]) highlights the importance of communicating about the actions each person can take to reduce his or her own risk exposure and that of others.

• Several evaluations deemed external communication to have been swift and regular enough, relying on a variety of both traditional and digital communication channels, and that the mix of high-level and technical messaging particularly contributed to increasing trust and acceptance within the population (Swiss Federal Chancellery, 2020[11]) (US Government Accountability Office, 2021[17]) (Danish Parliamentary Committee on COVID-19, 2021[8]). In fact, data from the OECD Survey on Understanding Public Communication indicate that most countries defined crisis communication protocols, which may explain why crisis communication was effective in many countries during the COVID-19 pandemic.

• Some governments paid special attention to producing communication material in multiple languages (whether in the form of social media posts, podcasts, or apps) and to targeting specific audiences, such as hard-to-reach populations (US Government Accountability Office, 2020[16]) (US Government Accountability Office, 2021[9]) (Australian National Audit Office, 2020[21]) (Danish Parliamentary Committee on COVID-19, 2021[8]). Evaluations stress that this is a good practice insofar as messaging should be inclusive and tailored for each audience to ensure that crisis communication is effective.

• Yet, governments encountered important issues linked to the consistency and clarity of messages relating to critical health issues. The increased frequency of communication and rapidly changing narratives may have led to mixed messages and hindered mitigation and recovery efforts (US Government Accountability Office, 2021[17]) (US Government Accountability Office, 2021[32]) (Belgian Health Care Knowledge Centre, 2020[12]) (Danish Parliamentary Committee on COVID-19, 2021[8]). For instance, rapid changes in guidance on wearing masks or use of PPE, meant that frontline staff did not always comply with them (UK National Audit Office, 2020[13]) (US Government Accountability Office, 2021[17]) (Belgian Health Care Knowledge Centre, 2020[12]). Evaluations underline that clear and consistent communication is essential during emergencies. Some evaluations note that limited and targeted messages may be more effective than large outreach strategies (US Government Accountability Office, 2021[32]) (Danish Parliamentary Committee on COVID-19, 2021[8]). This was also considered important for promoting public uptake of COVID-19 vaccines (US Government Accountability Office, 2021[32]). Furthermore, governments could consider implementing more collaborative forms of crisis communication, moving away from a one-way, top-down approach, and engaging in conversations with a variety of stakeholders to co-construct messages and learn from their concerns and experiences.

• Some evaluations also point out that the media may have received scientifically incorrect information from the government, which created uncertainties and may have affected the uptake of mitigation measures (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]) (Swiss Federal Chancellery, 2020[11]) (US Government Accountability Office, 2021[17]). This was the case regarding the effectiveness of masks as personal protective equipment, or the efficacy of hydroxychloroquine as a treatment for COVID-19 patients. In order to reduce confusion and increase trust in government responses, evaluations highlight the importance of disclosing the scientific rationale for health guidelines. The OECD Principles of Good Practice for Public Communication Responses to Help Counter Mis- and Disinformation stress the importance of transparency and honesty in communication (OECD, forthcoming[33]).

• Several evaluations point to the usefulness of publicising health and infection data on a single national dashboard to communicate the current status of the pandemic across different areas and show the scientific rationale for policy responses to the crisis (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]) (US Government Accountability Office, 2021[9]).

The pandemic has affected every sector of society and every level of government. First, sub-national authorities are responsible for crucial aspects of pandemic management, such as containment measures, healthcare, social services, and economic development (OECD, 2020[34]). Businesses and civil society organisations also supported national efforts in many countries – whether by repatriating citizens from abroad or distributing personal protective equipment to citizens. At the same time, in many countries, emergency measures have been accompanied by a restriction of civic freedoms and rights (OECD, 2020[28]). Indeed, at the start of the pandemic, 20 out of 38 OECD countries declared a state of emergency to give the executive special powers to prevent the spread of COVID-19 and to mitigate its effects on society. To address this range of issues, governments have sought to engage all relevant stakeholders in the effective design and delivery of programmes – with varying levels of success.

• Several contributing countries encountered major difficulties in ensuring a co‑ordinated response among all actors of the health sector and supply chain (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]) (US Government Accountability Office, 2021[17]). Indeed, health strategies such as vaccine delivery, for example, involve co‑operation between national and sub-national governments, as well as with practitioners, the private sector, civil society organisations, and/or the media. A lack of clarity in roles and responsibilities may have led to duplication, confusion, or gaps in delivery (Swiss Federal Chancellery, 2020[11]; French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Canadian Office of the Auditor General, 2021[7]).

• Several evaluations recommend establishing clear response strategies and processes for engaging with these stakeholders on pandemic responses in the future (US Government Accountability Office, 2021[9]) (US Government Accountability Office, 2021[17]) (Canadian Office of the Auditor General, 2021[7]).

• Several evaluations highlight the need to put in place mechanisms to mitigate the loss of democratic accountability relating to legislation passed by Parliaments that were given extraordinary powers (New Zealand Parliament, 2021[26]) (Danish Parliamentary Committee on COVID-19, 2021[8]). These evaluations do not question the necessity of these extraordinary power measures but suggest increasing the publicity of the underlying evidence for measures or allowing more time for evaluation of these measures ex post.

• Evaluations also underline the importance of involving citizens in decision-making, including in Parliamentary scrutiny committees (New Zealand Parliament, 2021[26]) and in scientific expert committees (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Belgian Health Care Knowledge Centre, 2020[12]).

Tax-based measures were an important part of governments’ efforts to directly support households and businesses during the crisis (OECD, 2020[38]). These measures have taken various forms across countries, including tax payment deferrals and more flexible tax repayment schedules, the removal of import duties and value-added taxes on medical products, and enhanced tax refunds and loss-offset provisions. These measures mostly sought to support corporate and household liquidity, employment and investment.

• Several evaluations underline administrative difficulties with rolling out tax schemes. In particular, evaluations mention the problems faced by countries in identifying eligible recipients for the support schemes, as the tax data and records used to identify eligible individuals and businesses were based on previous fiscal years (2019 and prior). According to some implementing authorities, limiting the schemes only to taxpayers with current and previous records was important to reduce the risks of fraud. However, this led to both the exclusion of potential beneficiaries and the inclusion of ineligible individuals in a number of countries (US Government Accountability Office, 2021[17]; UK National Audit Office, 2020[13]). Moreover, using tax records to identify eligibility did not appear to have reduced fraud in the countries that carried out these evaluations. Matching tax records with other government data became particularly important to avoid improper or duplicate payments (National Audit Office, 2020[39]; New Zealand Controller and Auditor General, 2021[14]; OAG of Canada, 2021[40]).

• Evaluations also mention that these measures were associated with significant budget costs. The United States Joint Committee on Taxation estimated that tax credits put in place to mitigate the cost of emergency paid sick leave, along with expanded family medical leave that certain employers had to provide under the Families First Coronavirus Response Act and the employee retention tax credits for employers under the Coronavirus Aid, Relief, and Economic Security Act of March 2020, would result in around USD 172 billion in lost revenue for fiscal years 2020-2030 (US Government Accountability Office, 2020[16]). The Australian Taxation Office also estimated the revenue foregone for the various tax-based measures under its responsibility (Australian National Audit Office, 2020[23]).

• Yet, evaluations agree that it is too early to know the true budgetary effects and impacts of these measures, or the potential cost of inaction. For instance, in the United States, the GAO indicates that, for several tax measures, the uptake by beneficiaries cannot be quantified yet (US Government Accountability Office, 2021[17]; US Government Accountability Office, 2020[16]). Evaluators in Norway indicated some concern that the use of short-term tax measures could weaken the tax base and affect the stability of the tax system if these changes become permanent, while acknowledging that it is too early to provide a robust assessment (Commission "Norway Towards 2025", 2021[41]).

Besides traditional spending and tax-based support, emergency fiscal packages have relied heavily on unconventional fiscal policy instruments such as government loans, guarantees and equity injections. In some countries, these measures made up the largest share, in nominal terms, of the government fiscal response (Moretti, Braëndle and Leroy, 2021[42]). The need to keep businesses afloat, especially SMEs, was the overarching concern of most countries (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; US Office of Management and Budget, 2020[43]; Commission "Norway Towards 2025", 2021[41]; Bank of Italy, 2020[44]). Governments provided access to liquidity and/or guaranteed loans to reduce the risk of insolvencies. Some countries also emphasised the contribution of these measures to employment support (Bank of Italy, 2020[44]; US Office of Management and Budget, 2020[43]). These measures tended to be aimed at specific sectors, such as aviation, agriculture, municipalities, or education.

• Evaluations underline the rapid implementation of balance sheet-based policies. In the United States, the Small Business Administration launched the Paycheck Protection Programme of forgivable loans within a week after the CARES Act passed (US Office of Management and Budget, 2020[43]). In France, the National Independent Mission on the assessment of the COVID-19 crisis management and on the anticipation of pandemic risks stressed how swiftly the loan guarantee scheme was implemented (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]).

• This speed may have come at the price of effective targeting. According to the United States Government Accountability Office (GAO), for instance, the lack of guidance and the complexity of the regulations on loan forgiveness under the Paycheck Protection Programme seem to have prevented the disbursement of loans to some recipients in need (US Government Accountability Office, 2020[16]). Loans worth USD 4 million were also approved even though borrowers had not yet received essential information regarding the eligibility for loan forgiveness.

• Contributors emphasised the risk and uncertainty around the longer-term budgetary costs of these measures. Unlike additional expenditures or tax-based support, balance sheet-based policies have a delayed fiscal impact. There are no immediate cash outflows for guarantees and immediate future cash inflows for loans. In France, the evaluation highlights the significant future risk to public finances of the loan guarantee scheme, citing estimates of a default rate on state-guaranteed loans of 5% (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]). In the United States, the cost of the Paycheck Protection Programme was deemed difficult to calculate due to the conditionality of loan forgiveness and changes in the level of loan forgiveness (US Government Accountability Office, 2020[16]).

• Evaluations identified positive short-term impacts, such as allowing employees to return to work and businesses to reopen (US Office of Management and Budget, 2020[43]) or limiting the number of bankruptcies (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]). Some institutions warn that estimating the impact of a given measure may be difficult given multiple forms of government support (UK National Audit Office, 2020[13]).

Economic measures targeting households seek to mitigate the impact of the crisis on income and purchasing power. They are often targeted at vulnerable groups, who face increased risks of income loss and insecurity as a result of the crisis. These measures are usually temporary and often rely on existing welfare systems (for funding or disbursement), and thus are dependent on the systems’ overall efficiency.

• Evaluations show that automatic stabilisers in the form of countries’ standard social protection systems had to be complemented with specific temporary measures to target those whose income and purchasing power suffered most from the shock (Commission "Norway Towards 2025", 2021[41]; French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]).

• State-funded job retention schemes, cash transfers, tax credits, income support for the self-employed, and extention of sick leave protections were seen as effective in limiting income losses incurred by households during the crisis (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Italian Ministry of Economy and Finance, 2020[46]; Australian National Audit Office, 2020[21]; Belgian National Social Security Institut for Self-employed Workers, 2020[47]; UK Department of Health and Social Care, 2020[48]; National Insurance Institute of Israel, 2020[49]; National Planning Department of Colombia, 2020[50]; US Office of Management and Budget, 2020[43]). Specifically, these measures were seen as generally successful in compensating the income loss of more vulnerable workers, and the households they live in, including self-employed workers, young people, large families and those below the poverty line, as well as in reducing their risks of poverty and liquidity problems (UK Department of Health and Social Care, 2020[48]; Italian Ministry of Economy and Finance, 2020[46]; US Government Accountability Office, 2021[17]; France Stratégie, 2020[51]).

• Evaluations recommended, however, that some households receive further state support in order to support their consumption levels (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Italian Ministry of Economy and Finance, 2020[46]; US Government Accountability Office, 2020[18]). This is particularly the case for households that were socio-economically fragile before the crisis and less well-protected by the existing social security system (Italian Ministry of Economy and Finance, 2020[46]; France Stratégie, 2020[51]). Indeed, according to several evaluations, these measures sometimes still excluded some of the most vulnerable groups of the population (such as unemployed youths, isolated elderly people or single-parent families), either due to their limited scope or to a lack of capacity to identify them (France Stratégie, 2020[51]; National Audit Office, 2020[39]). Evaluations recommend improving the scope and reliability of the data collected to identify households eligible for income-protection schemes in order to avoid excluding them in the future (National Audit Office, 2020[39]; US Government Accountability Office, 2021[17]).

• Non-recourse to emergency support was another issue reported by some countries. To ensure that eligible beneficiaries effectively access the offered support, application procedures should be simplified and access to rights should be more automatised (France Stratégie, 2020[51]; US Government Accountability Office, 2021[9]; Contraloría General de la República, 2020[52]). Awareness could also be raised through better outreach and communication efforts (US Government Accountability Office, 2021[17]; National Audit Office, 2020[39]; Belgian National Social Security Institut for Self-employed Workers, 2020[47]; National Planning Department of Colombia, 2020[50]), which could be targeted by tracking and reporting the number of eligible beneficiaries who were notified of the existing support and who have yet to file for it (US Government Accountability Office, 2021[17]; US Government Accountability Office, 2020[18]).

While a number of countries had automatic stabilisers in place to contain an large part of the economic shock resulting from the pandemic, governments had to adopt other measures as well, given the sudden, global nature and severity of the crisis. A number of businesses faced labour force and supply chain interruptions, mandatory closures, and other serious challenges. Across OECD countries, businesses thus benefited from various and sometimes overlapping measures: furlough and wage subsidy schemes, tax deferrals, asset write-offs, and mechanisms to offset additional paid sick leave expenditures or guaranteed loans. These measures were aimed at preserving employment and investments, providing job security and ensuring corporate liquidity and solvency (Demmou et al., 2021[54]; Demmou et al., 2021[55]).

• While schemes were, overall, rapidly implemented and timely (French Court of Accounts, 2021[56]; Belgian National Social Security Institut for Self-employed Workers, 2020[47]; US Government Accountability Office, 2021[9]), their efficiency and relevance were found to be more limited. Evaluations stress that governments had to adapt the objectives and modalities of these schemes often to account for changes in restriction measures or to respond to initial design failures and better target those in need (National Audit Office, 2020[39]; French Court of Accounts, 2021[56]; OAG of Canada, 2021[40]; Australian National Audit Office, 2020[23]; Belgian National Social Security Institut for Self-employed Workers, 2020[47]). As a result, some evaluations mention that these changes blurred messages about businesses’ eligibility and therefore diminished the measures’ effectiveness (Belgian National Social Security Institut for Self-employed Workers, 2020[47]; US Government Accountability Office, 2020[16]; New Zealand Controller and Auditor General, 2021[57]) (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Commission "Norway Towards 2025", 2021[41]; Belgian National Social Security Institut for Self-employed Workers, 2020[47]).

• Evaluators found that it was too early to assess the total impact of these schemes (National Audit Office, 2020[39]; US Government Accountability Office, 2020[16]; French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]). While they noted strong evidence that unemployment, output losses and business bankruptcies were contained, they also suggested that support schemes might have sustained non-viable firms (Commission "Norway Towards 2025", 2021[41]; National Audit Office, 2020[39]; OAG of Canada, 2021[40]) or firms that were not in need of support and exploited the rules (National Audit Office, 2020[39]; New Zealand Controller and Auditor General, 2021[57]), perhaps due to the speed of implementation and low conditionality (Commission "Norway Towards 2025", 2021[41]; OAG of Canada, 2021[40]).

• Anticipating a wave of insolvencies, some evaluations also suggested adapting and adjusting bankruptcy and restructuring laws and procedures (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Commission "Norway Towards 2025", 2021[41])to facilitate the restructuring of firms in temporary distress. For example, the French National Independent Mission on the assessment of the COVID-19 crisis management and the anticipation of pandemic risks recommended that the Finance and Justice Ministries study the trajectory of distressed companies to assess the impact of collective procedures on business bankruptcies or survivals and adjust these procedures accordingly.

The term ‘self-employed workers’ describes people who work for themselves and are registered as a business. This diverse group makes up a considerable part of the labour force in OECD countries (16.6% in the OECD countries on average in 2019) (OECD, 2020[58]). As this group is usually not eligible for unemployment insurance, they are one of the most financially vulnerable groups in times of crisis.

• Support to the self-employed has taken various forms across countries. Some countries extended their existing social security benefits to this group, e.g. by giving them access to unemployment insurance (Office of the Auditor General, Canada, 2021[59]) (US Government Accountability Office, 2020[16]). Other countries extended existing support systems for self-employed workers (Belgian National Social Security Institut for Self-employed Workers, 2020[47]), while others set up new programmes or tax deferments to compensate for loss of revenue due to the pandemic or restrictive measures (French mission to control the quality of the management of the health crisis, 2020[22]) (UK National Audit Office, 2020[13]) (Italian Ministry of Economy and Finance, 2020[46]) (Office of the Auditor General, Canada, 2021[59]); (Belgian National Social Security Institut for Self-employed Workers, 2020[47]).

• Overall, the employment support schemes for the self-employed were assessed positively, as they provided quick income support to self-employed people with encouraging short-term effects (UK National Audit Office, 2020[13]) (US Government Accountability Office, 2020[16]) (Office of the Auditor General, Canada, 2021[59]). Several contributions stress, however, that it is too early to evaluate long-term effects and impact. The UK National Audit Office claims that the UK’s scheme provided an effective bridge during the early phases of the pandemic to allow some people to return to work once the national lockdown eased. It supported 77% of potentially eligible recipients and the average grant came close to fully compensating beneficiaries for the effects of the pandemic. The HM Revenue and Customs also made their self-employment income support scheme available two weeks before schedule (UK National Audit Office, 2020[13]).

• However, many evaluations underline that the trade-off for rapid rollout was simplified eligibility procedures, which may have resulted in some self-employed workers being accidentally excluded from these measures (UK National Audit Office, 2020[13]) (US Government Accountability Office, 2021[17]) (Office of the Auditor General, Canada, 2021[59]) (New Zealand Controller and Auditor General, 2021[14]). Indeed, multiple contributing countries used tax data from previous fiscal years to verify claims by different recipient groups and to calculate the amount to pay (UK National Audit Office, 2020[13]) (US Government Accountability Office, 2021[17]) (New Zealand Controller and Auditor General, 2021[14]). This has excluded some self-employed workers who recently opened their businesses and who had not previously submitted tax returns. This also meant that post-payment controls were preferred to pre-payment controls and an increased risk of incorrect payments and fraud had to be consciously accepted (Office of the Auditor General, Canada, 2021[59]) (UK National Audit Office, 2020[13]).

• In order to minimise fraud and reach the right beneficiaries, some countries have adapted their measures several times – thus introducing greater complexity and minimising clarity for the public (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Belgian Health Care Knowledge Centre, 2020[12]).

• Some evaluations conclude with the need to improve the protection system for the self-employed for future economic shocks, both in terms of national solidarity and economic efficiency (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Belgian Health Care Knowledge Centre, 2020[12]).

The crisis has affected all parts of the population but its adverse effects are felt more strongly in vulnerable social groups, including people living in poverty, the elderly, homeless persons, children and young people (OECD, 2020[60]). To prevent social disruption and avoid an increase in poverty, inequality and exclusion, governments have expanded their social protection systems and implemented policies targeted at the most disadvantaged groups in society.

• Particular attention has been paid to poverty and inequality reduction. In Colombia, Israel and Italy, government interventions were seen as having had a positive impact on the poorest people and mitigating the impact of the crisis on overall inequality indexes (NPD and IADB, 2021[62]) (National Insurance of Israel, 2020[63]) (MEF, 2020[64]). However, several evaluations point to the challenges of identifying and protecting vulnerable beneficiaries under short time constraints (Contraloría General de la República, 2020[52]; Cour des Comptes, 2021[65]). In Costa Rica, for example, some people who qualified for the benefit did not receive the Bono Proteger programme, thus preventing it from having the desired impact on poverty levels (Contraloría General de la República, 2020[52]).

• Several evaluations point out that assessing the impact of the crisis on poverty and inequality could also allow governments to identify gaps in their social protection systems (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]) (US Government Accountability Office, 2020[18]). These findings are also in line with the recommendations from the OECD policy paper on “Youth and COVID-19” which suggests that tools to measure intergenerational impacts could also allow governments to anticipate the impact of response and recovery measures across different age groups and avoid exacerbating intergenerational inequalities, which have been accentuated by the crisis (OECD, 2020[60]).

• Policies aimed at mitigating the social impacts of the crisis also paid particular attention to the effects of remote learning and school closures, which were undertaken by a large number of countries to reduce the spread of the virus (UK NAO, 2021[66]; Knesset Research and Information Center, 2020[67]; Commission "Norway Towards 2025", 2021[41]; Cour des Comptes, 2021[68]). Ensuring continuous learning for students was a particular challenge for countries. The most vulnerable groups were particularly affected by such measures (HM Government, 2020[69]; Commission "Norway Towards 2025", 2021[41]; US Office of Management and Budget, 2020[43]; UK Department of Health and Social Care, 2020[48]; Knesset Research and Information Center, 2020[67]; Cour des Comptes, 2021[68]). At home for instance, vulnerable groups suffered from higher rates of infection and COVID-19 related deaths, more temporary layoffs in lower status occupations, a relatively greater loss of income compared to the general population, and a higher incidence of mental health issues. For disadvantaged children, schooling in this environment translated into more difficulty with devices and connectivity and more loss of contact with school and teachers (Thorn, W. and S. Vincent-Lancrin, 2021[61]). Generally, school closures also have a high cost in terms of student performance, well-being, physical and mental health, motivation and insertion in the labour market (HM Government, 2020[69]; Commission "Norway Towards 2025", 2021[41]; Cour des Comptes, 2021[68]; UK NAO, 2021[66]).The few evaluations available on this topic concluded that it was essential to keep schools open as much as possible (Danish Parliamentary Committee on COVID-19, 2021[8]; HM Government, 2020[69]; Cour des Comptes, 2021[68]) (GAO, 2020[70]).

In order to control the spread of the virus, OECD countries introduced more or less stringent infection control measures (mandatory use of protective masks, social distancing, quarantine measures, etc.) and deployed massive testing and tracing strategies. In addition, vaccination campaigns were launched starting from the end of 2020 to immunise the population. Multiple actors, both public and private, have been involved in these efforts. The pandemic not only had an impact on intensive care, but also on regular health care insofar as hospitals were overwhelmed with treating COVID-19 patients during the peak of the pandemic.

• A major challenge pinpointed by evaluations was the lack or poor quality of real-time data on intensive care unit beds available in hospitals in some countries (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]) (Belgian Health Care Knowledge Centre, 2020[12]). Yet, these data are crucial to assess hospital capacity and take steps to surge or de-surge this capacity, to decide on new admissions of COVID-19 patients, to create additional capacity, or assess the need for inter-hospital transport. Evaluations pointed to the importance of developing an inventory of critical care capacities both at the national and regional level, to assess capacities in normal times, and to better understand how to extend them in times of crisis. Evaluations also suggest working on shared IT systems among the health ministry, local health authorities and hospitals to allow for real-time reporting (US Government Accountability Office, 2021[9]) (see also Box 5).

• Evaluations underline that national governments encountered important challenges in deploying testing and tracing efforts, in particular as they required a high level of co‑ordination with local actors. As a result, the detection of large-scale outbreaks and the subsequent isolation of infected people was extremely difficult (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]). Evaluations recommend that governments establish clear protocols for authorities to work together to test and trace, in order to improve pandemic response capacities. Evaluations also highlighted the insufficient checks on quarantine and self-isolation orders imposed on infected people and travellers, given the lack of compliance with quarantine and testing requirements and public health agencies’ difficulty in performing controls. Some evaluations mention that a nationwide quarantine could be a solution to improve compliance with these rules (Canadian Office of the Auditor General, 2021[7]; Holden Expert Group, n.d.[73]).

• Evaluations emphasised that deploying vaccination strategies has posed important logistical challenges as they required the concerted effort of many stakeholders. Evaluations recommend a clear communication of vaccination plans to all involved entities to ensure that they have sufficient time to prepare and organise distribution (GAO, 2020[70]) (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]).

• Evaluations observe a reduction in medical consultation and in-person visits to clinics and hospitals for non-COVID-19 patients, and indicate that communications around regular health care were not clear in several countries (Danish Parliamentary Committee on COVID-19, 2021[8]; Cour des Comptes, 2021[74]). Online medical consultations were not always seen as an effective solution to maintain good levels of care for patients (State Comptroller of Israel, 2020[75]; Corona Investigative Committee, 2020[76]; Dutch Ministry of Sports and Health, 2020[77]). Evaluations pointed to the potential long-term negative consequences for population health outcomes due to challenges in maintaining care continuity, and suggest a further evaluation of the impacts of the pandemic in this regard (State Comptroller of Israel, 2020[75]; Corona Investigative Committee, 2020[76]; Dutch Ministry of Sports and Health, 2020[77]).

The lockdown and restriction measures implemented by countries generally include border closures and controls, travel restrictions, school closures, curfews and bans on public gatherings. Countries limited mobility within and across borders to curb the spread of the virus and indirectly diminish the detrimental economic, social and health impacts of the crisis.

• Evaluations mention that governments generally acted relatively quickly to close borders when it became apparent that the pandemic was no longer under control, although these findings may be very country-dependent. In general, border restrictions on international travel are beneficial in the short term to contain the spread of the virus, but their effectiveness lessens once the virus starts spreading within countries (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Canadian Office of the Auditor General, 2021[7]; Commission "Norway Towards 2025", 2021[41]; Danish Parliamentary Committee on COVID-19, 2021[8]) (Italian Ministry of Economy and Finance, 2020[46]). Countries tried to reallocate resources where to repatriate citizens residing abroad or at least ensure that their needs were met at a distance by swiftly and regularly informing them about the health situation and existing support (US Government Accountability Office, 2021[17]; Cour des Comptes, 2021[79]). Evaluations also show that countries within the European Union were able to co‑operate fruitfully to implement these measures (Commission "Norway Towards 2025", 2021[41]; French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]).

• Evaluations mention, however, that there were challenges with compliance with lockdown measures, party because they were sometimes difficult to understand for citizens or inconsistent. Evaluations recommend clearer provisions, more accurate control of their implementation, with a consideration for vulnerable groups that might be disproportionately affected by such measures (Canadian Office of the Auditor General, 2021[7]; UK Department of Health and Social Care, 2020[48]).

• The restrictions of visits in hospitals and retirement homes had high costs in terms of mental health and well-being of both patients and their relatives (US Government Accountability Office, 2020[18]; Belgian Health Care Knowledge Centre, 2020[12]). Evaluations therefore recommend that solutions such as building glass panes and organising more distance meetings be put in place (US Government Accountability Office, 2020[18]; Belgian Health Care Knowledge Centre, 2020[12]).

• While restrictive measures such as bans on public gatherings in restaurants and cultural places and mandatory teleworking had high economic, social and well-being costs, they were assessed as somewhat beneficial for the environment, with reduced carbon emissions associated with reduced national and international travel (Commission "Norway Towards 2025", 2021[41]; French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; US Government Accountability Office, 2021[32]; GOV.UK, 2020[80]; Italian Ministry of Economy and Finance, 2020[46]). Continuing to evaluate the costs and benefits associated with restrictive measures, such as strict lockdowns and curfews, would help countries optimise them in the future (French independent national mission to evaluate the COVID-19 crisis management and on the anticipation of pandemic risks, 2021[10]; Commission "Norway Towards 2025", 2021[41]; Danish Parliamentary Committee on COVID-19, 2021[8]).