This chapter analyses the institutional organisation of regulatory oversight as well as its evolution over time. It then focuses on existing oversight and quality control mechanisms for regulatory management tools, as well as on related performance assessment practices. The chapter concludes with a set of considerations on the meaning and determinants of well-performing regulatory oversight, with special attention to coordination-related functions and results-oriented approaches.
OECD Regulatory Policy Outlook 2021
3. Regulatory oversight
Abstract
Key findings
Robust oversight is crucial for effective regulatory policy and for implementing the 2012 Recommendation of the Council on Regulatory Policy and Governance. The Recommendation stresses the importance of establishing mechanisms and institutions to provide oversight of regulatory policy procedures and goals, support and implement regulatory policy, and thereby foster regulatory quality. Regulatory oversight bodies (ROBs) need to incentivise civil servants to use regulatory management tools, follow due process, and co-ordinate across the public administration to produce high-quality regulations, foster a whole-of-government perspective towards regulation and ensure a consistent approach to regulatory policy.
ROBs have a crucial role to play in promoting international regulatory co-operation and as in adopting innovative and forward-looking approaches to address uncertainty and enhance systemic resilience. In light of emerging phenomena requiring concerted global action, such as health threats, climate and environmental issues, and rapid technological change, ROBs can contribute to future-proofing rulemaking. Provided that they have adequate powers, resources and capacity, they can do so by adopting a holistic and anticipatory perspective in their scrutiny of regulatory management tools and acting as knowledge brokers vis-à-vis ministries and regulatory agencies.
OECD members remain invested in regulatory oversight. All members continue to have at least one ROB in charge of promoting regulatory policy and monitoring regulatory reform and regulatory quality, highlighting the crucial importance of dedicated mechanisms and institutions to ensure decision making is systematically grounded on the best available evidence.
A number of OECD members have continued to strengthen and institutionalise their existing oversight mechanisms. Since the beginning of 2018, five ROBs have had their mandate renewed and seven of them have become permanent. In addition, a number of them have assumed new responsibilities, which may be a sign of governments’ willingness to embed these ROBs further in the wider regulatory policy environment.
This chapter focuses on “core” functions of regulatory oversight. Compared with the snapshot presented in the previous edition of the Outlook, whose focus was broader, the current institutional framework for regulatory oversight appears less fragmented. Nevertheless, with responsibilities for certain oversight functions generally spread over more than one ROB, effective co-ordination on regulatory policy matters remains essential. Indeed, many countries choose to locate the main responsibility for that function as close to the centre of government as possible.
Among the four dimensions covered by the OECD composite indicators, oversight and quality control of regulatory management tools, which accounts for the role and attributions of ROBs as well as for publicly available evaluations, remains comparatively underdeveloped. Over the 2014-2020 period, this dimension has shown the lowest scores in each composite indicator, which highlights the need for stepping up efforts in this area.
While oversight of ex post evaluations is progressing in relative terms, this progress remains slow. Only about 30% of jurisdictions have a dedicated body for ex post evaluation scrutiny and just under one-third of ROBs focus on this function. In the same vein, only about half of the jurisdictions with formal requirements for stakeholder engagement have a designated body to oversee that related practices are up to standard. Oversight focus continues to lie primarily with the scrutiny of RIA quality, as the vast majority of jurisdictions have a ROB in charge of this function and about 75% of all ROBs count it among their responsibilities.
A number of OECD members are taking action to ensure that ROBs are able to adapt to, and help address, emerging needs and challenges. Increased uncertainty and complexity around decision-making means that regulatory oversight must evolve if it is to be effective. Steps are being taken in that direction: almost half of jurisdictions report having an oversight body focusing on innovation-friendly regulation, e.g. by helping ministries and regulators take into account the impacts of regulation on innovation. In addition, about 40% of jurisdictions report having a body overseeing regulatory quality during a crisis (e.g. in the context of emergency rulemaking).
Country examples show that, by promoting a co-ordinated and comprehensive approach to regulatory analysis, providing ongoing support and creating buy-in, ROBs can make a difference for regulatory quality. Beyond institutional design and related requirements, ROBs’ legitimacy, credibility, influence, and ability to elicit buy-in from those involved in better regulation across government are important factors for success. While some of them are often harder to track and measure given their rather “intangible” nature, they should by no means be overlooked.
Results-oriented, systematic performance assessment holds the key to improving regulatory oversight in the years to come. However, there is still scant evidence on the impact of regulatory oversight on regulatory improvement or, for that matter, on broader policy goals. In part, this is because ROBs’ reporting activity focuses primarily on process, implementation and conformity with formal requirements rather than effectiveness and outcomes. Certain jurisdictions are, however, showing the way by deploying considerable efforts to monitor and evaluate the results of ROBs’ work. In addition, there are promising examples of how new technology-based solutions and analytical tools can help improve our understanding of ROBs’ performance and its determining factors in order to maximise the value of regulatory oversight processes and structures.
Introduction
Robust oversight is a cornerstone of effective regulatory policy and practical implementation of the 2012 Recommendation of the Council on Regulatory Policy and Governance, which notably stresses the importance of establishing mechanisms and institutions to actively provide oversight of regulatory policy procedures and goals, support and implement regulatory policy, and thereby foster regulatory quality.
Regulatory oversight bodies (ROBs) need to incentivise civil servants to use regulatory management tools and follow due process to produce high-quality regulations, foster a whole-of-government perspective towards regulation and ensure a consistent approach to regulatory policy through appropriate co-ordination across the public administration. In light of emerging phenomena warranting concerted action globally, such as health threats, climate and environmental issues or technological change, ROBs also have a crucial role to play in promoting international regulatory cooperation as well as the adoption of innovative practices and forward-looking approaches to deal with uncertainty and enhance systemic resilience. The Recommendation of the Council for Agile Regulatory Governance to Harness Innovation and related practical guidance underscore the importance of regulatory oversight in that context (see also chapter on Regulatory policy 2.0).
Despite the critical role of regulatory oversight systems in enabling effective regulatory frameworks, available evidence points to significant room for improvement in this area. For example, among the four dimensions covered by the OECD composite indicators (see Chapter 2 on regulatory management tools for more details), oversight and quality control of regulatory management tools has remained comparatively underdeveloped over the 2014-2020 period. Challenges relating to the COVID 19 crisis raise, in turn, questions about the role of ROBs in emergency and high-uncertainty contexts. The present chapter builds upon previous OECD-led efforts to gain a better understanding of regulatory oversight systems. It provides updated information with a focus on “core” functions of regulatory oversight undertaken on a systematic basis: quality control of regulatory management tools; issuance or provision of relevant guidance on the use of regulatory management tools; co-ordination on regulatory policy and systematic evaluation of regulatory policy (see Box 3.1).
The main source of data and information is iREG 2020, which has fully integrated selected content from the 2017 OECD survey on regulatory oversight bodies for greater coherence and minimal burden for respondents. In addition, new data have been collected on oversight of ex post evaluation. Bodies outside the executive branch of government may be underrepresented in the sample given the strong focus on, and reporting by, government entities. In addition, respondents reported the smallest unit with responsibility for oversight function, which means several bodies may belong to same ministry or department. Data in this chapter are expressed using two basic complementary units: percentage (or number) of jurisdictions and percentage (or number) of all ROBs considered (across all jurisdictions). Unless otherwise stated, jurisdictions include OECD members and the European Union.
Box 3.1. “Core” functions of regulatory oversight
While previous analytical work by the OECD pertaining to regulatory oversight was broad in scope in order to capture a wide variety of situations, the 2021 Regulatory Policy Outlook focuses on selected core functions. These core functions have been identified in previous work carried out by the Secretariat based on analysis by Andrea Renda and Rosa J. Castro (Renda, Forthcoming[1]) as being essential for effective regulatory oversight.
The functions considered as core are:
Quality control of regulatory management tools (i.e. reviewing the quality of individual regulatory impact assessments, stakeholder engagement processes, and ex post evaluations);
Issuance or provision of relevant guidance on the use of regulatory management tools;
Co-ordination on regulatory policy; and
Systematic evaluation of regulatory policy.
Narrowing down the scope of functions aims to reduce survey burden for delegates, improve data quality and comparability, and enable robust analysis. It is also in line with the conclusion from the April 2018 meeting of the Steering Group on Measuring Regulatory Performance that, “for further analytical work, the identification of core and non-core functions of regulatory oversight may be helpful to refine and narrow the analysis”.
Although relevant actors of regulatory policy, a number of bodies’ contribution is ancillary to core regulatory oversight functions. For the sake of consistency, bodies that do not perform core oversight functions or do so only on an ad hoc basis are therefore not considered for analytical purposes. Below is a list of bodies that are excluded on those grounds:
Better regulation units inside ministries/departments;
Public think tanks and advisory bodies;
Behavioural Insights Teams;
Competition authorities;
Ad hoc task forces;
Permanent consultation bodies;
Public training schools for civil servants;
Budget and investment ministries/agencies;
Trade ministries/units;
Ministries of foreign affairs.
The chapter first analyses the institutional organisation of regulatory oversight as well as its evolution over time. It then discusses in more detail existing oversight and quality control mechanisms for regulatory management tools based on country-level data and relevant examples. The chapter concludes with a set of considerations on the meaning and determinants of well-performing regulatory oversight.
OECD members have continued to strengthen and institutionalise existing regulatory oversight systems
This section explores the institutional arrangements and organisation of regulatory oversight functions in OECD countries, including their evolution in comparison with the situation 2017, when data were last collected (and in 2014 whenever data are available).
OECD members remain invested in regulatory oversight. In 2020, as in 2017, all 39 reporting jurisdictions continued to have at least one dedicated body responsible for promoting regulatory policy as well as monitoring and reporting on regulatory reform and regulatory quality in the national administration from a whole-of-government perspective (this figure stood at 33 in 2014).1 This highlights the crucial importance of dedicated mechanisms and institutions to ensure decision-making is systematically grounded on the best available evidence.
In line with the 2012 Recommendation on of the Council on Regulatory Policy and Governance, a number of OECD members have continued to strengthen and institutionalise their existing oversight mechanisms. 2017 data showed that the mandates of the vast majority of ROBs were established in either law or statutory requirement or, alternatively, in a presidential or cabinet directive. According to 2020 data, ROBs continue to have a strong legal anchoring. As shown in Figure 3.1, since the beginning of 2018, five ROBs have had their mandate renewed and for seven of them it has become permanent. The latter include Denmark’s Government Economic Committee, the two bodies within Greece’s Secretariat General of Legal and Parliamentary Affairs, Latvia’s State Chancellery, Mexico’s CONAMER, Portugal’s Technical Unit for Legislative Impact Assessment, and Spain’s Regulatory Coordination and Quality Office. In addition, a number of ROBs have assumed new responsibilities, which may be a sign of governments’ willingness to embed these ROBs further in the wider regulatory policy environment.
Compared with the snapshot presented in the previous edition of the Outlook, which relied on a broader definition, the current institutional framework for regulatory oversight appears less fragmented - although there are generally several ROBs sharing certain functions. In total, 92 ROBs across all jurisdictions were reported as being in charge of performing at least one core regulatory oversight function on a systematic basis as of end 2020. This amounts to an average of nearly 2.4 ROBs per jurisdiction. By means of comparison, 163 ROBs (or more than four per jurisdiction on average) were considered for analytical purposes in the previous edition of the Outlook, which relied on a broader definition. Even so, with responsibilities for certain oversight functions often in the hands of more than one ROB, effective coordination on regulatory policy matters remains essential nevertheless. Many countries therefore choose to locate the main responsibility for that function as close to the centre of government as possible.
Figure 3.1. Changes to the mandate of ROBs
Note: Data is based on 38 OECD member countries and the European Union. Across all jurisdictions, changes to mandate were reported for 19 ROBs.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
As shown in Figure 3.2, more than three-quarters of ROBs are located within government, half of which at the centre of government, i.e. in a body that provides direct support and advice to the Head of Government and the Council of Ministers, for example: Heads of Prime Minister's Offices, Cabinet Secretaries, or Secretaries-General of the Government (OECD, 2020[2]). In addition, most jurisdictions have more than one ROB within government. The latest data also confirm the prominent role of non-departmental bodies in regulatory oversight. In most cases, these are arm’s length bodies, which are not subject to the direction on individual decisions by executive government but may be supported by government officials (OECD, 2018[3]). In a context where analytical rigour and credibility are paramount, these bodies are valuable sources of advice and scrutiny. In most jurisdictions where this kind of bodies operate, they also evaluate regulatory policy and contribute to the policy debate by putting forward suggestions for reform. For example, RegWatchEurope, a network of independent bodies, has issued recommendations for developing regulatory oversight further at EU level (RegWatchEurope, 2020[4]).
Among core oversight functions, the most widespread remains quality control of RIA: about 75% of all ROBs, across jurisdictions, have it among their responsibilities. By contrast, only about 45% and 30% of all ROBs are responsible for quality control of stakeholder engagement activities and ex post evaluation of regulation respectively. The other two core functions considered, guidance on the use of regulatory management tools and systematic evaluation of regulatory policy are within the remit of about 70% and 55% of all ROBs respectively.
ROBs carrying out core regulatory oversight functions typically perform “non-core” functions that are also important. The most frequent among these functions relate to the systematic improvement of regulatory policy and advocacy across government (e.g. by proposing changes to the regulatory policy framework, promoting the use of good regulatory practices or ensuring institutional relations), in which about 75% of all ROBs are involved. About two-thirds of ROBs are in turn responsible for training and capacity building activities regarding the application of regulatory management tools, and nearly half of them for identifying areas of policy where regulation can be made more effective, e.g. by gathering opinions from stakeholders, preparing reviews of existing regulation or analysing the stock and/or flow of regulation. Scrutiny of the legal quality of regulation under development, in turn, is among the functions of nearly 40% of all ROBs.
Figure 3.2. Location of ROBs (in % of total)
Note: Data is based on 38 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
Regarding the distribution of core functions across locations, the patterns observed in the OECD Regulatory Policy Outlook 2018 still seem to hold true to a large extent. ROBs located at the centre of government are entrusted with a relatively broad range of functions. As may be expected, they are by far the preferred location for functions where centrality is essential, such as coordination-related matters (e.g. promotion of joined up approaches to regulatory quality and the consistent application of relevant tools) and provision of guidance on the use of regulatory management tools. These functions are within the remit of about 80% and 75% of all ROBs at the centre of government respectively.
ROBs in other parts of government also have a diverse range of responsibilities. Those located in Ministries of Economy, Finance or Treasury tend to focus on quality control of regulatory management tools (about 80% of all ROBs in that location scrutinise RIAs) and are also involved in providing guidance and training as well as in identifying potential areas for improvement. ROBs located at Justice Ministries focus on reviewing the legal quality of proposals, although not exclusively: the vast majority of them issue guidance and more than 70% are involved in RIA scrutiny to some extent.
Non-departmental bodies have a clear focus on RIA scrutiny: all of them were reported to have it among their responsibilities. Approximately 45% and 35% of them scrutinise stakeholder engagement and ex post evaluations respectively. They are also heavily involved in the systematic evaluation of regulatory policy, as more than three quarters of them have this among their functions. ROBs external to government, in turn, focus on reviewing the quality of regulatory management tools, chiefly of RIA, as well as on the systematic evaluation of regulatory policy and the identification of areas for regulatory improvement. It should be noted that applying a revised definition has notably lowered the proportion of ROBs external to government (mainly in Parliament or part of the Judiciary) that have been considered for analytical purposes compared with 2017.
Figure 3.3. ROBs performing core oversight functions, by location
Note: Data is based on 38 OECD member countries and the European Union. Figures refer to the share (in %) of bodies in a given location performing each core function.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
Institutionalisation of regulatory oversight is also developing beyond the OECD membership. For example, in 2019, Thailand took major steps forward in strengthening their capacity to conduct effective oversight (see Box 3.2).
Box 3.2. Regulatory oversight in Thailand
The Office of the Council of State (OCS) was transformed by the 2019 Act on Legislative Drafting and Evaluation of Law from a legal scrutiny body into a regulatory oversight body. This was part of a broader regulatory reform stemming from the 2017 Constitution of Thailand, which enshrined the requirement to use regulatory management tools into the constitution and was implemented by the 2019 Act.
Positioned in the Prime Minister’s Office, the OCS had long served as a “gatekeeper” to the Council of State by providing opinions to the Council on legal quality of draft legislative and regulatory proposals. The new reforms gave them further power to both scrutinise regulatory impact assessments and stakeholder engagements that accompany proposals to the Council, as well as promote the use of these tools across the Government of Thailand through training sessions, advocacy and developing digital tools such as a new consultation platform for new laws (https://lawtest.egov.go.th).
The OECD worked with the OCS to conduct a report (OECD, 2020[5]) assessing their new role as oversight body. The report also provides a set of recommendations to support Thailand in maintaining momentum and iterating the reforms over the medium- to long-term to help cement the role of oversight and use of regulatory management tools.
Source: (OECD, 2020[5]).
Oversight of regulatory management tools has expanded further, although at varying speeds
This section focuses on quality control of regulatory management tools: RIA, stakeholder engagement and ex post evaluation. The analysis on oversight of ex post evaluation draws on newly collected variables.
Among the four dimensions covered by the OECD composite indicators, oversight and quality control of regulatory management tools, which accounts for the role and attributions of ROBs as well as for publicly available evaluations, appears to be comparatively underdeveloped when considering the 2014-2020 period: over time, this dimension has shown the lowest scores in each composite indicator, which highlights the need for stepping up efforts in this area. Despite recent improvements, oversight and quality control scores are particularly low for ex post evaluation. Indeed, quality control of both ex post evaluations and, to a lesser extent, stakeholder engagement, remains insufficient, as only slow progress has been achieved in recent years. Oversight focus continues to lie primarily with the scrutiny of RIA quality, with the vast majority of jurisdictions having a body in charge of this function.
Figure 3.4 provides an overview of the location of ROBs responsible for quality control of the different regulatory management tools. It reflects the overall distribution of ROBs across locations and confirms the pre-eminence of RIA in terms of coverage that was observed in previous analytical work. For ROBs in charge of quality control, RIA is their main focus (quality control of the other regulatory management tools being rarely dissociated from RIA’s), sometimes on an exclusive basis. In the case of ROBs at the centre of Government, quality control of RIA and stakeholder engagement practices related to the development of laws and regulations often go hand in hand.
Figure 3.4. Number of ROBs in charge of quality control, by location and regulatory management tool
Note: Data is based on 38 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
Countries resort to a combination of approaches including support and advice (more widespread) and formal opinions that can in some cases be made public and/or coupled with more stringent sanctioning mechanisms to request that quality be improved.
Based on the updates received regarding the mandate of ROBs, the general features regarding the scope and prerogatives of those in charge of quality control of regulatory management tools that were identified in the 2017 survey can still be assumed to apply; i.e. RIA quality control typically focuses on the quality of evidence (and on costs and impacts on businesses more often than on benefits and impact on citizens2) and compliance with applicable rules and procedures, whereas quality control of stakeholder engagement and ex post evaluation aims primarily at ensuring that formal and methodological requirements are met.
RIA quality control remains a cornerstone of regulatory oversight, but could be further strengthened by limiting exemptions
As of end 2020, about 80% of jurisdictions declared to have a government body outside the ministry sponsoring the regulation that is responsible for reviewing the quality of RIA. This confirms the central role of RIA scrutiny in oversight systems and, as shown in Figure 3.5, it continues the upward trend observed since 2014, when more than one-third of jurisdictions did not have such a body. Coverage is roughly equivalent for primary laws and subordinate regulation. Approximately 75% of all reported ROBs have RIA quality control among their responsibilities, and about more than 70% of these are located within government. Non-departmental oversight bodies also focus strongly on this function. In addition, about one-quarter of jurisdictions reported having a specific parliamentary committee or other parliamentary body with responsibilities for reviewing the quality of the RIA system as a whole.
Figure 3.5. Scrutiny of RIA quality (in no. of jurisdictions where each option applies)
Note: Data is based on 34 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Surveys 2014, 2017 and 2021.
In just under 40% of jurisdictions (in similar proportions both primary laws and subordinate regulations), the oversight body (or bodies) in charge of RIA scrutiny has some sort of sanctioning power, i.e. it can return the RIA for revision if it deems it inadequate. Figure 3.6 shows the reasons that ROBs can invoke to return a RIA in that case. Cost assessment deficiencies are the most widespread criteria, whereas lack of effective consultation and assessment of alternative options seems to carry comparatively less weight (further details on RIA requirements can be found in the section on Regulatory Impact Assessment earlier in this report). In all but a few cases, this sanctioning power can however be overturned by means of an active decision; e.g. from cabinet, a minister or a high-ranking official.
Figure 3.6. Reasons for returning RIAs (in number of jurisdictions where they are applicable)
Note: Data is based on 38 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
As far as RIA scrutiny is concerned, the number of jurisdictions in which ROBs have sanctioning power has not increased over the period 2014-2020. These figures do not capture, however, instances where quality control takes place in more consensus-oriented settings (e.g. recommendations from the ROB will be adhered to even if there is no prospect of a formal sanction). In addition, the number of jurisdictions with a ROB in charge of RIA quality control whose mandate is grounded on a legally binding document, which may be considered a proxy for these bodies’ influence, has grown from 18 in 2014 to 29 in 2020.
The effectiveness of RIA systems can be undermined in the absence of a systematic and effective obligation for legislative proposals to undergo RIA. If legislative proposals are arbitrarily exempted from ex ante impact assessment, or if RIA obligations can be circumvented easily, regulatory quality is bound to suffer. Only a few countries report that RIA is always conducted without exception: Austria, Canada, Estonia, Finland, France, Germany, Korea, Lithuania and Spain (in all but three of them, this applies to both primary legislation and subordinate regulation proposals). Decisions to waive RIA should therefore be exceptional, transparent and subject to systematic scrutiny. This is however an area where there is still room for improvement. As discussed in the Regulatory Impact Assessment section and shown in Figure 3.7, only a minority of OECD members currently publish decisions that RIA will not be conducted where it ought to have been. There also remains nearly two-thirds of OECD members without a body responsible for reviewing the decision made by officials about whether a RIA is required. This means that, in practice, in a majority of OECD members, exception mechanisms can be used to bypass RIA with little scrutiny on whether this decision is appropriate or proportionate to the regulatory proposal at hand.
Figure 3.7. Conditions for scrutiny of decisions not to conduct RIA (in no. of jurisdictions)
Note: Data is based on 34 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Surveys 2014, 2017 and 2021.
Quality control of ex post evaluation is developing among OECD members, but progress remains slow
Despite their critical importance for regulatory quality, oversight of ex post evaluations remains underdeveloped compared to quality control of RIA. The share of jurisdictions with a body outside the unit conducting the evaluation in charge of reviewing the quality of ex post evaluations has grown compared with 2014 for both primary laws and subordinate regulation. However, in both cases, these still account for under one third of jurisdictions. In addition, only a fraction of jurisdictions with a body in charge of reviewing the quality of individual regulations’ ex post evaluations report to do so for all of them.
Quality control of ex post evaluations is generally performed by ROBs at the centre of government, which often share this responsibility with bodies external to government or interdepartmental bodies. In most jurisdictions with a body in charge of reviewing individual regulations’ ex post evaluations, ROBs provide feedback or advice during the preparation of ex post evaluations and also issue formal opinions on their quality. As shown in Figure 3.8, uptake of both these practices has increased in relative terms since 2014, although absolute figures remain low and few OECD members make ROBs’ formal opinions publicly available.
Figure 3.8. Methods for quality control of ex post evaluations (number of jurisdictions)
Note: Data is based on 34 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Surveys 2014, 2017 and 2021.
There are promising examples of measures to strengthen oversight of ex post evaluations
Although overall progress in the development of robust and systematic oversight of ex post evaluations of regulation remains slow, some OECD members offer promising examples. Germany is one of the countries that have stepped up efforts in this area. In November 2019, the country adopted a more holistic and systematic approach requiring independent quality assurance for all internal ex post evaluations and all ex post evaluations of legislative proposals exceeding EUR 5 million in annual compliance costs. In the same vein, in 2020 Lithuania institutionalised the ex post assessment of regulations and designated the Ministry of Justice as dedicated function for ex post evaluation coordination.
When considering the period 2014-2020, some progress can also be observed, in relative terms, in the number of jurisdictions with a dedicated body in charge of reviewing the quality of ad hoc reviews of the regulatory stock (e.g. administrative burden reviews or in-depth reviews). The same applies to those with a body responsible for reviewing the quality of ex post evaluations of packages of legislation. However, in both cases, and both for primary laws and subordinate regulations, these only represent a small minority of surveyed jurisdictions (see Figure 3.9). Further efforts will thus be required to develop oversight in these areas, which are crucial for ensuring full implementation of the 2012 Recommendation and, ultimately, regulatory quality.
As of end 2020, approximately 30% of jurisdictions declared to have a mechanism (e.g. a body, unit or network) to co-ordinate ex post evaluation efforts across the public administration from a whole-of-government perspective, thus signalling their awareness of the importance of joined-up approaches for ensuring laws and regulations remain relevant and fit for purpose.
Figure 3.9. Quality control of ad hoc reviews of regulatory stock and packages of legislation (number of jurisdictions)
Note: Data is based on 34 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Surveys 2014, 2017 and 2021.
More systematic quality control is needed to ensure the effectiveness of stakeholder engagement practices
In about 60% of jurisdictions, regulators are formally required to consider consultation comments when developing the final regulation (see section on stakeholder engagement earlier in this report for more details). This proportion applies to both primary laws and subordinate regulation. A common approach to ensuring accountability in this regard consists of requiring a review by a standing or central body to oversee that related practices are up to standard. About half of the jurisdictions with formal requirements on stakeholder consultation declared to have such a mechanism – again, in similar proportions for primary laws and subordinate regulation. Quality control of stakeholder engagement activities in this context is often carried out by ROBs at the centre of government (more than 60% of them have it in their mandate), sometimes with the involvement of bodies external to government or non-departmental bodies. In only a handful of cases are regulators held accountable by means of judicial reviews. For both approaches, uptake has remained stable compared with 2017.
ROBs can help to enhance substantially the performance assessment of regulatory management tools, which is still not fully transparent or systematic
The OECD Recommendation of the Council on Regulatory Policy and Governance stresses the importance of assessing the functioning of regulatory management tools as part of governments’ efforts to evaluate the performance of regulatory policy. Principle 6 of the Recommendation encourages members to publish regular reports on the performance of regulatory policy and reform programmes and the public authorities applying the regulations, including information on how regulatory tools such as RIA, public consultation practices and reviews of existing regulations are functioning in practice. In the same vein, the OECD has developed a Framework for Regulatory Policy Evaluation to assess the success of regulatory policy in achieving policy objectives in the most efficient manner and bringing about improvements in growth and societal welfare (OECD, 2014[6]). ROBs have a decisive role to play in this context. In addition to engaging in evaluative work in their own right, they can notably promote concerted approaches across the public administration to ensure that relevant information already produced by regulatory management systems is collected systematically as indicators, and that measurement and assessment efforts encompass all relevant domains of regulatory reform instead of focusing only on certain aspects such as the cost of complying with administrative obligations (Radaelli and Fritsch, 2012[7]).
OECD members still need to step up their efforts to develop comprehensive monitoring and evaluation mechanisms for regulatory management tools. Although RIA is a cornerstone of regulatory management in most countries, performance measurement of this tool is still not fully transparent or systematic. Approximately one third of jurisdictions still do not publish online reports on the performance of their RIA system and in many of them reporting is not regular but ad hoc (see Table 3.1). In the same vein, monitoring the appropriate functioning of RIA systems by means of performance indicators or opinion surveys is still very far from being commonplace.
Table 3.1. Monitoring and assessment of RIA systems (in no. of jurisdictions)
Note: Data is based on 38 OECD member countries and the European Union.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
The need for improved monitoring and evaluation is even stronger for the other two regulatory management tools. Assessing the effects of ex post evaluations of regulation are particularly valuable to identify areas for improvement and reform. Their usefulness is notably illustrated by the work carried out by the EU’s Regulatory Scrutiny Board to draw forward-looking conclusions from its scrutiny of ex post evaluations, both from a methodological (e.g. recurrent design flaws) and an institutional perspective (e.g. potential biases and conflicts of interest) (Regulatory Scrutiny Board, 2019[8]).
However, only a handful of jurisdictions assess the effectiveness of ex post evaluations in improving the regulatory stock. In 20203, less than one-fifth of them declared to have done so in the previous five years and made findings publicly available. In the same vein, there are still few instances of reports being published online on the performance of the ex post evaluation system (i.e. to understand how it functions in practice) and many of them are not systematic. Moreover, the vast majority of jurisdictions do not use indicators on the percentage of ex post evaluations that comply with formal requirements/guidelines or opinion surveys to monitor the usefulness or quality of ex post evaluations. Although these figures are partly explained by the fact that relatively few countries undertake ex post evaluation on a systematic basis, the current level of effort remains insufficient.
Monitoring and evaluation of stakeholder engagement has not progressed in recent years either and it remains underdeveloped. Approximately one third of jurisdictions publish reports on the performance of consultation practices on improving draft regulations, nearly always on an ad hoc basis. Only seven jurisdictions reported to collect indicators on the percentage of consultations that comply with formal requirements or guidelines, and only three collect indicators on the results of surveys on the usefulness or quality of stakeholder consultations. Moreover, only the European Union declared to evaluate consultation of foreign stakeholders.
Previous OECD work focusing on EU Member States (OECD, 2019[9]) highlights that, although members rarely review the performance of their consultation systems and how they work in practice, evaluations have demonstrated that they can be a powerful tool when performed. Indeed, these evaluations provide insights to improve the effectiveness and ultimately also the acceptance of consultation channels amongst stakeholders. The Netherlands, for example, have reviewed the extent to which their online consultation system was valued by citizens, companies, and departmental staff, as well as whether the objectives of the legislative process were being achieved. Review results have also served to identify weaknesses in the consultation system, such as the relative lack of visibility for citizens and businesses as to how their inputs are taken into account. The European Commission, in turn, reviewed its stakeholder consultation practices prior to updating its consultation system in 2015 and, subsequently, in 2018-2019 as part of a broad-based stocktaking exercise on its commitment to Better Regulation (European Commission, 2019[10]).
Seamless coordination, adaptability and a holistic approach that creates buy-in are essential for effective regulatory oversight
Well-functioning regulatory oversight is crucial to bridge the gap between political commitments and formal requirements on the one hand and effective implementation on the other hand. In addition, it is necessary to ensure a whole-of-government approach and thus the effective uptake of the 2012 Recommendation in its entirety. Under the current, rather exceptional circumstances, ROBs have an essential role to play in ensuring transparency in emergency rulemaking as well as due scrutiny. In addition, they can also contribute to a sustainable economic recovery by ensuring that regulatory decisions are forward-looking.
Previous sections in this chapter refer to the importance of seamless coordination and well-conceived institutional arrangements obeying to efficiency and complementarity criteria to avoid unnecessary overlaps and fragmentation. It is equally important that ROBs can adapt timely to emerging needs and challenges, such as those stemming from rapid technological change and global threats, in order to help address them. Survey data show that a number of governments are taking steps in this direction. Almost half of jurisdictions declared to have at least an oversight body focusing on innovation-friendly regulation, e.g. by helping ministries and regulators take into account impacts of regulation on innovation. Moreover, nine ROBs have expanded their scope of intervention to cover additional areas (e.g. regulation of new technologies) and seven have otherwise had their mandate adjusted (in most cases, to adapt to changed institutional settings or improve workflows). In the same vein, about 40% of jurisdictions reported having a body in charge of overseeing regulatory quality during a crisis, for instance in the context of emergency rulemaking (see Figure 3.10 for further details).
The growing complexity and uncertainty levels affecting decision-making mean that ROBs need sufficient capacity to anticipate risks and understand potential innovation pathways and outcomes. The European Commission, for example, has decided to embed strategic foresight into its working methods (including to inform the design of new initiatives and the review of existing ones), and the mandate of its ROB, the Regulatory Scrutiny Board (RSB), has been expanded to include foresight (European Commission, 2020[11]). ROBs can indeed play a decisive role in promoting innovative approaches to regulatory policy and contributing to bring about the necessary changes in terms of institutional culture and mind-sets. Several developments at the country level are worth highlighting in this regard (see Box 3.3).
Figure 3.10. Areas reported to be covered by ROBs in charge of regulatory quality during the COVID-19 crisis
Note: Responses may relate to particular legislative arrangements within OECD members and should not be used as an indicator of the level of government intervention nor as an indicator of the severity of the COVID-19 pandemic. Data are self-reported by 21 OECD members generally as of 18 September 2020. The situation regarding covered areas may have evolved since that date.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
Box 3.3. Examples of ROBs’ innovative practices
In Canada, the Treasury Board has called upon the External Advisory Committee on Regulatory Competitiveness for advice and recommendations for “supporting the modernisation of Canada’s regulatory system into one that further enables investment and innovation”, including by “championing the use of pilots”.
Since 2017, the Norwegian Better Regulation Council has taken steps to scrutinise if regulatory proposals are innovation-friendly (e.g. 2019 statement on a set of proposed new rules governing the use of drones).
The goal of Denmark’s digital-ready legislation agenda is to “cut red tape by simplifying legislation and integrating public case processing and technology”. Unnecessary and complex legislation should be simplified and new legislation should be easily understandable and digitally compatible. To that end, the Danish Agency for Digitisation has issued Guidance on digital-ready legislation. In the same vein, a Secretariat for digital-ready legislation has been created. Its main purpose is to ensure that the public implementation impacts are properly described in new legislation and legislation is digital-ready.
In the UK, the Regulatory Policy Committee will be invited to scrutinise the application of an innovation test (under development at the time of writing), to “ensure that the impact of legislation on innovation is considered during the development of policy, introduction and implementation of legislation and its evaluation and review”. Moreover, the country’s White Paper on “Regulation for the Fourth Industrial Revolution” notably examines how the institutional framework needs to evolve to enhance regulatory oversight in areas relating to technological innovation.
Source: OECD Indicators of Regulatory Policy and Governance (iREG) survey, 2021, (Denmark's Agency for Digitisation, 2020[12]), (Government of Canada, 2019[13]), (UK Government, 2019[14]).
Appropriate execution of regulatory oversight functions also requires appropriate resourcing, especially in light of the above-mentioned additional needs in terms of analytical capacity. Reporting in this respect is however neither comprehensive nor fully standardised.4 Therefore, currently available evidence does not allow determining the extent to which resources at the disposal of ROBs are commensurate to either needs or the challenges they face. According to 2020 data (see Figure 3.11), across all jurisdictions, 23 ROBs reported changes in either financing or staff. In real terms, nine ROBs had their budget increased over the reporting period and three indicated that it had decreased. Staff numbers, in turn, grew for twelve ROBs and diminished for ten of them.
Figure 3.11. ROBs reporting changes in budget and/or staff endowments
Note: Data refer to 23 ROBs reporting changes in either financing or staff during the reporting period.
Source: Indicators of Regulatory Policy and Governance Survey 2021.
By promoting a coordinated and comprehensive approach to analysis, providing ongoing support and creating buy-in, ROBs can make a difference for regulatory quality
In addition to the institutional design elements and related requirements discussed earlier in this chapter, there are other factors that, while generally harder to track and measure given the “intangible” nature of some of them, should not be overlooked if regulatory oversight is to be effective. These factors include ROBs’ legitimacy and credibility as well as their influence and ability to elicit buy-in from those involved in better regulation across government through upstream work and the provision of ongoing guidance, advice and support.
In Portugal, for example, the Technical Unit for Legislative Impact Assessment (UTAIL) has developed strong and productive relationships with government officials involved in RIA, from high-level officials to technical staff. UTAIL is also responsible for RIA capacity building across the public administration (including methodological guidance) and carries out important upstream work in that capacity. Its advice is valued by members of the executive, which contributes to the development of better regulation in Portugal. UTAIL’s advice and networking efforts are also paying off in terms of the RIA procedure itself: while having no formal power to alter assessments whose analytical quality is deemed insufficient, in practice it is able to use its good relations and expertise to find compromises with the analytical services.
Australia’s Office of Best Practice Regulation (OBPR), in turn, offers valuable examples of how ROBs can support a coordinated and comprehensive approach to analysis and work upstream to demonstrate the value of better regulation tools for ministries across government (see Box 3.4).
Box 3.4. Australia’s Office of Best Practice Regulation: a holistic and responsive approach to oversight
To maximise RIA’s ability to identify the pathway to policy solutions with robust analysis of trade-offs, costs and benefits (and thus its influence), OBPR focuses on two areas: scanning efforts to identify upcoming proposals that require RIA, as well as proactive engagement with Ministries on the benefits of RIA. It uses information flows, decision-making processes of government, and its central position in the Department of the Prime Minister and Cabinet to assess if RIA is required for over 1 500 unique new proposals each year. However, much more effort is dedicated to the OBPR’s capacity-building focus. In 2019-20, it delivered over 2 250 structured training hours to public servants on how to conduct robust impact analysis and evidence-based decision making - in addition to emails, calls and meetings to provide agencies with the support and skills to produce high-quality impact analysis.
This support often involves the OBPR working with Ministries to identify the broad range of economic, social and distributional impacts of proposals before preferred options are settled. Using RIA early as a practical policy framework tool enables Ministries to consider the cumulative impacts of proposals. It also enables Ministries to consider how different policy levers interact with each other, well before a policy nears a decision point. This enables more interrogation of innovative options and discourages a siloed or narrow approach to solving policy problems: rather, it helps to focus on policy options that deliver net benefits to the community.
While it is a non-negotiable requirement that RIA is undertaken for all major decisions of the Government, the OBPR’s lived experience shows that Ministries who see value in using the RIA framework will generate higher quality impact analysis compared to Ministries who aim for minimum RIA expectations. From the earliest stages of policy development, the OBPR works with Ministries to identify the costs and benefits of options to solve policy problems, often with rapid response times. Where proposals have major impacts on business, individuals, or community organisations, it actively supports Ministries to develop in-depth analysis to inform decision-makers and adopts an agile approach to suit the support required. Assistance can take the form of interactive workshops, drafting advice on early analysis, or short-term secondments. As a result, Ministries are not only encouraged to adopt RIA, but are supported by the OBPR in practice. The OBPR also often works post-RIA, to partner with Ministries to showcase their analysis internally with their colleagues, and share lessons learnt and RIA tips.
Source: Exchanges with Australia’s Office of Best Practice Regulation (OBPR).
Results-oriented and systematic performance assessment holds the key to improving regulatory oversight
Maximising the value added of regulatory oversight processes and structures in the years to come will require a thorough understanding of their performance as well as its underlying factors. Despite promising examples and new opportunities offered by emerging analytical tools and methods, there is still much to be improved in that respect.
As of end 2020, approximately two-thirds of jurisdictions indicated that reports were prepared on the effectiveness of at least one ROB responsible for quality control of regulatory management tools; e.g. containing information on its activities, the fulfilment of its mission, or results of perception surveys on its performance and value added. The vast majority of them declared to publish those reports. In most cases, reporting takes place on an annual basis.
Generally, there continues to be relatively scant evidence on the impacts of regulatory oversight on regulatory improvement, let alone broader policy goals. Among other reasons, this owes to the fact that ROBs’ reporting activity focuses primarily on implementation (e.g. number of items scrutinised, turnaround times) and conformity with formal requirements (which tend to be easier to track and measure) rather than effectiveness and outcomes.
Certain jurisdictions are however showing the way by deploying considerable efforts to monitor and evaluate the results of their ROBs’ work. In its 2019 annual report, the Norwegian Better Regulation Council published performance indicators seeking to capture, among other aspects, the effect of the Council’s statements in which it had deemed RIAs not to be fit for purpose. This report also included and assessment of the general trends and developments regarding RIAs within the Council’s remit and any recurring problems, as well as an overview of the Council’s guidance and information activities to foster effective regulations. Mexico’s CONAMER, has in turn developed an "indicators for results" approach encompassing indicators to assess its contribution to reducing regulatory burden (Comisión Nacional de Mejora Regulatoria, 2019[15]). In Korea, white papers for Regulatory Reform are published on an annual basis including a regulatory reform satisfaction index, and the EU’s Regulatory Scrutiny Board publishes key performance indicators including on quality improvements subsequent to interactions with European Commission services in its oversight capacity.
Moreover, as discussed in OECD case study work (OECD, 2018[3]), a number of non-departmental bodies use surveys to collect feedback on the quality and impact of their work. Moreover, in some jurisdictions ROBs must undergo external evaluation on a periodic basis. Box 3.5 presents selected examples of both practices.
Box 3.5. Selected examples of assessment of ROBs’ work
The UK’s Regulatory Policy Committee conducts a quarterly survey of departments and regulators who have submitted cases to offer feedback on their service received and the quality of the opinion returned to them. In a similar vein, the Swedish Better Regulation Council surveys ministries’ and government agencies’ perception of the Council’s opinions and their impacts and makes the information available in its annual reports. The Dutch Advisory Board on Regulatory Burden (ATR) also gathers feedback from mechanisms, e.g. on the fast-track procedure it introduced in 2019 (ATR, 2019[16]).
In some jurisdictions, ROBs must undergo external evaluation on a periodic basis. For example, under Dutch law, the oversight body must be evaluated every four years. These evaluations have informed adjustments to the body’s mandate over the past two decades. Furthermore, the mandate of ATR was designed based on an evaluation by two independent researchers. In the UK, the National Audit Office and the Public Accounts Committee produce independent reports and studies on the evidence and analysis around regulatory measures, including an assessment of the effectiveness of the institutions involved in the development of regulatory policies. Moreover, the Swedish Agency of Public Management carried out assessments of the Swedish Better Regulation Council in 2012 and in 2018, in view of a possible mandate change.
Source: (OECD, 2018[3]).
In addition, there are promising examples of how new technology-based solutions and analytical tools can help improve our understanding of ROBs’ performance and its determining factors in order to maximise the value added of regulatory oversight processes and structures. A recent study, for example, uses supervised machine learning algorithms to identify major change requests in RSB opinions and text similarity measures to identify changes between draft and final versions of impact assessment reports (see Box 3.6).
Box 3.6. “Meet the critics: Analysing the EU Commission’s Regulatory Scrutiny Board through quantitative text analysis”: a promising approach to impact evaluation of ROBs
In their study, Roman Senninger and Jens Blom-Hansen, both of Aarhus University, analyse the impact of the Regulatory Scrutiny Board on the European Commission’s policy preparation process. Using machine learning techniques and quantitative text analysis, the study examines RSB opinions on all draft impact assessment reports from the period 2010–2017 (673 in total) and compares almost 100 draft and final policy proposals.
It concludes that the RSB is an “active watchdog” insofar as “it seems to apportion its critical scrutiny evenly in the Commission system” and “no units seem to be spared”. It also concludes that the RSB is “taken seriously” by the Commission’s departments, as “the more changes the RSB asks for, the more the draft impact assessment reports are changed by the responsible department”. However, it points out that this result mainly holds if the RSB’s overall judgment is negative (it can ask for major changes in draft reports but still adopt an overall positive judgment, in which case the responsible Commission department does not need to submit a revised report and can proceed).
The study highlights a number of caveats and limitations as well as areas deserving further investigation. For example, the analysis does not directly show whether or how the RSB influences the substance of policy proposals or their success (but just how RSB’s opinions influence the formulation of impact assessments), and it does not examine the type of changes requested or the parts of the impact assessment that are the most thoroughly revised. Additional aspects worth exploring in future include less formal interactions (e.g. physical meetings, oral exchanges) and additional factors (e.g. political context) determining the extent to which departments revise impact assessments thoroughly.
Source: (Senninger, 2020[17]).
In the same vein, Australia’s OBPR are developing a bespoke IT system for RIA aimed at improving workload management related to RIA scrutiny (e.g. via automated processes, enhanced information management and system notifications for better work prioritisation) as well as the quality of impact analysis advice. This system is expected to provide a system-wide perspective by capturing a richer set of data and allowing for reporting and analysis of aggregate data (e.g. to identify recurrent issues in RIAs from a given ministry and tailor capacity building accordingly). Crucially, it is conceived to help OBPR understand what kind of feedback is more effective at each stage of the policy cycle and target their efforts accordingly. By doing so, the system can also contribute to improving impact analysis across Government.
It could be extremely valuable to exploit the full range of opportunities offered by emerging analytical tools and methods, such as those illustrated by the above-mentioned examples, to improve the better regulation community’s understanding of ROBs’ impact and, ultimately, enhance regulatory quality.
References
[16] ATR (2019), Dutch Advisory Board on Regulatory Burden. Annual report 2019, https://www.atr-regeldruk.nl/wp-content/uploads/2020/06/2019-ATR-annual-report.pdf (accessed on 5 March 2021).
[15] Comisión Nacional de Mejora Regulatoria (2019), Informe de avances en la implementación de la Estrategia Nacional de Mejora Regulatoria y de la Comisión, https://www.gob.mx/conamer/documentos/informe-de-avances-de-en-la-implementacion-de-la-estrategia-nacional-de-mejora-regulatoria-y-de-la-comision-nacional-de-mejora-regulatoria.
[12] Denmark’s Agency for Digitisation (2020), Secretariat for digital-ready legislation, https://en.digst.dk/policy-and-strategy/digital-ready-legislation/secretariat-for-digital-ready-legislation/.
[11] European Commission (2020), Strategic foresight, https://ec.europa.eu/info/strategy/priorities-2019-2024/new-push-european-democracy/strategic-foresight_en (accessed on 15 December 2020).
[10] European Commission (2019), Better regulation: taking stock and sustaining our commitment, https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how/better-regulation-taking-stock-and-sustaining-our-commitment_en (accessed on 5 March 2021).
[13] Government of Canada (2019), External Advisory Committee on Regulatory Competitiveness, https://www.canada.ca/en/government/system/laws/developing-improving-federal-regulations/modernizing-regulations/external-advisory-committee-regulatory-competitiveness.html.
[2] OECD (2020), Network of Senior Officials from Centres of Government (CoG), https://www.oecd.org/governance/cog.htm.
[5] OECD (2020), Regulatory Management and Oversight Reforms in Thailand: A Diagnostic Scan, OECD, Paris, https://www.oecd.org/regreform/RIA-Thailand.pdf.
[9] OECD (2019), Better Regulation Practices across the European Union, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264311732-en.
[3] OECD (2018), Case Studies of RegWatchEurope regulatory oversight bodies and of the European Union Regulatory Scrutiny Board, https://www.oecd.org/gov/regulatory-policy/regulatory-oversight-bodies-2018.htm.
[6] OECD (2014), OECD Framework for Regulatory Policy Evaluation, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264214453-en.
[7] Radaelli, C. and O. Fritsch (2012), Measuring Regulatory Performance – Evaluating Regulatory Management Tools and Programmes, http://www.oecd.org/gov/regulatory-policy/2_Radaelli%20web.pdf.
[8] Regulatory Scrutiny Board (2019), Annual Report 2019, https://ec.europa.eu/info/sites/info/files/rsb_report_2019_en.pdf (accessed on 5 March 2021).
[4] RegWatchEurope (2020), Further Development of Regulatory Oversight at EU-level. Joint Discussion Paper by RegWatchEurope, https://www.regwatcheurope.eu/?p=282.
[1] Renda, A. (Forthcoming), Defining and Contextualising Regulatory oversight and Co-ordination.
[17] Senninger, R. (2020), Meet the critics: Analyzing the EU Commission’s Regulatory Scrutiny Board through quantitative text analysis, http://dx.doi.org/doi:10.1111/rego.12312 53.
[14] UK Government (2019), Regulation for the Fourth Industrial Revolution: White Paper, https://www.gov.uk/government/publications/regulation-for-the-fourth-industrial-revolution.
Notes
← 1. Approximately one third of jurisdictions also reported the existence of networks of regulators that are involved in the exchange and dissemination of good regulatory practices. These networks are however not considered for analytical purposes here.
← 2. As discussed later in this section, cost assessment deficiencies are the most widespread criterion allowing oversight bodies to return RIAs for review. This can be considered a proxy for their relative importance in RIA oversight mechanisms.
← 3. Based on data from 38 OECD member countries and the European Union.
← 4. For example, in the 2017 OECD survey on regulatory oversight bodies, data on resources were only reported for about half of all ROBs in charge of quality control functions. In addition, comparison is not straightforward given that in some cases figures were reported for an entity as a whole rather than actual oversight functions.