Building on the OECD 2012 Recommendation on Regulatory Policy and Governance (OECD, 2012[1]), the following best practice principles have been devised in relation to ex post evaluation.
Reviewing the Stock of Regulation
Report
OECD Best Practice Principles for Regulatory Policy
Table of contents
Share
Facebook
Twitter
LinkedIn
Best practice principles
Overarching principles
Regulatory policy frameworks should explicitly incorporate ex post reviews as an integral and permanent part of the regulatory cycle.
A sound system for the ex post review of regulation would ensure comprehensive coverage of the regulatory stock over time, while “quality controlling” key reviews and monitoring the operations of the system as a whole.
Reviews should include an evidence-based assessment of the actual outcomes from regulations against their rationales and objectives, note any lessons and make recommendations to address any performance deficiencies.
System governance
There need to be oversight and accountability systems within government administrations to ensure that key areas of regulation are not missed and that reviews are conducted appropriately.
There are benefits in institutional arrangements that combine oversight of the processes for ex ante as well as ex post evaluation processes, and that do so across the whole-of-government.
The type of ex post review, and its timing or “trigger”, are best determined at the time regulations are made.
Departments and agencies should provide advance notice of forthcoming reviews of regulation (ideally in the form of an annual “forward regulatory review plan”).
There should be explicit provision in agency budgets to cover the costs of reviewing regulations for which they are responsible.
Broad approaches to reviews
A “portfolio” of approaches to the ex post review of regulation will generally be needed. In broad terms, such approaches range from programmed reviews, to reviews initiated on an ad hoc basis, or as part of ongoing “management” processes.
“Programmed” reviews
For regulations or laws with potentially important impacts on society or the economy, particularly those containing innovative features or where their effectiveness is uncertain, it is desirable to embed review requirements in the legislative/regulatory framework itself.
Sunset requirements provide a useful “failsafe” mechanism to ensure the entire stock of subordinate regulation remains fit for purpose over time.
Post-implementation reviews within a shorter timeframe (1 to 2 years) are relevant to situations in which an ex ante regulatory assessment was deemed inadequate (by an oversight body for example) or a regulation was introduced despite known deficiencies or downside risks.
Ad hoc reviews
Public “stocktakes” of regulation provide a periodic opportunity to identify current problem areas in specific sectors or the economy as a whole.
Stocktake-type reviews can also employ a screening criterion or principle to focus on specific performance issues or impacts of concern.
“In depth” public reviews are appropriate for major regulatory regimes that involve significant complexities or interactions, or that are highly contentious, or both.
“Benchmarking” of regulation can be a useful mechanism for identifying improvements based on comparisons with jurisdictions having similar policy frameworks and objectives.
Ongoing stock management
There need to be mechanisms in place that enable “on the ground” learnings within enforcement bodies about a regulation’s performance to be conveyed as a matter of course to areas of government with policy responsibility.
Regulatory offset rules (such as one-in one-out) and Burden Reduction Targets or quotas need to include a requirement that regulations slated for removal, if still “active”, first undergo some form of assessment as to their worth.
Review methods should themselves be reviewed periodically to ensure that they too remain fit for purpose.
Governance of individual reviews
The governance and resourcing of reviews, and the approaches employed, need to be proportionate to the nature and significance of the regulations concerned. While needing to be cost-effective, arrangements should be such as to facilitate findings that are sufficiently well supported to be publicly credible.
For many regulations, evaluations will be best conducted within the departments or ministries having policy responsibility. Enforcement bodies normally should not conduct reviews themselves, but are uniquely placed to offer relevant information and advice and should be closely consulted.
The more “sensitive” a regulatory area, and the more significant its economic or social impacts, the stronger the case for an “arm’s-length” or independent review process. This in turn requires, at a minimum, that those leading a review are not beholden to the agency concerned, and have no perceived conflicts of interest.
Transparency is paramount for in-depth reviews. Reviews should be publicly announced, with scope for stakeholder input (see Public consultation) and the findings/recommendations as well as the government’s response made publicly available.
Key questions to be answered in reviews
Appropriateness: reviews should address as a threshold question whether a valid rationale for regulating still exists.
Effectiveness: reviews should determine whether the regulation (or set of regulations) actually achieves the objectives for which it was introduced.
Efficiency: reviews need to determine whether regulations give rise to unnecessary costs (beyond those needed to achieve the regulatory goal) or other unintended impacts
Alternatives: reviews should consider whether modifications to regulations, or their replacement by alternative policy instruments, are called for.
Methodologies
Evaluations should be conducted within a cost-benefit framework that firstly identifies and documents impacts of relevance and then assesses their relative magnitudes.
Quantification should be encouraged where feasible, as it brings additional rigour to assessments of impacts and potential outcomes.
Data requirements are best considered at the time a regulation is being made, as part of wider consideration of the type of ex post review that would be most appropriate.
The observed impacts of a regulation should ideally be compared with “counterfactuals” – how things might have turned out otherwise.
Public consultation
All reviews should involve consultations with affected parties, and to the extent possible, be accessible to civil society.
The nature and extent (coverage, duration) of consultations should be proportionate to the significance of the regulations and the degree of public interest or sensitivity entailed.
Prioritisation and sequencing
High priority should be given to reviewing regulations that have a) wide coverage across the economy or community and b) potentially significant impacts on citizens or organisations – i.e. “breadth and depth” – and for which there is c) prima facie evidence of a “problem”.
Attention to sequencing can be important to maximise the realised gains from reforms.
There are benefits in reviewing regulations as a group, rather than in a piecemeal fashion, where they are interactive or operate jointly to achieve related policy objectives.
Capacity building
Having in-house capability in evaluation and review methods is essential, both in order to conduct reviews internally as well as to oversee those commissioned externally.
Capacity enhancement needs to be pursued through the training of existing staff as well as through recruitment, with on-the-job learning an important element.
Consultants can usefully supplement the expertise available within government, but how they may best contribute in specific cases needs careful consideration, and they should not be over utilised to the detriment of internal capability.
Committed leadership
Support from political leaders is essential to the establishment and ongoing effectiveness of systems for the ex post review of regulation.
Senior officials within the bureaucracy need to promote a culture of evaluation within their organisations and be vigilant in ensuring that good practice is actually followed “on the ground”.
Reference
[1] OECD (2012), Recommendation of the Council on Regulatory Policy and Governance, OECD Publishing, Paris, https://www.oecd.org/governance/regulatory-policy/49990817.pdf.