Industries and businesses are becoming increasingly digital, and the COVID-19 pandemic has further accelerated this trend. Regulators around the world are also experimenting with data-driven tools to apply and enforce rules in a more agile and targeted way. This report maps out several efforts undertaken jointly by the OECD and Italian regulators to develop and use artificial intelligence and machine learning tools in regulatory inspections and enforcement. It provides unique insights into the background processes and structures required for digital tools to perform predictive modelling, risk analysis and classification. It also highlights the challenges such tools bring, both in specific regulatory areas and to the broader goals of regulatory systems.
Data-Driven, Information-Enabled Regulatory Delivery
Abstract
Executive Summary
Risk-based regulatory delivery can be made more efficient through data management tools and information technology
This report draws upon the results of three projects in different regions in Italy, covering distinct regulatory areas, to assess the increasingly important role played by data analytics in applying and enforcing rules.
The importance of risk-based approaches to regulatory inspections and enforcement is well known. However, regulators seeking to incorporate risk-based approaches still encounter roadblocks in terms of insufficient data generally and inadequate data management tools specifically. While using some risk analysis is more efficient than no risk analysis at all, data-related roadblocks have made it more difficult to identify risk factors. This problem has become more pronounced during the COVID-19 pandemic, where regulatory inspections and enforcement activities and related interventions had to be prioritised in order to balance safety concerns. However, recent developments have shown that data management can help improve inspection systems quite quickly, in an easier and cheaper way than in the past, because costs for equipment are lower, less specialist staff time is required, and computing power has increased. Developments in machine learning have also made the analysis of large volumes of data faster.
The role of machine learning is already showing promise in risk analysis and predictive modelling to improve regulatory inspections and enforcement
The use of digital tools is already well accepted in revenue fields such as tax and customs, where data is numerical and has been digitalised for longer. The initiatives covered in this report focus on regulatory sectors where data is not yet collected in a precise fashion and is often from different sources, including the subjective perception of the regulators. For machine learning to work well, data coherence is crucial to predict the level of risk, as well as attribute relative “weights” to various risks, as precisely as possible. Three projects in Italy focusing on three different regulatory areas – construction, occupational health and safety, and food safety -- used machine learning techniques to predict non-compliance. Results show that machine learning helped in identifying predictors of risk based on a business’ characteristics, including its previous record of non-compliance. In the case of the construction industry, which is traditionally a high-risk sector, the machine learning tool is programmed to identify sites with the greatest risk. This can help ensure limited resources are used most effectively. Machine learning systems are also helping regulators integrate operational and strategic functions while co-ordinating with multiple stakeholders in a given regulatory sector. This can help with risk prediction and, through data collection and machine learning techniques, with the monitoring of operators.
Data-centric tools can be used to create a single register for different inspectorates
In Trento, the Single Register of Provincial Controls on Businesses (RUCP) collects data from the enforcement activities of different inspectorates as well as data on objective business characteristics. Scorecards will be used to develop a risk profile for regulated establishments. The goal of the RUCP system is to help inspectorates better plan their control activities. With the Rating Audit Control (RAC) engine, the RUCP will be able to carry out risk analysis and rate operators according to risk. Moreover, the system allows sourcing of data from multitude of agencies, and thus can help prevent overlapping among inspection activities. Performance to date of the RUCP has highlighted the need for concise definitions and functions – currently developed using the Inception Deck methodology. Once scorecards of existing risk parameters are available, the system can then perform predictive modelling.
The report describes the system architecture that the RUCP would use to assess the risk level of a given operator. It explains how data is exported from different sources to the RAC engine after which it is cleaned, aggregated and put into a simple format for running in the algorithm, which will then produce the prediction and risk classification to be used by each inspectorate. Currently, this system is managed by the regional public IT company– Trentino Digitale, with support from other suppliers outside the company.
Areas for future development
The last part of the report outlines some of the current challenges and scope for future improvements in the performance of data-driven regulatory delivery. Easier and cheaper access to data and data processing, in addition to time-tested knowledge of inspectors, will allow regulatory delivery activities to be carried out in a more risk-based and targeted way, thereby reducing burdens. However, lessons from the current projects in the three Italian regions have also shed light on future issues that need to be addressed. At present, predicting the impact dimension of a given risk is difficult. Furthermore, artificial intelligence and machine learning assess the greatest risk of non-compliance, but do not always contribute to making the regulatory system better at identifying the “unknowns”. The quality of data still remains a challenge, especially in regulatory areas traditionally non-revenue based. Currently, there is a need to build capacities in the design and development on digital systems and use of data; inspectorates without adequate training are not able to reap the maximum benefit from the system. Privacy and data protection concerns are also on the rise. The OECD framework for digital talent and skills in the public sector, as well as its data governance guidelines, provide support to countries in addressing several such issues.
