The impact of the work of supreme audit institutions (SAIs) largely depends on the ability and willingness of the audited entities to implement the changes suggested in the audit reports issued by the SAIs. Applying behavioural insights (BI) can help SAIs promote the uptake of their audit reports and recommendations. BI is a perspective based on psychology, cognitive science, and social science that takes into account how humans actually make choices, and can be used to help design better policies. The report analyses how the SAI of Chile, the Comptroller General of the Republic (CGR), produces audit reports and monitors their follow-up. Based on this analysis, the report identifies and discusses challenges hampering the timely follow-up and identifies opportunities to use behavioural insights to address them. Ultimately, a behaviourally informed review of the auditing and follow-up process could help significantly improve the added value of the CGR’s work and thus the effectiveness and efficiency of the public administration in Chile.
Enhancing the Oversight Impact of Chile’s Supreme Audit Institution
Abstract
Executive Summary
Supreme Audit Institutions (SAI) are a key part of the institutional framework of democratic and accountable states. Audit reports are one of the main vehicles through which SAIs can induce change in the public sector. To do so, audit reports and their observations and recommendations must be relevant, read and understood by the right people, made available at the right time, and the presented in the right way.
Understanding the behaviour of both auditors and auditees is thus a crucial part of analysing the follow-up of audit reports -- or the lack thereof. Applying behavioural insights can identify avenues for improving the uptake of audit reports and help enhance the impact of SAIs. The report identifies relevant behaviours, perceptions and attitudes of the involved stakeholders to understand the behavioural drivers underlying non-compliance with their audit reports and provides concrete recommendations to address these issues.
Main findings
The report first reviews previous research on behavioural barriers and biases of both auditors and auditees that can help explain the success or the failure of audits to achieve the desired impact. Auditing is principally a matter of human judgement and, as such, is not immune to typical biases. In turn, auditees, too, may react differently to audit results depending on the way these are collected, processed, presented and communicated.
The CGR has made significant progress over the past years in monitoring the follow-up of audit reports. In 2012, the CGR established audit follow-up units, followed in 2014 by the online follow-up system SICA (Sistema Integrado para el Control de Auditorias) and, in 2016, the Compliance Support Programme. Thanks to these initiatives, the CGR found out that, on average, around 49% of the audit observations are addressed by the audited entities. Applying the behavioural lens to understand the implementation rate, the project confirmed several insights suggested by research and that help explain the implementation rate in Chile:
The quantity of audit observations and the way they are presented to auditees undermine their perceived relevance, can trigger “decision fatigue” and sometimes produce a negative attitude towards audits in general. In particular, the quantity of detailed observations makes it difficult to for auditees to see the bigger picture and understand the underlying issues driving these observations.
Perceptions of unfairness by auditees and weaknesses in communication between auditors and auditees during the audit process can lead to observations that could have been avoided and may undermine the willingness of auditees to address audit findings. Some of the challenges in communication are driven by an informal culture within the CGR that still primes primes auditors to take a “severe” approach to auditees and informally pressures them to include a large number of findings in the reports.
Heads of services and public managers in the audited services sometimes show low levels of interest in, ownership of, and motivation to address audit results. This can be partly explained by the fact that non-compliance with audit observations seldom has negative consequences. However, the technical or legal nature of the observations helps explain a lack of interest by leaders, both at senior and middle management levels, as the strategic value of the audits remain unclear.
Main recommendations
Based on these findings, the report suggests undertaking a behaviourally inspired review of existing procedures and practices to enhance the uptake of audit recommendations. The measures are complementary and aim for impact at three levels.
First, the CGR could seek to improve the auditor-auditee relationship by increasing interactions between them and making these interactions more constructive. In addition, the CGR could continue promoting an internal cultural change towards more supportive attitudes towards auditees.
Second, focusing on the drafting and communication of the audit reports could help improve their uptake. In particular, this requires a review of the way audit reports are drafted and could be complemented by testing different messages to communicate about the audit report.
Third, the CGR could target the follow-up process and allow for better planning by auditees, by introducing some flexibility in deadlines and by tracking progreess. Such steps should help reduce frustration and promote better quality responses.
The report proposes a detailed theory of change and implementation design to test a combination of two specific measures in a pilot to be carried out in audits at municipal level. The first is a pre-follow up meeting during which the CGR explains and clarifies the audit reports. This should reduce the cognitive burden caused by audit reports that are too complex. The second would allow some flexibility in the deadlines to address the audit observations. This should reduce the stress and perceived unfairness reported by auditees.
