This chapter discusses strategic and operational issues for the General Comptroller of the State Administration (Intervención General de la Administración del Estado, IGAE) and the National Audit Office (Oficina Nacional de Auditoría, ONA) to further improve the continuous supervision system (Sistema de Supervisión Continua, SSC). It offers recommendations for the IGAE and the ONA to enhance the strategy and capacity for data-driven monitoring, as well as to improve the transparency, communication and co-ordination concerning the SSC.
Enhancing Public Accountability in Spain Through Continuous Supervision
OECD Public Governance Reviews
2. Strategies for data-driven and transparent continuous supervision in Spain
Abstract
Introduction
The current form of continuous supervision in Spain has its roots in the 2013 reform package of the Commission for the Reform of the Public Administration (CORA), which proposed regular monitoring and evaluation of the ”rationality” of public sector entities in Spain (OECD, 2014[1]). This concept was soon codified in the Public Administration Legal Regulation Act of 2015 (Ley 40/2015 de 1 de octubre de Régimen Jurídico del Sector Público)1 and outlined further in a 2018 Directive (Orden HFP/371/2018) that stipulated the methodology for performing continuous supervision. While the legal and regulatory framework developed over this span of five years, the General Comptroller of the State Administration (Intervención General de la Administración del Estado, IGAE), and more specifically, the National Audit Office (Oficina Nacional de Auditoría, ONA) have had comparatively less time to implement the continuous supervision system (sistema de supervisión continua, SSC).
At the time of the project with the OECD, the SSC was barely a year old, having been implemented for the first time in 2020. Given these time constraints, the ONA has had to advance on many strategic and operational priorities in parallel, while developing the risk assessment methodology itself, as described in Chapter 1. Indeed, the SSC is not simply a risk assessment methodology. It has implications for IGAE’s strategies for enhancing data governance and its own capacities for data-driven monitoring, as well as its approach to communication and co-ordination.
This chapter builds on the background and recommendations for the ONA in chapter 1 that focused on the risk assessment process, and it considers other aspects and challenges related to the SSC. It shares experiences from Turkey, Austria, Italy and Canada to support the ONA in addressing these issues. In particular, the chapter discusses issues and opportunities for enhancing the ONA’s continuous supervision strategy and capacity, including recommendations for:
institutionalising feedback loops to ensure continuous improvement to the SSC, as well as considering further automation and the use of dashboards
taking additional steps to assess data quality with respect to the SSC
improving the tracking of conclusions and recommendations from its continuous supervision activities
enhancing the SSC by further investing in the ONA’s capacity and specialised data skills in the continuous supervision context.
Beyond the methodology and the processes for implementing the SSC, other considerations can have an impact on the effectiveness and relevance of continuous supervision in Spain. These issues and challenges broadly reflect notions of transparency, communication and co-ordination. This section in the chapter draws inspiration from the Institute of Internal Auditors as well as supreme audit institutions, and encourages the IGAE to consider:
improving the transparency of the SSC, including publishing the annual report and establishing audit committees
enhancing co-ordination with key oversight institutions to ensure the effectiveness of the SSC and avoid duplication
further developing its communication strategy to demonstrate the value of the SSC to government entities and oversight bodies.
The issues presented in the chapter are not exhaustive, but they represent some of the most immediate challenges facing the IGAE and the ONA as they advance the SSC. Addressing these issues can help to position the IGAE and the ONA to take advantage of the digital transformation that is underway in Spain’s government and society. Subsequent versions of the SSC can be a driver and example for the IGAE’s own modernisation in a digital age.
Enhancing strategies and capacity for data-driven monitoring
The IGAE could institutionalise feedback loops to ensure continuous improvement to the SSC, including consideration of further automation and the use of dashboards
The ONA and the IGAE’s Office of Finance and Information Technology (Oficina de Informática Presupuestaria, OIP) developed the analytics tool that supports the SSC using internal personnel and native infrastructure. The ONA and OIP designed and developed the first version of the tool, currently in use, in 2018. The following year it was used for the first time to collect and report on information and data from self-assessment questionnaires and accounts of public entities. The year 2020 marks only the second year the ONA has used the tool for continuous supervision.
As noted, for efficiency and in accordance with the Directive, the tool itself relies on existing interactive web applications (CICEP.red and RED.coa) that allow public entities to send information concerning their financial and accounts data to the IGAE. The OIP developed both applications in-house using Microsoft.NET and JavaScript as a programming framework. The data from CICEP.red and RED.coa, along with the self-assessment questionnaires, are merged into an SQL database (“El Cubo”) using Microsoft Power BI for extracting, transforming and loading (i.e. ETL processes) as well as reporting.2 The output of this process is an Excel spreadsheet with various worksheets that summarise the information. The ONA can continue to access the detailed questionnaires and files through the CICEP.red and RED.coa. The SSC simplifies the work of ONA analysts; however, there are opportunities for further efficiencies.
While the tool has been piloted and is in use for a full financial reporting cycle, the ONA indicated that elements of the SSC strategy related to automation are still being implemented (OECD, 2020[2]). Desired criteria for the analytics tool included importing data from IGAE’s internal systems and executing a risk-based analysis to identify high-risk entities. However, at this stage, the tool has only partially automated the process. Specifically, the collection of responses to self-assessment questionnaires as well as the financial indicators are fully automated, but the collection of data to assess “other qualitative risk factors” is only partially automated. Moreover, the ONA captures data in a complex series of Excel spreadsheets, and analysts manually perform the risk assessment to select entities for a further control review (ONA, 2020[3]). The risk assessment tool itself is essentially a data repository, with limited analytical functionality. The ONA also envisages using the tool to automate the evaluation reports for the entities selected for further review and in the annual reporting of results to the Ministry of Finance.
Having completed one full cycle of the SSC, the ONA could create an internal feedback loop between the OIP developers and ONA analysts as a mechanism to begin systematically monitoring the most challenging, time-consuming, and potentially error-prone aspects of the current process on an ongoing and iterative basis. In the short-term, this feedback loop could include an assessment of the need and options to adapt indicators that are partially-automated, as these have the highest likelihood of creating additional burden for analysts during the data collection phase. For continuous improvement of the SSC, the feedback loop can be institutionalised with both formal and informal channels of communication. In addition, as part of this process, the ONA could consider how statistical software, risk dashboards and other visualisation tools can support analysis, relieving some of the burden on analysts who now analyse over 420 entities using “dynamic” pivot tables in Excel to calculate ratios.
Box 2.1 presents the experience of the Turkish Court of Accounts (TCA) for assessing financial risks in municipalities. TCA created a business intelligence system called “VERA,” which uses data visualisations to facilitate the identification of risks. VERA relies on a robust web-based, centralised system (Oracle Business Intelligence Enterprise) that is customisable and allows for a secure connection, even when auditors are working remotely. In addition to risk analyses for municipalities, the system allows for analysis of financial statements, accounting entries and salaries. It also supports data verification. This includes analysis of the reliability of financial and accounting data. All data that VERA collates for analysis is facilitated by different public financial management systems and data warehouses that provide the critical infrastructure for the collection, processing and reuse of data. TCA institutionalised its own feedback loop by creating a “Data Analysis Group.”
Box 2.1. Turkish Court of Accounts Data Analysis System (VERA) for monitoring financial risks
In 2017, the Turkish Court of Accounts (TCA) created a “Data Analysis Group” to design methodologies for using computer-assisted audit techniques (CAAT) and enhance the capability of the TCA to assess risks in municipalities. The group had other aims, including decreasing auditors’ workload, analysing big data, identifying mistakes and errors in data processing, and automation of analyses to facilitate continuous monitoring. Their efforts resulted in “VERA”, TCA’s Data Analysis and Business Intelligence System.
VERA provides auditees a standard, automated tool for risk-based ranking of over 1 400 municipalities. VERA allows management to take into account risks before the TCA’s annual audit programming and supports the creation of the audit strategy. In addition, auditors use the results of the risk analyses to plan audits, as well as identify possible material misstatements in financial reports that could represent errors and fraud. All auditors have access to VERA, and are able to assess the results of VERA’s automated analyses related to risks and financial indicators in a dashboard or automatically generated reports.
Risk profiles for municipalities reflect budget size, investments, incomes, transaction numbers and volumes, size of their expenses and demographics. Scoring of individual risks generally follows a 5 point scale. For instance, VERA assesses municipal data for liabilities and calculates debt to assets ratios. VERA assigns municipalities with the highest ratios (i.e. the highest financial risk) a score of 5.
Source: OECD Interview with TCA officials and TCA presentation.
There is a cost-benefit trade-off when investing in analytics tools for public sector entities. New systems can be costly, both in terms of the outlay in public funds to install and maintain them as well as in the time to train staff on how to use the new tools to realise the expected benefits. Moreover, the cybersecurity risks associated with using free or open source platforms or statistical software packages are particularly heightened given the privacy and ethical concerns of the types of data that public sector entities handle. However, the supreme audit institution of Austria, the Court of Audit (ACA), has successfully designed a digital tool using R, a free open source statistics package, to assess the financial risk of municipalities in the country.
Following the enlargement of its audit mandate in 2011, the ACA designed a tool to prepare a profile of each municipality (there are over 2 100) using indicators to assess financial risk, and analyse the significance of the municipality from an audit perspective. The ACA found that R software was better equipped for analysing big data than Excel, was less prone to error and the R codes could be readily re-used in future evaluations, with minor adaptations. However, the learning curve for ACA analysts was significant given the level of detailed technical expertise required. Box 2.2 gives more detail on how the ACA addressed these challenges.
Box 2.2. Use of open source statistical software (R) at the Austrian Court of Audit
Since 1929, the Austrian Court of Audit (ACA) has been entitled to audit municipalities with more than 20 000 inhabitants. However, in recent years, municipalities in Austria have progressively been entrusted with more budgets to deliver services in such areas as social affairs, education and healthcare. This has resulted in an increase in the financial and economic significance of municipalities, and since 2011 the ACA has been entitled to audit those with more than 10 000 inhabitants. The extended audit responsibility has prompted the ACA to develop a tool to monitor the financial health of Austrian municipalities.
The tool operates mainly through the statistics software “R” and enables municipalities to be compared using different criteria, as well as observation of changes in municipalities and select the ones with the highest financial risk. The ACA obtains raw data from the country’s statistical body. The data include detailed information on the closed accounts of the municipalities, statements of debts and liabilities, and socio-demographic data.
By ranking the municipalities according to their financial risk based on certain indicators, the tool allows the ACA to profile each of the 2 356 municipalities in Austria and to assess them with regard to their significance for the audit activities. The tool is used for audit planning and for the preparation of audits at the operational level (e.g. for the selection of peers). Upon request, the ACA also provides the relevant fact sheets to the respective municipalities.
Source: OECD interview of ACA officials and OECD (2020[4]), Auditing Decentralised Policies in Brazil: Collaborative and Evidence-Based Approaches for Better Outcomes, OECD Publishing, Paris.
Improvements to the SSC concerning IT, data and tools do not occur in isolation of IGAE’s broader IT environment or digitalisation strategy. Legacy technology and capability gaps are often obstacles to digital transformation in the public sector. The OIP team designing the SSC tool were limited to building a tool from existing in-house systems at the IGAE. The process began with CICEP.red and RED.coa and OIP indicated that data sources will be expanded to include other systems such as CINCO.net (for budget and accounting data) and AUDI.net (for audit and internal control reports etc.). Taking advantage of the introduction of the momentum surrounding the pilot phase, the OIP and the ONA could use the SSC as a catalyst for making more systemic advancements to the broader digitalisation strategy of the IGAE. This could include strengthening key elements that have direct implications for the SSC, such as the data strategy, data management and automation of analysis, as well as IGAE’s oversight and control activities more broadly. From an institutional perspective, it could also include an assessment of the Enterprise Architecture (EA) of the IGAE.
EA is a practice that focuses on the alignment of an entities strategy and the IT infrastructure it has to achieve goals and objectives (Canada Border Services Agency, 2019[5]). It guides the process of planning and designing IT capabilities to meet objectives. EA defines the current- and target-state architectures, aligning with the entity’s strategy, priorities, and IT assets and capabilities (Canada Border Services Agency, 2019[5]). Contemporary approaches to EA go beyond a focus on improving processes to include a consideration of outcomes. The early stages of the SSC provides a concrete application for considering IGAE’s EA in a broader context and promoting further digitalisation in the coming years. Box 2.3 provides some insights from the internal audit function in the Canadian Border Services Agency and its review of the EA Programme.
Box 2.3. Auditing of Enterprise Architecture of the Canadian Border Services Agency
In 2009, the Canadian Border Services Agency (CBSA) created an Enterprise Architecture (EA) Programme to align its strategy with its IT Infrastructure. The EA Programme has a dedicated team, CBSA’s Enterprise Architecture Division (EAD), which is responsible for delivering and managing the programme. As an entity-wide programme, stakeholders include all CBSA Branches. The EA Programme adopted the Open Group Architecture Framework (TOGAF), a generally-accepted framework for enterprise architecture. The framework provides guidelines for the successful development and execution of an EA Programme strategy.
According to the internal audit (IA) function of the CBSA, the EA Programme can play a significantrole to ensure that Information Management/Information Technology (IM/IT) tools and capabilities are aligned with the overall strategy of the CBSA and the Government of Canada (GC). “An effective EA Programme should result in efficiencies and cost savings through the reuse of shared services, elimination of redundant operations, and optimisation of service delivery through the streamlining of business processes, data standardisation and systems integration.”
The objective of the audit was to assess whether the CBSA established an EA Programme that adds value, is effectively governed and is aligned with the CBSA’s current needs and priorities and the future direction of the CBSA.
The CBSA’s IA function conducted in audit in 2019 that focused on the activities of the EA Programme during the period between April 1, 2017 and March 31, 2019, including an examination of the following:
CBSA governance processes, including architectural governance processes, to support adoption of the EA Programme within the CBSA.
The adoption of EA solutions by the CBSA for business processes and transformational activities.
Performance measurement and reporting for the EA Programme.
The IA function found that the CBSA established an EA Programme that is aligned with the objectives and priorities of the Government of Canada and the CBSA itself. However, it identified key areas for improvement of the EA Programme to enhance its value for the CBSA. Key findings of the audit included:
Governance committees are active in fulfilling their responsibilities related to the EA; however, there was a need for an Architecture Review Board (ARB) to regularly discuss architecture issues and oversee the governance of “architecture variances.
There is a need for more systematic ongoing review processes and communication of EA artefacts to all stakeholders.
Embed EA considerations early in planning processes to enhance uptake of solutions. The IA function found that the EAD was generally consulted, as required, for IT-enabled projects, but there was no process to consult EAD for non-IT enabled projects.
Governance processes were inadequate for holding individuals accountable for non-compliance CBSA’s approved EA standards governance.
Performance measures for the EA Programme were not established or tracked.
The IA function’s audit underscored the need for improving the EA programme so that it added value to the agency. It also noted the risks for the CBSA in implementing solutions that are counter to the EA programme. Specifically, it could result in bypassing requirements for security, privacy, interoperability, accessibility and open information, as well as a lost opportunity for costs avoidance and efficiencies.
Assessing the IGAE’s EA was beyond the scope of this project. Many resources that offer frameworks to support the IGAE in this assessment are decades old, but are still based on fundamental principles and practices that are relevant today. For instance, the IGAE may draw inspiration for assessing its own EA from the U.S, Government Accountability Office’s Organizational Transformation: A Framework for Assessing and Improving EA Management (US Government Accountability Office, 2017[6]). This framework draws from an even older source, Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity (US Government Accountability Office, 2004[7]). These frameworks offer tools and insights to inform IGAE’s assessment of its own digitalisation strategy.
The IGAE could take additional steps to assess data quality with respect to the SSC
Spain is among the vanguard of countries that have embraced a digital government approach, employing digital tools and information technology to modernise public administration and deliver better policies and public services.3 This long-standing commitment to harnessing the benefits of technology is reflected in Spain’s top ten ranking in the OECD’s Digital Government Index 2019 (OECD, 2020[8]). Governments are increasingly embracing a data-driven approach are leveraging technology to not only achieve cost-savings and administrative efficiencies but as a tool to inform managerial decision making and take preventive actions to respond to risks. Moreover, international standards have evolved to reflect the emergence of a public sector that is data-driven and risk-based. For example, OECD instruments and standards recognise the added value of investment in developing effective data policies, data governance models, skills and capacity (OECD, 2019[9]).
Deriving meaning from data through analytical tools or techniques, commonly referred to as “data analytics”, has been transformational for public sector entities who apply this approach in service delivery and design, monitoring and evaluation of the performance of programmes and policies or for oversight purposes (Fazekas, M., Ugale, G, & Zhao, A., 2019[10]). While data analytics can be applied in diverse ways, there are common principles and practices to maximise its effectiveness that are relevant in multiple contexts. These include having a strategy for analytics with clearly defined objective, as well as ensuring effective institutional and data governance, technology, people and skills, and project-level planning.
Data governance includes standards and controls to help ensure availability, consistency, security and integrity of data. The IGAE applies European and national data protection and information security regulations in its use of data and digital tools (OECD, 2020[2]). While data for the indicators and ratios in the automated reviews of the SSC is captured from the financial reporting systems, the IGAE indicated that it does not perform independent systems audits or edit checks to verify the reliability of data. In addition, the ONA relies on attestations from senior management at the relevant entity regarding the accuracy and validity of the data reported. Officials said the control reviews provide an opportunity for following up on any questions regarding the data, but at this point the entity has already been selected for review.
Given the weighting of 40% assigned to the responses of the self-assessment questionnaires in the preliminary risk assessment, validation or at least some corroboration of this data is vital. IGAE could develop a plan to improve data validation and corroboration of self-reported data as part of its data quality management process, including spot-checking and formalised guidance for analysts to ensure systematic checking of facts during control reviews. Table 2.1 shows the European Commission’s good practices on data quality management, which the IGAE could consider as part of future efforts to ensure data quality.
Table 2.1. European Commission’s Data Quality Management Process
The ONA could improve its tracking of conclusions and recommendations from its continuous supervision activities
Having piloted the SSC in 2019, the ONA can consider additional measures to monitor and track the status of recommendations it makes. As discussed, the ONA’s conclusions following its control reviews, which it documents in an evaluation report, can include one of three actions: maintain, merge or dissolve. When the ONA’s determination is to maintain an entity, it may also provide management with recommendations for improvements that it can make to its policies and processes. For instance, the ONA may recommend that the entity prepare strategic planning documents in accordance with the Public Administration Legal Regulation Act of 2015 (Ley 40/2015 de 1 de octubre de Régimen Jurídico del Sector Público), or to further specify the activities and functions the entity performs.
As of March 2020, the ONA had begun monitoring the extent to which entities had implemented its recommendations made following the control reviews in 2019. The ONA conducts monitoring by means of a letter that it sends to the line ministry of the audited entity. Through this letter, the ONA requests information that describes the progress the entity has made in implementing recommendations the ONA has made. The monitoring focuses on recommendation that managers of the audited body had previously accepted in response to the provisional report or the audited entity’s own ministry. However, audited entities may disagree with the ONA’s recommendations, in which case the Ministry of Finance makes a decision to accept or reject the recommendation. There is currently no mechanism in place to track the decision of the Ministry of Finance when the ONA and the audited entity disagree on recommendations.
In addition, the ONA can improve its processes for tracking if and how the Council of Ministers acts on its recommendations. The SSC is a tool that informs decisions of policymakers. Decision to dissolve or merge entities are highly political, and in some cases, policymakers decide not to adopt the measures that the ONA recommends. This phase of decision-making is opaque. Once the ONA sends the evaluation report, A formal mechanism to track and communicate the judgement and actions of the Ministry of Finance and the Council of Ministers does not exist. As a result, the ONA does not know how policymakers use and act on its evaluation reports.
International auditing standards call for audit institutions to follow-up on audit recommendations as a critical element for enhancing the impact of their reports. “Follow-up is a process by which internal auditors evaluate the adequacy, effectiveness, and timeliness of actions taken by management on reported observations and recommendations,” as well as whether management or the Board has assumed the risk of not taking actions (The Institute of Internal Auditors, 2009[12]). Systematic tracking and monitoring of the uptake of recommendations facilitates effective follow-up. As discussed, in Spain, the ONA and SSC stakeholders, including the Ministry of Finance, could improve two types of follow-up and tracking mechanisms. They include: 1) processes for tracking recommendations for audited entities in relation to its control/evaluation reports, including the Ministry of Finance’s judgement when the ONA and the audited entity disagree; and 2) tracking decisions made or actions not taken by policymakers in response to its evaluations reports.
Recommendation tracking can take many forms, including online and internal databases, as well as narrative reports that summarise recommendations. The aforementioned work of the US Government Accountability Office’s to assess duplication, fragmentation and overlap in government offers an example of an online tracker for recommendations. The Action Tracker is an online tool that tracks the progress made by both the Congress and federal agencies in response to GAO’s recommendations to reduce duplication, fragmentation and overlap. The categorisation of the status of recommendations includes: New-Pending, Not Addressed, Partially Addressed, Addressed, Consolidate or Other, and Closed-Not Addressed (US Goverment Accountability Office, 2021[13]). The recommendations can be downloaded in XLSX or CSV formats, thereby promoting greater public use, analysis and awareness of its work.
For policymakers and decision-making bodies, such as the Council of Ministers in Spain, the narrative around recommendation tracking can be a useful input for understanding the nature of recommendations and impact of action or inaction. A narrative can provide the context to complement a status list of recommendations. As noted by the Institute of Internal Auditors (IIA) in its White Paper, Reporting on the Status of Audit Recommendations, narrative reports are particularly beneficial for audit committees, if Spain were to develop these, as described in the next section. Table 2.2 provides considerations for the ONA to consider in terms of the style and focus of such reports, contrasting what the IIA sees as low-value versus high-value narrative reporting styles. recognising that any narratives on recommendation tracking would need tailoring to ONA’s context, and potentially integrated with existing reports that result from the SSC.
Table 2.2. Reporting styles for follow-up of audit recommendations
Source: Adapted from (The Institute of Internal Auditors Australia, 2020[14]).
The IGAE could enhance the SSC by further investing in the ONA’s capacity and specialised data skills in the continuous supervision context
The IGAE—specifically, a small team in the ONA and the OIP—absorbed the responsibilities of the SSC as a result of the CORA reform proposals and the Public Administration Legal Regulation Act of 2015. The design, implementation and continuous improvement to the SSC and supervision of 420 public entities creates a demand on internal resources that did not exist prior to the reforms. Moreover, in response to the OECD’s questionnaire, officials of the ONA highlighted the increase in workload from the SSC, coupled with the need for further specialisation. To ensure the ONA can effectively deliver its SSC mandate, the IGAE and the ONA could consider establishing a dedicated team or unit within the ONA. This team could consist of a multi-disciplinary group of experts, including auditors and data experts, who could address the demands of the SSC as a unique work stream in the ONA.
The ONA could take inspiration from other audit entities that recognised the need for a dedicated workforce to develop data-driven risk assessments. For instance, in the aforementioned example from the Turkish Court of Accounts (TCA), the TCA created a “Data Analysis Group” to design methodologies for using computer-assisted audit techniques. In Italy, the Court of Audit harnessed the expertise of a cross-functional team of auditors and technicians to deliver on its own monitoring and oversight activities, called the “Data Analysis Competency Centre” (see Box 2.4). The ONA could contemplate in more detail how best to develop and structure its team as part of its efforts for taking a strategic approach to digitalisation and assessing its Enterprise Architecture, as previously described. The ONA may also consider targeted training programmes for OIP data technicians and ONA analysts to expand their skillsets, as well as recruitment targeting new staff with the requisite data or analytical skills.
Box 2.4. Data-driven monitoring and supervision at the Italian Court of Audit
The digital strategy of the Italian Court of Audit (Corte dei Conti, CdC) focuses on three main areas: knowledge sharing and data integration; its business intelligence system (called ConosCo); and digitalisation. ConosCo supports the CdC’s mandate to monitor public finances and expenditures and enhance reports to Parliament. The CdC launched ConosCo in 2008 as a set of methodologies, processes, architectures, and technologies to transform raw data into meaningful and information for decision-making and control purposes. The tool is not meant for risk-based audit selection, which is a process that is primarily used for the CdC’s performance auditing portfolio.
ConosCo relies on financial data sources from both central and local governments, as well as external parties like the European Commission, which feed into a data warehouse and is then transmitted to dashboards for auditors to use during the audit process. For instance, member states of the European Union (EU) are required to communicate to the Irregularities Management System (IMS) of the European Anti-Fraud Office (OLAF) any fraud and irregularities over EUR 10 000 related to European Union funds. The CdC is able to access this data through one of its own systems, and through ConosCo, make this information available to auditors and regional offices. The dashboards available to auditors in ConosCo support the analysis of indicators for detecting fraud and irregularities, as well as broader issues for monitoring the financial performance of government entities.
According to CdC officials, key features that enable the Court to carry out its function include:
Clear objectives to drive the activities related to ConosCo.
Access to reliable and trustworthy data at national and regional levels. Memorandums of Understandings with key ministries facilitate this access.
A level of automation that reflects the standard architecture of a data warehouse with requirements adjusted to meet the needs of the end users.
Automation that allows for regular reporting, based on reporting requirements found in laws and regulations.
The right mix of software, tailored to the CdC, including Microsoft Strategy, Power BI and visualisation to aid users in understanding the data and quickly drawing insights or create reports.
In addition, the CdC developed a “Data Analysis Competency Centre,” which is a new cross-functional team that brings together business and technical competencies to support the effective implementation of ConosCo. The Centre will support users of ConosCo to make better decision using machine learning, analytics, predictive analysis and other data analytics techniques. At the time of writing this report, this Centre is still in development and intends to be a multi-disciplinary team with knowledge and skills that span levels of government (i.e. national and regional) as well as technologies. According to CdC officials, this effort signals a recognition that any data-driven tool is not static, and requires a capacity-building strategy to support its development and evolution.
Source: OECD interview with CdC officials.
Improving transparency, communication and co-ordination
The IGAE could take steps to improve the transparency of the SSC, including publishing the annual report and establishing audit committees.
The Open Budget Survey (OBS) for 2019, the most recent year available, measures three key areas of governance: transparency, public participation and budget oversight. Transparency metrics in the OBS focus on public access to information as to how the government raises and spends public resources (International Budget Partnership, 2019[15]). According to the International Budget Partnership (IBP), a transparency score of at least 61 out of 100 indicates a country is “likely publishing enough material to support informed public debate on the budget.4 Spain’s score for 2019 was 53, as shown in Figure 2.1.
Figure 2.1. Transparency score for Spain in the 2019 Open Budget Survey
The OBS also covers questions related to budget oversight, and concerning questions related to audit oversight, Spain scores comparatively well (95 out of 100). However, specific questions provide further context that are relevant for the IGAE and accountability actors as key standard-bearers for the state of transparency in the Spanish government. In particular, when asked, “Does the executive make available to the public a report on what steps it has taken to address audit recommendations or findings that indicate a need for remedial action?,” the response was the same as many OECD member countries and others OBS surveyed. “No, the executive does not report on steps it has taken to address audit findings.” (International Budget Partnership, 2019[15]).
Against this backdrop, the IGAE can help to enhance transparency in government and meet public demand by making public its annual reports from the SSC. This is particularly critical given the greater attention to the effectiveness, efficiency and economy of government in the wake of the COVID-19 pandemic and increased public spending. IGAE officials are required to maintain confidentiality regarding control activities (Government of Spain, 2003[16]); however, exceptions are made in related laws, such as the possibility for public investigative bodies to publish summary reports about their activities (Government of Spain, 2019[17]). Although it follows separate standards for supreme audit institutions, the Spanish Court of Accounts has the power and mandate to publish its reports and audit findings. To ensure such transparency, the IGAE and the ONA could identify and make use of existing legal exceptions to share the results of the SSC externally. At a minimum, while respecting its requirements of confidentiality, this could include making summary reports of key findings and recommendations that stem from the SSC accessible to other oversight bodies. According to ONA officials, key stakeholders with whom it could share its reports from continuous supervision include:
The Court of Auditors to support its external control activities of the economic-financial management of the public sector.
The General Directorate of Budgets to inform processes and discussions for determining the state budget and allocating resources.
The General Inspections of Services within line ministries to improve its control activities.
Other relevant entities, such as the Independent Authority for Spanish Fiscal Responsibility, an entity responsible for fiscal control (La Autoridad Independiente de Responsabilidad Fiscal, AIREF).
Sharing the results of the SSC more broadly could have broader implications for transparency of internal audit and IGAE’s work in general. This would promote good practices for reporting transparency within the Spanish administration, and further align the IGAE with international standards for publishing reports. Box 2.5 provides context on reporting transparency in the internal audit context, drawing from a seminal publication and survey data of the Institute of Internal Auditors. In many countries, publishing internal audit reports has been a long-established practice.
Box 2.5. Survey of the Institute of Internal Auditor’s on audit report transparency in the public sector
The Institute of Internal Auditors (IIA) conducted a study to identify global internal audit report transparency practices in the public sector, and help public sector internal auditors and decision makers benchmark transparency practices. The IIA’s International Standards for the Professional Practice of Internal Auditing (Standards)—Standard 2440: Disseminating Results—notes that the chief audit executive “must communicate results to the appropriate parties.”
In 2012, the IIA Public Sector Committee surveyed internal audit entities in the public sector to obtain information about their reporting transparency practices. The Committee received 160 responses for a survey that was a combination of 17 multiple-choice and open-ended questions. The survey respondents represented 14 countries across five continents. Bearing in mind that the survey was conducted in December 2012, some of the key findings still have relevance for Spain today given the current state of transparency per the Open Budget Survey and the relevance to recommendations in this report:
Most of the public sector entities disseminate the internal audit report to an audit committee or senior management.
Federal/national governments are less likely than lower levels of government to disseminate internal audit reports to internal parties, including impacted management, except to the board or audit committee.
Most entities disseminate reports to external auditors.
All entities that disseminate internal audit reports to an external party also disseminate these reports to senior management and/or legal counsel.
Entities that are subject to public information laws are more transparent in the publication of the internal audit report.
The IIA asked respondents about the internal and external recipients of surveys. Of the 146 respondents that answered the question concerning internal dissemination of reports, 102 (70%) indicated that the internal audit report is disseminated to the board or audit committee, 93 (64%) to the executive director or president and 81 (55%) to impacted management. Regarding external dissemination of reports, 129 respondents answered the question and stated the supreme audit institution (61%), legislative auditor (22%), legislature/Parliament (22%) and the comptroller/Treasurer (14%) and other (19%), were among the main external recipients of the internal audit report.
The IGAE faces more systemic, broader capacity issue to ensure that recommendations and results of the SSC are monitored and that stakeholders have shared priorities for supervision. The tracking system and follow-up reporting described above are critical mechanisms to promote accountability and transparency, but they are technical means for managing operations and informing stakeholders. The IGAE could benefit from mechanisms that also convene relevant stakeholders and promote co-ordination. There are several modalities to accomplish this. The first step is for the IGAE to clarify its objectives in this regard, and then decide on the form, function and stakeholders. In particular, an objective to engage formally and frequently with political actors and key decision-makers of the SSC, such as the Council of Ministers, can lead to a different co-ordination mechanism than an objective that is more technical in nature, such as involving audit subjects to ensure the uptake of IGAE’s recommendations within ministries.
Depending on the ultimate objective, one option is for the IGAE to establish a working group of oversight bodies, including those listed above, with a mandate to support the delivery, improvement and dissemination of results from continuous supervision. In the short-term, this would be the most effective and efficient way for the IGAE to advance constructive partnerships and information sharing in the context of the SSC. As a more robust, formal mechanism, the IGAE could consider taking the lead on the establishment of an audit committee(s) within the Ministry of Finance (and/or across Ministries) to bolster co-ordination during the next phase of developing the SSC.
In the public sector, independent audit committees are board-level committees with a majority of independent member charged with providing oversight of management practices in key governance areas (The Institute of Internal Auditors, 2014[20]). Audit committees have a mutually beneficial relationship with internal audit, as they hold management accountable for assessing and implementing, where appropriate, internal audit recommendations (The Institute of Internal Auditors Australia, 2020[14]). Audit committees can help to define priorities, promote the flow of information and insights between different stakeholders and advise on the adequacy of resources. Audit committees can add value to an entity in other ways, including:
Facilitate well-informed, efficient, and effective decision-making.
Promote and monitor an ethical culture.
Ensure compliance with a well-designed code of conduct.
Oversee an effective system of risk oversight and management.
Oversee an effective and efficient internal control system.
Oversee internal and external reporting of financial and nonfinancial information.
Promote effective communication with audit activity and external assurance providers and respond appropriately to matters they raise (The Institute of Internal Auditors, 2014[20]).
In Spain, public sector audit committees are rare. There is no legal requirement for public entities to establish an audit committee, except for state mercantile companies (The European Confederation of Institutes of Internal Auditing, 2019[21]). The Good Governance Code addresses listed companies but it does not affect public sector entities, which can voluntarily set up an audit committee. While not legally obligated to do so, the IGAE could consider spearheading the development of an audit committee that would have the SSC as part of its responsibilities. The audit committee could be made up of internal stakeholders in the Ministry of Finance, Council of Ministers and other oversight bodies, as well as the General Inspection of Service, among others. It could provide a forum for sharing the results of the SSC and support improvements in the future. The mandate of any audit committee could be broader than the SSC, and would be informed by existing laws, regulations and policies. An audit committee charter would define its mandate and establish its authority with respect to its activities.
The IGAE could enhance co-ordination with key oversight institutions to ensure the effectiveness of the SSC and avoid duplication
To perform continuous supervision, the IGAE is legally mandated to leverage available financial and economic data, information provided by the entities to comply with the new requirements and recommendations of the Inspector General of Services within the line ministries (Government of Spain, 2018[22]). Similar to countries such as France, Spain has Inspectors General of Services whose role as internal audit functions involves reviewing services and entities affiliated with the relevant ministry and making proposals to improve and simplify administrative procedures. This role also encompasses reviews of services for quality assurance purposes, effectiveness and value for money5 (OECD, 2014[1]).
The role of the Inspectors General was also expanded under the Public Administration Legal Regulation Act. It now includes an effectiveness control (control de eficacia), which is an assessment of the extent to which an entity has met its objectives and evaluates the use of its resources in line with its strategic action plan (Government of Spain, 2015[23]).6 The IGAE indicated that the effectiveness control complements the continuous supervision process, with the work of the Inspectors General serving as inputs to the evaluation of the entity’s rationality (racionalidad).
As summarised in Table 2.3, co‑ordination is one of the guiding principles for effective continuous supervision stated in the SSC Directive. The Inspectors General can co‑ordinate their effectiveness control activities with the IGAE and establish a channel of communication with the oversight bodies of the entities subject to supervision. Putting this principle into practice, the IGAE has met periodically with the Inspector General of the Ministry of Finance to raise awareness of the model of continuous supervision. It has also collaborated on a guide for Inspector General of Services performing the effectiveness control with the Directorate General of Public Governance within the Ministry of Public Administration and Civil Service (OECD, 2020[2]). This guide incorporates elements related to continuous supervision for inspectors to consider as they conduct effectiveness control reviews.
Table 2.3. Co-ordination is one of the guiding principles of Spain’s continuous supervision system
Note: Article 5, Guiding Principles of the System Principios orientadores del Sistema.
Source: Ministerio de Hacienda y Función Pública (Government of Spain, 2018[22]).
In keeping with the guiding principle on co‑ordination stipulated in the Directive for performing continuous supervision (Government of Spain, 2018[22]), the ONA planned a series of activities to raise awareness of the SSC with other control and oversight bodies. This included clarifying roles and responsibilities of the IGAE vis-à-vis the Inspectors General of Services and bodies supervising the public sector entities (órganos de tutela), as well as building relationships with these bodies to support effective delivery of the SSC (ONA, 2018[24]).
IGAE applies International Auditing Standards adapted to the Spanish public sector. In line with ISA 610 and NIA-ES-SP 1610, the ONA must evaluate the objectivity and competency of the work of the internal audit function, the Inspectors General of Services, before it uses it (International Auditing and Assurance Standards Board, 2013[25]; IGAE, 2019[26]). While such safeguards are critical, international good practice on internal auditing in the public sector also recommends information sharing, co-ordination of activities and even reliance by internal audit functions on the work of other assurance providers depending on the circumstances (The Institute of Internal Auditors, 2019[27]). This co-ordination can be particularly advantageous when resources are limited for all parties involved, including the entity being reviewed. This guidance to internal auditors explicitly recognises the integral role that inspectors and external auditors play in public sector oversight.
The mandates of the audit and control bodies in Spain are defined in law, and current regulation does not allow for information exchange between them. The ONA therefore currently has informal communication channels with other oversight bodies. For example, it shares planned public audit activities with the Tribunal de Cuentas in advance of the plans being approved to minimise duplication or overburdening of public sector entities.7 From interviews with the Inspector General of Services, while co-ordination of control activities with the ONA occurs, this is on an informal basis (OECD, 2020[28]). A provision to facilitate information exchange between the internal and external audit bodies was submitted and approved in 2020 as a modification of the 2003 General Budgetary Law (Government of Spain, 2003[16]).
However, in the absence of audit committees or other forms of intra- and inter-ministerial co-ordination, there are opportunities for the audit bodies to improve sharing of relevant risk information generated for or as a result of the SSC, without impeding their autonomy or independence. Closer co-ordination between the internal audit, external audit institutions and other assurance providers is crucial for achieving the following complementary objectives:
exchanging information, audit plans and reports between the internal auditors and the SAI, to help conduct audits, including evaluations of the effectiveness of internal control and risk-management arrangements
achieving economies of scale as audit entities co-operate on methodological and training matters
SAIs advising or acting as an observer, taking part in regular meetings of the heads of internal audit units (as happens in Austria, Bulgaria, Denmark, Hungary, Latvia, Netherlands, Poland and the United Kingdom)
streamlining interactions and communication with both external and internal audit bodies
agreeing common standards, tools and procedures to facilitate effective co‑operation.
Strengthening and formalising the co-operation and co-ordination mechanisms between the different control, internal audit and external audit institutions is crucial. Improved co-operation between internal and external control and audit institutions relies on a number of factors, first and foremost being a commitment to take an active role and the willingness to make necessary changes. Both INTOSAI and the IIA have issued international standards and guidance relating to the co-ordination and co-operation between SAIs and internal auditors in the public sector, including INTOSAI GOV 9150 Coordination and Cooperation between SAIs and Internal Auditors in the Public Sector (INTOSAI 2010) and IIA IPPF Standard 2050 (IIA 2016) and Practice Advisory 2050-1 Coordination (IIA 2009). Moreover, a paper prepared jointly by the European Entity of Supreme Audit Institutions (EUROSAI) and the European Confederation of Institutes of Internal Auditing (ECIIA) elaborates the main trends in the co-ordination between external and internal audit institutions (see Box 2.6).
Box 2.6. EUROSAI and ECIIA Study: Co‑ordination between external and internal auditors
In 2014, EUROSAI and ECIIA jointly published a study that elaborated the mechanisms and challenges for co-operation and co-ordination between external and internal audit entities. The following are some of the key findings from the report:
A very large majority of SAIs are using international standards or international references regarding co-ordination and co-operation with internal audit institutions. Most of them refer in general to the International Standards for Supreme Audit Institutions (ISSAIs), International Standards on Auditing (ISA) and INTOSAI’s GOV standards, such as ISSAI 1610, ISA 610, INTOSAI GOV 9140 and INTOSAI GOV 9150. Only a minority have explicit, written SAI internal rules, such as auditing manuals, standards, guidance, procedures or checklists, documenting and formalising the co-ordination and co-operation channels.
Co-ordination and co-operation between SAIs and internal auditors is often described as “informal”, which can be difficult to assess or ensure the quality of its implementation. The most common benefits of co-operation and co-ordination cited include:
promoting good governance by exchange of ideas and knowledge
more effective and efficient audits based on a clearer understanding of the respective audit roles with better co-ordinated internal and external audit activity
resulting from co-ordinated planning and communication
refined audit scope for SAIs and internal auditors.
However, almost half of the responding SAIs stated they experience risks or identify potential risks in relation to co-ordination and co-operation. A majority of SAIs pursued co-ordination and co-operation largely in the following areas:
evaluating the audited entity’s internal control framework and risk-management arrangements
evaluating the entity’s compliance with laws and regulations
documenting the entity’s systems and operational processes.
Source: EUROSAI and ECIIA (2014), Coordination and Cooperation between Supreme Audit Institutions and Internal Auditors in the Public Sector.
The IGAE could further develop its communication strategy to demonstrate the value of the SSC to government entities and oversight bodies
While the SSC is a legal requirement for entities, the IGAE could benefit from adopting an approach that promotes this added value of the process to entities beyond compliance. Stakeholders from the line ministries indicated an interest in being able to discuss and share recommended good practices as a result of the SSC with peers or other government institutions. They also welcomed more informal discussions and communication with the ONA.
The Ministry of Finance reflected on the benefits to entities and affiliated line ministries in having access to the results of the risk assessment ranking in the automated review component of the SSC. This information was seen as potentially useful to entities for their own risk assessment purposes. It also provides line ministries with greater visibility on the performance of entities that are not often on the radar, even if there is no immediate concern over its financial sustainability (OECD, 2020[28]). Increased automation and real-time information were cited as areas that the ONA could improve to better support entity’s efforts to provide information for the SSC (OECD, 2020[28]).
The ONA can therefore consider leveraging the broader application of the results of the SSC to promote its benefits as a valuable tool for improving the strategy setting, decision-making, and daily management and operations of public entities. Management of public entities have the primary responsibility to establish and use their internal control system to identify and effectively mitigate programme and risks (The Institute of Internal Auditors, 2019[27]). The people who are responsible for achieving the entity’s objectives and delivering its services should also take responsibility over risk management, including identifying and putting in place control activities to mitigate risks. “Management ownership” implies not only that management and staff understand the institutional reform, but also that they embrace it. Sharing the results of the risk assessment component of the SSC could enable this.
As this is still a new process and a legal requirement, a learning curve for entities is to be expected. However, ONA should consider opportunities to raise awareness of the process and the expectations so that entities can better prepare and organise limited resources to respond. The ONA indicated that training on the requirements of the SSC was envisaged to be incorporated as a module on public administration training for civil servants and for new recruits to the IGAE in 2021 (OECD, 2020[2]). The ONA could consider including targeted reports of the results of the SSC that reflects the information needs of its various stakeholders where permitted by regulation.
Conclusion
In a short timeframe, the IGAE and the ONA have developed an effective methodology for continuous supervision in Spain that reflects the original spirit of the 2013 CORA reform proposals and subsequent regulations. As the ONA develops the SSC, in addition to enhancements to the risk assessment methodology described in Chapter 1, it could strengthen its strategy and capacity to make use of data and ensure that processes are in place for continuous improvement. This could include establishing feedback loops to begin systematically and iteratively monitoring the most challenging, time-consuming and error-prone aspects of the current process.
Data governance, data management and data skills are all key factors that can influence the effectiveness of the SSC. The chapter makes several recommendations and sub-recommendations for the IGAE and the ONA that touches on these areas. For instance, the chapter highlights opportunities for the ONA to automate processes for importing data and analyses, some of which are now partially automated. It also suggest that the ONA take additional steps to validate and corroborate self-reported data to provide further assurance of the quality of the data inputted into the SSC. In addition, the chapter considers the full cycle of the SSC, and recommends improvements to the ONA’s processes for tracking conclusions and recommendations from its continuous supervision activities. If the ONA moves towards a more data-driven, automated approach, additional expertise and specialised data skills will also be needed to enhance the SSC.
Finally, the chapter recognises the political economy and overall context in which the ONA and the SSC operate. Specifically, the ONA is breaking ground on continuous supervision in Spain and this has implications for stakeholders that have a direct impact on the SSC, or on institutions who could benefit from knowing the results of the monitoring for enhancing their own governance. The chapter therefore offers recommendations to promote the transparency, communication and co-ordination for continuous supervision. Specifically, the ONA could strengthen the transparency of its efforts by publishing the annual reports of the SSC. An audit committee with the SSC as part of its responsibility could also help to promote transparency as well as ensure input from stakeholders across government, including the Ministry of Finance, Council of Ministers and other oversight bodies. The chapter also encourages the IGAE and the ONA to enhance co-ordination and communication with key oversight institutions, in part to avoid duplication but also to demonstrate the value of the SSC as it evolves.
References
[5] Canada Border Services Agency (2019), Audit of Enterprise Architecture, https://www.cbsa-asfc.gc.ca/agency-agence/reports-rapports/ae-ve/2019/ea-ae-eng.html.
[11] European Platform Undeclared Work (2016), Data Mining for more Efficient Enforcement.
[10] Fazekas, M., Ugale, G, & Zhao, A. (2019), Analytics for Integrity: Data-Driven Approaches for Enhancing Corruption and Fraud Risk Assessments, OECD, Paris, https://www.oecd.org/gov/ethics/analytics-for-integrity.pdf.
[17] Government of Spain (2019), Royal Decree-Law 3/2019, https://www.boe.es/diario_boe/txt.php?id=BOE-A-2019-1782.
[22] Government of Spain (2018), Official Gazette (Boletín Oficial del Estado), https://www.boe.es/eli/es/o/2018/04/09/hfp371.
[23] Government of Spain (2015), Official Gazette (Boletín Oficial del Estado), https://www.boe.es/buscar/act.php?id=BOE-A-2015-10566.
[16] Government of Spain (2003), Law 47/2003, of November 26, General Budgetary, https://www.boe.es/buscar/act.php?id=BOE-A-2003-21614&p=20201231&tn=6.
[26] IGAE (2019), Norma Internacional de Auditoría (NIA-ES-SP) 1610: Utilización del trabajo de un experto del auditor, https://www.igae.pap.hacienda.gob.es/sitios/igae/es-ES/Control/CFPyAP/Documents/Copia_Electr%C3%B3nica_NIA-ES-SP%201620%20%20NOTA.pdf.
[25] International Auditing and Assurance Standards Board (2013), International Standard on Auditing (ISA) 610, https://www.ifac.org/system/files/publications/files/ISA-610-(Revised-2013).pdf.
[15] International Budget Partnership (2019), Open Budget Survey: Spain Country Summary, https://www.internationalbudget.org/open-budget-survey/country-results/2019/spain.
[4] OECD (2020), Auditing Decentralised Policies in Brazil: Collaborative and Evidence-Based Approaches for Better Outcomes, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/30023307-en.
[8] OECD (2020), “Digital Government Index: 2019 results”, OECD Public Governance Policy Papers, No. 3, OECD Publishing, Paris, https://dx.doi.org/10.1787/4de9f5bb-en.
[28] OECD (2020), OECD Fact-Finding Interview with the Inspector General of Services for the Ministry of Finance (Inspección General de Servicios del Ministerio de la Hacienda).
[2] OECD (2020), OECD Fact-finding interviews with the National Audit Office (Oficina Nacional de Auditoría, ONA).
[9] OECD (2019), The Path to Becoming a Data-Driven Public Sector, OECD Digital Government Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/059814a7-en.
[1] OECD (2014), Spain: From Administrative Reform to Continuous Improvement, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264210592-en.
[3] ONA (2020), Informe de Evaluación Supervisión Continua 2017 y 2019.
[24] ONA (2018), Estrategia del Sistema de Supervisión Continua (2018-2020).
[21] The European Confederation of Institutes of Internal Auditing (2019), Audit Committees in the Public Sector., https://www.eciia.eu/wp-content/uploads/2019/07/Audit-Committee-Paper-8th-draft-15.7-disp.pdf.
[27] The Institute of Internal Auditors (2019), Supplemental Guidance: Unique Aspects of Auditing in the Public Sector.
[18] The Institute of Internal Auditors (2017), International Professoinal Practices Framework: Implementation Guide 2440--Disseminating Results, https://na.theiia.org/standards-guidance/Public%20Documents/Transparency%20of%20the%20Internal%20Audit%20Report%20in%20the%20Public%20Sector.pdf.
[20] The Institute of Internal Auditors (2014), Global Public Sector Insight: Independent Audit Commitees in Public Sector Organisations, https://global.theiia.org/standards-guidance/Public%20Documents/Independent-Audit-Committees-in-Public-Sector-Organizations.pdf.
[12] The Institute of Internal Auditors (2009), Practice Advisory 2500.A1-1: Follow-up Process, https://www.iia.nl/SiteFiles/IIA_leden/Parktijkadviezen/PA%202500A1-1.pdf.
[19] The Institute of internal Auditors (2012), Leading Practice: Transparency of the Internal Audit in the Public Sector, https://na.theiia.org/standards-guidance/Public%20Documents/Transparency%20of%20the%20Internal%20Audit%20Report%20in%20the%20Public%20Sector.pdf.
[14] The Institute of Internal Auditors Australia (2020), Reporting on the Status of Audit Recommendations, https://iia.org.au/sf_docs/default-source/technical-resources/2018-whitepapers/iia-whitepaper_reporting-on-the-status-of-audit-recommendations.pdf?sfvrsn=2.
[13] US Goverment Accountability Office (2021), GAO: Duplication and Cost Savings, https://www.gao.gov/duplication-cost-savings (accessed on 21 March 2021).
[6] US Government Accountability Office (2017), Organizational Transformation: A Framework for Assessing and Improving Enterprise Architecture Management, https://www.gao.gov/assets/gao-10-846g.pdf.
[7] US Government Accountability Office (2004), Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity, https://www.gao.gov/assets/gao-04-394g.pdf.
Notes
← 1. Part II, Chapter II, Organisation and Functioning of the state institutional public sector. Article 81.2 requires public administrations to establish a system of continuous supervision of their dependent entities, justifying the reasons for their existence and financial sustainability and include proposals to maintain, transform or dissolve the entity. Article 84 defines the categories of public sector entities in scope for continuous supervision and efficiency control reviews while Article 85 defines the roles and responsibilities of the Hacienda, the IGAE and the ministerial inspection units.
← 2. General data about public entities from INVESPE/INVENTE, also developed and maintained by the IGAE, is transmitted to “El Cubo” using automated processes.
← 3. OECD (2020[8]), “Digital Government Index: 2019 results”, OECD Public Governance Policy Papers, No. 3. Spain was ranked seventh overall of 33 countries and fourth on the digital by design, data-driven public sector and proactiveness dimensions.
← 4. The OBS assesses the “online availability, timeliness, and comprehensiveness of eight key budget documents using 109 equally weighted indicators and scores each country on a scale of 0 to 100” (International Budget Partnership, 2019[15]).
← 5. Box 8.1 Audit, evaluation and inspection in the context of the Spanish control framework.
← 6. Article 85 Effectiveness control and continuous supervision.
← 7. Interview with the Gabinete Técnico, Tribunal de Cuentas, December 2020.