Strengthening Oversight of the Court of Auditors for Effective Public Procurement in Portugal

Executive summary

The Court of Auditors of Portugal (Tribunal de Contas, TdC), as the Supreme Audit Institution of the country, plays a crucial role in safeguarding the legality, regularity, cost-effectiveness and efficiency of public procurement, which represents a significant share of Portugal's GDP (10.3% of in 2021). To fulfil its mission, every year the TdC conducts a significant number of audits related to public procurement processes (ex ante, concomitant and ex post), requiring extensive human and financial resources.

To enhance its audit activities, the Organisation for Economic Co-operation and Development (OECD) and partners at NOVA University supported the TdC in developing a risk assessment methodology for its audit selection for public procurement that relies on data, advanced analytics and artificial intelligence. At the core of this initiative, was the need to map risks and data sources, examine the digital maturity of the TdC to conduct such work, and assess the quality of potential databases that could be used for building a new risk methodology. This report describes this work, which was a precursor to the next phase of the project of developing a proof-of-concept for a data-driven model for the TdC to assess risks in public procurement. The report looks at the key role of the TdC in promoting a risk-based approach in public procurement, at key aspects of the TdC’s data ecosystem and, at key variables from stakeholders’ databases that can be employed to assess risks and irregularities in public procurement,

Key findings and recommendations

• The public procurement regulatory framework of Portugal has undergone multiple amendments that may impact TdC audit activities. The TdC must align its audits with the updated regulatory framework, and contracting authorities may face challenges in ensuring the compliance of their procurement procedures, which may be audited or controlled.

• The public procurement system in Portugal faces key challenges that could inform the TdC’s risk-based approach to public procurement. These challenges include a low level of competition and potential transparency and integrity issues. In addition, the TdC’s risk-based approach should consider different categories of risks, beyond those having a financial implication and including environmental ones.

• Public procurement risks are considered as part of the strategic plan of the TdCt. The teams in charge of concomitant and ex post control implement some kind of risk-based approach which would benefit from being formalised. Implementing a risk-based approach is not seen as a priority by the a priori audit team, given that all tenders and contracts falling under the scope defined by law should be audited. However, such an approach would enable this department to identify red flags (irregularities and specific issues) in contracts.

• The implementation of a data-driven risk-based approach to public procurement activities requires identifying and working with the relevant databases. The following external databases could be considered: BASE, managed by Institute of Public Procurement, Real Estate and Construction (Instituto dos Mercados Públicos, do Imobiliário e da Construção, IMPIC), the registry of beneficial ownership, managed by the Portuguese Institute of Registries and Notaries (Instituto dos Registos e do Notariado, IRN); and databases managed by the Tax and Customs Authority (Autoridade Tributária e Aduaneira, AT). TdC access to external databases can represent a challenge given data protection requirements. TdC has been granted access to IMPIC data. However, access to IRN and AT databases, which would enable TdC to identify additional risks, is more challenging, as it requires the signing of protocols between each of these institutions and TdC.

• In Portugal, there is no dedicated strategy on risk management in public procurement covering different categories of risks. To infuse a risk management culture in public procurement, the government of Portugal could consider developing a dedicated and comprehensive public procurement risk management strategy encompassing all categories of risks.

• TdC established the Department for Innovation, Technology and Methodologies (CITM), a central entity to lead its digital transformation. Good governance processes are in place to drive greater coherence between TdC’s digital initiatives. The TdC’s Digital strategy benefited from external self-assessments as well as internal collaboration and analysis.

• A framework for monitoring and evaluating TdC’s digital initiative for data-driven risk assessments in public procurement would help strengthen its goal of moving towards a more automated, AI-driven form of risk assessment. Another important consideration is for TdC to evaluate the return on investment in the development of digital tools.

• TdC’s digital initiatives and digital transformation depend on change management and continuous learning to ensure the planning, execution and sustainability of digital transformations, such as data-driven risk assessments. There is scope to strengthen the data literacy and digital skills of TdC staff assessing public procurement risks, and this can be achieved and sustained through targeted training and internal knowledge-sharing.

• The TdC has implemented many improvements to IT infrastructure, such as the creation of the “ModInAudit’’ application. However, there is scope to develop and implement other improvements to achieve interoperability with other public administration organisations, better automation of internal verifications, and improve data quality.

• To create a robust data-driven risk assessment tool, the main challenge relates to data preprocessing to overcome data quality issues. Three analytical requirement categories – rule-based, inference-based and model-based were examined in context of the TdC’s audit activities and needs. Key challenges were identified in relation to optimising the processes to extract the best value from semi-structured and structured information.

• To develop a data-driven risk-based model will require the identification of all relevant risk indicators. A thorough assessment of the quality of each indicator will need to be undertaken before inclusion in the risk model.