Public procurement measurement frameworks are essential, both to assess progress and achievements periodically and consistently and to identify gaps in progress against objectives and targets. This report assesses the public procurement performance measurement framework in Hungary and provides recommendations for its improvement. It also explores aspects to consider in developing a public procurement measurement framework and communicating the results of the framework.
Enhancing the Public Procurement Performance Measurement Framework in Hungary
Assessing Efficiency, Compliance and Strategic Objectives
OECD Public Governance Reviews
Abstract
Executive Summary
Public procurement measurement frameworks are essential to assess progress and achievements periodically and consistently and identify gaps in progress against objectives and targets. In Hungary, where public procurement represents 16% of GDP in 2021, no comprehensive public procurement measurement framework was in place before 2022.
The Hungarian government committed, in its Recovery and Resilience Plan and within the conditionality procedure for EU funds, to set up a comprehensive measurement framework to regularly assess the performance of the public procurement system and to analyse the reasons behind low levels of competition in certain sectors. The development and implementation of this framework, which was inspired by the OECD comprehensive measurement framework, was subject to Government Decision 1425/2022 (5. IX.) published in September 2022, and its results for 2019-2022 were published in February 2023. The measurement framework developed by Hungary includes 77 indicators and 57 sub-indicators and goes beyond the areas foreseen in the government decision and the commitment made to the European Commission.
Additionally, the Hungarian government requested the support of the OECD to assess the developed framework and to provide key recommendations to enhance it, given the Organisation’s work on the measurement of the procurement function. This report provides an assessment of the Hungarian public procurement measurement framework by i) exploring key aspects to consider for the development of the framework: ii) providing a detailed assessment of the indicators included in the framework and suggesting additional ones; and iii) discussing the communication of the results of the framework, as well as a proposed timeline for implementing the recommendations.
Key findings and recommendations
Going beyond its initial objective, the Hungarian measurement framework for public procurement covers key aspects of the procurement system, including compliance issues (e.g. legal compliance and the remedies system), efficiency issues (e.g. competition, capacity and centralisation), and the use of public procurement as a strategic tool for achieving wider policy objectives, such as the green transition and social inclusion. The framework could be further improved by integrating indicators focusing on integrity and monitoring/oversight of the public procurement system.
The Government Decision established a working group to consult with experts and independent non-governmental organisations active in the field of public procurement. While the collaboration with the working group brought significant benefits to the development process, more stakeholders within and outside the public sector (beyond the working group) could be involved in the development and analysis of the procurement measurement framework.
The different indicators of the public procurement measurement framework are based on 3 types of data sources: i) the national e-procurement system, known as EKR (39% of indicators), ii) data provided directly by relevant stakeholders (42.8%) and iii) surveys targeting relevant stakeholders (contracting authorities, economic operators, public procurement experts and central purchasing bodies- CPBs).
On EKR, while the quality of the data has improved, there are still issues related to the manual entry of information by contracting authorities or economic operators. Therefore, the Prime Minister’s Office and the Public Procurement Authority should raise awareness among contracting authorities of the importance of ensuring the quality of data entered manually. Furthermore, in line with EU legislation, Hungary had to implement eForms, which is a new data standard expected to improve the availability, quality and (re)usability of procurement data (TED data) at EU level. The implementation of eForms is key for data collection and thus for the access and reliability of data.
Stakeholders, including the Public Procurement Arbitration Board (PPAB) and CPBs, directly provided data to the PMO for the calculation of some indicators. While EKR, CPBs systems and the PPAB system could remain independent, their integration could contribute to enhancing the efficiency of the monitoring of the public procurement system.
Given the unavailability of some data, two surveys were launched as part of the measurement framework of Hungary. One targeted economic operators, contracting authorities and public procurement experts (covering 14.3% of indicators), and another targeted CPBs (covering 28.6 % of indicators). Some issues were identified with these surveys such as survey design and the relatively low number of respondents. To enhance the efficiency of surveys, it is key to improve the design, the number of respondents, and to consider the frequency of launching surveys.
While the establishment of the Hungarian public procurement performance measurement framework already represents an achievement, some indicators could be improved, either in terms of their scope, the methodology or the possibility of providing sub-indicators to provide a better understanding of the system in line with policy priorities.
The implementation of a performance indicator requires ensuring that different users and stakeholders of the public procurement system understand its aim and the relevance. The Hungarian framework does not provide such information for all indicators. Therefore, the framework could benefit from adding a detailed methodology for calculating each indicator and sub-indicator as well as the reason for focusing on certain sub-categories or for choosing certain indicators.
As for possible additional indicators, the framework could be improved by providing, in addition to integrity and monitoring/oversight indicators, i) general data on the main breakdowns envisaged in indicators to understand their relevance; ii) indicators on “values” of public procurement spent (not only in numbers of procedures); iii) indicators to assess the level of competition; and iv) indicators on strategic procurement (considering data availability).
In terms of communicating the results of the framework, the annual report on the performance of the public procurement system could recall the indicators and sub-indicators and provide their results in a more consistent way and improve the analysis of the results. Furthermore, given that the framework is supposed to help identify areas requiring further policy action, the government could identify the necessary actions in the report and mention them in relevant governmental documents (when these actions fall under its remit).
To disseminate and promote the results of the annual report, the government could use channels such as relevant websites and the Public Procurement Authority mobile application.
