Enhancing the Public Procurement Performance Measurement Framework in Hungary

4. Communicating the results of a performance measurement framework

4.1. Improving the readability and structure of the report

4.1.1. Providing the list of indicators and their value in a consistent way

The Government published the results of the Hungarian performance measurement framework for the first time on 28 February 2023. Before its publication, the OECD was asked to provide comments on the draft report. Due to the short deadline, the OECD was not able to provide in-depth comments to the draft report but provided initial comments mainly focusing on the structure of the report, the presentation of the findings and policy recommendations and the readability of the report. The OECD acknowledged that the draft report included plenty of valuable and relevant information and figures on the performance of the Hungarian public procurement system based on the indicators and sub-indicators part the Hungarian framework but also on other information collected through surveys. It was also highlighted that for the reader, it might not be clear which exact indicators are part of the measurement framework and which data is only used to support the analysis. For instance, the report includes data on the “statutory fees” that contracting authorities must pay when conducting procedures, and “fees payable to central government purchasing bodies” which are not part of the indicator framework but represent a useful source of information for the reader. During the finalisation of the report, the PMO took some of the OECD comments on board and improved the structure and readability of the report. However, due to the time constraints, the report could have not been restructured completely.

To guide readers, the report on the performance of public procurement that is based on the Hungarian framework should at least recall the indicators and sub-indicators used as part of this framework and provide their results. A table providing an overview of all indicators could be added to the executive summary of the report. In addition, each chapter could start with the list of related indicators for readers interested in a specific area or section of the report to have a clear picture of the indicators covered. Furthermore, an independent document providing the list of indicators and their values could be published online.

Furthermore, some indicators and the related figures are well detailed in the report and others are more detailed in annexes of the report. The report should be consistent in the way it covers the different indicators. For instance, there is no figure in the report on some indicators (e.g. ratio of estimated value to contract price, number of staff who received a training in a specific year).

4.1.2. Providing concise findings of the report in the executive summary and the different chapters

An executive summary is intended to provide readers with a general idea of what they can expect to learn from the report. It should also give readers who are more closely involved with the issue under discussion an overview of the main findings and recommendations they can digest easily and quickly. The executive summary is also of value as a document that can be used to promote the report to a wider audience (OECD, 2015[58]).

In line with good practices, the report includes an executive summary with all relevant information. To enhance the readability of the report, it could be recommended to organise the executive summary by categories and sub-categories of the indicators of the Hungarian framework and to focus only on the main findings.

4.2. Clarifying how the results of the framework will be used for policy actions

One of the objectives of the Hungarian framework is to contribute to the identification of areas that require further policy action to achieve the policy objectives (Hungarian Government, 2022[6]). When identifying these areas, it could be used by the government to identify the necessary actions to enhance the system in place when these actions fall under its remits (without a detailed description of the actions) and to mention them in relevant governmental documents including the action plan to increase competition in public procurement and relevant government decisions. When these actions do not fall under its remits, the government can support other stakeholders in taking the necessary actions.

The analysis of the report shows different areas where the Hungarian system is underperforming and that would require tailored policy actions. However, in the report the government does not make suggestions for the necessary actions to remedy the situation except for the low performance in terms of single bids where the government committed to develop a dedicated action plan that was published in March 2023 (Hungarian Government, 2023[15]) and that will be updated in 2024. For instance, the report highlights a relatively low uptake of green and social procurement (see Figure below), however the report is silent about potential policy actions. Another relevant example is the relatively low number of public procurement challenges initiated by market players. While the root cause might be diverse, a policy action is needed to enhance the efficiency of the remedies system.

Figure ‎4.1. Uptake of green and social procurement in Hungary

Source: (Hungarian Government, 2023[7])

Furthermore, in some cases, the report identifies some actions but does not specify the responsibilities of all relevant stakeholders. For instance, regarding the low numbers of employees receiving trainings in public procurement, the report concludes that “the continuous training of public procurement professionals and the improvement of the quality of the pool of professionals will remain an important task for contracting authorities in the future”. However, the PMO and the PPA could also contribute to ensuring further trainings to government officials (through government decisions mandating regular trainings, the provision of trainings, etc.). Therefore, the report should consider actions for all stakeholders to ensure the reliability of its conclusions.

In addition, the PMO should take the lead in some actions. For instance, the report highlights that none of the CPBs have a framework to monitor the effectiveness of public procurement, but half of them are willing to develop one. CPBs could ask for the support of PMO to develop a monitoring framework in line with the Hungarian measurement framework. As mentioned in Section ‎2.2.2, further collaboration with CPBs and CAs (beyond the provision of data) could strengthen the measurement framework and also ensure its use by key public procurement players.

4.3. Improving the analysis of the results of the indicators

Analysing the results of the various indicators is particularly relevant to take the necessary policy actions in case of need. The PMO used relevant data that goes beyond the value of indicators to analyse the results of some indicators. It used for instance feedback and complementary information through the surveys addressed to different stakeholders (contracting authorities, economic operators, procurement experts and CPBs) and used information available in legal documents (e.g. statutory fees regarding the remedies system).

For instance, to further understand the root causes behind the low level of competition in the public procurement market, the PMO addressed a specific question to the different stakeholders. The results of this questions are useful to support the analysis on single bids.

The analysis of the results of indicators could be improved in two regards. First, there could be further links between the results of the different indicators. Indeed, while the Hungarian framework includes 77 independent indicators and 57 sub-indicators, some of them could be considered as “integrated”, meaning that it is relevant linking indicators between them for a more meaningful analysis. For instance, it could be relevant to assess jointly the single bid indicator in the last years with the evolution of remedies fees, the number of infringements identified by the PPAB in relation to competition issues, the share of unsuccessful tenders due to single bids, etc.

Secondly, the results of some indicators would benefit from further explanations and analysis. For instance, the share of lots with social aspects decreased by around 60% between 2019 and 2020 (it moved from 2.6% to 1%). However, the report does not provide any insights or explanation.

4.4. Publishing information on the performance framework

The OECD Report on public communication highlights that the provision of accurate and timely information, alongside the opportunity for stakeholder participation and feedback, are essential elements of the democratic policy-making process. In addition, it has proven to be critical for policy implementation and compliance, and a prerequisite for transparent, accountable, and responsive public administrations (OECD, 2021[59]).

Public procurement is an essential governmental activity representing a significant share of GDP impacting economies, societies and the environment. Therefore, in addition to ensuring available data and indicators on this key activity, it is essential for governments to ensure an adequate communication and promotion of the results of the measurement framework.

The report on the measurement framework was published in the EKR on 28 February 2023 in line with the commitments of the Hungarian Government. However, there was no communication to promote the publication of the results of the framework. As mentioned in Section ‎2.3.4, the Hungarian government could leverage different channels to disseminate and promote the results of the report including i) relevant stakeholders’ websites like the websites of the PPA, the Integrity Authority, the Competition Authority, the Hungarian Chamber of Commerce and Industry, etc. ii) the mobile application managed by the PPA, the “Daily Public Procurement” (Napi Közbeszerzés in Hungarian), iii) social media and press release sent to relevant medias. Furthermore, communication around the report could be tailored to the main audience of each communication mean and their level of knowledge on public procurement.

Furthermore, all documents related to measurement framework including the report are published on EKR. However, the website does not include a formal “webpage” dedicated to this topic that interested parties can consult to find all relevant information and documents. It is therefore, recommended to develop in EKR a dedicated webpage listing all relevant documents related to the public procurement performance measurement framework.