This chapter discusses public procurement processes in Malta throughout the procurement cycle. It highlights the need to strengthen procurement planning with a focus on procurement plans, and needs and market analysis. It also reviews procurement processes for open tender and highlights the need to simplify the vetting process, to further use multiple award criteria, and to improve processes for the tendering and contract management phase with a focus on the digitalisation of processes. Lastly, the chapter discusses the use of efficiency tools in Malta, with a focus on framework agreements, dynamic purchasing systems and centralisation schemes.
Public Procurement in Malta
2. Enhancing public procurement processes
Abstract
2.1. Strengthening public procurement planning
2.1.1. Preparing and publishing procurement plans
Public entities have different mandates contributing to provide public services to citizens and businesses. To deliver on their mandate, those entities require the adequate human capital and the procurement of goods, services and public works. Therefore, the strategic planning of each entity is closely linked with procurement planning (OECD, 2021[1]). Procurement planning has two key benefits: i) ensuring a better management of time and resources for all stakeholders including the procurement departments within contracting authorities and the DoC (for entities under schedule 2 and 16) and ii) improving market engagement by providing economic operators with enhanced visibility on upcoming procurement opportunities (when the procurement plans are published), therefore increasing access to public tenders and ultimately promoting more competition. Discussions with contracting authorities in Malta highlighted the lack of strategic planning at the entity level and the lack of alignment and coordination with procurement planning, when available.
Procurement planning is formalised using procurement plans which usually include key information such as the procurement category, the estimated value, the procurement procedure, the estimated launch timeline, etc. (see Box 2.1). In some countries, such as Croatia and Latvia (European Commission, n.d.[2]), the preparation and publication of procurement plans is mentioned in the regulatory framework or in procurement guidelines, which is not the case of Malta and other European Union countries. Making better use of procurement planning was already mentioned as key area of improvement in the previous OECD assessment (OECD, 2019[3]). In Malta, discussions with stakeholders highlighted that the preparation of procurement plans is not a widespread practice. Only a few contracting authorities interviewed mentioned the preparation of an annual procurement plan that was requested by the SPD unit in charge of administering their tenders. The main benefit of such practice was to provide a better visibility for both the contracting authority and the SPD unit on the workload throughout the year. However, requesting procurement plans is not a homogeneous practice across the SPD units and DoC. Interviewed procurement officials recognise the benefits of such practice but mentioned challenges to collect data and information from the technical departments. Aware of those benefits, the DoC is working on regulatory changes to request procurement plans from contracting authorities.
In addition to the gaps related to the preparation of procurement plans, issues also exist regarding the publication of procurement plans which significantly decreases the benefits of such practice: on the one hand contracting authorities are not providing visibility to the market on upcoming procurement opportunities, and on the other hand it impacts the quality of procurement plans as well the accountability of contracting authorities. The need to publish procurement plans was confirmed by the 2021 report of the Malta Chamber of Commerce that reveals that 89% surveyed economic operators consider that contracting authorities should disclose their procurement plan in advance1 (The Malta Chamber, 2021[4]). This report also highlights that Malta Information Technology Agency – MITA is the only organisation that publishes on its website key information for the market. Therefore, in addition to reinforcing the strategic planning at the entity level, Malta should continue its efforts in order to integrate in the regulatory framework provisions related to the development and publication of procurement plans. These procurement plans could be published in the e-procurement system ePPS and in the entities’ websites. To streamline the process, the DoC should consider developing a procurement plan template to be used by all contracting authorities (Schedule 2, 3 and 16).
Box 2.1. Key information included in procurement plans
Procurement planning is defined as a process carried out by contracting authorities to plan purchasing requirements for a defined period. Procurement plans usually include:
Details on the stakeholders involved in the preparation of the plan, in terms of roles, department/s and any necessary sign-offs, including any other stakeholders consulted during the process
Description of procurement requirements – divided by the respective category; supplies, services, works.
Respective section/department requiring the procured goods (if relevant) and any areas affected by the procurement needs
Estimated budget and project implementation deadlines
Strategy including established timelines – for expected award, delivery or implementation
Action Plan – delineating key tasks to be completed by each identified stakeholder, with timeframes for completion. This may also incorporate the approach the project team will adopt to select the product/s or service together with the procurement type/procedure and any respective selection criteria
Source: (SIGMA, 2015[5])
To further engage the market, it could be beneficial to engage and inform the private stakeholders in advance. This is particularly relevant for complex and/or strategic procurement operations. In line with the European procurement directives and international good practices, the PPR provides the possibility to publish prior information notice (PIN). A PIN can be published up to 12 months ahead of the estimated publication of the tender and includes at least basic information regarding the goods or services to be purchased (EU Parliament and Council, 2014[6]). In Malta, the use of PINs is not a widespread practice. Between January 2020 and March 2022, only 8 PINs have been published (European Commission, 2022[7]). The DoC should consider raising the awareness of contracting authorities to the existence of such practice and its benefits. It could also promote using the PIN in selected tenders as pilots in order to demonstrate the value in its use.
2.1.2. Reinforcing needs and market analyses in contracting authorities
The information reflected in procurement plans relies on a sound needs analysis and market analysis. Indeed, those analyses reflect the first steps of the public procurement process. Their impact goes beyond the sound preparation of procurement plans, as it impacts the overall efficiency and success of public procurement operations (OECD, 2021[1]).
A needs analysis refers to the identification of end-users needs. It is recommended to follow a functional and performance-based approach meaning identifying the performance, functionalities, quality and quantity of the solution required. It should not be oriented towards products, services and brands available in the market. In Malta, based on discussions with contracting authorities, needs analysis are not conducted systematically. When conducted, they are mainly performed by the technical departments within contracting authorities (79% of surveyed contracting authorities).
To understand the extent to which the market can meet the identified needs, the needs analysis process should go hand in hand with a sound market analysis (OECD, 2021[1]). A sound market analysis is key to understand the characteristics, capacity and capability of the supply market and their capacity to respond to priorities and policy objectives of the procuring entity. In Malta, similarly to needs analysis, market analysis is mainly performed by technical departments within contracting authorities (71% of surveyed contracting authorities). Answers to the OECD survey and discussions with contracting authorities highlighted that in practice conducting market analysis is limited and concerns mainly new services, works, and supplies. The weaknesses regarding market analysis have also been identified by economic operators. A survey launched by the Malta Chamber of Commerce highlighted that 61% of respondents consider the level of market research and studies made by contracting authorities prior to the issue of tenders is “inadequate” (The Malta Chamber, 2021[4]).
In addition, different tools and methods can be used to engage and collect information from the market, including the publications of procurement plans, meetings with key suppliers and preliminary market consultations (see Box 2.2). The use of these methods depends on different elements such as the complexity of the procurement and the estimated value of the contract. In Malta, DoC provides a preliminary market consultation template that includes relevant questions to consider to assess the market. However, further guidance could support entities in choosing the appropriate method. For instance, Ireland developed a specific guidance on supply market analysis (NPPPU, n.d.[8]) (Office of Government Procurement, 2020[9]).
In Malta, the gaps to the needs and market analyses are often linked to the lack of capacity of the technical departments within contracting authorities in charge of performing these tasks. In this context, the DoC should consider providing further guidance and capacity building activities to contracting authorities on these tasks.
Box 2.2. How to engage the market and collect information on the market?
Different methodologies can be used to engage the market.
Direct engagement mechanisms:
Publish procurement plan.
Organise public events to meet with suppliers.
Meet with key suppliers (taking into account integrity risks).
Request for quotation (RfQ)/questionnaires.
Publish Prior Information Notices (PINs).
Getting information from third parties
Commission a consultant (public and transparent selection).
Use market analysis or sector study reports published by specialised companies or trade unions.
Consult other contracting authorities with experience in similar procurement.
Source: (OECD, 2020[10])
2.2. Enhancing the use of the open tender procedure
The public procurement regulatory framework should provide an appropriate range of procurement methods comprising competitive and less competitive procedures, when appropriate (MAPS initiative, 2018[11]). Competitive procedures should be the standard method for conducting procurement as a means of driving efficiencies, fighting corruption, obtaining fair and reasonable value for money and ensuring competitive outcomes (OECD, 2015[12]). Indeed, they can bring many benefits to contracting authorities and to the economy in general (OCDE, 2019[13]). The main procedure within competitive tendering is the open tender procedure which has become the default approach when procuring goods and services and public works (OECD, 2019[14]). In Malta, different procedures are used: open procedure, open accelerated procedure, restricted procedure, competitive procedure with negotiation, negotiation procedure, etc. As described in Figure 2.1., the most common used public procurement procedures are the open procedures. Open procedures represent 97% of all contracts in 2018, 96% of all contracts in 2019, and 94% of all contracts in 2020. The second most used procurement procedure in 2018 and 2019 is the open accelerated procedure (representing 3% of all contracts in 2018 and 2019). Negotiated procedures, restricted procedures and other procedures such as simplified procedures represent a very small share of Malta’s total public procurement procedures. Innovation partnership and design contest procurement were not used between 2018 and 2020.
In Malta, the open tender procedure entails a specific process that includes six main stages from tender preparation to the contract management and the conclusion of the contract (see Figure 2.2. ). During the tender preparation, usually contracting authorities’ respective technical department develop the technical specifications and the procurement section gathers all information required to compile the tender documentation. In some contracting authorities, officials are in charge of vetting draft tender documents to improve their quality. However, this practice is not adopted across all contracting authorities. Schedule 2 and 16 entities, submit draft tender documents to OD and SPD generic email respectively (according to the estimated value threshold) for vetting. Schedule 3 entities review their own documents without the involvement of DoC. With reference to tenders including the Best Price-Quality Ratio (BPQR) award criterion, the contracting authorities are to seek approval of the criteria from DoC DPPQA prior to vetting of the rest of the tender documents. Upon BPQR approval, the vetting process is assigned to a vetting officer within the OD or SPD or for Schedule 16 and 2 entities, according to the threshold and case. For the SPD documents are uploaded on Microsoft Teams and vetting officers review all tender documentation. The exchange of documents between the DoC and the CA can be iterated (depending on the quality of documents) until the final documents are approved by DOC.
The publication and clarification stage entails the publishing of the finalised tender documents on ePPS which is accessible by all interested economic operators. This is performed by DoC on behalf of Schedule 2 and 16 entities, and by contracting authorities themselves in the case of Schedule 3 entities. Economic operators may request clarifications through ePPS. In the case of Schedule 3 entities, contracting authorities access the ePPS and reply directly to interested bidders. In the case of Schedule 2 and 16 contracting authorities, SPD or OD (depending on the threshold) access the ePPS and relay the clarification requests to the contracting authorities via email. After, a review by the respective contracting authorities, responses are then sent via email to DoC (SPD or OD), who upload and publish replies on ePPS.
Tender opening and evaluation require coordination with contracting authorities and evaluation committees, coordinated through DoC for Schedule 2 and 16, and directly by Schedule 3 contracting authorities. Evaluation committees may request clarifications and/or rectifications to bidders, who respond accordingly through ePPS. Once the tender evaluation committee evaluates the administrative, technical, and financial components of the tender, a tender evaluation report is developed and reviewed by the SPD or OD and then submitted for approval to the Departmental Contracts Committee (DCC) for tenders within the SPD threshold, i.e. EUR 10 000 – 750 000 or General Contracts Committee (GCC) for tenders above EUR 750 000 and/or within the OD threshold. GCC is the permanent committee established within the DoC, while DCC is set up at each ministry.
Contract awards are then issued whereby bidders accept award through the ePPS following the standstill period. This is followed by the publication of contract notices and subsequent issuance of contracts. In doing so, performance guarantees, any relevant forms required and contracts are coordinated between the successful bidder and the contracting authority. Whilst this is coordinated themselves by Schedule 3 entities, the contract is prepared by DoC in the case of Schedule 2 and 16 entities.
Following the award of the contract, contract management is managed by the respective contracting authority. In the case of Schedule 2 and 16 contracting authorities, modifications of contracts are approved either by the respective head of contracting authority (in the case of tenders administered by SPD) or by the General Contracts Committee (in the case of tenders administered by the OD and therefore contracts signed by the Director General of DoC), depending on the threshold. Contract management and amendments are approved by the respective head of contracting authority in the case of Schedule 3 entities. For all schedules, the respective Permanent Secretary has either to approve or endorse the modification requests.
The process for the restricted procedure is similar, with an added step for Schedule 2 and 16 entities, whereby the contracting authority is to provide a justification to be approved by SPD or OD Director depending on the situation.
2.2.1. Simplifying the vetting process of procurement operations at the pre-tendering phase
Several entities involved in vetting procurement processes
The procurement process at the pre-tendering phase involves several entities, for vetting and approval of tender documents. In the case of approvals, different stakeholders are involved, depending on i) the estimated value of the procurement, ii) the procurement procedure and iii) the schedule of the entity (2, 3 or 16).
Open tender procedures for Schedule 16 entities might involve several vetting processes. While some vetting processes are specific to each entity such as internal vetting, vetting from a Ministry (for some entities under the supervision of a ministry); other vetting processes are common to all Schedule 16 entities. Indeed, as open and restricted tenders between EUR 10 000 and 750 000 are administered by the SPD, those tenders are also vetted by different SPD officials such as the SPD vetting official, the SPD assistant director and/ or the SPD director. Some contracting authorities allocate resources to vet the tender documents prior to submission to DoC, with the aim of increasing quality of tender documents.
In addition, open and restricted tenders above EUR 750 000 are administered and vetted by the OD of the DoC (OD vetting assistant and OD director). Last but not least, the use of BPQR is vetted by the DPPQA of the DoC for all schedules.
In addition to internal vetting processes, open tenders issued by Schedule 2 entities, are subject to the vetting of the OD when their value is above EUR 140 000 (OD vetting assistant and OD director). Table 2.1 summarises the different actors involved in internal and external vetting processes.
Table 2.1. The vetting process for open tender procedures
Schedule 16 |
Schedule 2 |
Schedule 3 |
|
---|---|---|---|
Internal |
Specific to each contracting authority |
Specific to each contracting authority |
Specific to each contracting authority |
External |
SPD (for tenders between EUR 10 000 – EUR 750 000 ) |
OD (for tenders = or > EUR 140 000) |
No vetting |
OD (for tenders > EUR 750 000) |
|||
DPPQA for BPQR criteria |
Source: the PPR
Streamlining the vetting process in place for the pre-tendering phase
For schedule 2 and 16 entities, the vetting process starts when the SPD or the OD receive all procurement documents and relevant supporting information, including necessary approvals and the nomination of the Evaluation Committee and Contract Manager. As described in the previous section several stakeholders are involved in the vetting process of open tenders.
Indeed, when the vetting is performed by the DoC, a vetting officer from one of the SPD departments or OD department is in charge of the procedure. The officer registers and assigns a reference number to the procurement documents, which are usually then uploaded on Microsoft Teams (for SPD) for vetting purposes. SPD Vetting Officer vets submitted documents and provides feedback via tracked changes on Teams to the contracting authority. Once the first vetting is concluded, in the case of the SPD, the officer informs the contracting authority via email that the vetted documents can be accessed via Microsoft Teams and a meeting is set between both parties to clarify issues. In the case of OD, officers send the documents with track changes via emails. Following this, there are additional layers of vetting depending on the DoC directorates involved. In the case of SPD, documents (without track changes) are then sent to SPD Assistant Director for review, comments and approval. This cycle is repeated until procurement documents are finalised for publication. Before publication, procurement documents are referred to contracting authorities via email for final approval by Vetting Officer. Once approval is received from contracting authorities, the Assistant Director emails SPD/DoC Operations Director for final approval to publish. Lastly, Director SPD/OD addresses any issues with the contracting authority and the tender is published on the ePPS by SPD/OD on behalf of the contracting authority.
The analysis of the process and discussions with contracting authorities highlighted different issues: first, the need to streamline the vetting process as it is lengthy and administratively burdensome, second the need to harmonise the process between the SPD and the OD and lastly the necessity to improve communication tools used between DoC and contracting authorities.
Regarding the need to streamline the vetting process, DoC requires sending around 11 documents including the list of evaluation committee members that needs to be signed by the head of each contracting authority and the respective permanent secretary (for entities under SPD remits), to start the vetting process (see Box 2.3). Some contracting authorities mentioned that given the length of the process, the evaluation committee members are likely to change and the internal processes to approve new committee members is burdensome. Therefore, internal processes to get the approval from the head of the contracting authority and the respective permanent secretary (when applicable) should be streamlined to enhance the efficiency of the process.
Box 2.3. List of documents required for the vetting process
1. Signed Procurement Originators Form (POF)
2. Signed Commitment of Funds (when relevant)
3. Budget Office and/or any other Ministry for Finance and Employment approval/s (when relevant)
4. Budget estimate breakdown
5. Evaluation committee’s nominees (including respective CVs)
6. Contracts Manager nominee
7. Green Public Procurement (GPP) approval
8. Permanent Secretary approval covering: publication, evaluation committee, contracts manager
9. Planning Authority (PA) permits (in the case of Works tender), including drawing/s and drawings schedule as applicable
10. Tender document including:
technical specifications and terms of reference
technical offer form including questionnaire
literature list (in line with technical offer)
sample list (in line with technical offer)
financial bid form and/or bill of quantities
key experts form including statement of exclusivity and self-declaration form (as required)
11. BPQR approval if applicable.
Source: Responses of the DoC to the OECD questionnaire
In addition, the lack of standardisation in procedures between SPD and OD in relation to vetting was noted for Schedule 16 entities. Indeed, the SPD has a 3-layer vetting system (Vetting Officer, Assistant Director and Director), whilst OD has mainly a 2-layer vetting process (Vetting Officer, Director). On the one hand, discussions with DoC indicated that repeated vetting cycles are required due to the differing levels of quality of tender documents received by the different directorates of DoC. On the other hand, discussions with contracting authorities highlighted the approval layers in the current vetting system could represent in some cases a source of inefficiency which could lead to significant delays in the vetting process, generally taking 2 to 3 months for finalisation, and even longer periods for tenders integrating the BPQR evaluation method. In addition, the last round of vetting does not involve the same level of detailed feedback whether it is a procedure under the OD or SPD remits.
Furthermore, differences or conflict in opinions between the first and other officials in charge of vetting was noted amongst some contracting authorities, in particular for the SPD. Discussions with contracting authorities indicated issues relating to capacity of some SPD vetting officials. In addition, tracked changes or comments made by one vetting officer and addressed by the contracting authority in the procurement documents, cannot be seen by the other vetting officials. This lack of tracking and visibility regarding the changes made to the procurement documents might lead to conflicts of opinions between the different vetting officers. Such inconsistencies between vetting officials further increase the length and the burden of the process. As a result of delays experienced across the process, some contracting authorities mentioned that they tend to use direct orders, particularly when funding deadlines are concerned.
To address delays and inefficiencies in the process, DoC should consider implementing several actions, as follows:
i. strengthening public procurement officials’ capacity in contracting authorities in order to enhance the quality of tender documents received;
ii. encouraging contracting authorities to validate their own tender documents prior to submission to DoC in order to facilitate the vetting process;
iii. standardising and reducing the layers of the vetting process in order to limit delays and;
iv. reinforcing vetting officials’ capacity to ensure coherent feedback throughout the process
v. promoting communications and modifications savings (through track changes or other relevant digital tools) between officials in charge of vetting and contracting authorities in order to keep track of previous modifications
Reinforcing the capacity of DoC officials could be done through training sessions and other internal workshops to communicate issues and share best practices in order to further standardise the type of feedback provided (see Chapter 5). Once vetting officers are sufficiently trained to the expected level, the number of vetting officials may be further reduced by empowering only one (1) official (rather than 2 or 3 individuals) to vet a tender document. In addition, streamlining the vetting process may also be achieved by rethinking the organisational structure of the DoC in the long term through a more centralised structure that would unify and facilitate the vetting process (see section 1.1.1).
While contracting authorities mentioned potential delays resulting from DoC processes, both the SPD and the OD also mentioned the lack of responsiveness of some contracting authorities to validate comments and finalise the vetting process. In this regard, SPD and OD are now setting deadlines to cancel procedures when no feedback is received from the contracting authorities. However, SPD and OD should also consider setting minimum and maximum timelines for each process under their responsibility and communicating them to contracting authorities to manage expectations accordingly. This will enable DoC to better monitor the timing of their internal processes and to further assess the potential sources of delays. Visibility on vetting timelines will also help contracting authorities for planning (see section 2.1) and monitoring their own internal processes (see Chapter 6).
Moreover, discussions with contracting authorities indicated that vetting of tenders utilising the BPQR evaluation method further prolongs the vetting process. This is due to the additional heavy scrutiny of scoring criteria by the DPPQA of the DoC. In efforts to increase efficiency, DoC should consider removing this additional layer of vetting to the use of the BPQR method. In tandem, the capacity of officials in establishing adequate BPQR criteria should be reinforced.
Lastly, based on discussions with selected contracting authorities, of multiple communication tools are used in the vetting process between DoC and contracting authorities. The vetting process is not taking place via ePPS or any specific dedicated platform. During this stage, multiple communication channels and systems are used by the different participants in the process. Documents are received from contracting authorities via generic email, registered, and then the vetting process occurs via shared documents in Teams and online meetings. For re-engineering public procurement processes in Malta, the previous OECD report highlighted the need to review the process and to track tender status (OECD, 2019[3]). According to discussions with DoC, the shift from email to Teams was as a result of process reengineering efforts to increase efficiency at DoC. However, the use of Teams and emails for vetting purposes might lead to several issues related to safety, efficiency, workflows, and traceability. Indeed, for instance, some contracting authorities highlighted the absence of formal acknowledgment of receipt after sending the required documents at the start of the process. DoC directorates are still tracking tender progress through an excel file which does not enable to send alerts or notifications. Additionally, the shifting of documents between channels each time is inefficient as it disrupts the workflow on each occasion. It may also result in traceability issues, whereby the latest version of documents may not be reflected in the workflow. In addition, despite the numerous functionalities offered by collaborative tools such as Teams (shared documents, co-editing etc.), they are linked to non-national clouds under different regulations and are therefore subject to many data safety risks. Public procurement data could include sensitive information related to government priorities and strategies. In this regard, France for example only authorises the use of national or European clouds in public administrations and has therefore prohibited the use of Teams (linked to a cloud outside the EU) (Acteurs publics, 2021[15]).
In order to enhance the vetting process, communication means with contracting authorities should be digitalised, centralised and secure. The choice of the ePPS or a specific platform should be based on a cost-benefit analysis. The system should enable to notify contracting authorities of i) the receipt of documents ii), the start of the process and, iii) the stage of the process. The system could also send to both contracting authorities and DoC directorates alerts when approaching key milestones and deadlines.
2.2.2. Improving procurement processes for the tendering and contract management phase
In Malta, in line with international good practices, the process for tendering and clarifications is predominantly digitalised. However, similarly to the vetting process, multiple communication channels and systems are used by multiple stakeholders for tenders administered by SDP and OD: ePPS, emails, etc. Discussions with stakeholders highlighted that the use of multiple communication channels and systems by multiple stakeholders and the “intermediary role” of SPD or OD could create inefficiencies in this process.
Indeed, once clarifications are raised from bidders through the ePPS, these are then relayed by DoC (in the case of Schedule 2 and 16 entities) to the respective contracting authority via email. The contracting authority then replies to clarifications and provides any minutes to clarification meetings/site visits via email to SPD / OD. The SPD/OD then upload the contracting authority’s response after reviewing it, and publish it on ePPS. DoC also sends the evaluation committee a deadline to conclude the evaluation process (four weeks). DoC may consider providing contracting authorities with access to the ePPS for the clarification stage (at the publication stage).
During the course of evaluation, the evaluation committee may request any clarification or rectification from the economic operators through DoC. Such clarification or rectification requests are subject to vetting and approval by DoC prior to publishing on ePPS. Tender evaluation committees evaluate the offers and submit the final evaluation report with annexes via email to DoC. The evaluation report, which incorporates the evaluation committee’s recommendations for award, is forwarded to the DCC or GCC (depending on the previously mentioned thresholds), who approves recommendations for award via email since the Covid 19 pandemic. In its previous report, the OECD already recommended to fully integrate the evaluation report within ePPS to simplify the process (OECD, 2019[3]). Furthermore, it was noted that whilst the SPD is composed of eight units representing different contracting authorities, DCCs exist within each ministry, whereby currently Malta has a total of 18 ministries. For a more structured process, consideration should be given to aligning the number of DCCs with those of the SPD. In doing so, the number of members within the DCCs should be strengthened in order to; (i) have the necessary capacity to shoulder the responsibility of various ministries, and (ii) enhance efficiency due to potential delays in approving evaluation reports for each tender. Similarly, any future structural changes to the DOC should also be reflected in the roles and remits of the DCC or GCC.
Moreover, DOC should reconsider their role in the line of communication between the economic operator and contracting authority. Indeed, DoC could allow contracting authorities to be at the frontline of the communication with economic operators at the clarification and evaluation stages. Moreover, and similarly to the vetting stage, the use of multiple communication channels in tender evaluation stage should also be addressed by having one centralised digital channel across the various stakeholders. DoC should also reconsider the need to review and approve each clarification or rectification request put forward by the evaluation committee. In doing so, evaluation committees’ capacity should be reinforced.
Furthermore, in the request for clarification stage, Schedule 2 and 16 contracting authorities experience lack of flexibility in the number of times that clarifications or rectifications may be issued. Indeed clarifications may be issued only once according to the DoC policy, irrespective of the topic or nature of the tender. After the clarification stage, contracting authorities should therefore carefully examine tender documents and ensure that the potential changes are clear and might not lead to additional clarification needs from potential bidders (EU Parliament and Council, 2014[6]) (Clarkslegal, 2017[16]).
Additionally, online evaluation on ePPS carried out by the contracting authorities’ evaluation committee does not allow for a detailed description of the process (due to lack of characters) on the system. Whilst the restriction of characters facilitates more clear and concise explanations, it may give rise to potential misunderstandings by the SPD or OD, which may in turn lead to the incorrect issuance of award letters by DoC through the ePPS. In such occurrences, the DoC may consider improving or enabling enhancements to the ePPS. In addition, DoC should reconsider its intermediary role, and empower contracting authorities to issue letters of award through the ePPS.
Regarding the award of the contract, upon elapse of the standstill period prior to contract award notice, in line with international good practices, it is necessary to check whether objections are filed before the relevant bodies. In Malta objections are filed before the Public Contracts Review Board (PCRB). While ePPS includes a functionality to file challenges directly online, this functionality is not yet made mandatory. Therefore, the SPD, the OD and schedule 3 contracting authorities check with the PCRB whether objections were filed for each tender via email, which may be administratively taxing. It is therefore recommended that the use of the ePPS is made mandatory for challenging decisions. This would not only reduce the use of multiple systems, but also render all information relating to each tender in one location, facilitating better information management, data collection and reporting thereafter.
If no appeals are lodged, the Contract Award Notice is published once the Contractor accepts the award on ePPS. For Schedule 2 and 16 contracting authorities, the contract is prepared by DoC and sent to contracting authorities or the DoC Director General for signatures. In the case of Schedule 3, contracts are prepared and coordinated by the respective contracting authority.
Schedule 2 and 16 contracting authorities mentioned a lack of standardisation in i) contract signing and ii) contract modification by the different Directorates within DoC (SPD and OD). In fact, in tenders administered by SPD, the SPD issues contracts, and signatures are sought from both the Head of the contracting authority (Permanent Secretary) and the successful bidder. However, when the OD is in charge of the process, the OD issue contracts, which are signed by the DG Contracts (on behalf of the contracting authority) and the successful bidder. Once contracts are signed, the contracting authority provides electronic copies of the performance guarantee and the signed contract agreement to the Contractor, and retains the original copies. Similarly, with reference to contract modifications and amendments for tenders administered by the SPD, approval is provided by the respective Head of the contracting authority and permanent secretary. However, for tenders administered by the OD, the approval of modifications is provided by the General Contracts Committee (GCC). At times, such practices result in delays experienced by various contracting authorities, particularly with contracts issued by the OD. In the short term, whe■ OD is i■❖o●❖ed, dead●i■es for co■tract modificatio■ appro❖a● by the GCC sho♦●d be estab●ished, to i■crease efficie■cies a■d red♦ce de●ays. I■ the medi♦m term, it is ■ecessary to sta■dardise co■tract sig■i■g towards sig■i■g by the respecti❖e co■tracti■g a♦thority. This wo♦●d ser❖e to e●imi■ate i■co■siste■cies betwee■ the directorates withi■ DOC a■d i■crease the acco♦■tabi●ity of co■tracti■g a♦thorities.
In addition, whilst it is acknowledged that procedures are primarily digital through the use of the ePPS, as noted from contracting authorities’ feedback, remaining items to be integrated and/or used in digital format include; incorporating e-signatures, the use of the ePPS module to challenge public procurement decisions, the use of the contract management module, and the provision of digital guarantees. For instance, forms such as the tender originator forms and the contracts following the award are signed manually and then saved in electronic format. In addition, all required external approvals, particularly in the contract management stage, with the SPD or the DoC are obtained through email. In order to enhance efficiency and reduce the administrative burden, Malta should consider further digitising all processes throughout the procurement cycle. This includes integrating additional features such as the e-signature (eliminating hard copies of procurement documents) and using ePPs or other relevant platforms for exchanges between DoC and contracting authorities. It will also ensure that all information relating to public procurement procedures is available and accessible online, in one centralised location (see 6.4) on the availability of relevant data to assess the efficiency and quality of procurement processes).
2.2.3. Promoting the use of multiple award criteria
Award criteria are used to evaluate bidders’ offers and to award the contract to the offer that provides the best value for money (OECD, 2021[17]). The European directives offers three options to assess bids: i) the best price-quality ratio (BPQR); ii) life cycle costing (LCC); and iii) the lowest price. The use of BPQR criteria enables contracting authorities to assess bids not only based on the price criterion but also on other aspects such as quality, technical merit, social and environmental characteristics, qualification and experience of supplier staff, after-sales service and technical assistance and delivery conditions. Furthermore, using the BPQR criteria along with the LCC method can also support innovation outcomes and enhance competition (OECD, 2019[18]). In addition, the use of BPQR can serve as a strategic lever to stimulate innovation and competition. Indeed, with minimum technical specifications, and using the lowest price criteria, bidders might not be encouraged to go beyond the minimum requirements and innovate (OECD, 2019[18]).
In Malta, in line with the European directives, the public procurement regulatory framework enables the use of BPQR. Despite the encouraging fact that both contracting authorities and economic operators seem to be aware of the benefits of BPQR (The Malta Chamber, 2021[4]) the use of this evaluation method is limited in practice. At the European level, for above threshold procedures, Malta is among the countries that are using the lowest price criteria the most (European Commission, 2020[19])) (see Figure 2.3). To use BPQR method in their tenders, contracting authorities have to seek DPPQA approval for the use of the BPQR criterion method.
In addition, DoC have developed procurement policy notes to guide contracting authorities opting to use BPQR (Government of Malta, 2016[20]). However, feedback from contracting authorities highlighted the need to provide further guidance with concreate examples to operationalise the use of this evaluation method. The European Commission provides such guidance for practitioners, which include examples of award criteria of the BPQR approach as well as practical tips and delineation of good and bad practices when defining such award criteria (European Commission, 2018[21]).
Furthermore, data provided by DoC shows that in the last 3 years, on average, only 3.2% of Call for Tenders (CfTs) used BPQR (see Table 2.2). The poor and decreasing uptake of BPQR in Malta is due to several reasons including the administrative burden associated with the use of this method which may deter contracting authorities from utilising this method, and the lack of practical guidance on how to use it. Indeed, discussions with contracting authorities highlighted that the use of BPQR can be cumbersome as it is subject to hefty review and approval of DPPQA. DoC explains this control by the need to ensure that the criteria used by contracting authorities are not subjective and will not lead to increased likelihood of challenges of procurement decisions.
Table 2.2. Share of Call for tenders in volume using BPQR in 2018-2020
Year |
Number of CfTs |
Number of BPQR CfTs |
Percentage |
---|---|---|---|
2018 |
5 882 |
198 |
3.37% |
2019 |
5 858 |
186 |
3.18% |
2020 |
6 165 |
188 |
3.05% |
Note: CfTs= Call for tenders
Source: Data provided by DoC in 2022
In addition, while the use of BPQR is authorised and has a positive impact for all procurement categories (see Box 2.4), in Malta, contracting authorities are mainly encouraged to use it for the procurement of services. Furthermore, discussions with stakeholders revealed issues relating to the evaluation of tenders using BPQR criteria. Evaluators are encouraged to allocate different scores from each other to avoid being accused of manipulating results. To enhance the uptake of BPQR, Malta should consider gradually removing the approval of the use of this evaluation method by the DoC for schedule 16 and 2. This will also reinforce the accountability of contracting authorities. In addition to promoting the use of BPQR for different procurement categories, Malta should consider developing the existing guidelines on the use of BPQR with detailed breakdown of criteria and concrete examples from different procurement categories. This will enable to enhance the capacity of contracting authorities in using the appropriate award criteria and to reduce the subjectivity in using those criteria. It will also ensure the transparency and fairness of scores provided by evaluators at the evaluation stage.
Box 2.4. Using BPQR criteria in public procurement tenders
Using BPQR, as opposed to the lowest price criterion, presents a series of advantages. It allows contracting authorities to take into account qualitative considerations. The BPQR criteria are typically used when quality is important for the contracting authority. Some cases where it may be considered appropriate to use BPQR are as follows.
The procurement of supplies: For public supply contracts that involve significant and specialised product installation and/or maintenance and/or user training activities, the contract award is usually made on the basis of BPQR criteria. For this type of contract, in fact, quality is generally particularly important.
The procurement of works: For the procurement of works that involve a lengthy and/or complex project management process, quality of materials used and works delivered are key elements that can be reflected in the award criteria.
The procurement of services: For the procurement of consultancy services and more generally intellectual services, quality is normally very important. Experience has shown that when procuring this type of service, the best results in terms of the best value for money are achieved when BPQR criteria are used.
Source: (OECD, 2021[1])
2.3. Towards the strategic use of public procurement efficiency tools
The OECD Recommendation calls adherents to develop and use tools to improve procurement procedures, reduce duplication and achieve greater value for money, including framework agreements and dynamic purchasing systems (OECD, 2015[12]). A framework agreement means an agreement between a fixed number of one or more contracting authorities and one or more economic operators, with the purpose of establishing the terms governing contracts to be awarded during a given period, in particular with regard to price and, where appropriate, the quantity envisaged. Unlike the Dynamic purchasing system (DPS), no additional economic operators can join throughout the duration of a classical framework agreement. It is to be noted that in 2019, 97% of OECD countries responding to a survey have implemented framework agreements as a means of increasing efficiency and realising savings (OECD, 2019[14]).
A DPS is somehow a framework agreement to which potential suppliers can join any time during its period of validity, thus enabling further competition over time. The DPS can streamline procurement for both suppliers and authorities (LPP NHS, n.d.[22]). The contract award process can also be conducted more rapidly than under other procedures. A DPS offers flexibility in fast-paced, constantly changing markets. The use of needs aggregation coupled with those collaborative instruments enables to achieve further efficiency gains. (OECD, 2021[1])
2.3.1. Further promoting the use of DPS and framework agreements
In Malta, in line with the European directives, the Public Procurement Regulation provides the possibility to implement framework agreements (FAs) and DPSs. The Public Procurement Regulation 108 of S.L. 601.03 state that DPS may be set up for procurement estimated above the threshold of EUR 140 000 excluding VAT as it entails using a restricted procedure. In other countries, there are no thresholds to use DPS and framework agreements, such as Ireland and Lithuania (The Parliament of the Republic of Lithuania, 2022[23]). To enhance the uptake of DPS, Malta should consider removing thresholds to use DPS.
Data provided by the DoC shows that the uptake of framework agreements and DPS is relatively low. A total of 106 FAs were published between 2018 and 2020. The Ministry for Transport Infrastructure and Capital Projects published the greatest number (37) of FAs between 2018 – 2020, representing 34.9% of the total number of FAs in the country, followed by DoC (28.3% of the total number of FAs) that is mainly publishing FAs on behalf of contracting authorities, and the Ministry for Justice, Culture and Local Government (12.3% of the total number of FAs). Similarly, the share of contracts awarded through framework agreement against the total procurement volume was 0.72% in 2018, 2.28% in 2019 and 0.61% in 2020. Consultations with DoC indicated that in 2019, policy shifted to increase framework agreement uptake, hence such figures reflect the increased establishment of framework agreements in 2019 that were still in place, in 2020.
The share of contracts awarded through DPS was 0.03% in 2020. Data covering 2018 – 2020 in Malta identifies that a total of 11 DPSs were published, with DPSs administered by DoC on behalf of contracting authorities representing the majority (58.3%) of shares against all contracting authorities, followed by MITA (25%) and Infrastructure Malta (16.7%), the latter two being Schedule 3 entities. With reference to SPD, a total of 17 DPSs were administered by SPD on behalf of contracting authorities in 2021, with a total volume of EUR 674 thousand. Whilst none related to works, 52.9% related to supplies and 47.1% related to services. The majority of the volume (59.7%) related to services. Additionally, in 2022, there are five (5) operational DPSs issued by DoC on behalf of contracting authorities, relating to procurement of: examination gloves, travel agents, professional engineering services, professional architect services, and supply of books. In addition, seven (7) operational DPSs were issued by three (3) Schedule 3 entities in the technology, construction and infrastructure industries.
Some contracting authorities mentioned that they are not using DPSs or framework agreements because i) it is not adapted to their procurement categories, ii) they do not see the added value and iii) the process is burdensome. Regarding the administrative burden, for schedule 2 and 16 entities, when the procedure is administrated by the OD or the SPD, documents need to be resubmitted to these directorates for each mini-competition. Therefore, the administrative burden strongly limits the benefits of using these tools. In this regard, it is necessary to rethink the process for competitions derived from signed framework agreements and DPSs to enhance the efficiency of the process, by reducing or eliminating the involvement of DoC directorates at the mini competition stage.
On the other hand, a number of contracting authorities are seeking to increasingly make use of efficient procurement tools. Malta exhibits high potential towards the use of public procurement efficiency tools, particularly in collaborating with other contracting authorities for increased centralisation. In addition to the IT field, contracting authorities as well as the DoC mentioned the high potential of some procurement categories to implement framework agreements and dynamic purchasing systems, which include: Cleaning services, Security services, Travel agencies, Communication and Vehicles. At the European Union level, office supplies and IT equipment are the top categories for DPS, followed by education and training services (see Table 2.3). In this context, the DoC should raise awareness on the benefits of such tools and the most adequate procurement categories to use them.
Table 2.3. The Top CPV codes when using DPS in the European Union
Main division |
Occurrences |
---|---|
Office and computing machinery, equipment and supplies except furniture and software packages |
338 |
IT services: consulting, software development, Internet and support |
211 |
Education and training services |
184 |
Transport equipment and auxiliary products to transportation |
162 |
Construction work |
137 |
Business services: law, marketing, consulting, recruitment, printing and security |
126 |
Furniture (incl. office furniture), furnishings, domestic appliances (excl. lighting) and cleaning products |
125 |
Architectural, construction, engineering and inspection services |
125 |
Medical equipment, pharmaceuticals and personal care products |
98 |
Radio, television, communication, telecommunication and related equipment |
71 |
Source: TED data (all contract notices from the two Directives up until April 2020)
In Malta, some guidance has been developed on the use of efficiency tools such as FAs and DPS. The Procurement Policy and Quality Assurance Directorate (DPPQA) within DoC developed a manual of procedures delineating the framework agreement and DPS procedures (DOC, 2016[24]), and guidance notes on DPS are made accessible to contracting authorities through the ePPS. Moreover, the DoC has also published a document on rules governing DPS (DoC, 2018[25]). Whilst this document is i) accessible online and ii) provides a detailed description of the various aspects of the procedure, it lacks the necessary visual guidance and concrete examples needed for effective uptake of the procedure. In fact, a number of contracting authorities mentioned the need to improve the guidance as it was not always operational. DoC should increase the uptake and effective adoption of the procedure. For instance, the Scottish Government published detailed guidance on the use of DPS, including on i) points to consider before using a DPS, ii) how to create a DPS, iii) awarding contracts under a DPS, iv) implementation of a DPS (see Box 2.5). The Scottish Government guidelines also include a video guide, and a checklist (Scottish Government, n.d.[26]).
Box 2.5. Example of the Scottish Government guidelines on using DPS
Definition of a Dynamic Purchasing System
The Scottish Government guidance starts with a definition of DPS.
Points to consider
The Scottish Government guidelines list key characteristics of what to consider when setting up a DPS. This includes for example :
i. Large volume or suppliers
ii. Large volume of transactions
iii. Total anticipated spend
A decision making checklist is also provided with many suggestions, including:
i. Speak to others who have set up or operate a DPS
ii. Engage with key stakeholders
iii. Don’t underestimate the potential challenge around cultural change
iv. Build a robust business case
Creation of a DPS
The guidance provides key information and useful links on the procedure to use (Restricted procedure), suppliers, period of validity, communication issues, regulations concerning potential changes, selection criteria and other best practices related to DPS creation.
Validity of the DPS
The guidance highlights key elements related to the validity of the DPS including the applicable timescale to evaluate requests for participation in a DPS and the available extensions for evaluation.
Awarding contracts under a DPS
The guidance provides information about key requirements regarding the award criteria, the “standstill” period, and the publication of the contract award notices.
Implementation suggestions
The guidance provides key steps for DPS implementation including:
i. Piloting – undertaking a trial run prior to implementation of a DPS incorporating various categories
ii. Ensure sufficient resources – establishing a DPS requires resources particularly at initial stages, if it involves higher numbers of suppliers. It also involves administrative efforts throughout the duration of the DPS.
iii. Involve suppliers – DPS success factors requires available suppliers to actually be interested to bid. Ways to support suppliers include communication of eligible factors apriori, demonstrating the way the DPS works, and eliciting feedback at various stages of the cycle.
Source: (Scottish Government, n.d.[26])
2.3.2. Considering the centralisation of procurement for further efficiency gains
Aggregating needs is a key lever to enhance the efficiency of public procurement systems. This holds particularly true in times of fiscal austerity when all levels of governments are focusing efforts on rationalising public spending (OECD, 2019[18]). It has several benefits including the achievement of economies of scale and administrative savings by reducing duplications and the faster uptake of the use of public procurement to achieve policy objectives. The use of needs aggregation coupled with framework agreements or DPSs enables to achieve further efficiency gains (OECD, 2021[1]).
There are different ways of aggregating needs: through centralisation of procurements from different contracting authorities usually with the lead of a single contracting authority or by doing joint procurements between two or more contracting authorities (OECD, 2021[1]). In Malta, both options exist but at a limited scale.
Indeed, regarding centralisation, the DoC is leading two DPSs on supply of Winter and Summer uniforms respectively, through which all contracting authorities may procure. The larger initiative related to the aggregation of needs is the one of the Malta Information Technology Agency (MITA) who is leading several DPSs in the Information Technology (IT) field, which include the provision of temporary IT resources, the procurement of printers, scanners and related supplies or services, the procurement of software development services, workstations for the government of Malta, and the procurement of software development services for modernisation initiatives.
In this context, a number of documents and templates are readily available and accessible through the Agency’s website for the DPS procurement of printers, scanners and related supplies/services, as well as procurement of software development services (MITA, 2022[27]). Specifically, the following documents were available online for some DPSs: i) Guidance note for contracting authorities on the use of the DPS, ii) DPS Overview, iii) Specific contract template, iv) Baseline hardware specifications, v) Mandatory general requirements, and vi) Purchase order template (MITA, 2022[27]). Contracting authorities interviewed provided a positive feedback on DPSs managed by MITA as it decreases their administrative burden and does not reduce competition for a certain period of time. However, DoC has not promoted or raised awareness on the benefits of centralisation schemes. Given the benefits of needs aggregation, DoC should consider raising awareness on the advantages of using such centralised procurement schemes, with tangible examples. It should also consider promoting MITA’s contracts and exploring the possibility of centralising some relevant procurement categories gradually, considering its limited capacity, in terms of human resources. This should be done following a methodological approach including the assessment of most frequent purchases, and the assessment of purchases where contracting authorities may face some difficulties or may require some support. Finally, relevant stakeholders could be engaged in this process. For instance, in Ireland, the Office of Government Procurement (OGP) supported since 2014 the education sector during a budget restriction period by centralising some specific procurement categories. The benefits of such centralisation could not be achieved without the involvement of relevant actors from the education sector. In this context, a Schools Procurement Unit (comprising of four individuals) was established between the OGP and its clients, emphasising the advantages of centralised procurement and the benefits of cost savings (Irish Management Institute, 2017[28]).
References
[15] Acteurs publics (2021), L’État freine le “tout-Microsoft” dans ses administrations, https://acteurspublics.fr/articles/letat-freine-le-tout-microsoft-dans-ses-administrations.
[16] Clarkslegal (2017), Perfecting Imperfect Bids Part 1 - When is a clarification not a clarification?, https://www.clarkslegal.com/Blog/Post/Perfecting_Imperfect_Bids_Part_1_When_is_a_clarification_not_a_clarification.
[24] DOC (2016), Manual of Procedures Public Procurement Regulations (SL 174.04), https://contracts.gov.mt/en/Resources/Documents/Manual%20of%20Procedures%20-%20PPRs%202016%20-%20v1%201.pdf.
[25] DoC (2018), General Rules Governing Dynamic Purchasing Systems, https://procurement.mita.gov.mt/media/110254/general-rules-governing-dynamic-purchasing-system-v10.pdf.
[6] EU Parliament and Council (2014), “Directive 2014/24/EU of the European Parliament and of the Council”, Official Journal of the European Union, http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32014L0024&from=EN (accessed on 31 January 2018).
[7] European Commission (2022), Search result - TED Tenders Electronic Daily, https://ted.europa.eu/TED/search/searchResult.do (accessed on 9 March 2022).
[19] European Commission (2020), Single market scoreboard, https://single-market-scoreboard.ec.europa.eu/policy_areas/public-procurement_en.
[21] European Commission (2018), Public Procurement Guidance for Practitioners on avoidinng the most common erros in projects funded by the European Structural and Investment Funds, https://ec.europa.eu/regional_policy/sources/docgener/guides/public_procurement/2018/guidance_public_procurement_2018_en.pdf.
[2] European Commission (n.d.), Publish annual procurement plans, https://razvoj.gov.hr/ (accessed on 9 March 2022).
[20] Government of Malta (2016), Procurement Policy Notes, https://contracts.gov.mt/en/ProcurementPolicyNotes/Pages/ProcurementPolicyNotes.aspx.
[28] Irish Management Institute (2017), Case Study on the Office of Government Procurement (OGP), https://www.reformplan.per.gov.ie/2014/downloads/files/Case%20Study%20OGP%20FINAL%2004.07.17.pdf.
[22] LPP NHS (n.d.), Dynamic Purchasing Systems - all you need to know, https://www.lpp.nhs.uk/for-suppliers/dynamic-purchasing-systems-all-you-need-to-know/.
[11] MAPS initiative (2018), MAPS Methodology, https://www.mapsinitiative.org/methodology/MAPS-Methodology-ENG.pdf (accessed on 21 September 2021).
[27] MITA (2022), Procurement of Printers, Scanners and related Supplies/Services - DPS 020/20, https://procurement.mita.gov.mt/resources/dps/dpsprinters-scanners/.
[8] NPPPU (n.d.), Supply Market Analysis Guidance Note General 2, http://etenders.gov.ie/Media/Default/SiteContent/LegislationGuides/23.%20Supply%20Market%20Analysis.pdf.
[13] OCDE (2019), Revue du système de passation des marchés publics en Algérie: Vers un système efficient, ouvert et inclusif, Examens de l’OCDE sur la gouvernance publique, Éditions OCDE, Paris, https://doi.org/10.1787/49802cd0-fr.
[17] OECD (2021), Public Procurement in the State of Mexico: Enhancing Efficiency and Competition, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/cc1da607-en.
[1] OECD (2021), Unlocking the Strategic Use of Public Procurement in Bratislava, Slovak Republic, OECD Publishing, Paris, https://doi.org/10.1787/d616e4d9-en.
[10] OECD (2020), Training on Market analysis.
[14] OECD (2019), Enhancing the Use of Competitive Tendering in Costa Rica’s Public Procurement System, https://www.oecd.org/costarica/costa-rica-public-procurement-system.pdf.
[3] OECD (2019), Public procurement in Malta Re-engineering the Department of contracts, https://www.oecd.org/governance/public-procurement/publications/public-procurement-in-malta.pdf.
[18] OECD (2019), Reforming Public Procurement: Progress in Implementing the 2015 OECD Recommendation, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/1de41738-en.
[12] OECD (2015), OECD Recommendation of the Council on Public Procurement, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0411.
[9] Office of Government Procurement (2020), Public procurement guidelines for goods and services, https://www.gov.ie/en/publication/c23f5-public-procurement-guidelines-for-goods-and-services/.
[26] Scottish Government (n.d.), Dynamic Purchasing System (DPS) Use for Procurement Exercises above Threshold, https://www.procurementjourney.scot/additional-resources/dynamic-purchasing-system-dps.
[5] SIGMA (2015), Public Procurement Training Manual, https://www.sigmaweb.org/publications/Module-C-%202015-clean.pdf.
[4] The Malta Chamber (2021), Report on Public Procurement Reform, https://www.maltachamber.org.mt/loadfile/c51f072e-897a-485f-9a7f-0248fbb887d5.
[23] The Parliament of the Republic of Lithuania (2022), Law on Public Procurement of the Republic of Lithuania, https://www.e-tar.lt/portal/lt/legalAct/TAR.C54AFFAA7622/asr.
Note
← 1. Based on the 100 responses received in a survey conducted by The Chamber of Malta.