This chapter discusses existing requirements, tools and guidance instruments used in policy development processes to assist policy practitioners in their day-to-day work. It provides suggestions on how to close gaps in support and guidance and increase the use and implementation of policy development tools, instruments and methodologies.
Strengthening Policy Development in the Public Sector in Ireland
OECD Public Governance Reviews
6. Policy Methods and Tools in Ireland
Abstract
A policy development system needs to be supported by generic processes and practical policy tools – including for quality assurance – to assist policy practitioners in their day-to-day work and to avoid continually reinventing the wheel. These tools typically support one aspect of the policy development process, such as impact analysis, public engagement or data modelling. The quality, availability and socialisation of tools reflect the ability of the civil service to enable and support policy development and to deliver on the three pillars of evidence, implementation and feasibility, and legitimacy. This chapter critically discusses the existing tools and guidance instruments and provides suggestions on how to increase their use and implementation. The tools discussed in this chapter will contribute to the forthcoming policy development toolkit, as part of a broader good practices hub, as an output of this project.
Existing requirements, guidance and tools for policy development
While Ireland does not yet have an integrated standard, benchmark or guidance instrument for policy development, a range of separate process requirements, formal guidance and tools for policy development exist in the Irish system and are used by policymakers across the government (see Figure 6.1).
In addition to the Cabinet Handbook, a detailed guide to the legislative process from the Revenue Legislation Services exist. For project management, the new Public Service Project Management Handbook builds on the Civil Service Project Management Handbook to support managers by giving them an easy reference guide to the core principles and methodologies of project management and to promote a standardised and consistent approach to the governance of project management across the public service.
With regard to the requirements related to public spending and regulatory impact, a number of formal documents and processes are in place. The most frequently mentioned document in the OECD survey is the Irish Public Spending Code. It includes a set of rules and procedures to ensure that the best possible value for money is obtained whenever public money is being spent or invested across the Irish public service. It is frequently used by 30% of the surveyed policymakers. The Code provides guidance related to evaluating, planning and managing public investment and current expenditure and to the use of public-private partnerships. Its Technical Guidance Series further offers guidance and tools to assist departments and agencies inter alia with the efficient management of public investment through reviews of past practice; to support programme evaluation methodologies (such as value-for-money reviews and focused policy assessments) used across the civil service; to share appraisal methods and techniques; and to foster cost-benefit analysis. A Financial Appraisal document further provides a guide to carrying out a financial analysis as part of the Public Spending Code. It provides supplementary guidance for public sector bodies conducting the financial appraisal element of the Preliminary and Detailed Business Case stages of a public investment proposal.
The Public Spending Code also includes detailed guidelines in relation to regulatory impact assessments (RIAs). The requirement to undertake RIA for new regulations was first introduced in 2005 through formal guidelines that were integrated into the Cabinet Handbook and procedures for the development of regulations.
In June 2009, the Department of the Taoiseach published a set of revised RIA Guidelines that made minor changes to the process, including the removal of separate advice on screening and full RIA processes and providing a more detailed consideration of methodological issues, particularly where qualitative data are involved. However, there have not been any updates to the RIA Guidelines since then and none appear to be planned in the near future. Since 2011, the RIA Guidelines have formed part of the Public Spending Code overseen by the Department of Public Expenditure, NDP Delivery and Reform (DPENDR), as part of the Technical Guidance Series – alongside guidance on Value for Money and Policy Reviews, Focused Policy Assessments, a Spending Review, Performance Reporting, and Tax Expenditure Evaluation Guidelines (which may provide a useful model for developing guidelines on similar issues). Importantly, the RIA Guidelines cover the legislative activities of the executive but do not formally cover regulatory agencies or local authorities, although these agencies and authorities are encouraged to conduct RIA where appropriate.
The Capital Works Management Framework (Government of Ireland, 2023[1]) is a structure that has been developed to deliver the government’s objectives in relation to public sector construction procurement reform. The Framework consists of a suite of best practice guidance, standard contracts and generic template documents.
DEPR also provides guidance for public bodies that are considering carrying out a public consultation. The “Public Consultation Principles & Guidance” offer a detailed overview of different forms and steps of consultation processes. As mentioned earlier, this document dates from 2016, and a number of innovations and initiatives for public consultation have been adopted since (such as the Citizens’ Assembly). While no specific data exist for the uptake and use of these guidelines, 45% of the surveyed policy practitioners reported the frequent use of different stakeholder participation methodologies.
Aside from general cross-departmental guidance documents such as the Cabinet Handbook, the Public Spending Code and the Capital Works Management Framework, some departments have also developed sector-specific guidance tools, for example to apply the Public Spending Code to business cases from cultural institutions and sports organisations that are applying for funding. A total of 39% of the surveyed policymakers said that they occasionally or frequently deploy department-specific manuals for policy development.
In terms of uptake of tools, 50% of the surveyed policy practitioners also reported making occasional or frequent use of instruments and tools provided by external stakeholders such as international organisations, academia or civil society (Figure 6.1).
Figure 6.1. Use of tools and instruments for policy development
Note: n=168. Respondents to the OECD survey were asked, “In your current role, how frequently are you currently using the following tools and guidance instruments for policy development?”. Response options included “never”, “occasionally”, “frequently”, “very frequently” and “not applicable for my function”.
Source: (OECD, 2022[2])
Interviews and research also highlighted a number of tools that were perceived as useful in the policy-making process. For example, the National Risk Register led by the Department of the Taoiseach is perceived as a helpful tool for enhancing policy development, as are the National Risk Assessments led by the Department of Defence. Both exercises provide a basis for establishing priorities for mitigating key risks identified at the national level and inform government decisions regarding resource allocation.
A number of other relevant policy development tools and instruments (beyond the Cabinet Handbook) were mentioned during the interviews. For example, the detailed drafting guidelines on the preparation of Statutory Instruments from the Office of the Attorney General and the Department of Public expenditure with quite detailed guidance in relation to the preparation of regular RIAs. Other policy development tools and instruments include spending review papers, Irish Government Economic and Evaluation Service (IGEES) analytical papers, IGEES-led policy discussions, ex post policy evaluations, the Prevention & Early Intervention Series, Focused Policy Assessments, stakeholder analysis, as well as department-specific manuals.
All these instruments and guidance tools are relevant and useful to some extent and may help bolster a more overarching policy infrastructure and platform for policy development.
Improving the use and implementation of policy development tools, instruments and methodologies
Despite the existence of cross-departmental policy development tools and guidance, interviewees and survey respondents described their application and use as uneven across departments and sometimes inadequate. In some cases, formal policy analysis is overridden by political or administrative imperatives, potentially compromising policy development processes. Informal or ad hoc practices are often used across the government, which while helping to deal with issues at short notice, may undermine the longer-term use and mainstreaming of tools. This may be exacerbated by the loss of institutional memory as a result of the near-term ”retirement cliff”.
The OECD survey findings (OECD, 2022[2]) show uneven levels of awareness of what guidance and tools exist and where to find them. Asked about standards, benchmarks and guidance instruments for policy development in different departments and organisations, many responses to the survey highlight that staff are unaware of guidance documents or find them difficult to locate, underscoring the need for an overview (or one-stop shop) for guidance materials and for increasing awareness of existing tools. Knowledge of policy development tools, instruments and methodologies is mainly transmitted within departments; staff members often rely on institutional knowledge being shared informally with many standards implicitly learned from colleagues. Seventy-three percent of the surveyed policymakers use informal contacts within their departments to consult or learn more about policy development tools, instruments and methodologies.
Despite the existence of these tools, the interviews highlighted the lack of an overarching approach to policy development in the tools available to civil servants. While there are sources of guidance, including requirements included in the Cabinet Handbook, the Public Spending Code and other accountability tools and while some departments have their own bespoke manual for policy development or have piloted a policy oversight committee (such as the Department of Health), there is currently no well-socialised whole-of-government model or framework of policy development. The Civil Service Management Board has recently taken steps to fill this gap. The report “Strengthening Policy Making in the Civil Service” is structured around the pillars of evidence, implementation and feasibility, and legitimacy and offers guidance on enablers and best practices for policy development. However, although the framework has been endorsed by the Civil Service Management Board (CSMB), it has not been implemented or shared widely across the civil service.
A number of responses stress the added value that general guidance documents could bring to the work in the different departments. In that regard, a policy framework, handbook or platform on policy development that is accessible to all civil servants and includes both standard guidance (cabinet requirements, public spending code requirements, regulatory impact analysis, etc.) as well as a number of innovative methods (behavioural insights, design thinking, system thinking, data analytics, participatory processes, co-design methods, strategic foresight tools and methods) can help raise awareness and ensure quality throughout the policy development process. The guidance document(s) could propose suggestions on navigating the political-administrative interface, including how to present advice to decision makers, and would ideally be linked to the overall public policy cycle mode. It could also offer guidance on when to use the various methods (at which stage of the policy cycle) and how.
Respondents to the OECD survey (OECD, 2022[2]) highlighted the existence of gaps in support and guidance for areas such as behavioural insights, user-centric insights and design methods, planning tools for policy development, foresight tools, and stakeholder consultation. A policy framework could cover these areas and provide guidance for policymakers.
For areas where guidance documents may exist, e.g., in the case of stakeholder participation, respondents mentioned that more practice-oriented handbooks that also detail how to operationalise high-level guidance could be helpful. For instance, one respondent stressed that the Public Spending Code is useful for outlining the process but found that many elements of it do not give sufficient detail on how one should implement it when faced with a spreadsheet. Overall, respondents underlined the importance of operational documents, e.g., in the form of checklists, rather than lengthy documents that require time and in-depth knowledge to navigate.
Against this background, many respondents suggested more targeted training and specific capacity-building and upskilling on policy development in general and certain policy development tools, instruments and methodologies in particular. The importance of sharing good practice examples to illustrate the use and potential of tools and instruments was proposed by policymakers. There is thus a documented interest for the leadership to share good practices and take them to scale. In addition, some policymakers also wish to receive, as learning opportunities, examples and honest appraisals of situations where something went wrong.
One frequently mentioned element for improving the use of policy development tools, instruments and methodologies was the need for easy access to information and documents. A single website with easily accessible information, guidance and activity tools could be useful to spread awareness and enhance consistency across departments. A further collation of standards and best practices of policy development and implementation from across the civil service was also suggested by a number of respondents.
A common form of sharing guidance on policy development in different departments takes place through coaching and mentoring from colleagues and line managers, in addition to formal guidance tools and the Cabinet Handbook. Providing peer exchange and learning opportunities both within departments and between different departments through mentoring and informal networks of policymakers can also help foster knowledge about policy development; cross-pollinate tools, instruments and methodologies; and share best practices on what has worked well. To create greater opportunities for on-the-job learning, adequate overlap with previous post holders and policy development manuals from former incumbents could also be helpful.
Ex ante regulatory impact assessment
Regulatory management tools, including ex ante RIAs, stakeholder consultations and ex post evaluations of regulations, play an important role in supporting evidence-informed policy development. The 2012 OECD Recommendation on Regulatory and Policy Governance (OECD, 2012[3]) states that an RIA should be integrated into the early stages of the policy process for the formulation of new regulatory proposals.
The fundamental infrastructure of a functioning RIA framework remains in place within Ireland’s policy-making process. The requirement to undertake an RIA for new regulations was introduced in 2005, through formal guidelines which were integrated into the Cabinet Handbook and procedures for the development of regulations. From that point on, the guidelines stipulated that all government departments and offices were required to conduct an RIA when undertaking the following legislative processes:
proposals for primary legislation involving changes to the regulatory framework
significant statutory instruments
proposals for European Union directives and significant regulations when they are published by the European Commission
policy review groups bringing forward proposals for legislation.
In June 2009, the Department of the Taoiseach published a set of revised RIA Guidelines that made minor changes to the process. However, there have not been any updates to the RIA Guidelines since and none appear to be planned in the near future.
According to the Guidelines, the RIA process should be started as early as possible in the regulatory proposal development process and be used as the basis for consultations, where possible. Specifically, an RIA must be attached to the draft memorandum and outline of the bill on its way to the cabinet for approval prior to the stage of drafting the bill itself. It stresses that the RIA is a “living document” subject to continuous change and that there can be numerous drafts before the final version of the RIA is complete. The following eight steps are conducted in the RIA process – as per OECD best practice:
1. summary of the RIA
2. statement of the policy problem and objective
3. identification and description of options
4. analysis of costs, benefits and other impacts for each option
5. consultation
6. enforcement and compliance
7. reviews
8. publication.
The Guidelines also draw attention to the relevant analytical techniques – multi-criteria analysis and cost-benefit analysis – as well as signposting readers to more detailed technical guidance in the Public Spending Code. Importantly, the level of analysis undertaken should be proportionate to the significance of the proposal, i.e., a formal cost-benefit analysis should be conducted in respect of the most significant proposals, whereas the vast majority of RIAs are expected to comprise a qualitative multi-criteria analysis approach. Exceptions to the RIA obligation are also stipulated, e.g., it is not compulsory to apply an RIA to the Finance Bill or to emergency, security and certain criminal legislation.
The issue of proportionality is prominent in the guidelines, which point out that, for each stage of the RIA, the exact analytical approach and level of detail required have to be decided on a case-by-case basis, having regard to the significance of the proposal – the greater its importance, the more analysis will be required. Information is provided on how to analyse the impacts of draft options and how to estimate the costs and benefits of each. It is stated that where possible impacts should be monetised, and where this is not possible, they should be quantified (expressed numerically, e.g., number/proportion of lives saved, reduction in traffic volumes).
In 2016, DPENDR published a “Consultation Principles & Guidance” document. Furthermore, Ireland is developing, and currently trialling as a prototype, a single central government website on which some of the ongoing consultations are published. However, it is not apparent that Ireland’s consultation practices operate on a systematic basis across government departments, particularly for statutory instruments – the evidence from the OECD’s survey and interviews was mixed on this point. However, there is a problem with the transparency of the RIA system, as many regulatory proposals are being published online without an accompanying RIA document being published (although there are occasions where RIAs are not required, in light of the exceptions referred to earlier).
The RIA Guidelines also encourage departments to undertake periodic reviews (or ex post reviews) of regulatory measures to evaluate the extent to which they are achieving the objectives and intended benefits. Performance indicators should be identified to show the extent to which the regulations are meeting their objectives. The Guidelines point to possible review mechanisms, including “reporting on performance within Annual Reports, consulting with stakeholders, establishing Review Groups and regular appearances of the relevant Minister or Regulator before Oireachtas Committees”. Standing orders from parliament state that the minister responsible for implementing a law must provide an assessment of its functioning within a year. However, it is not clear to what extent ex post reviews of legislation are taking place in Ireland and how effective the institutional framework for RIAs and the Guidelines have been at incentivising policy teams to actually use RIAs as a core policy tool. The OECD survey results suggest that ex post reviews are one of the least used sources of evidence in the problem identification and policy development process. This is unsurprising as, according to the OECD iREG data, ex post evaluation systems remain rudimentary in most member countries, and it is still not mandatory to conduct an ex post review in one-third of OECD countries.
Furthermore, as mentioned earlier, many regulatory proposals are being published online without an accompanying RIA document being published alongside – this makes it unclear whether the RIA was undertaken at all or whether it was undertaken early in the policy process and also misses the opportunity for stakeholder input into the RIA development.
Various challenges around the access to data by policy officials were reported, e.g., high cost of data; outdated and low-quality data, and long procedures to obtain data (see earlier discussion on evidence and data). Furthermore, it is difficult to determine whether there has been any significant improvement in the quality of RIAs since 2005 – the OECD examined a number of RIAs published online and found that they generally tended to be descriptive in nature and lacking in quantification of regulatory impacts. According to the OECD survey, there is a strong desire among policy officials for a greater level of training in RIA and the associated methodological tools (e.g., cost-benefit analysis). While the civil service has contracted out RIA training to certain outside consultants, it is not clear how systematically policy officials undergo RIA training and how many officials are able to actually carry out RIA effectively. Furthermore, it is not clear if officials can easily access advice and help for producing RIAs – a function previously provided by the RIA Network.
Areas of opportunity to strengthen policy tools and frameworks
Based on the OECD survey on policy development tools and instruments and on the fact-finding interviews, this chapter highlighted a range of useful process requirements and formal guidance tools for policy development in the Irish system that are used by policymakers across the government. They include the 2006 Cabinet Handbook, the Public Spending Code with its Technical Guidance Series, the Capital Works Management Framework, DPENDR’s “Consultation Principles & Guidance”, along with sector-specific guidance tools developed by individual departments and agencies. The chapter articulated ways to further mainstream the use of these tools and methodologies across departments and leverage the RIA Guidelines.
In this context, a number of areas of opportunity can be identified:
A consistent, repeatable and scalable approach for policy design and delivery of advice to decision makers would have multiple benefits, including to help ease cross-departmental work (working to the same process model), to clarify quality standards (articulating what good policy advice looks like) and to ensure that policy staff have a common playbook that can be applied to any policy work (thereby mitigating some of the downsides of staff turnover).
There is a need for an overarching toolkit (or one-stop shop) as part of a broader good practices hub for guidance materials and to ensure an increasing awareness of existing tools.
Collect, consolidate and promulgate the best tools across research, evidence, impact assessments, data, public engagement, government communication and policy design methods to feature on the policy toolkit as part of a broader good practices hub so these are widely available for policy practitioners. The CSMB may envisage leading this effort.
The Cabinet Handbook could be updated to reflect recent tools (e.g. eCabinet) and practices.
The RIA Guidelines could be reviewed by DPENDPDR or the regulatory oversight function to determine whether they remain up to date with current developments in regulatory policy, e.g., the ways to integrate assessments of innovation and the distribution of the incidence of costs and benefits between different social groups.
References
[1] Government of Ireland (2023), Capital Works Management Framework, https://constructionprocurement.gov.ie/capital-works-management-framework/ (accessed on 25 April 2023).
[2] OECD (2022), Survey for Policy Development Assessment Report of Ireland (unpublished).
[3] OECD (2012), “Recommendation on Regulatory and Policy Governance”, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0390.