Ex ante appraisal of new spending programmes is used to strengthen programme design including key performance indicators, and to facilitate processes of monitoring and ex post evaluation.
Ex post evaluations of major spending programmes are carried out on a rolling basis and the findings are systematically fed back into the budget preparation process.
Spending reviews are used in conjunction with performance budgeting to review the justification for spending and to identify budgetary savings that can be redirected to support priority goals.
OECD Good Practices for Performance Budgeting
Good Practice 6: Performance budgeting complements other tools designed to improve a performance orientation, including programme evaluation and spending reviews.
Ex ante programme evaluation
The quality of performance budgeting can be improved through the systematic use of ex ante appraisal of new spending programmes. This should include definition of programme objectives and alignment with strategic goals, an explanation of the programme logic, and identification of costs and indicators of programme impact, including baseline and target values. Such analysis serves to improve the quality of programme design, weed out less effective programmes and provide a basis for subsequent performance monitoring and impact evaluation. Chile provides a good practice example of a systematic approach to ex ante programme appraisal (see Box 15).
Box 15. Chile – ex ante evaluation system
The process of ex ante evaluation of programmes began in 2008. This process provides relevant information on the design of new and reformulated programmes, and provides inputs to the budget formulation process.
Its specific objectives are to:
Explain for each programme:
the public issue to be addressed,
the populations to be considered,
the strategy to be followed,
the goods and/or public services to be provided and the expected results.
Facilitate the subsequent monitoring and evaluation of the results of the funded programmes.
Contribute to improve the coherence of the Programme Offer of Public Services.
Contribute to transparency in the allocation and management of public resources.
Source: Government of Chile (2016)
Ex post evaluation
The effectiveness of public spending, and the extent to which taxpayers receive value for money, are matters of continuing concern to governments and the public. Programme evaluation should ideally take place at an advanced stage of implementation or completion. The results of ex post evaluations should feed back into the strategic budget decision-making process and, at the same time, boost transparency and accountability for the management of stewardship of public funds.
Despite quite widespread use of both ex ante and ex post evaluations, evidence from OECD surveys suggests that evaluations of all types have limited impact on budget decisions, with the greatest impact being at the programme level, with less influence on decisions by the CBA, suggesting less impact on strategic resource allocation across government (see Figure 8).
Figure 8. Assessed Impact of Evaluation on budgetary decision making
Notes: Data for Australia, Denmark, Greece, Italy, Turkey, Portugal and Turkey are not available for the impact of evaluation processes on budget decision making.
Data for Israel and the United States are not available; Information on data for Israel: http://dx.doi.org/10.1787/888932315602.
Source: OECD (2018), OECD Performance Budgeting Survey, Question 34, OECD, Paris.
The fact that the process of ex post evaluation is normally on a multi-year track, delinked from the annual budget process, makes it challenging to ensure that the results of evaluations influence future spending decisions. There is also the issue that, by the time the evaluation is completed, government’s attention and energy are already focused on future policies and spending initiatives. The issue is how to ensure that evaluations findings are relevant and inform budget decision making.
In OECD countries evaluations are mainly carried out by the line ministries (see Figure 9), although a wide range of other institutions are active in this space, including the CBA, SAIs and external bodies such as universities and consulting firms. In practice, while a majority of OECD countries have some form of central guidance for performance evaluation less than half have a legal requirement and a significant number of countries have neither.
Figure 9. Governance of ex ante and ex post evaluation
Note: Data for Israel and the United States are not available.
Source: OECD (2018), OECD Performance Budgeting Survey.
Due to the complex relationship between budget resources and results the job of evaluation should involve both technical and budget experts. The reasons for under or over performance may be due to policy choices, programme design, programme management, external factors beyond management control or inappropriate selection of indicators that do not accurately reflect performance. All these factors need to be examined in order to identify the appropriate actions needed.
Good practice is that budget regulations require that all spending programmes should be evaluated on a rolling basis. High value, high risk and politically important programmes should be prioritised. The CBA should provide central guidance on evaluation policy and standards. Good practice also dictates that there should be a requirement that evaluation findings are factored into the budget decision-making process. In Chile the budget law requires that the results of evaluations are considered as part of the budget process. In Canada, ministries are encouraged to present evidence from evaluations as part of their budget submissions to the Treasury Board (see Box 16). The submissions include a section for ‘evidence’ where the findings of evaluations, studies, audit reports etc. can be presented.
Box 16. Canada - Linking performance measurement and evaluation
Canada has had policies governing programme evaluation in government since 1977. In 2016, Canada released a new Policy on Results intended to improve how departments and agencies (i.e. ministries) plan for, measure, evaluate and report on results.
The new policy created a distinction between departmental-level results – as summarised in a Departmental Results Framework – and the results of programmes, identified in a separate Programme Inventory. The new policy relaxed the requirement in earlier legislation that all direct programmes should be evaluated every 5 years. Rather, continuing the principle that all spending programmes should be evaluated periodically, government departments build an evaluation schedule based on risk and other considerations. In addition the Treasury Board also has the right to initiate evaluations independently of line ministries.
Departments are required to make public not only their evaluations, but also their rationales for not evaluating other programmes.
Source: Government of Canada (2018b)
Spending Reviews
Spending reviews are now commonly used in OECD countries and have established their value as a component of a performance-based approach to budgeting. Their twin aims are to scrutinise the purposes and value of existing programme expenditures and to improve their efficiency or to create fiscal space for new priority spending initiatives. To be effective, spending reviews must be backed by sufficient authority to overcome forces of inertia and entrenched interests and bring about significant reallocations of resources and/or efficiency improvements, where justified.
The structure and conduct of spending reviews differs among countries and from year to year, reflecting the political sensitivity of the exercise and cuts to established budgets. For this reason it is generally difficult to establish systematic links between spending reviews and performance budgeting. However, the general principle is clear that the results of any spending reviews carried out will be a key input to the budget. Equally the budget information generated by performance budgeting, including programme definition, and identification of performance indicators and targets is very helpful to those conducting spending reviews, looking for less effective programmes and poor value for money.