Methods of dealing with uncertainty – specifically probabilistic risks – in CBA have typically focused on expected utility theory which provides a strong theoretical basis for deviating from the simple use of expected values in a deterministic framework, towards estimating welfare corrections for use in CBA. However, estimating the resulting certainty equivalent values requires assumptions about the nature of society’s utility function, and some demanding estimates of the probability distributions of the risky quantities associated with any given project. Even so, practitioners are increasingly prepared to use these methods, given emerging evidence about the errors associated with simpler approaches. That said, more ad hoc ways of addressing this such as sensitivity analysis and Monte Carlo simulations have their place, and the chapter shows how a nuclear power project appraisal might utilise and interpret a Monte Carlo analyses. Nor should a focus on formal economics ignore the fact that there are many other principles that could be applied in CBA to make decisions in the face of uncertainty, such as “safety first” and “precaution”.
Cost-Benefit Analysis and the Environment
Chapter 9. Uncertainty
Abstract
9.1. Introduction
So far it has been implicitly assumed that the costs and benefits of projects are known with certainty. The reality is that estimates of costs and benefits are often very uncertain and subject to random variation over time. The uncertainty can stem from several sources. Technical issues are a chief source of uncertainty. For instance, the precise geological conditions that will arise in the process of implementing large infrastructure projects, like hydro-power schemes or nuclear power stations, are not known with certainty beforehand. In addition to a wide range of project-specific technical matters, the prices of goods and services are also subject to variation over time, and are also therefore uncertain.
Uncertainty matters from the perspective of CBA because the welfare effects of two projects which have identical paths of expected costs and benefits (i.e. the average costs and benefits) will have very different welfare effects if a project has uncertain costs and benefits and the other does not. Typically people behave as if they are averse to uncertain situations. Evidence for this comes from the large and ubiquitous demand for insurance, and a burgeoning collection of experimental results and other results (e.g. Holt and Laury, 2002; Andreoni and Sprenger, 2012; Harrison et al., 2002; Groom and Maddison, 2017). For this reason there are strong arguments for aversion to uncertainty to be reflected at the societal level in the appraisal of public projects. Other things equal this would lead to projects with lower levels of uncertainty to be selected. Put another way, projects with more reliable and certain outcomes would be preferred from a welfare perspective.
Beyond the uncertainty in the costs and benefits there is also uncertainty in the macro economy, so-called systematic uncertainty in the level of growth. As discussed in more depth in the discounting chapter, the presence of project-related uncertainty and systematic uncertainty also changes the welfare consequences of public projects if these risks are correlated with one another. If a project has high pay-offs during boom times, the same benefit is valued less in welfare terms than if it occurred during a recession and essentially acted as an insurance policy. The magnitude of the penalty that should be applied to projects with payoffs that are positively correlated with growth depends on the project itself, the societal preferences, and the extent to which a government can spread risks in society.
Economists tend to think about decisions under uncertainty from the perspective of expected utility theory. Doing so leads to some elegant theoretical expressions for welfare changes under uncertainty. However, some of the information needed to operationalise these measures is difficult or expensive to obtain, and the methods are open to debate. For this reason, practitioners of CBA have a number of other methods in their toolkit. These include sensitivity analysis, Monte Carlo simulations, and so forth. These methods are not especially based in the welfare theory that economists are used to, but they do provide a practical way of thinking about the effect of uncertainty, and the sensitivity of the NPV calculation to particular assumptions within project appraisal.
This chapter will start with the theory behind welfare analysis of risky projects. This will entail a brief discussion of the Arrow-Lind theorem (Arrow and Lind, 1970). The chapter will explain the use of certainty equivalence, risk premiums and the willingness-to-pay to reduce or eliminate risks, such as flooding. Indeed, an example of flood risk is presented to show how to use the valuation methods under risk. The chapter then discusses more ad hoc methods of dealing with uncertainty: sensitivity analysis and Monte Carlo simulations. The example of nuclear power is used to illustrate how these methods work. In order to proceed, a more precise definition of uncertainty is needed.
9.2. Risk and uncertainty: Some definitions
If costs or benefits are uncertain it means that ex ante, before the project is implemented, they could take on many different possible values. Prior to flipping a coin, the outcome could be heads or tails, for instance. Prior to excavating the foundations of a nuclear power station, the geology could be very favourable, or less so. A key distinction here is between risk and uncertainty. In the case of flipping a coin, one can be pretty sure that the probability of heads or tails is close to 0.5. When it is possible to assign probabilities to the possible events, and indeed know the full range of possibilities, then this is usually referred to as a risky situation. When this information is not available, as it might be in relation to the geology example, or in defining the likelihood of a technological failure, this is defined as uncertainty. Clearly from an analytical perspective it is easier to deal with risky situations than uncertain ones since one can calculate summary statistics from the distribution of possibilities: expected values, variance, etc., by these definitions.
The remainder of this chapter deals with risk rather than uncertainty, and it is assumed that possible outcomes have some probability distribution associated with them, like the 0.5 chance of getting heads or tails. All the possible outcomes are known, and there are no unknown unknowns. Indeed, the numerical examples make use of such information to undertake some welfare analysis. Importantly though, the terms uncertainty and risk are used interchangeably from hereon.
9.3. Welfare under uncertainty
In a deterministic world, CBA would evaluate the social desirability of a project in terms of the standard Utilitarian Social Welfare Function (SWF) as discussed in Chapter 2:
[9.1]
and project evaluation tests whether or not a given project, and the changes it implies for the current level of consumption across time, ct, increases this measure of inter-temporal welfare. When consumption is uncertain, and the net benefits of the project are uncertain, society must take a position on how the welfare measure should take into account uncertainty.
The standard approach in economics is to use an expected utility function:
[9.2]
where consumption is now a random variable, 
, and at time t, the current (non-discounted) value of expected utility is:
where f(ct) is the probability density function of consumption and cmax to cmin are the maximum and minimum values of 
. For instance, in the discrete case, if there were two possible outcomes for consumption, ct1 and ct2, with respective probabilities p1 and p2, then 
and 
and expected consumption at time t would be:
while expected utility at time t would be:
The fact that the range of outcomes for 
and the probabilities associated with them are defined means that strictly one is in the realm of risk rather than uncertainty as defined above.
The change in the way welfare is evaluated under uncertainty, using expected utility, means that using expected values of consumption in the deterministic framework of (1) is no longer sufficient for measuring changes in well-being. Defining expected consumption as 
, it follows that if U(.) is non-linear then:
.
Using a deterministic SWF and expected values (the LHS of the above equation) provides the wrong measure of well-being compared to the expected utility framework (the RHS). In fact, if the utility function reflects the preferences of a risk-averse agent, then it is the case that:
[9.3]
This would be the case if the utility function is concave in ct. So using the expected values of uncertain consumption overestimates the value of a risky level of consumption when agents are averse to risk. The reason is obvious. Using expected values as if they are certain ignores the risk associated with them. If agents are risk-averse, one would expect a welfare measure that accounts for risk to be lower. As is shown in the following section, using expected values for project net benefits, which is a common short-cut taken in the appraisal of risky projects in some quarters, will also prove to be an inadequate method for evaluating the welfare contribution of a risky project. The standard measures of the welfare value of uncertain projects show this clearly and provide a means of either a) estimating corrections to incorporate the effect of risk; or, b) show the welfare value of reducing or removing risks altogether.
9.4. Certainty equivalence and risk premiums: Definitions
One way in which one can use the deterministic framework in [9.1] to evaluate well-being and account for risk aversion is to calculate the certainty equivalent value of the uncertain variable. From [9.3] it is clear that a risk-averse expected utility maximiser would be willing to accept some value of consumption with certainty that is lower than the expected value 
. The certain value of consumption that equates both sides of [9.3] is known as the certainty equivalent, cE. It is defined as follows:
. [9.4]
For a risk-averse agent, the certainty equivalent is less than the expected consumption: 
. This leads to another concept useful for CBA, the risk premium, RP. RP measures the amount by which the certainty equivalent consumption is less than expected consumption, and is defined as:
. [9.5]
A simple expression for the risk premium can be derived for small risks. Annex 9.A1 shows that the risk premium can be approximated by:
[9.6]
where 
is the elasticity of marginal utility, also known as the Coefficient of Relative Risk Aversion, which measures the curvature of the utility function, and 
is the variance of consumption. The risk premium measures the willingness-to-pay to receive the expected value of c with certainty, rather than face the risk.
Risk premiums can be estimated in a number of ways. One way would be to estimate each element of [9.6] individually. This pragmatic approach requires an assumption about the specific form of the utility function, as well as knowledge of the variance parameters (see Kind et al., 2016, for an example of this pragmatic approach). The risk premium could also be estimated via suitably designed experiments as the willingness-to-pay to have the expected value of the lottery, rather than the lottery itself. A related measure of risk aversion is the willingness-to-pay to reduce risk. This can be estimated using revealed or stated preferences when the payoffs and changes in risk can be observed.
So, in principle, one way in which to capture the uncertainty in CBA is to use certainty equivalent values in the standard deterministic framework. These values can be estimated in a number of ways.
Up until now, the analysis has been in terms of aggregate consumption. The following section introduces the project net benefits into the analysis, with and without project risks. The concepts of certainty equivalence and risk premiums are used to show how project-specific risks can be incorporated into project appraisal.
9.5. Certainty equivalence: Application in CBA
One way in which to embed the welfare costs of the uncertainty associated with the net benefits of a project is to calculate the certainty equivalent of these net benefits. When the net benefits are compared to the baseline level of income, the certainty equivalent then captures two aspects of uncertainty: i) the uncertainty in the net benefits themselves (due to, say, uncertainty in the amount of a good or service that will be delivered); and, ii) uncertainty in the background level of income/consumption at the time the net benefits accrue. Once calculated the certainty equivalent values can be treated “as if” they are certain. They can then be discounted in the normal way to calculate the Net Present Value using a risk-free discount rate. The certainty equivalent is now defined and an example in project appraisal is developed.
Suppose that a project provides an uncertain net benefit in cash terms of NB. Suppose also that the background level of income, Y, is also uncertain. I.e., it is not known how rich society will be when the net benefits arrive. The current value of additional welfare at a particular point in time with the project is given by its expected utility:
and expected utility without the project is given by:
The welfare change associated with this project is given by the difference between the two:
[9.7]
The Annex shows that by means of Taylor series approximations, the change in welfare for a small value of NB, measured in terms of units of consumption/income, is given by:1
[9.8]
where 
, and 
. In the context of a public project, Y can be thought of as
national income (or sometimes the portfolio of public projects). Equation [9.8] shows that the welfare change in terms of consumption is basically equivalent to the summation of 3 terms: 1) the expected value of the change in net benefits, 
; 2) A risk premium associated with the pure variance of the project net benefit, 
; and, 3) a risk premium reflecting the covariance of the net benefit with national income, Y: 
. As discussed below, in the context of evaluating public projects, the latter two components can be thought of as diversifiable and non-diversifiable sources of risk. In essence, [9.8] is the certainty equivalent value of the uncertain net benefit, NB, measured in units of consumption. It is the sure change in net benefits that would give the same welfare change as the uncertain NB of the project. If it is greater than zero, then the project is worthwhile.2
In principle, equation [9.8] provides a means of correcting the expected values of net benefits for the fact that society is risk-averse and the project is risky. In contrast to the definitions above, though, in equations [9.4]-[9.6], the certainty equivalent here contains a risk premium with two components. The first is:
[9.9]
which measures the willingness-to-pay to avoid the variation in the net benefits alone. For a risk-averse agent this will be positive. The second component of the risk premium is:
. [9.10]
This component reflects the fact that the net benefits may be correlated with the uncertain background national income. This is important because if a project is positively correlated with national income: it has high payoffs when one is rich and low (possibly negative) payoffs when one is poor, then the project clearly contributes to the overall risk that society faces. Projects that add to overall risk should be penalised when society is risk-averse. Inversely, a project’s net benefits might be negatively correlated with income. In this case, the project has high net benefits when income is low, and low payoffs when income is high. Such a project reduces risk in society. Such projects should be rewarded in a risk-averse society since they essentially act as insurance policies.
So RPY,NB represents either the cost of additional risks that a project entails when its net benefits are positively correlated with income (RPY,NB>0), or the benefit of the reduced risks that a project provides if its net benefits are negatively correlated with income (RPY,NB < 0).
Put together these two risk premiums show that using the expected values of the net benefits as a means of dealing with uncertainty could lead to misleading project appraisals because it would ignore the preferences for risk reduction of various types. Expected net benefits should be corrected to account for these two sources of uncertainty in CBA. All that is needed is an estimate of the relevant parameters for these two risk premia. In order to make practical progress here, there are several routes one could take. How to proceed depends on the data that is available. The next section considers how in principle these terms could be estimated. The following section explains how relevant these elements of risk are to public policy decision.
9.5.1. Certainty equivalent net benefits: Estimation and implementation
In order to estimate [9.8] one could take a direct approach, by first estimating the risks that are involved for a particular net benefit, and for income, and then estimating the preference parameter η. Specifically this requires estimates of the variance of the net benefits, 
, the variance of background income, 
, and the covariance of NB and Y, 
. Estimation of the joint probability distribution of Y and NB is needed, from which one can obtain the marginal distribution of Y and NB to estimate the preference parameters. The exposition so far has focussed on risk aversion, i.e. the preferences associated with the variance or spread of outcomes. In principle society will have preferences such as downside risk aversion (aversion to skewness), and aversion to kurtosis (See e.g. Groom et al., 2008). This chapter solely focusses on risk aversion: aversion to spread.
Typically, and mainly for analytical convenience, it is assumed in applied theory and in empirical applications that society has iso-elastic preferences: 
, which is a constant relative risk aversion (CRRA) utility function where the coefficient of
relative risk aversion is given by the constant: 
. Many empirical estimates of
this parameter exist, obtained in different contexts. While in the contexts of private decisions individuals behave as if they have values of risk aversion of often as high as η > 10, experimental studies tend to show that on average agents are risk-averse with η » 1 (e.g. Holt and Laury, 2002; Harrison and Rutstrom, 2009). Groom and Maddison (2017) analyse aggregate risk aversion in insurance markets in the United Kingdom. Such studies have the ubiquity that is probably more relevant to CBA than individual experimental studies. Their estimates show that η » 1.5. Once a parameter is estimated, and the risk characteristics of a project are known, then it is possible to calculate the certainty equivalent values of the net benefits for CBA. Before a numerical example is provided some criticism of certainty equivalence is discussed.
In practice the calculation of the welfare change of an intervention in (8) could proceed as follows. Suppose, rather than looking at the impact to the economy as a whole, the case of an individual farmer who wishes to invest in flood defences is considered. The defences have a net payoff of GBP 350 in the event of a flood, due to avoided damages, and a cost of GBP 100 in the event that no flood occurs. In the background, the farmer’s income is GBP 4 000 in the event of a flood, and GBP 5 000 when there is no flood. The flood occurs with a probability of 0.2. These payoffs and their expected values and variances are presented in Table 9.1.3
Table 9.1. The state-dependent project payoffs and incomes
Source: Elaboration of Dinwiddy and Teal (1996).
The payoffs of this project have a negative covariance with the income levels. The project therefore has insurance properties since it has a high payoff in the bad times (low income) and a low payoff in the good times (high income). The final state-dependent income has a lower variance in the presence of the project than in the absence (48 400 compared to 160 000). As such the risk premiums in [9.8] will be of the opposite sign to one another. Table 9.2 shows this to be the case and that the insurance effect of the project can dominate. The first risk premium is negative as expected, but the second risk premium is strongly positive. The welfare value of this project can be much higher than the expected value of the project, which is negative (-10).
Table 9.2. The welfare change and risk premiums of the flood defence project
The importance of the correction for risk increases with the level of risk aversion. Table 9.2 shows that in this case, as risk aversion (η) increases, the risk premium that reflects the insurance properties of the project, equation [9.10] starts to dominate. As the risk aversion parameter η increases beyond η= 0.8, the welfare impact of the project becomes positive as a result. This illustrates the importance of dealing carefully with project risk and the need to understand the level of risk aversion when evalutating projects under uncertainty.
While in principle these welfare effects are important when considering the well-being of an individual farmer, as is the case here, other considerations are required when public projects are considered in the round at the aggregate level. Concepts of risk sharing and risk pooling become important in this case. In some cases it can be argued that the elements of risk discussed above are irrelevant for public policy appraisal. Irrespective of that, what the previous analysis clearly states is that, when project net benefits are uncertain, at a minimum expected values should be calculated and used for the appraisal of projects. Issues of risk pooling and risk sharing are discussed below.
Before discussing these issues in more depth, a different dimension of project appraisal under risk is addressed: the value of eliminating risk. The following section discusses this in the context of removing flood risk.
9.5.2. Willingness-to-pay to eliminate flood risk
The previous example undertook a valuation of an investment with risky payoffs which were correlated with the background income. In the example above, the correlation with background income was negative and the welfare change associated with the project was typically larger than the (negative) expected value. Similar methods could have been applied to the evaluation of a project that is positively correlated with background income, and therefore increases risks in the economy.
Yet, many public projects aim to eliminate the risk entirely, and so have large, non-marginal effects on expected well-being. Flood defences are a good example. In such cases, what may be required is a measure of the welfare benefits of the complete elimination of the risks, rather than a correction for the riskiness of costs and benefits discussed above.
Kind et al. (2016) set out clearly the procedures for undertaking a CBA of the removal of the risk of flooding while taking into account risk aversion using the tools described above. This example illustrates how all the concepts described so far can be used. It also illustrates the potential mistakes that can be made by using expected values of uncertain variables rather than explicitly evaluating the welfare effects of risk.
Suppose society is confronted by a risk of flooding, which would cause a loss of goods and services. Table 9.3 shows the details of the example. In the absence of flooding, households do not suffer any damages and enjoy a consumption level of 100. In the event of the flood, they incur losses of 90 and consume only 10. Flooding happens with a probability p = 0.2. The expected consumption is 82, and so the expected damage is 18 (100-82). Assume that utility is iso-elastic with a value of η= 1.2.
Table 9.3. Flood risk example: Payoff matrix, expected values and certainty equivalents
Table 9.3 provides enough information to evaluate the willingness-to-pay, and hence benefits of, eliminating the flood risk by building flood defences. First, notice that the certainty equivalent is calculated as follows:4
The risk premium is therefore:
The large positive risk premium shows that there is a large willingness-to-pay for removing the risk associated with the flood. Yet, since removing the entire risk would also remove the expected damages, the value of the removing expected damages should also be added to the risk premium to obtain the total welfare gain. Total willingness-to-pay (TWTP) to eliminate the risk of flooding is therefore given by:
TWTP = Expected Damages + Risk Premium
= 18 + 24.5
= 42.5
In essence, even with quite modest risk aversion (η = 1.2) focussing only on the expected benefits of flood defences would capture only some of the welfare benefits of the project. Kind et al. (2016) explain that a good summary measure of the error involved in only looking at expected values is given by what they call the “Risk premium multiplier” (RPM). In this numerical example the ratio is 2.3 (= 42.5/18). This is a measure of how much one needs to multiply up expected damages in order to obtain the appropriate welfare measure for a risk elimination project. The measure is defined as follows when utility is iso-elastic (see Annex for derivation):
[9.11]
where p is the probability of a flood, and z is the proportion of consumption that is lost due to the flood: in this case z = 90/100=0.9. Figure 9.1 shows how this error varies with risk aversion and with the risks faced by society (the proportion of income lost in a flood). Figure 9.1 shows that ignoring risk aversion in the welfare analysis will underestimate the welfare gains from risk reduction, especially when the risks (potential damage) and risk aversion are high.
Figure 9.1. Risk premium multiplier
As a function of proportion of: Left panel: income lost in flood; Right panel: Relative risk aversion
9.6. Risk in the public sector: The Arrow-Lind theorem
Returning to equation [9.8] above, which is reproduced here for convenience:
this section discusses the relevance of the two risk premia to public policy appraisal (the second and third terms on the RHS of [9.8]). The first risk premium relates to the variance of the project itself, and the second risk premium relates to the correlation of project risk with, in the public policy case, national income, Y. Two arguments are typically used to make the case that each risk premium is not relevant to the appraisal of public projects.
First, as hinted at above, the first risk premium represents risks which are diversifiable across the entire portfolio of projects that a government has. That is, the effect of such risks cancels out across the many projects implemented such that in the aggregate such risks are unimportant. Second, the Arrow-Lind theorem (Arrow and Lind, 1971) states that since the aggregate risk is shared across many individuals in society, at the aggregate level these risks become vanishingly small.
The basic idea can be seen in relation to the example of flood defences seen in Tables 9.1and 9.2.5 If the risk associated with the project alone were shared between two parties, so that in the bad times the losses were GBP 50 and in the good times the gains were GBP 175, then the variance of this risk is quartered. The risk reduces by a power of 2 for each increase in the number of people sharing the risk, until it disappears for all practical purposes, so the argument goes.
The second risk premium concerns the correlation of the project net benefits with, in the public policy case, the national income or the macro-economy. Rather than being diversifiable, such risks are referred to as non-diversifiable or systematic risks. Historically, it has been argued that this element of risk is likely to be small given the small size of many projects in relation to the macro-economy as a whole. For this reason, this term is frequently ignored. The UK Treasury’s “Green Book” takes this position on systematic risk for instance. Yet many countries do take systematic risks into account.
The Arrow-Lind theorem has been hugely influential in the realm of CBA and is the motivation. Yet the Arrow-Lind theorem has always been called into question because of some of the assumptions required for public sector risk sharing. Some argue that it is unrealistic to assume that the diversifiable risks of particular projects will be shared in the way Arrow and Lind (1970) suggest. Furthermore, Baumstark and Gollier (2014) argue that the assumption that benefits of private and public investments are independent of one another is also unrealistic. In short, such arguments imply that both the risk premiums in [9.8] should still be considered in CBA. Ignoring them would lead to poor public project selection and potentially the government taking on a portfolio of projects which are add to macroeconomic risk: e.g. in transport and energy.
9.7. Sensitivity analysis
If there is some uncertainty about the value of some of the key parameters in CBA, then a sensitivity analysis can be used to gain an understanding of how sensitive the NPV of a particular project is, or some particular cost and benefit is, to changes in that parameter. The approach is somewhat arbitrary and ad hoc, and does not have any welfare significance of the kind demonstrated in the previous sections, but practitioners can obtain some idea of the importance of some assumptions in calculating the baseline NPV.
The following discussion uses nuclear power generation as an example of a project where there are uncertainties in the flow of costs and benefits over time, particularly decommissioning costs. Furthermore, due to the long time-horizons associated with nuclear power generation and decommission, such projects are likely to be sensitive to the choice of the discount rate.
A sensitivity analysis in relation to these two parameters identifies two important concepts in sensitivity analysis: switching values and switching ratios. The former is value of the parameters at which the NPV changes sign; the latter shows the proportional change in the parameter required from the baseline for the NPV to change sign. The following example undertakes what is known as a gross sensitivity analysis, which looks at the sensitivity of the NPV.
9.7.1. Nuclear power: Sensitivity to the discount rate and decommissioning costs
Figure 9.2 shows the cash flows estimated for a nuclear power plant. These values should be considered purely illustrative since the estimates come from the early 2000s. Their source is the UK Cabinet Office’s Performance and Innovation Unit Energy Report from 2002 (PIU 2002).6 The Figure shows that there is a long period of around 6 years of investment costs, followed by a 40 year period of production during which the net benefits are positive. This is followed by a lengthy period of decommissioning costs. Overall, the time horizon for this CBA is a period of around 120 years.
Figure 9.2. The cash flows of a nuclear power plant
Source: Pearce et al., 2003; PIU, 2002.
Figure 9.3 shows the sensitivity analysis associated with the nuclear power plant. The sensitivity analysis tell us that the NPV is highly sensitive to the discount rate, but in unexpected ways. At 0% the NPV is negative, meaning that the raw sum of the cash flows is negative, mainly due to the duration of the decommissioning costs in the future. As a larger discount rate is applied, the NPV increases, however, as the present value of these decommissioning costs is discounted at ever higher rates. The switching value is at 2.25% above which NPV is positive. This positive relationship between the discount rate and the NPV stems from the fact that the net benefits of the nuclear power project change sign twice. Beyond some value, an increasing discount rate starts to reduce the NPV, as it would in a standard investment project with now tail-end costs. As the future benefits are discounted more and more eventually, at 5.25%, the NPV changes sign once more. The sensitivity analysis has revealed aspects of the project that the analyst might not have known before. First, that there are two switching points for the discount rate, and second that there is only a narrow range of discount rates for which NPV > 0.
Figure 9.3. Sensitivity analysis: Discount rate
As for the switching ratio, there are two of these also. The UK government uses 3.5% as its basic discount rate, so the lower switching ratio is 0.64 (2.25%/3.5%), and the upper is 1.5 (5.25%/3.5%). These are quite close to 1, indicating that the project is very sensitive to the discount rate.
A similar analysis can be done for the decommissioning costs, the present value of which is crucial for the NPV. Figure 9.4 undertakes a gross sensitivity analysis in relation to decommissioning costs which has a monotonic relationship, and a single switching point.
Figure 9.4. Sensitivity analysis: Decommissioning costs
More complicated sensitivity analyses are possible, and better measures of the sensitivity can also be calculated. For instance, one could subject the project to a stress test in which NPV is tested against the worst and best case outcomes for singular variables or for all variables together.
Sensitivity analysis provides useful information on the robustness of the NPV to various assumptions concerning variables that are deemed to be uncertain (prices, costs, time horizons for construction, etc.). However, the changes in the variables are rather ad hoc and down to the analyst. For instance, one would have no idea from the previous analysis of the likelihood of the discount rate being between 2.25% and 5.25%, nor of the decommissioning costs being lower than GBP 100 per kWh. A better approach might be to characterise the likelihood of variables taking on particular values, and obtain a picture of the likelihood of the variables approaching the switching values. Monte Carlo analysis provides a means of doing this.7
9.8. Monte Carlo Analysis
Monte Carlo analysis uses estimates of the probability distributions of costs and benefits, and other parameters used in CBA, to undertake a probabilistic analysis of the likely NPV to emerge from a particular project. The probability distributions of, say, costs or the discount rate, provide information on the likelihood of different scenarios emerging, e.g. high decommissioning costs, and use this information to build up a probability distribution for NPV. The steps to Monte Carlo analysis are as follows:
Estimate the probability distributions for the parameters of interest. Where parameters are likely to be correlated, the joint probability distributions are estimated;
Take a random draw of the parameters of interest of sample size n;
Estimate the NPV n times using the parameters drawn;
Calculate the mean NPV across the n estimates and store the value;
Repeat m times until one can plot the probability distribution of mean NPV conditional on the uncertain parameters with sample size n, with m repetitions;
Evaluate the likelihood of a positive or negative NPV.
The difficulty in Monte Carlo analysis is accurately reflecting the probability density functions associated with the parameters of interest. One can look at historical data, expert opinion or experimental evidence when looking at preference parameters, like risk aversion. Typically the analysis is undertaken using the deterministic representation of welfare, rather than the expected utility approach. But this is not always the case, and it is also possible to include the preference parameters, such as risk aversion, into the Monte Carlo analysis. This is the approach taken in many integrated assessment models (e.g. Stern, 2007).
To illustrate the technique, the nuclear power example above is continued, with a focus on the two variables that NPV is evidently sensitive to in this case: the discount rate and the decommissioning costs.
9.8.1. Nuclear power: Monte Carlo simulation of discount rates and decommissioning costs
To undertake the Monte Carlo analysis, the joint probability distribution for the discount rate and decommissioning costs is defined according to Table 9.4. While the numbers for decommissioning costs are centred on the values presented in Figure 9.2, the standard deviations associated with them are just illustrative. One could imagine obtaining a distribution of these costs from expert opinions. Such opinions would reflect the nature of the project and expectations of technological change in the future. The discount rate, however, is centred around the 3.5% used by the UK Treasury, and the standard deviation is obtained from the range of expert opinions on the SDR found in Drupp et al. (2017). One could just as easily use historical interest rate data to estimate these parameters if the policy was to use interest rates for social discounting, as in the United States (Groom and Hepburn, 2017). The STATA code for the Monte Carlo simulation can be found in Annex 3. The data are available upon request.
Table 9.4. Parameter values for Monte Carlo simulation
Finally, two simulations which differ in the correlation between discount rates and decommissioning costs are represented. In the first simulation the variables are assumed to be positively correlated, in the second they are assumed to be negatively correlated. Again, there is no clear source of information on this matter, so the simulations simply illustrate the implications of positive or negative correlations between the two random parameters. The sample size n, and the number of repetitions is chosen to be 1 000. Figures 9.5and 9.6 show the simulated distributions of the mean NPV.
Figure 9.5. NPV with negatively correlated discount rate and decommissioning costs (r = -0.7)
Figure 9.6. NPV with positively correlated discount rate and decommissioning costs (r = +0.7)
Figure 9.5 shows the distribution of NPV in Simulation 1 which assumes a negative correlation between the discount rate and the decommissioning costs.8 Here the NPV values are centred below zero. The mean value of the mean NPV is approx. GBP-200 and the median is approx. – GBP150. So, in more than 50% of the simulations, the mean NPV is negative. This shows that there is considerable uncertainty around a mean value of zero. This does not look like a convincing project.
Figure 9.6 shows the results of Simulation 2 where a positive correlation between the discount rate and decommissioning costs is assumed. With a positive correlation, high values of the discount rate are coupled with high values of the decommissioning costs, and vice versa. This tends to lengthen the tail of the distribution of NPV, so that there are is a positive probability of some very bad outcomes: large and negative NPVs. The mean and median values are not tremendously different to in Simulation 1, but the long left skewed distribution is a warning to a risk-averse planner.
9.9. Optimism bias
Work by Flyvbjerg (2009) showed the empirical regularity with which public projects would end up being more expensive than anticipated in the original project document. The HM Treasury “Green Book” (HMT 2003, Ch7, p. 85) devotes an entire section of its chapter on Uncertainty to this topic. Optimism bias is often thought to be the chief feature of uncertainty, and the chief fear of policy makers: that the benefits of the project turn out to be lower than expected, or the costs higher. That this happens systematically in project appraisal is the motivation for labelling this type of uncertainty a “bias”.
Optimism bias mainly affects the cost side, but can also affect the benefit side of project appraisal. Capital costs are often poorly defined or sometimes overlooked in the planning phase. The duration of works is also frequently underestimated.
The fear of optimism bias has led to all sorts of proposed solutions, some more crude than others (HMT, 2003, p. 85-87):
Collection of the best evidence on net benefits;
Performance management systems:
Competent project managers;
Break large projects into more manageable smaller projects;
A premium on the discount rate to reflect optimism bias.
These methods are discussed in HMT (2003). Using the discount rate to control for the optimism bias is not generally to be recommended. Discounting in this way would treat all project net benefits in the same way, ignoring the fact that optimism bias varies from one project to another.
9.10. Conclusions
This chapter has provided an introduction to methods of dealing with uncertainty in CBA. The focus has been on expected utility theory which provides a strong theoretical basis for deviating from the simple use of expected values in a deterministic framework, towards estimating welfare corrections for use in CBA. The use of certainty equivalent net benefits is routinely recommended by economists for the analysis of the public projects. Several practical examples have been explained which show how these welfare adjustments can be made. However, certainty equivalent values require some heroic assumptions about the nature of society’s utility function, and some demanding estimates of the probability distributions of the risky quantities associated with any given project.
More commonplace, although more ad hoc, are methods which investigate how the NPV of a project is affected by changes in some crucial parameters. Sensitivity analysis provides an indication of how sensitive NPV can be to some parameters. Monte Carlo analysis can be used to evaluate sensitivity of NPV to multiple parameters based on the likelihood that particular parameter combinations arise. While slightly less ad hoc, Monte Carlo simulations are demanding when it comes to estimating the probability distributions associated with some of the parameters that determine NPV. Examples from a nuclear power project have illustrated how one can use and interpret Monte Carlo analyses.
Of course, the focus on formal welfare economics and expected utility theory should not ignore the fact that there are many other principles that could be applied in CBA to make decisions in the face of uncertainty. “Safety first” approaches and the precautionary principle represent such alternative approaches. Even then, economic analysis can help in defining what is meant by these principles, and the trade-offs involved. In relation to the precautionary principle, the next chapter shows that this principle can be interpreted within an option value framework.
References
Andreoni, J. and C. Sprenger (2012), “Estimating time preferences from convex budgets”, American Economic Review, Vol. 102/7, pp. 3333-3356, http://dx.doi.org/10.1257/aer.102.7.3333.
Dinwiddy, C. and F. Teal (1996), Principles of Cost Benefit Analysis for Developing Countries, Cambridge University Press, Cambridge.
Drupp, M. et al. (2017), “Discounting Disentangled: An Expert Survey on the Components of the Long Term Social Discount Rate”, Forthcoming in the American Economic Journal: Policy. Working paper version: Drupp, M. et al. (2015), Grantham Research Institute on Climate Change and the Environment Working Paper No. 172, www.lse.ac.uk/GranthamInstitute/wp-content/uploads/2015/06/Working-Paper-172-Drupp-et-al.pdf.
Stehr, K. (2006), “Risk and uncertainty in cost benefit analysis”, Toolbox paper, Environmental Assessment Institute, Copenhagen, www.ttu.ee/public/k/karsten-staehr/2006_Staehr_-_Risk_and_uncertainty_in_ cost_benefit_analysis.pdf.
Flyvbjerg, B. (2009), “Survival of the unfittest: Why the worst infrastructure gets built – and what we can do about it”, Oxford Review of Economic Policy, Vol. 25/3, pp. 344-367, http://dx.doi.org/10.1093/oxrep/grp024.
Groom, B. and D.J. Maddison (2017), “Four New Estimates of the Elasticity of Marginal Utility for the UK, forthcoming in Environmental and Resource Economics. Working paper version: Groom and Maddison (2013). “Non-identical Quadruplets: Four New Estimates of the Elasticity of Marginal Utility for the UK”, Centre for Climate Change Economics and Policy Working Paper No. 141, Centre for Climate Change Economics and Policy, www.lse.ac.uk/GranthamInstitute/publication/non-identical-quadruplets-four-new-estimates-of-the-elasticity-of-marginal-utility-for-the-uk-working-paper-121/.
Groom, B. et al. (2008), “The story of the moment: Risk averse Cypriot farmers respond to drought management”, Applied Economics, Vol. 40, pp. 315-326, http://dx.doi.org/10.1080/00036840600592916.
Groom, B. and C. Hepburn (2017), “Looking back at social discount rates: The influence of papers, presentations and personalities on policy”, Review of Environmental Economics and Policy, Vol. 11, pp. 336-356, https://doi.org/10.1093/reep/rex015.
Harrison, G.W. et al. (2005), “Eliciting risk and time preferences using field experiments: Some methodological issues”, in Carpenter, J.P., G.W. Harrison and J.A. List (eds.), Field Experiments in Economics, Elsevier, Amsterdam.
Harrison, G.W. et al. (2002), “Estimating individual discount rates in Denmark: A field experiment”, American Economic Review, Vol. 92/5, pp. 1606-17, http://dx.doi.org/10.1257/000282802762024674.
HM Treasury (2003), The Green Book: Appraisal and Evaluation in Central Government, HM Treasury, London, www.gov.uk/government/uploads/system/uploads/attachment_data/file/220541/green_book_complete.pdf.
Holt, C.A. and S.K. Laury (2002), “Risk aversion and incentive effects”, American Economic Review, Vol. 92/5, pp. 1644-55, http://dx.doi.org/10.1257/000282802762024700.
Kind, J. et al. (2016), Accounting for risk aversion, incoe distribution and social welfare in cost-benefit analysis for flood risk management, WIREs Climate Change, e446, http://dx.doi.org/10.1002/wcc.446.
Pearce, D. et al. (2003), “Valuing the future: Recent advances in social discounting”, World Economics, Vol. 4/2, pp. 121-141, www.world-economics-journal.com/Pages/Download.aspx?AID=141.
PIU (2002), The Energy Review, Cabinet Office, Performance and Innovation Unit, London, www.gci.org.uk/Documents/TheEnergyReview.pdf.
Stern, N. (2007), The Stern Review on the Economics of Climate Change, Cambridge University Press, Cambridge.
annex 9.A1. Risk premiums
The welfare change:
can be re-written as a Taylor series:
This can be converted into units of consumption, in which NB and Y are measured, by dividing through by the marginal utility U¢(Y) to give:
[9.A1.1]
which is equivalent to equation [9.8] in the text. See Dinwiddy and Teal (1996) for further details of this (Annex to Chapter 14).
annex 9.A2. The risk premium multiplier
The risk premium multiplier (RPM) is derived from the ratio of Total Willingness-to-Pay (TWTP) to Expected Damages. TWTP is given by the difference between income in the absence of a flood and the certainty equivalent 
, so the ratio to expected damages is:
[9.A2.1]
The certainty equivalent in the case of iso-elastic preferences is given by:
Dividing by M gives;
where z = D/M. Expected damages per unit of income is given by 
, and per unit of total income it is there for 
. Having divided the numerator and the denominator of [9.A2.1] by M, the RPM can be re-written as in the text:
[9.A2.2]
annex 9.A3. Monte carlo simulation: STATA code and distribution
*****************************************************************************************************************************
*Start the simulation by assuming iterations and sample size of 1000*9
*****************************************************************************************************************************
forvalues j=1(1)1000 {
*generate random sample j of 1000 for the discount rate and decommissioning costs*
cap drop Discount *discount factor*
cap drop Decomm *discount rate*
*assume discount factor has a mean parameter of -3.5, and sd of 2*
*decommissioning costs have mean parameter of 4.5 and sd of 0.7*
*assume jointly normally distributed with correlation coefficient -0.7,*
mkbilogn Discount Decomm, r(-.7) m1(-3.5) s1(2) m2(4.5) s2(.7)
*generate discount RATE*
cap drop DRate
gen DRate=-ln(Discount)/100
*Simulate Cost benefit analysis using parameters for discount rate and decomm costs**
forvalues i =1(1)1000 {
tempvar DR NB DC DC2 PV
cap drop DC2
gen DC2 =0
local DC= -Decomm in `i’
*DC costs kick in after 46 years*
replace DC2=`DC’ if _n>46
cap drop NB
*use netcash flow data from dataset on costs and benefits: Time horizon 1-118 years*
gen NB=netcash if _n<118
*Replace decommissioning costs with those from the random sample drawn above*
replace NB=DC2 if _n>46&_n<118
*Use random sample of the discount rates to calculate the preent value*
local DR5 = DRate in `i’
replace DFactor=1/((1+`DR5’)^yearnumber)
*Calculate the PV of NB for each case i for each time period*
cap drop PV1
gen PV1 = NB*DFactor if _n<118
sum PV1 if _n<118
*Calculate the net present value over time horizon and record*
replace NPVsim=r(sum)+1000 in `i’
*repeat this 1000 times for each sample j of parameters*
}
*Take the mean of the i=1000 NPVs and record the PV*
sum NPVsim if NPVsim!=0, d
replace PVmean = r(mean) in `j
*repeat process 1000 times*
}
Figure 9.A3.1. Distribution of decommissioning costs
Figure 9.A3.2. Distribution of the discount rate
Notes
← 1. Footnote by Turkey:
The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of United Nations, Turkey shall preserve its position concerning the “Cyprus issue”.
Footnote by all the European Union Member States of the OECD and the European Union:
The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus.
← 2. Note, a second-order Taylor series expansion has been used here. Higher order expansions could be used if preferences for higher-order moments of the distribution are thought to be important. Groom et al. (2008) show the theory and provide an application to agriculture.
← 3. Another way of saying this is to note the correspondence of 
with the definition of the certainty equivalent of NB:
← 4. This example is an elaboration of the example shown in Chapter 13 of Dinwiddy and Teal (1996).
← 5. Utility is rescaled by adding 4 units of utility in the following numerical example.
← 6. The example comes from Dinwiddy and Teal, 1996, p. 230.
← 7. In fact, for illustrative purposes, in the numerical example GBP 1 000 per kWh has been added to the NPV in each case so that the NPV is positive for some range of the sensitivity analysis. The raw PIU (2002) data do not support a positive NPV.
← 8. Staehr (2006) provides a general source of further details on sensitivity analysis.
← 9. The distribution of the parameters is shown in Annex 9.A3.
← 10. Cost and benefit data available on request.