Delivering Better Policies Through Behavioural Insights

Chapter 2. Real-time information and consumer decisions on energy consumption

Abstract

This chapter investigates ways that behavioural insights (BI) can be used to improve the effectiveness of smart meters and induce energy savings. It describes the results of an experiment in Ontario, Canada, that tests the provision of real-time feedback on energy consumption through in-home displays.

Introduction

New technologies are quickly developing in the electricity market, encouraging consumers to make well-informed decisions about their electricity consumption.1 Unlike analogue electricity meters that typically only allow for flat electricity tariffs and information on aggregate usage, the digital (“smart”) meters provide consumers with information on dynamic electricity pricing and consumption in real time. Smart meters are replacing analogue meters in many regions of the world. As discussed in Rivers (2018), in Canada and the United States, approximately half of all residential meters have been replaced by smart meters as of 2016. In Italy, smart meters have been introduced for the totality of residential accounts (about 26 million). In France and the United Kingdom, the rollout of smart meters to households lags behind the rollout in North America. In Ontario, the focus of part of this report, the rollout of smart meters to residential customers was completed by 2010, making it an interesting case study for understanding the potential impacts of smart meters and associated feedback technologies on consumer electricity demand.

Paired with in-home feedback technologies, smart meters have two distinguishing features that can impact the environment through both behavioural and market-based avenues.2 First, unlike analogue meters, digital meters record electricity consumption at a fine-grained interval, potentially enabling households to be exposed to prices that vary over time of day. On the other hand, with a standard electricity meter, consumers only find out their consumption when electricity bills arrive at monthly or bi-monthly intervals. Second, smart meters can communicate electricity prices and electricity consumption in real time to households, which provides them with a better informational basis on which to make electricity consumption decisions.

These are behavioural avenues through which smart meters and feedback technologies can change consumer behaviour. In addition, smart meters enable time-varying electricity pricing. Economists have long advocated for time-varying wholesale prices to be passed on to consumers, arguing that the flat tariffs normally used in the residential sector suppress potentially cost-effective demand response (Borenstein et al., 2002). Because they are digital devices, smart meters can facilitate the implementation of virtually any type of tariff structure, including those that vary over time. In contrast, with a standard electricity meter, implementing time-varying rates is difficult or impossible. Smart meters enable consumers to conserve electricity when supply is constrained by facilitating dynamic electricity pricing. This is a market-based avenue through which smart meters affect the market by influencing electricity demand.

This report, which draws on the findings in Rivers (2018) and where further detail can be found, focuses on a particular type of real-time feedback technology, in-home displays (IHDs), which provide consumers with real-time information about electricity consumption, price and expenditures. Such high-quality information should make electricity consumption more salient to households and therefore increase the ability of the consumer to optimise decisions relating to electricity consumption. However, it is unclear how consumers will respond to the installation of IHDs. In particular, optimising consumers may respond by either increasing or decreasing electricity demand, depending on the nature of their perceptions of electricity consumption and price before the installation of the IHDs. Likewise, IHDs may make consumers more or less responsive to changes in electricity price, depending on how consumers’ pre-IHD beliefs reflected actual electricity prices. Moreover, the installation of an IHD may also increase the attention that consumers devote to their electricity consumption and cause changes in consumption as a result. Understanding how consumers respond to more information, therefore, rests more on empirical than on theoretical results.

The empirical literature on the impact of real-time feedback via IHDs on electricity demand has produced mixed results. Early pilot programmes developed by electric utilities typically suggest that providing households with real-time feedback on electricity demand causes a substantial reduction in electricity consumption. However, these early studies often do not use methods that would be considered appropriate today or they do not report enough information on methods, leading to doubts about their findings. More recent studies use high-resolution (e.g. hourly) data to compare electricity consumption from households with and without in-home displays, using either quasi-experimental or experimental research designs. These studies suggest that IHDs can induce meaningful reductions in electricity consumption in contexts where the price for electricity is high. However, there are few such high-quality studies and most of those that have been conducted focus on particular contexts such that results may not necessarily generalise to a wider population.

This report also provides a review of a recent study that sheds new light on the effect of real-time IHD feedback on consumer electricity demand. The study evaluates a programme that resulted in approximately 7 000 households in Ontario, Canada, being provided with an in-home electricity display. It uses a quasi-experimental approach to assess the impacts of real-time IHD feedback on household electricity demand, by leveraging the fact that IHDs are rolled out to households over a one-year period. This context enables a longitudinal approach to estimating the impact of IHD feedback, in which household electricity consumption with an IHD is compared to consumption in the same household before receipt of an IHD, controlling for trends experienced by other households whose IHD status does not change.

Based on this approach, several important findings are reported. First, the receipt of an IHD results in a reduction in electricity demand of around 3% overall. This result suggests that either: i) households underestimated their expenditures on electricity prior to receiving an IHD and the additional information caused them to reduce consumption; and/or ii) the receipt of the IHD caused electricity consumption to become more “visible” to households and led them to conserve electricity independently of the response to improvements in the quality of information. The results also suggest that household electricity conservation in response to real-time feedback provided via IHDs is concentrated in the autumn and winter heating seasons. The response by households is roughly uniform throughout the day and does not appear to be caused by the time-of-use pricing schedule.

The study found that household electricity conservation in response to IHD feedback persists for at least five months following the receipt of the display. Although it is not possible to confidently identify the mechanisms by which households respond to the IHD with the data available in this study, this finding suggests that households respond to real-time feedback in part by adjusting thermostat settings downwards or investing in durable energy efficiency improvements that result in lower space heating demand.

The chapter will be structured as follows: first, a background is provided on smart meters, including how they provide feedback and allow for time-varying prices. This is followed by a brief overview of findings from previous studies that measured the impact of real-time feedback on residential electricity demand. Next, the report summarises the theoretical model underlying the experiment, followed by a description of the quasi-experimental design of a case study in which real-time feedback was provided to residential customers in an electricity distribution area in Ontario, Canada. Results of the case study are then discussed, including the impact of real-time feedback on electricity consumption and how it varies by season, time of day and outdoor temperature. The chapter is then concluded with a summary of the findings.

Context and problem setting

Real-time feedback and time-use electricity pricing

Smart meters differ in two important ways from traditional analogue electricity meters. First, they record electricity consumption using a digital, rather than analogue, technology. Electricity consumption on smart meters is also recorded with a corresponding timestamp, indicating the time of use with hourly or higher frequency. On an analogue meter, in contrast, it is not possible to know when electricity was consumed within a billing period. This difference between the two technologies entails that smart meters enable flexible pricing (i.e. varying within day and across days) while analogue meters do not provide this opportunity. Second, smart meter infrastructure allows communication between the meter and the electricity distribution company. This eliminates the requirement for manual in-place meter reading that is associated with analogue meters. Most smart meters additionally allow communication between the smart meter and the household.

These two differences between smart and conventional meters – regular recording of electricity consumption and communication ability – allow for important changes both in the way that electricity consumption is communicated to households and in the way that electricity consumption is billed. The following sections discuss each of these potential changes. It is worth noting that when smart meters are adopted, households and their electricity distributors can make choices about using these features of smart meters or not. Upon adopting smart meters, certain jurisdictions and households have chosen not to change the way electricity is priced or to make use of feedback on household electricity consumption

Smart meters allow real-time feedback on household electricity consumption

Like an analogue meter, a smart meter is installed outside the house and does not typically display information on electricity consumption in an accessible, intuitive or easy-to-read manner for the average household. On its own, a smart meter provides limited information to a household about electricity consumption. However, most smart meters include features to allow communication between the electricity meter and the household, typically using wireless technology. Using these features, or using near-real-time data relayed by the smart meter to the electricity distribution company, households can obtain feedback on their electricity consumption.3 The provision of this information may encourage households to change electricity consumption behaviour, possibly inducing energy conservation. There are a number of technologies that have been adopted to provide households with real-time information on their electricity consumption, outlined below.4

Text message or email

Irregular text messages or email messages can be used to highlight to consumers unusual consumption or changes in prices. For example, Gleerup et al. (2010) analyse a feedback scheme in Denmark in which emails or SMS messages are sent to participants when electricity consumption deviates from average levels by a pre-specified amount and find a 3% reduction in electricity demand as a result.

Internet site or mobile application

It is possible to display information in a useful graphical format by linking a mobile application or Internet website to the distribution company repository of consumption data. For example, Schleich et al. (2013) analyse an Austrian field trial in which consumers were provided with access to a website that displayed useful information relating to electricity consumption (with a one-day lag). They find limited impact of website feedback on consumer electricity demand.

In-home display

In-home displays (IHDs) use a wireless or optical reader to display information from the smart meter in a convenient and accessible manner to the household. Typical in-home displays feature graphics that display electricity consumption and price over the day and month, as well as indicators showing the current price of electricity. For example, Houde et al. (2013) analyse a programme that provided Google employees with an in-home display and find that electricity consumption was reduced by about 5% for several weeks following the receipt of the device.

Smart meters allow dynamic electricity pricing regimes

Smart meters record electricity consumption on an hourly or higher frequency and recording occurs with a time stamp. As a result, smart meters enable the electricity distribution utility to use prices that change over the course of a day or change from one day to the next.5 Changes in prices to reflect different costs of electricity provision over time are a market-based mechanism for encouraging energy conservation.

There are a number of pricing schemes that are enabled through the use of smart meters.

Real-time pricing

In a real-time pricing programme, residential consumers are exposed to the wholesale price of electricity. This can provide them with an incentive to conserve electricity during periods when demand is high or when supply is reduced. Real-time pricing is rarely applied to residential customers. Allcott (2011a) examines a case where selected Chicago consumers were exposed to real-time prices.6 He finds a reduction in peak-period consumption and a welfare gain for consumers on real-time prices.

Critical period pricing

Under a critical-period pricing tariff, customers pay a flat price for electricity except for during a certain number of “critical” periods during the year, when the consumer electricity rate increases substantially. These critical periods are times of particularly constrained supply, such as hot summer afternoons, when air conditioning demand peaks. The large increases in electricity price during a limited number of hours provide consumers with a substantial incentive to reduce demand during these periods. Jessoe and Rapson (2014) examine a critical peak period electricity scheme and find that consumers indeed respond by reducing demand.

Time-of-use pricing

In a time-of-use pricing scheme, the consumer electricity tariff changes by a predictable amount at predictable periods during the day. For example, during the summer season, a utility might declare the hours of noon to 7 p.m. on weekdays as “peak” periods, in which the price of electricity is double the price in other periods. Time-of-use pricing obtains some of the benefits of real-time pricing without exposing consumers to the fluctuating wholesale price of electricity.

Literature review

This section briefly reviews the findings from articles that focus on the relationship between real-time feedback and consumer electricity demand in the residential sector. A more detailed summary can be found in Rivers (2018). For a recent review of real-time pricing studies, see Faruqui and Sanem (2010).

Studying the effects of varying the price of electricity, Ireland’s Commission for Energy Regulation (2011), now the Commission for the Regulation of Utilities (CRU), conducted a behavioural trial to gauge customer response to various time-of-use tariffs and demand- side management stimuli (enabling technologies). Time meters were installed in 5 028 participating households, which were then assigned to treatment and control groups with the former receiving various combinations of time-of-use tariffs, in-home displays and fridge magnets and stickers that outlined different electricity use time bands and cost per band. The study found that participants equipped with in-home displays reduced their overall energy consumption by an average of 3.2% and their peak demand by 11.3%.

Faruqui and George (2005) looked at California’s Statewide Pricing Pilot involving 2 500 residential and small- to medium-sized commercial and industrial customers. They found that customers who received the Critical Peak Pricing (CPP) rate intervention saw peak electricity use reductions between 8% and 15%. However, when smart thermostats were added to the CPP intervention, peak reductions were even greater, reaching 25% to 30%.

Delmas et al. (2013) conduct a meta-analysis of 59 studies across multiple disciplines in the academic literature, all of which use randomised controlled trials (RCTs) to estimate the impact of information provision on electricity consumption. The study covers a wide variety of behavioural interventions that affect electricity demand, including real-time feedback, social norm comparisons, delayed feedback, audits and other interventions. The results of the meta-analysis suggest that real-time feedback causes a reduction in electricity consumption of about 11% on average. However, the authors caution that estimates of the effects of feedback are inflated in poor quality studies (for example, those that do not control for weather or other confounding factors). Across all types of feedback, they find that the treatment effect in high-quality studies (which represent only a small fraction of all studies) is only about one‑quarter as large as the treatment effect for all studies. However, Delmas et al. (2013) neither provide an estimate of the effect of real-time pricing across high-quality studies in their data set nor clarify whether they consider any of the real-time feedback studies in their survey to be high quality.

Faruqui et al. (2010) summarise findings from several pilot experiments using real-time electricity feedback, most of which were published in non-peer-reviewed outlets. The pilots use a number of different interventions, including different types of IHD, different types of payment for electricity and different electricity tariffs, making it somewhat difficult to compare across studies. Faruqui et al. (2010) report that providing real-time feedback through an IHD to consumers is associated with a reduction in electricity demand of 3% to 13%. However, some of the reviewed pilot projects use very small samples, and the methods used to estimate the treatment effect and design the experiment are not clearly presented in the paper (owing to the large number of interventions surveyed), so it is difficult to ascertain the validity of the results.

Further studies on the effect of real-time electricity feedback include Faruqui and George (2005), who find that critical peak electricity demand in Maryland was reduced among a pilot group of 1 021 households by 18% to 21% with education materials and different rate structures alone, and 23% to 27% with an IHD combined with dynamic pricing programmes. Another study by Faruqui and Akaba (2014), this time in Connecticut, found that the same IHD as tested in Maryland did not reduce energy consumption. However, when all enabling technologies, including A/C switches and in-home displays, were combined with dynamic pricing, customers reduce their energy consumption by 23%.

A report produced by Karkkainen (2004) summarises results from several energy efficiency pilot projects conducted in Europe, including one in Norway that included a sample of 10 894 participants divided into treatment and control groups. The pilot tested the effectiveness of “Ebox” load control relays, which allowed for direct two-way communication of consumption data via the Internet. The pilot saw average peak demand reduced by 11%.

While some studies found no significant effects, others faced methodological issues, related to randomisation, sample size or biases for example, affecting the validity of results. Allen and Janda (2006) studied households in Ohio that received electricity monitors displaying both real-time and historical electricity consumption in kilowatt hours or USD. Baseline data was also collected from the households using utility bill records and semi-structured interviews. Researchers found no statistically significant effect in electricity consumption between the treatment and control groups. Nilsson et al. (2014) presents the results of two field experiments in Sweden, which tested the effects of IHDs on energy consumption. Both studies showed statistically insignificant effects but were limited by small sample sizes of 32 and 42 households. The researchers also note that prior interest in environmental sustainability, energy savings and knowledge of IHDs – as well as the aesthetics of the IHDs themselves – all contribute to the impact these enabling technologies have on consumer behaviour. Westskog et al. (2015) also fail to find statistically significant effects on energy consumption a year after IHDs were installed in a pilot programme in Norway, probably hampered by the small sample size of 33 participants.

Furthermore, Xu et al. (2015) tested IHDs in two recently built apartment buildings in Shanghai, People’s Republic of China. Their sample consisted of 131 respondents, 76 of whom received the IHDs (the other 55 served as control). The treatment group reduced their energy consumption by an average of 9.1% over the control group, and researchers found that introducing IHDs also led to a 12.9% reduction in average standby power usage when compared to the control group. However, there was no randomisation in the sampling and treatment assignment, in addition to little to no discussion of the methodology used. Fenrisk et al (2014) found large effects for 2 groups who opted into an advanced metering infrastructure (AMI) – 27% reduction in electricity demand – but the authors do not discuss disaggregated effects and believe self-selection bias affected the opt-in groups.

In the last few years, several high-quality studies have been published that examine the effect of real-time feedback on consumer electricity demand. Gans et al. (2013) use a quasi-experimental approach based on the roll-out of smart meters with real-time feedback to a subset of Northern Irish households for this purpose. The context they examine, in which customers pre-pay for electricity and experience some of the highest electricity prices in Europe, is likely to produce large conservation impacts. They find that real-time feedback generates a large (11%-17%) reduction in electricity consumption for treated households, which is sustained over several years. It is emphasised that these large impacts are likely context-specific.

Three studies stand out in the domain leveraging IHD. Houde et al. (2013) report on a randomised controlled trial, in which real-time feedback on electricity consumption – with an IHD – was provided to a randomly assigned group of volunteering Google employees. They report a 5% reduction in electricity consumption due to the provision of an IHD but find that the effect does not persist more than a few weeks. Again, the particular context of the study (Google employees) makes it difficult to understand how IHDs might affect consumption in a broader population.

Similarly, Jessoe and Rapson (2014) sampled 437 households to examine the impact of providing an IHD in a context in which households are also exposed to critical peak period pricing (in which prices increase by 2 to 6 times for several hours at a time). They find that households with an IHD are significantly more responsive to critical peak prices than other households. Customers in the group who received both the price and IHD treatments saw their energy consumption decline from 8% to 22%. In contrast, those who only received the price intervention reduced their energy consumption from only 0% to 7% relative to the control group. Researchers attribute the increased energy savings of the IHD group not to price salience but to “consumer learning”.

Some studies also found more modest results in response to real-time feedback. Schliech et al. (2013) ran a trial in Austria with 1 525 residential customers randomly selected into treatment and control groups for a field trial examining the effects of real-time feedback on energy consumption. Feedback group participants reduced their average energy consumption by 4.5% relative to the control group. The researcher’s findings also suggest that electricity consumption is inversely correlated with the frequency of billing and metering. Sulyma et al (2008) conducted a pilot programme testing the efficacy of different price-signalling regimes and technologies in British Columbia, Canada. Two thousand residential customers were randomly assigned to three treatment groups and a control group, with Treatment Groups A and B receiving advanced meters and different communication packages and Group C receiving the same as B as well as an IHD. Group C experienced a 5% reduction in their overall energy consumption and a 9% reduction in peak demand – both attributable to the effect of in-home displays.

Finally, Harding and Lamarche (2016) analyse how the provision of real-time feedback technologies impacts consumer response to time-of-use (TOU) pricing. They find that households with IHDs do not significantly alter their profile of hourly electricity consumption compared to households without them in response to modest price changes.

In sum, the existing literature appears to consist of a fairly large number of studies of questionable quality, which finds varying but often large impacts of real-time feedback on electricity demand. More recently, several high-quality studies have been produced but while the internal validity of these studies appears to be high, it is not clear how well the results from these studies will transfer to other contexts because most have used rather idiosyncratic populations or treatments. As a result, there remains a relatively significant gap in the understanding of how real-time feedback affects electricity consumption.

Methodology

Theoretical model

Real-time feedback technologies have the potential to reduce electricity demand by providing higher quality and quantity of electricity information to consumers. Based on this assumption, the model for this study reflects the consumption decisions of a “rational” consumer who makes optimal decisions when given information about price and efficiency. A short summary of the theoretical model is outlined below. The paper by Martin and Rivers (2015) provides a more detailed discussion of the model.

The model is based on a representative consumer who has the ability to choose how much electricity to consume. It assumes that the consumer optimises electricity consumption in response to more information on price and efficiency of electricity provided by a single service that is differentiated by time. Perfectly informed, the consumer makes decisions that maximise utility. In contrast, the imperfectly informed consumer misinterprets the price and efficiency, and his/her consumption deviates from optimal levels. In this model, utility maximisation occurs under constraints related to demand for energy services, the consumption of electricity and the consumer budget constraint. The model does not account for a consumer who makes non-optimising decisions even with perfect information.

Experimental design

This section presents a case study on the implementation of time-of-use electricity rates and in-home real-time electricity feedback technologies. The results and analysis presented in this section are based on the paper by Martin and Rivers (2015), which provides a more detailed discussion.

The case study presents an evaluation of a natural experiment in which in-home electricity displays are rolled out quasi-randomly to about 7 000 households served by an electricity distribution company in Ontario, Canada. This section first describes the context in which the programme was offered. It then describes the empirical approach used for understanding the causal effect of real-time feedback on household electricity consumption. Finally, it presents the results of the analysis.

Context

Households within the service area of an Eastern Ontario local electric distribution company (EDC) were offered the opportunity to participate in “peaksaverPLUS”, a demand response programme. Upon agreeing to participate in the programme, the EDC activates a device on the home’s electric hot water heater that allows the utility to remotely reduce the electricity consumption of the water heater during certain high-demand periods of the year (for up to four hours at a time and only between May and October).7

It is important to emphasise that the pre-condition for programme participation is ownership of an electric hot water heater. Since there is a very strong correlation between owning an electric hot water heater and using electricity as the primary space heating energy source (i.e. baseboard heaters), it is likely that the vast majority of the households in the sample primarily use electricity for both space and hot water heating.8 The effect of real-time feedback on electricity consumption shown estimated in the report should, therefore, be interpreted as the effect of feedback on households with electric heat and hot water. In addition, it is important to emphasise that households that participate in the programme are not randomly drawn from the population but instead select into the demand response programme. The statistical implications of this selection are addressed below but here it is important to emphasise that the results obtained in this paper reflect the subset of households with electric water heaters that select into a demand response programme. It is not clear how generalisable the results are to the full population since demographic information on households was not available for this study.

In-home display

In return for participating in the demand response programme, participating households received an IHD. The IHD is wirelessly connected to the house’s digital electricity meter (all Ontario households have been converted from analogue to digital electricity meters). It displays, in real time, the power consumption by the household in physical units (in kW), the current retail electricity price (in CAD/kWh) and the implied current expenditure on electricity (in CAD/day). It also shows the consumption of electricity over the previous 24 hours as well as over the previous month. Additionally, the IHD is equipped with an LED display, which glows a different colour depending on the current electricity price (e.g. green is off-peak; yellow is mid-peak; red is on-peak).

The IHDs were sent from the utility by mail to each participating household, with instructions for activation. The utility had already pre-paired each IHD with the electric meter at the residence so that upon receiving the IHD, the household could activate the device simply by plugging it into a standard electrical outlet (information on electricity consumption is then transferred wirelessly from the digital electricity meter to the IHD). The data indicates the date that the device was couriered to the customer, and this date is used as the start of the “treatment effect” associated with the IHD. It is important to note that there is no way of knowing if or when the consumer actually installs the IHD and so the effect that estimated throughout the report is an intent-to-treat effect, rather than a treatment-on-the-treated effect. The intent-to-treat effect is a lower bound on the treatment-on-the-treated effect.

Time-of-use electricity prices

In the Electricity Restructuring Act, 2004, the Ontario Energy Board (OEB) was mandated to implement a regulated price plan that included a TOU (Time of Use) pricing structure to more accurately convey the real costs of generation to consumers and to encourage customers to shift demand away from peak periods. Italy and Ontario are the only jurisdictions in the world to implement smart meters for all residential customers as well as an associated TOU pricing plan (Faruqui and Lessem, 2014). The roll-out of the smart meters and implementation of the TOU pricing plan were complete prior to the beginning of the period covered by this study.9

Ontario’s TOU pricing structure divides each hour into one of three blocks representing off-peak, mid-peak, or on-peak periods. Weekends and holidays are off-peak periods, as are the hours from 7 p.m. to 7 a.m. each weekday. In the summer, hours from 7 p.m. to 7 a.m. are off-peak each weekday. Hours from 7-11 a.m. and 5-7 p.m. are mid-peak, while hours from 11 a.m. to 5 p.m. are on-peak.10 In the winter, the daytime blocks are switched, such that peak periods are during the morning (7-11 a.m.) and evening (5-7 p.m.), while the mid-peak period is from 11 a.m. to 5 p.m.

The OEB adjusts TOU prices every six months in response to changes in electricity load as well as the profile of electricity generators in the province. During this period, real electricity prices have been trending upwards in Ontario.11 The ratio of peak/off-peak prices has changed slightly during the study period but has remained between about 1.5 and 2.12

The impact of IHDs on electricity consumption is estimated by making use of the staged roll-out of IHDs to electricity consumers. In particular, the impact of the IHD on electricity consumption is determined by comparing a household that has just received an IHD with the same household just before receipt of the IHD and controlling for unobserved confounders using households that are just about to receive an IHD as a control group. Both of these households are programme participants and so are likely similar in important respects (at minimum, both have electric hot water heaters and likely have electric space heaters, for reasons discussed in the prior section).

The research design imposes the assumption that households that are enrolled in the IHD programme early in the year are equivalent to those that are enrolled in the programme later in the year. The identification approach might be compromised if these two types of households are significantly different. There are two reasons to think that the assumption is likely to be valid. First, although the roll-out of IHDs is long enough to exploit it for empirical purposes, from a household’s perspective it is still relatively short; there is no reason to think that there is a significant difference between a household that enrols in a demand response programme a few months before another household. Second, the phased roll-out was in part a response to resource constraints at the utility and this provides a source of exogenous variation in adoption date that is exploited in the analysis.

In addition to these qualitative arguments that suggest the timing of the roll-out is exogenous, it is possible to provide quantitative evidence. To do this, observations of electricity consumption prior to any households receiving an IHD are used (IHD roll-out began in January 2013 and the data on electricity consumption starts in September 2012). A comparison between pre-programme electricity consumption in these households is used to determine if there is any difference between early adopting and late adopting households that could contaminate the estimated treatment effects.

To operationalise this, the data are split into two groups: early adopters and late adopters. Households are split according to the median date of adoption (21 August 2013). Pre‑programme electricity consumption in early and later adopters is then compared. Daily electricity consumption is clearly very similar between early-adopting and late-adopting households in the pre‑treatment period, following the qualitative arguments above. Additional evidence on this point comes from a regression of pre-programme electricity consumption on the date of IHD receipt. There is no statistical relationship between these two variables. Martin and Rivers (2015) provide more formal statistical evidence that pre-treatment consumption in early-adopting and late-adopting households are identical.

Results and discussion

The main finding of the analysis is that households reduce electricity demand by an average of about 3% once they receive an in-home display (the result is “statistically” different from zero at conventional significance levels).13 As described below, this reduction in electricity demand is maintained for at least several months following receipt of the device. The result is estimated based on a comparison of daily household electricity consumption within the same household before and after receiving an IHD, and controlling for temporal shifts in electricity consumption experienced by all households in the small service area of the utility, for example, due to holidays or changes in weather. The average effect is similar when controlling for household-by-season fixed effects, and also when hourly rather than daily data is employed for estimation. Tables showing this result and other relevant tests are provided in Rivers (2018) and more detail is available in Martin and Rivers (2015).

Temporal variation in household response

The hourly metering data produced by smart meters in Ontario allows for the possibility of breaking down the response by hour of the day.

For most hours of the day, the hourly effect of an IHD is very similar to the average effect over all hours of the day. fact, the hourly effect is only statistically different from the average effect for 2 hours of the day: the hour up to 7 a.m. and the hour up to 7 p.m.

This is notable for two reasons. First, the stability in the effect across all hours of the day suggests that households are not dynamically responding to real-time information over all hours in the day but rather are permanently adjusting behaviour in a way that generates a relatively uniform response across hours of the day. Second, the result provides preliminary evidence that changes in the time-of-use price within a day are not driving major changes in the response to the IHD (a point explored further below). In particular, the largest reduction in electricity demand is in the hour leading up to 7 a.m., which is on off-peak price. The smallest response is in the hour leading up to 7 p.m., which is on mid‑peak or on-peak price, depending on the season.

The study also observes the response according to the season of the year. Unlike the relatively flat response over the course of the day, there is a distinctive seasonal effect of the IHD on consumption. In particular, during the spring and summer months, there is a small and statistically insignificant impact of the IHD on electricity consumption. In contrast, during the winter and fall heating seasons, the IHD causes a roughly 4% reduction in the demand for electricity. This is suggestive evidence that households respond to the IHD in part by reducing the demand for space heating. Further evidence on this point is provided in the following section.

Household response by outdoor temperature

To provide additional evidence on the mechanism through which households are responding to the IHD, an additional regression was performed to examine how the hourly outdoor temperature interacted with the IHD dummy variable. Temperature is divided into equally-sized bins that span the range of temperatures in the data set, in order to enable visualisation of the potentially non-linear relationship between outdoor temperature and the impact of the IHD. This enables the possibility of establishing whether the presence of an IHD produces a differential response at different outdoor temperatures and helps to establish the mechanism by which households respond to the IHD.

The analysis shows that when the outdoor temperature is low, the presence of IHD results in a significant reduction in electricity consumption. In particular, at an outdoor temperature of -8°C or below, household electricity consumption is reduced by 4% to 6% due to the presence of an IHD (with the larger reduction at lower temperatures). The effect of the IHD on electricity consumption declines near-monotonically as temperature increases until the outdoor temperature is between 2°C and 7°C, at which point the IHD appears to have no effect on electricity demand. At temperatures above 17°C, there is weak evidence that the IHD reduces household electricity demand.

The analysis provides additional evidence that households respond to receiving an IHD by adjusting the thermostat setpoint. When temperatures are extremely cold, suggesting a large heating load, the effect of the IHD is larger. Similarly, when temperatures are extremely hot, there is some evidence that households with an IHD consume less energy than households without. In contrast, when temperatures are less extreme, such that there is little requirement for heating or cooling, the IHD does not appear to have any effect on electricity consumption.

It is possible to make an estimate of the shift in thermostat setpoint that would give rise to the effects observed in this study. To provide an estimate, the HOT2000 building simulation model that is developed by Natural Resources Canada, is used to simulate household heating requirements for different indoor temperature setpoints and outdoor temperatures. Based on model simulations with different indoor setpoints and based on the weather in Ontario, a 1°C reduction in the indoor temperature setpoint is estimated to reduce building energy consumption by about 4% during the heating season. The United States Department of Energy suggests a reduction of 0.6°C (1°F) is sufficient to reduce energy consumption by about 3%.14 These two studies suggest that a possible interpretation of the findings here is that households responded to an IHD by reducing the thermostat setpoint by about 1°C or slightly less.

Persistence of household response

To establish whether IHDs can be (part of) a cost-effective strategy to encourage households to reduce their electricity consumption, it is critical to know whether the impact of the IHD on consumption is transitory or persistent. Prior studies have shed some light on this (e.g. Gans et al., 2013; Houde et al., 2013) but many have not followed households for sufficiently long periods to observe whether the response is transitory or persistent.

To test the persistence of household responses, a regression is performed in which the IHD treatment dummy is interacted with a variable indicating the number of weeks since the IHD has been received. As above, bins are used to enable the identification of a possibly non-linear response. The results show that the effect of the IHD appears to increase over time, from roughly 2% upon initial receipt to around 4% after households have had the IHD for several months. Importantly, the effect of the IHD on electricity consumption does not appear to be transitory but rather appears to increase fairly steadily over the five-month period over which households are observed following receipt of the IHD. Although it is again not possible to pin down the precise mechanism explaining this response, it is plausibly linked to the increased salience of electricity consumption under IHD adoption, leading consumers to shift habits in a persistent manner, for example by acquiring more energy efficient appliances or by permanently adjusting thermostat setpoints.

Real-time feedback and time of use electricity prices

The programme under study is in Ontario, a province with time-of-use electricity pricing for nearly all residential customers. During the period covered by the data, on-peak prices for electricity were about twice as high as off-peak prices. During the period covered by the data and the rollout of IHDs to customers, the time-of-use tariff for residential households changed twice: once in Spring 2013 when it switched from the winter to summer tariff structure, and once in Fall 2013 when it switched from summer to winter structure. At each switch, prices for electricity were also increased for each block of electricity. It is possible to use these two tariff changes to identify the impact of changes in electricity prices on electricity consumption, both for households with an IHD as well as for households that have yet to receive an IHD. As explained above, it is theoretically not clear whether households with real-time feedback should respond more or less to a price change than households without real-time feedback.

For households without an IHD, the estimated short-run elasticity of electricity demand with respect to price is between -0.17 and -0.37, depending on the time period and the model specification. This is well within the range of other estimates of the short-run elasticity for electricity demand (Lijesen, 2007). For households with an IHD, the estimated elasticity of demand is about -0.2 and does not change appreciably across different time periods. Interestingly, this implies that the elasticity estimated for households with an IHD is sometimes higher and sometimes lower than that estimated for households without an IHD. It is therefore not possible based on this study to conclude that real-time feedback appreciably increases or reduces the sensitivity to time-of-use electricity prices.

Conclusion

This chapter summarises the empirical literature on the effect of real-time feedback on electricity consumption decisions, develops a simple analytical model that describes how an optimising consumer responds to real-time feedback and presents results from an empirical study based on a large-scale roll-out of IHDs to electric utility customers in Canada. Taken together, the results suggest that real-time feedback is likely to cause consumers to reduce electricity consumption. The results also suggest that consumers are unlikely to shift patterns of electricity consumption (i.e. the timing of electricity demand throughout the day) substantially in response to receiving an IHD if differences in prices throughout the day are modest. Finally, the results suggest that households respond to receiving real-time information on electricity price and consumption in part by making one-time decisions of a durable nature – such as adjusting thermostat setpoints, hot water heater settings or upgrading the energy efficiency of household equipment – rather than by responding in real-time to the real-time information.

References

Allcott, H. (2011a), “Rethinking real-time electricity pricing”, Resource and Energy Economics, Vol. 33(4), pp. 820-842.

Allcott, H. (2011b), “Social norms and energy conservation”, Journal of Public Economics, Vol. 95(9), pp. 1082-1095.

Allen, D. and K. Janda (2006), “The effects of household characteristics and energy use consciousness on the effectiveness of real-time energy use feedback: A pilot study,” in Proceedings of the ACEEE Summer Study on Energy Efficiency in Buildings, pp. 1-7.

Borenstein, S., M. Jaske and A. Rosenfeld, “Dynamic pricing, advanced metering, and demand response in electricity markets”, Center for the Study of Energy Markets.

Commission for Energy Regulation (2011), “Electricity smart metering customer behaviour trials (CBT) findings report”, Technical Report, Commission for Energy Regulation.

Delmas, M.A. et al. (2013), “Information strategies and energy conservation behavior: A meta-analysis of experimental studies from 1975 to 2012”, Energy Policy, Vol. 61, pp. 729-739.

Department for Business, Energy and Industrial Strategy (2016), Smart Meters: Quarterly Report end June, Great Britain, Statistical Release: Experimental National Statistics, www.gov.uk/government/uploads/system/uploads/attachment_data/file/555971/2016_Q2_Smart_Meters_Report_FINAL.pdf.

Eurostat (2016), Private Households by Household Composition, 2006-2015 (Nb of Households in 1 000 and % of Household Types), Database, http://ec.europa.eu/eurostat/statistics‑explained/index.php/File:Private_households_by_household_composition,_2006‑2015_(number_of_households_in_1_000_and_%25_of_household_types)_new.png#file.

Faruqui, A. and L. Akaba (2014), “The impact of dynamic pricing on residential and small commercial and industrial usage: New experimental evidence from Connecticut”, Energy Journal, Vol. 35(1), pp. 137-160.

Faruqui, A. and S. George (2005), “Quantifying customer response to dynamic pricing”, The Electricity Journal, Vol. 18(4), pp. 53-63.

Faruqui, A. and N. Lessem (2014), Year Two Analysis of Ontario’s Roll-out of TOU Rates, Technical Report, The Brattle Group.

Faruqui, A. and S. Sanem (2010), “Household response to dynamic pricing of electricity: A survey of 15 experiments”, Journal of Regulatory Economics, Vol. 38(2), pp. 193-225.

Faruqui, A., S. Sanem and A. Sharif (2010), “The impact of informational feedback on energy consumption – A survey of the experimental evidence”, Energy, Vol. 35(4), pp. 1598-1608.

Fischer, C. (2008), “Feedback on household electricity consumption: A tool for saving energy?”, Energy Efficiency, Vol. 1(1), pp. 79-104.

Gans, W., A. Alberini and A. Longo (2013), “Smart meter devices and the effect of feedback on residential electricity consumption: Evidence from a natural experiment in Northern Ireland”, Energy Economics, Vol. 36, pp. 729-743.

Gleerup, M. et al. (2010), “The effect of feedback by text message (SMS) and email on household electricity consumption: Experimental evidence”, The Energy Journal, pp. 113-132.

Harding, M. and C. Lamarche (2016), “Empowering consumers through data and smart technology: Experimental evidence on the consequences of time-of-use electricity pricing policies”, Journal of Policy Analysis and Management, Vol. 35(4), pp. 906-931.

Houde, S. et al. (2013), “Real-time feedback and electricity consumption: A field experiment assessing the potential for savings and persistence”, The Energy Journal, Vol. 34(1), p. 87.

Jessoe, K. and D. Rapson (2014), “Knowledge is (less) power: Experimental evidence from residential energy use”, American Economic Review, Vol. 104(4), pp. 1417-38.

Karkkainen, S. (2004), Results from the EFFLOCOM Pilots, Technical Report, EU/SAVE.

Lijesen, M.G. (2007), “The real-time price elasticity of electricity”, Energy Economics, Vol. 29 (2), pp. 249-258.

Martin, S. and N. Rivers (2015), “Information provision, market incentives and household electricity consumption: Evidence from a large-scale field trial”, Technical Report, SSRN Working Paper, No. 2703947.

Metering.com (2017), “Enedis issues paper on status of AMI rollout project”, https://www.metering.com/reports/linky-smart-meter-enedis/.

Nilsson, A. et al. (2014), “Effects of continuous feedback on households electricity consumption: Potentials and barriers”, Applied Energy, Vol. 122, pp. 17-23.

Office for National Statistics (2016), “Statistical bulletin: Families and households in the UK: 2016”, www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/families/bulletins/familiesandhouseholds/2016 (last accessed in May 2017).

Office of the Auditor General of Ontario (2014), Annual Report, www.auditor.on.ca/en/content/annualreports/arreports/en16/v2_111en16.pdf.

Rivers, N. (2018), “Leveraging the Smart Grid: The Effect of Real-Time Information on Consumer Decisions”, OECD Environment Working Papers, No. 127, OECD Publishing, Paris, http://dx.doi.org/10.1787/6ad4d5e3-en.

Schleich, J. et. al. (2013), “Effects of feedback on residential electricity demand – Findings from a field trial in Austria”, Energy Policy, Vol. 61, pp. 1097-1106.

Smart Energy International (2017), “Enedis issues paper on status of AMI rollout project”, Smart Energy International, https://www.smart-energy.com/resources/reports-and-white-papers/linky-smart-meter-enedis/ (accessed on 27 March 2019).

Sulyma, I. (2008), “Experimental evidence: A residential time of use pilot,” Notes, Vol. 100(6.33), pp. 6‑33.

U.S. Energy Information Administration (2017), Table 10.10. Advanced Metering Count by Technology Type, Database, www.eia.gov/electricity/annual/html/epa_10_10.html (last accessed in May 2017).

Uribe-Pérez, N. et al. (2016), “State of the art and trends review of smart metering in electricity grids”, Applied Sciences, Vol. 6(3), p. 68.

Westskog, H., T. Winther and H. Sæle (2015), “The effects of in-home displays: Revisiting the context”, Sustainability, Vol. 7(5), pp. 5431-5451.

Xu, P. et. al. (2015), “Case study of smart meter and in-home display for residential behavior change in Shanghai, China”, Energy Procedia, Vol. 75, pp. 2694-2699.

Zhou, S. and M.A. Brown (2017), “Smart meter deployment in Europe: A comparative case study on the impacts of national policy schemes”, Journal of Cleaner Production, Vol. 144, pp. 22-32.

Notes

← 1.  This chapter draws on Rivers (2018). Some of the research presented in that article was conducted by Steve Martin and the author of the report, which is also reported in the paper by Martin and Rivers (2015).

← 2.  This report focuses only on the direct impacts of smart meters on household electricity consumption. Smart meters also confer other benefits, such as improved ability by the electricity distribution company to detect electricity theft, improved ability to manage electricity flows on the electricity network, and reduced costs for electricity meter reading. These benefits do not accrue to the household directly, and are not the focus of this report.

← 3.  While smart meters record electricity consumption on an hourly or higher frequency, they typically relay that information to the electric distribution company on a lower frequency, such as daily.

← 4.  It is important to note that there have been a number of efforts to supply households with feedback on their electricity consumption that do not rely on real-time consumption information (e.g. Allcott, 2011b; Fischer, 2008). This document focuses on real-time feedback.

← 5.  Prices that change over the course of a season are also possible with analogue meters.

← 6.  In the case examined by Allcott (2011a), consumers were exposed to the day-ahead forecast of the wholesale price.

← 7.  For a household to be eligible for the programme, it must have an electric hot water heater. During the two-year period covered by the data, the utility only implemented load control events for two four-hour periods. Because this report focuses on the response to real-time feedback and not the response to the load control interventions, days on which loads are controlled are removed from the sample. Load control events were declared by the Ontario Power Authority on 24 June and 16 July 2013, from 2 to 6 p.m.

← 8.  Using a separate data set – the US Residential Energy Consumption Survey – shows that single family households with electric hot water heaters have roughly an 80% probability of also using electricity for space heat.

← 9.  See: www.ontarioenergyboard.ca/OEB/Industry/Regulatory+Proceedings/Policy+Initiatives+and+Consultations/Smart+Metering+Initiative+(SMI)/Smart+Meter+Deployment+Reporting. Roll out of smart meters to Ontario residential customers was monitored by the Ontario Energy Board in monthly progress reports until June 2012, at which point 99% of eligible customers had smart meters installed. Some Electric Distribution Companies in Ontario implemented time of use pricing as early as 2009 and all EDCs had implemented time of use pricing by 2012. This study uses data from the period September 2012 to 2014.

← 10.  Summer is defined as the months from May to October.

← 11.  Independent Electricity System Operator, Ontario.

← 12.  As in most utilities, the cost of the electricity commodity is just one component of the electricity bill received by the customer. Customers also pay a charge for delivery of electricity, as well as a regulatory charge, debt retirement charge and a service charge. Some of these additional charges scale with usage, while others are fixed. In total, the all-in electricity price varies less over price blocks than the electricity commodity charge.

← 13.  The identification of this effect leverages the quasi-experimental roll-out of the IHDs to consumers. Martin and Rivers (2015) estimate this effect by performing a regression of the log of electricity consumption on a dummy variable indicating whether the household has received an IHD. The variable is equal to one starting on the date when the household received the IHD and equal to zero on days prior to the receipt of the IHD. In their regression, they control for factors that remain constant for a household and for temporal shocks affecting electricity demand. The paper provides further details on the controls and fixed effects involved in the analysis.

← 14.  See http://energy.gov/energysaver/thermostats.