This chapter provides an overview of the origins, theoretical background and results from online experiments that examined how regulators can foster elements of a culture of safety in the energy sector with the use of behavioural insights. The chapter concludes with guidance on how regulators can apply these lessons to future projects on enhance safety.
Behavioural Insights and Organisations
Fostering Safety Culture
1. Overview and guidance for policy makers
Abstract
Human behaviour can be a mystery. Policy makers often assume that humans make “rational” decisions and build policy based on this model. However, social context and behavioural biases systematically influence people’s abilities to act rationally, and often counter to these models of rational decision making.
The core concepts are easy to relate to: we have limited ability to attend to all aspects of life, our choices are shaped by our context, we often have difficulty making sense of the complex world around us and we have bounded willpower that limits our determination to stick with decisions over time (OECD, 2019[1]). These behavioural issues are increasingly being considered when analysing, designing, implementing and evaluating policies.
Policymakers around the world are turning to the field of behavioural insights (BI) for a clear methodology that generates evidence on how people “actually” behave. The field of BI is fundamentally about analysing policy problems based on lessons derived from the behavioural and social sciences, collecting evidence of which solutions works and which do not, and applying these findings to improving the outcomes of public policy.
A key feature of the BI methodology is its empirical approach, driven by experimentation and piloting. This approach also allows policy makers to experiment and test solutions at smaller scale to determine the best course of action. As a result, governments can test multiple policy solutions with the beneficiaries at once before committing to resources to implementing full policy solutions that may need to be revisited later (OECD, 2019[1]).
This approach has been used around the world to make policies better (OECD, 2017[2]). This publication presents the findings and guidance for policy makers from new research into the application of BI to fostering elements of safety culture in the energy sector. The findings have enhanced the practical understanding of both organisational behaviour of regulated entities, as well as pushed the frontier towards a more frequent integration of the field of BI and safety. This chapter presents an overview of the project, a summary of the key findings, and actionable guidelines for policymakers.
Applying behavioural insights in public policy: from individuals to organisations
Behavioural insights is a powerful tool for understanding human biases in decision making across a number of policy areas and disciplines (see Box 1.1). As BI started to be used more systematically by governments around the world, its effectiveness at addressing policy issues had to be demonstrated. Some governments created “nudge units” within the public administration to demonstrate positive return on investment within a short timeframe. Famously, the UK Government’s Behavioural Insights Team (BIT) – the first such unit – began with a mandate that required them to demonstrate a tenfold return on investment in two years (Halpern, 2015[3]).
A scan of the use of BI by governments, across sectors, shows that most applications have been aimed mostly at correcting individual-level biases in decision-making (OECD, 2017[2]). This has enabled governments to simultaneously impact thousands, or in some cases millions, of people at once.
However, policy problems are not solely confined to problems of individual behaviour – the behaviour of organisations are omnipresent in most sectors of the economy. This is true, for example, for economic regulation that has the purpose of regulating and supervising the behaviour of regulated entities that are organisations, rather than individuals. As the field of BI evolves to tackle more complex policy issues, the widespread perception is that BI can and should go beyond the study of individual-level decision processes for higher impact (OECD, 2018[4]). Governments around the world are starting to study how the field of BI can be broadened to include meso- and macro-level applications aimed at affecting the decisions of groups of people, including organisations.
Box 1.1. The “ABCD” of behavioural insights in public policy
Have you ever missed an important appointment because you had too much to do and forgot? Given up on properly filling out a form because it was too cumbersome and hard to understand? Driven a little above the speed limit because all the other drivers were going fast as well?
These are everyday examples of how context and behavioural biases can influence decision-making.
A better understanding of human behaviour can lead to better policies. Drawing from rigorous research from behavioural economics and the behavioural sciences, behavioural insights (BI) can help public bodies understand why citizens behave as they do and pre-test which policy solutions are the most effective before implementing them at large scale. By integrating BI into policymaking, policymakers can better anticipate the behavioural consequences of policies and ultimately design and deliver more effective policies that can improve the welfare of citizens.
The “ABCD” framework focuses on four key drivers of behavioural policy problems: Attention, Belief Formation, Choice and Determination (see Table 1.1).
Table 1.1. The ABCD framework with examples
Source: (OECD, 2019[1]), Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit, Paris, https://doi.org/10.1787/9ea76a8f-en.
Impacts of organisational behaviour on public policy and regulation
Discovering how to change organisational behaviour is important for regulatory policy in two key ways. First, economic regulatory authorities1 act as “market referees” by protecting market neutrality and fostering competition to ensure access to and quality of public utilities (OECD, 2016[5]). This function necessarily means interacting with firms who deliver utilities to consumers. While BI has been applied successfully to improving consumer choices in regulated markets (Lunn, 2014[6]); (OECD, 2016[5])), regulators also have the remit to engage with regulated entities and their behaviour. There is an opportunity to discover behaviourally-informed solutions that can both improve the functioning of firms in regulated sectors, as well as the interaction between the regulator and regulated entities.
Second, there is the possibility of large economic gains from improving the functioning of firms in regulated sectors. It is well-recognised that the organisational culture within firms has a powerful effect on both the performance and long-term success of those entities (Cameron and Quinn, 2011[7]). Commonly, less prominent failures in organisational culture can lead to pervasive economic impacts. (Cialdini, Petrova and Goldstein, 2004[8]) argues that the negative consequences of poor organisational culture reduce not only the health of the organisation but of the people inside them as well. This can lead to poor performance and productivity stemming from a poor reputation, mismatching of employee values leading to absenteeism, low job satisfaction, high turnover, and theft, as well as increased surveillance from both the regulator and management that foster a resistance to control and a negative perception of the entire system. In one estimate, employee theft in the United States alone led to about USD 40 billion in losses annually, which is nearly 10 times the cost of all street crimes including burglary (US Chamber of Commerce, in (Thau, Pitesa and Pillutla, 2014[9]).
Applying BI to organisational behaviour: origins of the safety culture project
At first glance, it may seem as though applying behavioural insights to organisations is different to individuals. However, research consistently argues that while the venue for decisions has changed, choice-making is still fundamentally the same. Research presented in this publication (see Chapter 2) makes the case that organisations are made up of individuals and group decisions are affected by individual input. They are susceptible to many of the cognitive and behavioural tendencies at work that we are influenced by outside of work, although the nature of groups and organisations can alter those tendencies. Organisational literature has highlighted the fact that perceptions about an organisation are strongly related to perceptions of individuals – such as supervisors – in that organisation (Eisenberger et al., 2002[10]). Psychological research on teams has shown similarities with individual psychology and behaviour (Wieber, Thürmer and Gollwitzer, 2012[11]); (Schoemaker, 1993[12]).
Therefore, where groups function similarly to individuals, many of the same behavioural insights could apply. Where groups and organisations function differently than individuals, it will be important to tailor any intervention to the behavioural insights that are unique to them (Sunstein and Hastie, 2014[13]); (2015[14]) that businesses and organisations can be influenced using behavioural insights as has been documented in reviews of their application to public policy (OECD, 2017[2]) and in reviews of the literature on strategy and management (Stingl and Geraldi, 2017[15]); (Gavetti et al., 2012[16]). Sample case studies can be found in Box 1.2.
Box 1.2. Case studies on the application of BI to changing organisational behaviour
As part of the early scoping work on applying BI to organisational behaviour, (Shephard, 2017[17]) produced a series of cases that either directly involve organisations or can inform interventions at an organisational level. Some examples include:
Social norms: In the United Kingdom, the Chief Medical Officer sent a letter to select general practices notifying them that they were prescribing more antibiotics than 80% of the practices in their NHS Local Area Team. As a result, 73 406 fewer prescriptions were made across 791 intervention practices, compared to the control group of 790 practices. While the letters were sent to individuals, organisational level impacts were noted (Hallsworth et al., 2016[18]).
Implementation intentions: For groups, the formation of implementation intentions can counteract weaknesses of group dynamics by increasing their likelihood of selecting an optimal decision and decreasing the likelihood of escalating commitment to a failed course of action. (Thürmer, Wieber and Gollwitzer, 2014[19]) demonstrated in a laboratory setting that adding an implementation intention to a group resulted in that group being more efficient and more likely to choose the best alternative.
Combining top-down and bottom-up accountability: An RCT in India compared 60 schools in which each teacher was given a camera that students would use to take a time-stamped photo of the teacher and other students at the beginning and end of each day. The teachers’ salary was linked to their attendance. This combination of a classic incentive with bottom-up social pressure from students resulted in teacher attendance rates increasing from 58% in control schools to 78% in treatment schools (Duflo and Hanna, 2005[20]).
Priming identity and ethical salience: In the USA, the Social and Behavioural Sciences Team (2015) worked with the General Services Administration (GSA) to test a change in the online reporting used by vendors to report their sales, which was used to calculate their Industrial Funding Fee paid to the GSA. A signature box was placed at the beginning stating “I promise that the information I am providing is true and accurate”. The amount of fees collected from the vendors rose in a single quarter by USD 1.59 million). The effect dissipated in future quarters; however, the initial increase in receipts remains a substantive finding.
Reference class forecasting: Ample behavioural research has shown how the escalation of commitment, the planning fallacy and over-confidence can lead to decision-making and forecasting errors. (Flyvbjerg, 2006[21]) found that the average inaccuracy in cost forecasting was 44.7% for rail, 33.8% for bridges and tunnels, and 20.4% for roads. Usage of rail was underestimated by 51.4% and 9.5% for roads. Examples using these as “reference cases” demonstrated how forecasting can be at least partially debiased or used as a due diligence process to stop projects with a high likelihood of cost overruns and underperformance before they start.
Source: (Duflo and Hanna, 2005[20]), “Monitoring works: Getting teachers to come to school”, NBER Working Papers, No. 11880, National Bureau of Economic Research, www.nber.org/papers/w11880; (Flyvbjerg, 2006[21]), “From Nobel Prize to project management: Getting risks right”, Project Management Journal, Vol. 37/3, pp. 5-15, https://www.pmi.org/learning/library/nobel-prize-project-management-risks-2545; (Hallsworth et al., 2016[18]), “Provision of social norm feedback to high prescribers of antibiotics in general practice: A pragmatic national randomised controlled trial”, The Lancet, Vol. 387/10029, pp. 1743-1752, http://dx.doi.org/10.1016/S0140-6736(16)00215-4; (Shephard, 2017[17]), “Applying behavioural insights to organisations: global case studies”, EC-OECD Seminar Series on Designing better economic development policies for regions and cities, 10 May 2017, Paris, https://www.oecd.org/cfe/regional-policy/Shepard_Applying-Behavioural-Insights-to-Organisations_Case-Studies.pdf; (Social and Behavioral Sciences Team, 2015[22]), Annual Report, Office of the President National Science and Technology Council, Washington, DC.; (Thürmer, Wieber and Gollwitzer, 2014[19]), “A self-regulation perspective on hidden-profile problems: If-then planning to review information improves group decisions”, Journal of Behavioral Decision Making, Vol. 28/2, pp. 101-113, http://dx.doi.org/10.1002/bdm.1832.
Fostering elements of safety culture in the energy sector: Overview and key results for Canada, Ireland, Mexico and Oman
The OECD Network of Economic Regulators (NER) brings together over 70 economic and network regulators from across the world that operate in different network sectors, such as communications, energy, transport and water. The NER allows regulatory agencies to share their experiences, challenges and innovative solutions to regulatory policy design and delivery and to enhancing their performance.
In the energy sector, regulators have found clear evidence that many high profile incidents have occurred – at least in part – due to poor organisational behaviour, including safety culture. These include, for instance, the nuclear safety system failure at the Fukushima Daiichi plant in Japan in 2011 (OECD/NEA, 2013[23]), and behavioural and cultural factors that contributed to the BP Deepwater Horizon oil spill in 2010 (Reader and O’Connor, 2014[24]); (Corkindale, 2010[25]). Smaller scale accidents, which are inherently more frequent, can also be partly motivated by poor safety culture. For instance, one study analysed 15 major petrochemical accidents between 1980 and 2010 noted that poor safety culture contributed to 12 of the 15 accidents (Fleming and Scott, 2012). Major incidents are usually the product of a simple (i.e. minor) decisions or behaviours and system complexity. Safety culture is about changing these simple decisions and behaviours to prevent incidents (both minor and major).
Prevention of incidents like these strongly supported further research into safety culture in regulators and regulated entities. On the one hand, regulators have a role to play in advancing safety culture across the industries that they oversee. This includes a duty to lead the way by understanding their own organisational culture, as well as that of the regulated entities that they oversee. On the other, changing organisation culture requires an understanding of the barriers and opportunities for changing elements of safety culture within regulators and regulated entities (see Chapter 3).
In both cases, regulators may look to behavioural insights as a powerful tool for understanding and enhancing organisational behaviour to foster elements of safety culture in the energy sector. Through the NER, the OECD has been able to develop an extensive knowledge of BI applied to regulatory policy and regulation across a number of sectors and countries (Lunn, 2014[6]); OECD 2016, (OECD, 2017[26]). Within this context, energy regulators from four countries – the Canadian Energy Regulator2 (CER, formerly the National Energy Board of Canada, NEB), Canada, the Commission for the Regulation of Utilities (CRU), Ireland, the Agency for Safety, Energy and Environment (ASEA), Mexico, and the Authority for Electricity Regulation (AER), Oman – joined together for the first-ever comparative application of BI to safety culture.
Overview of the project
In 2017, the OECD began work with the project partners investigating the application of BI to changing the culture of organisations, with a focus on strengthening a culture of safety in the energy sector. While research notes the lack of a concise internationally agreed upon definition of safety culture in the literature (see Chapter 3), it is noted that – at its core – safety culture is about the organisation’s values, beliefs, norms, practices, competencies and behaviours related to safety (TRB, 2016[27]). There is also a clear understanding that safety culture impacts safety performance (Smith, Emma and Wadsworth, 2009[28]).
It is widely acknowledged that regulators have a central role in promoting safety culture. However, a number of reviews and commissions established to investigate accidents have highlighted the importance of the responsibility of the industries in combination with regulators to promote a safety culture, acknowledging limits of regulation and that regulators cannot create a safety culture on their own (TRB, 2016[27]). There are also a number of barriers and enablers to safety culture, which are discussed further in Chapter 3.
Figure 1.1. Theoretical interaction of behaviour and safety culture
More broadly, safety culture is related to BI as behaviours for safety are an essential and visible component of safety culture (OECD, 2019[29]). Safety culture itself is the result of a series of safety innovations over the last 70 years that have focused on standards, compliance frameworks and systems to predict risk. Embedded within these innovations have been behavioural aspects of safety culture, including awareness raising, discussions on safety, clear boundaries for behaviour and required consideration of safety-critical behaviour. Behaviours can also have a feedback loop to affecting culture; that is, as a behaviour becomes an entrenched and visible part of an organisation, it could influence the culture of that organisation as well. In this sense, BI and safety culture could interact as a bi-directional cycle (see Figure 1.1). Context can also be an external influence and impact both safety culture and behaviours.
What was found lacking, however, was the issue of “person centred problems”, and research notes that the application of BI is worth considering as possible enablers of safe behaviour and decision making (Krause, Sellers and Horn, 2001[30]). While BI is not a “silver bullet” – or cure-all – that can create safety culture, it can be used in a complementary fashion for improving workplace safety (DeJoy, 2005[31]). Noting this limitation, the intention of this project was to see how BI could amplify efforts to promote safe behaviour via individuals within organisations. The project was implemented in two phases: first a set of comparative online experiments with regulators and regulated entities in all four countries, and second a follow up experiment with registered electrical contractors and gas installers in Ireland. The results are presented in Chapter 4.
Testing BI applied to safety culture in a two-phased approach
This study sought to explore the application of behavioural insights to fostering elements of safety culture through two phases: First, experiments testing the same behavioural insights in Canada, Ireland, Mexico and Oman at the same time amongst regulators and regulated entities in the energy sector and, second, a follow up study in Ireland further testing one behavioural insight from Phase 1 as well as improving inspections reports. A full breakdown of the project partners and participants can be found in Figure 1.2.
Figure 1.2. Overview of project partners and sample participants
Phase 1: Experiments in Canada, Ireland, Mexico and Oman
The first phase brought together energy regulators and regulated entities in Canada, Ireland, Mexico and Oman to test the potential importance of BI to fostering elements of a safety culture. Three key BI principles were tested, based on a review of the literature and qualitative research to understand the specific contexts with regulators and regulated entities:
Messenger: People process the same information differently depending on who they received it from ( (Clark et al., 2013[32]); (Eckel and Gintis, 2010[33]).
Social benchmarking / feedback: People pay attention to feedback in almost everything they do and often cannot adjust their behaviour without it. Providing a benchmark can reduce mistakes and make the consequences of decisions more salient.
Social norms: People tend to survey their social and physical environment for attitudinal and behavioural cues and they care deeply about what their neighbours do. This is especially true when their neighbours belong to their same social in-group.
It should be noted that feedback and social norms are subsets of the larger domain of social influences. This domain notes that humans are social creatures and look to the behaviour of others for information on how they themselves should behave (Bicchieri, 2005[34]); (Goldstein, Cialdini and Griskevicius, 2008[35]).
The experiment consisted of an online questionnaire that assessed respondents perceptions of safety culture, presented them with vignettes3 presenting three scenarios (new guideline regarding Personal Protective Equipment, reports of a bad lost-time injury rate, and a situation where a supervisor asks a worker to carry out a task in an unsafe manner), asked qualitative follow ups, and collected basic unidentifiable demographic data. A complete overview of the methodology for the phase one experiments is published in (OECD, 2019[29]).
Phase 2: Experiments with registered electrical contractors and gas installers in Ireland
The second phase extended this work to small-scale regulated entities in Ireland under the auspices of the Commission for the Regulation of Utilities (CRU). The CRU oversees the Safety Supervisory Bodies (SSBs), which administer the safety schemes for Registered Electrical Contractors (RECs) and Registered Gas Installers (RGIs). These are, respectively, the Safe Electric Register of Electric Contractors (hereafter Safe Electric) and the Register of Gas Installers of Ireland (RGII).
Phase 2 continued to explore some of the findings from Phase 1, particularly the messenger effects that had the strongest effect for Ireland in Phase 1. This was done by looking at messages from a trainer (a high status peer), from the regulator, and from SSBs with whom RECs and RGIs register and whom also mediate messages coming from the regulator. This behavioural insight was chosen because it was the most effect behavioural insight for Ireland in Phase 1.
Phase 2 investigated new areas as well, particularly in regards to applying BI to improving the effectiveness of inspections and the feedback associated with such inspections. The experiments investigated three behavioural insights (in addition to messenger effects) through adjustments made to the forms used to conduct inspections and provide feedback after inspections designed to improve the salience of safety:
Implementation Intention: prompting the commitment to a plan of action with a specific time
Primacy: changing the order of inspections to focus attention on areas by placing them first
Personalisation: the use of specific information about the recipient and their needs
Similar to Phase 1, the survey experiments were carried out through an electronic questionnaire sent out to the regulated entities, staff of the SSB and the staff of CRU. A complete overview of the methodology and results are located in Chapter 4 of this publication.
Summary of the results and lessons learned for Canada, Ireland, Mexico and Oman
The two phases of this study demonstrate that behavioural insights can influence attitudes and behaviours of organisations related to safety culture. This section summarises these results by each phase and provides lessons learned for the project partners that could help inform the development and delivery of regulations in the future.
Full details on the results of both phases of the experiments can be found in Chapter 4.
Phase 1
The first phase analysed the results from two perspectives: a comparative perspective that sought to tie together findings across countries, and a country-by-country perspective highlighting how the behavioural insights worked for each project partner.
Overall, the results reveal interesting insights for each country, and regulatory policy in general. Further research for Canada, Mexico and Oman, like that conducted in Phase 2 in Ireland, would help to better understand the effect of behavioural biases in their given context, and which behavioural solution is best suited in response to achieve desired outcomes.
A key finding from the comparative analysis of the data was on the perception of safety culture. Within regulated entities, frontline workers had the lowest perception of safety while this perception increased as the management level increased. Regulators had the lowest perception of safety of all survey participants. These results make sense given that frontline workers are more likely to witness unsafe activities first hand, and the role of regulators is to be analysing their sector for potential risks. This does not suggest any influence on the capabilities, expectations or actions of regulators or regulated entities, as this was not part of the study and evidence would be needed to support such a claim. What this finding does suggest is the need to understand the audience and how they perceive the issues at hand, which may influence their understanding and alter the way in which messages are delivered and received.
With regards to the three behavioural insights tested, it was difficult to draw conclusions from a comparative perspective as each insight operated differently in each country context. Broadly, it was found that the source of messages mattered, highlighting the need to ensure messages about safety are transmitted through the appropriate channels. Some feedback was overall better than no feedback, but the results were inconclusive as to which type of feedback was best. Finally, social norming was perceived as the least effective across the sample, with the exception of descriptive norms in Mexico.
Detailed findings, as well as suggested focus areas are included in Table 1.2 and OECD (2019[29]).
Table 1.2. Key comparative findings and suggested areas of focus by country
Note: For Canada, due to a small sample size (n=28) and a 6 to 1 response rate by regulators over regulated entities, these results should be seen as indicative and validated by additional research. For Ireland, a larger but still relatively small sample size (n=92) means the results should be interpreted with caution.
Phase 2
Behavioural insights were effective in improving the likelihood that respondents believe safety-related non-conformances would be resolved. More specifically, the use of behavioural insights was effective in the relatively new regulatory scheme for registered gas installers and was effective with organisations (compared to self-employed, individual installers).
Results showed that inspections can be improved by placing areas of more common non-conformance first in the inspection form and combining that primacy with an implementation intention (planning prompt) to designate a specific time to resolve the problem. When sending notices about safety problems (‘non-conformances’), the study found that combining personalisation reduced the responsiveness of individual installers while improving the responsiveness of firms. Meanwhile, as was the case with inspections, the use of implementation intentions was effective at increasing the likelihood that respondents believe that safety problems will be addressed.
Finally, it was found that trainers are particularly effective messengers for installers because they share a social community with recipients while also having higher status and more expertise.
Suggested areas of focus include:
Feedback: Test the use of inspections for the provision of safety related feedback combined.
Primacy: In both inspections and communications, carefully select the material that appears first as this is most likely to receive the most focus.
Implementation intentions: Wherever possible, prompt organisations to make explicit, concrete implementation intentions for when, where, and how they will follow through on the desired behaviour. Such implementation intention planning prompts can be included in existing forms and communications to regulated entities.
Messenger: Explore the use of trainers as messengers among small-scale regulated entities and test the effects through a field experiment.
Guidance for policy makers: Towards a toolkit for fostering safety culture
The following section provides guidance on how regulators can apply behavioural insights to enhance safety. These are presented in the form of action points and implications, informed primarily by the results of Phase 1 and Phase 2 studies alongside the relevant literature. Taken together, this guidance forms the beginning of a “toolkit” on how BI can help foster elements of safety culture in the energy sector.
The results presented in this report point to the contextual nature of the insights gathered. This guidance proposes a series of steps to regulators around the world for applying BI to issues with safety culture that can be adapted to their setting. This comes with a word of caution: more research is needed in each given context to understand what works, and what does not, so that the methods and approaches can be fine‑tuned for maximum effectiveness. This guidance provides a beginning that should be updated and amended as new evidence is gathered.
This section offers five lessons learned that provide guidance for policy makers looking to apply BI to safety culture:
1. Utilise a multi-staged process for applying behavioural insights to safety regulation.
2. Understand the group you are trying to influence and adapt the behavioural insights accordingly.
3. Apply and test combinations of the six highlighted behavioural insights principles most effectively.
4. Recognise the limitations of the state of knowledge in this field and its impact, especially given the fact that this is frontier work.
5. Continue research in areas identified by this study so that energy regulators and researchers can move forward and expand the collective knowledge of this sector.
Utilise a multi-staged process for applying behavioural insights
Findings from both phases of research demonstrate the value of regulators exploring new, behaviourally informed strategies to address issues around safety culture. This study found that behavioural insights can be applied to organisations through the people within them. This work highlights both how behavioural insights influence elements of safety culture and how the level of safety culture can influence which behavioural insights are most effective. The varied responses within and between the four countries highlight the importance of testing behaviourally science applications and of avoiding the temptation to apply the same techniques in all cultural contexts.
This work followed a multi-stage blueprint that can inform continued work by regulators to apply behavioural insights to improve safety. By following the four stages, regulators can increase the efficiency and effectiveness of the application of behavioural insights in this sector and others. :
1. Follow a clear methodology that invests time upfront to fully scoping the behavioural problem, such as the BASIC approach (see Box 1.3). This includes the combined use of literature reviews, structured conversations with policy communities such as the NER, and discussions with experts to gain a nuanced understanding of the safety problem, its behavioural drivers, and what behaviourally-informed policy solutions are feasible.
2. Qualitative research is valuable for identifying behavioural barriers and solutions and to reveal differential perspectives between groups that may highlight biases and avenues of potential intervention. Working with researchers in the academic domain and including the participation of regulators and regulated entities is advisable when conducting such research. If different groups of respondents perceive different problems (e.g. regulators and regulated entities), such disagreements may indicate productive areas for applying behavioural insights (e.g. biased understandings of the frequency of specific safety behaviours that could be corrected through the use of descriptive norms). The use of semi-structured interviews and focus groups that include open-ended questions are useful research tools in this situation.
3. Survey experiments allow regulators to test a menu of behavioural interventions in the context of hypothetical behaviour. This enables lowering costs, conducting more tests, and generating results more quickly. As demonstrated through the two-phases of studies in Ireland, several survey experiments can be done to target new groups and explore behavioural insights in more depth and with new entities. The findings can then help regulators select final insights to test through a subsequent field experiment.
4. Field experiments are effective as a final step, as they can be informed by previous work to increase the probability of their success. These experiments should be designed to measure observed (or reported) behaviour. Whenever possible, it is advisable to use existing administrative data sources to decrease the cost burden and enable continuous testing and improvement using current systems. Ideally, it would only be after the results of such a field experiment that a procedure change or policy change could be made.
Specific suggestions for future research that regulators may wish to explore to fill gaps in current knowledge of the use of behavioural insights to improve safety are found at the end of this chapter.
Box 1.3. The BASIC approach to behavioural insights
In 2019, the OECD released the BASIC framework, which is an acronym for the 5-step process of applying BI throughout the policy cycle. The steps are:
1. Behaviour: Identify and better understand the policy problem, including its structural and behavioural drivers.
2. Analysis: Review the available evidence to identify the behavioural drivers of the problem.
3. Strategies: Translate the analysis into strategies to address the behavioural problem.
4. Intervention: Design and implement an intervention to test which strategy best addresses the problem.
5. Change: Develop plans to scale what works into full policy solutions, sustain behaviour and communicate results.
OECD (2019[1]) contains a detailed description of the framework in a toolkit that provides the policy makers with best practice tools, methods and ethical guidelines for conducting BI projects from beginning to end.
Source: OECD (2019[1]), Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit, Paris, https://doi.org/10.1787/9ea76a8f-en.
Understand the group you are trying to influence
Different groups experience different barriers to action. As such, different behavioural insights should be tested with different subgroups. This study has identified specific dimensions of difference that should be considered to inform the targeting of behavioural insights and contribute to elements of safety culture—especially its behavioural elements. Key dimensions to consider are: 1) the perceived safety culture of the organisation and sector; 2) the role and status of individuals within organisations; and 3) the type and size of the organisation.
1. Perceived level of safety culture of organisations and sector
The perceived level of safety culture of respondents influenced their responses to the tested behavioural insights. Overall, respondents who perceived a higher safety culture were more responsive to the behavioural insights as indicated by a more optimistic perception of the effect that the behavioural insights would have on safety-culture related behaviours. More importantly, the types of behavioural insights that were most effective differed between those who perceived a low safety culture compared to those who perceived an average or high safety culture. On the low end, the use of norms was more effective. On the higher end, messengers and feedback were more effective (see Table 1.3).
This suggests the importance of the relative position of the respondent in terms of their perception of safety culture and safety behaviour. If they perceive safety culture as low, then they may be influenced to improve when they are shown a descriptive norm that is superior to their perceived position. Meanwhile, among those who already have a positive perception of safety culture, it can be more effective to provide feedback from a person or organisation with higher perceived status and expertise regarding safety.
Implications for policy makers:
1. Measure a proxy for safety culture and then investigate ways to use this to differently target regulated entities with low and high safety culture.
2. Test the use of behaviourally informed feedback for organisations with high safety culture.
3. Test the use of descriptive norms among organisations with low safety culture.
Table 1.3. Target BI based on perceived safety culture
Note: P1 = Phase 1. All groups in Mexico reacted more strongly to the norm vignettes; however, as a country and sector, Mexico had a low safety culture overall (5.10) and therefore the use of positive descriptive norms may be effective country-wide in a low perceived safety culture context.
2. Role and status of individuals within organisations
The status of individuals within an organisation made a difference for targeting behavioural insights. Specifically, the most effective messenger differed by the relative status of the targeted regulated entity (see Table 1.4).
Taken together, the evidence from Phase 1 and Phase 2 suggest that the messenger should be of higher status while being as socially close as possible – essentially the ideal messenger is one step ‘above’ but not far removed. As such, small-scale entities were more responsive to messages from their trainers, front-line staff of large-scale entities more responsive to messages from their managers, and managers were more responsive to messages from the regulator.
Implications for policy makers:
1. Map the different levels of social status among regulated entities and regulatory bodies, ideally keeping this to a small number such as three or four tiers of social groups with varying status levels.
2. Test the matching of message recipients with messengers who are from the closest social subgroup with higher social status.
3. Highlight what the messenger holds in common with the recipient and why they are an expert related to safety.
Table 1.4. Target BI based on status of recipient
Note: P1 = Phase 1, P2 = Phase 2. The most robust finding was that Registered Gas Installers (RGIs) in firms were more responsive to trainers as messengers. However, the Registered Electrical Installers (RECs) in firms were more responsive to messages from the CRU.
3. Type and size of the organisation
Results suggest that the type and size of organisation should also inform how to target behavioural insights (see Table 1.5). Differences in entities responsiveness to behavioural insights include differences between regulators and regulated entities in both phases, differences between sectors in Phase 2 (gas and electric), and differences between the size of the regulated entities (comparing Phase 1 and Phase 2 results).
Encouragingly, regulated entities were more responsive to behavioural insights than regulators. This suggests that although the staff of a regulator may be sceptical about the added utility of behavioural insights, the regulated entities are likely to be more responsive, and therefore regulators should avoid premature scepticism. The smaller regulated entities in Phase 2 were more responsive to messenger effects and this was also mentioned in the qualitative responses in Phase 1. Meanwhile, the provision of feedback and the use of managers and regulators as messengers were more effective for larger regulated entities.
Implications for policy makers:
1. Test different behavioural insights with regulated entities of different sizes.
2. Changing the messenger may be particularly effective for smaller regulated entities.
3. Providing feedback from managers and regulators to larger regulated entities can be effective.
Table 1.5. Target BI based on organisation size and type
Note: P1 = Phase 1, P2 = Phase 2. These findings only held for the gas installers not electricity installers.
Targeting specific subgroups will increase the effectiveness of the application of behavioural insights to change factors related to safety culture. Not only does such targeting influence the choice of which behavioural insights to apply (see the three tables above), it also changes how those behavioural insights should be constructed. For example, each group would have a different high-status messenger that is socially proximate, feedback would be benchmarked to the relevant subgroups were possible, norms would be associated with that subgroup (or a higher performing, socially similar subgroup), the behaviour to list first (primacy) and how to leverage implementation intentions needs to align with the most important actions (and gaps) for that subgroup, and personalisation should be tailored accordingly. Therefore, these findings suggest the importance of using targeted behavioural insights – with targeting based on baseline safety culture, status, and organisation.
Finally, the apparent discrepancies in findings across this research can be resolved by considering the above target groups and the importance of the relative position of respondents and entities. Most notably, for messenger effects, peers were the least effective in Phase 1, but peer-trainers were the most effective in Phase 2. However, in Phase 1, the “peer” messenger had no heightened status and the message was “by word of mouth”. Meanwhile, in Phase 2, the size of the entities were smaller, the “peer‑trainers” were higher status (not direct peers), and the hypothetical message was delivered in writing. So a peer with higher status seems effective. This is also supported by the fact that the regulator was not the most effective messenger for frontline staff in Phase 1, but rather managers who were socially closer but still of higher status.
Apply the behavioural insights principles most effectively
This study (Phases 1 and 2) explored the effects of six different behavioural insights principles on hypothetical behaviours related to safety culture of regulated organisations within the energy sector. The following provides the findings with regards to each of the six and how they might be productively applied by regulators to improve safety culture.
1. Norms
Norms are a central part of any conceptualisation of safety culture. Where there is data on a desirable safety behaviour across an energy sector with multiple regulated entities, descriptive statistics about people’s actions can be generated to provide evidence-based descriptive norms of what entities are doing. These descriptive norms can then be sent to regulated entities who are performing below the norm to encourage them to improve and therefore move closer to the norm.
Where data do not exist or where there are few entities, the use of descriptive norms is not possible. However, where the data exist, they can be an effective means to improve the safety behaviour of underperforming entities. The results do not support sending descriptive norms to entities who are performing better than the norm. The study also did not find evidence to support the use of injunctive norms that simply state what the behaviour should be.
2. Messengers
Regulators choose who communicates regulations and recommendations to regulated entities. Those entities must choose who to communicate those messages to their management and staff. Each of these messages has a messenger, and the social proximity and social status of that messenger vis-à-vis the recipient matters. Who the messenger is can increase or decrease the likelihood that a safety related directive or recommendation is noticed or enacted.
This study recommends segmenting potential message recipients by social group and status and then assigning messengers who are as socially close to the recipient subgroup as possible while having higher status. The use of different messengers can and should be combined with a consistent underlying message – even if the framing differs, the content should be consistent. The results suggest avoiding informal communication methods and the use of direct peers as messengers unless they have a higher status (for example, due to being trainers).
3. Feedback
Feedback is key part of any learning and continuous improvement process, including safety. Where there is no feedback given to regulated entities on safety, such a practice could be institutionalised within the regulatory framework. In addition to creating new opportunities for feedback, regulators can productively make use of existing interactions and communications to provide feedback. One example, is providing behaviourally informed feedback during and after inspections.
Because this study does not find direct evidence that a particular mode of feedback is more effective, it is recommended to combine feedback with one of the other behavioural insights such implementation intentions. Providing feedback is particularly significant for regulated entities who are performing above the norm across desired safety behaviours, but in those cases it should not be paired with descriptive norms due to the risk that they will decrease their exceptional efforts when they see that they are exceeding the norm.
4. Primacy
The first item in a list, or the first activity in a plan, often gets more attention than latter items. This insight can productively be applied to the design of safety inspections and the ordering of content in safety-related communications from regulators. This insight can help ensure that sufficient attention is paid to address weak safety areas by placing them first in inspections and communications.
Periodically the ordering of such actions and communications could be adjusted based on shifting priorities – as the needs change, the order may need to change. However, such dynamic changes should be done carefully, should be tested, and should not be done too frequently. Frequent changes to the order of procedures and communications may confuse those it is meant to help.
5. Implementation intention
Making a concrete, time-bound plan to take an action to improve safety can increase the likelihood of that safety behaviour being implemented. This is the power of implementation intentions, and it can be productively linked to multiple behaviours. This insight can be particularly useful in supporting the initial choice to act and the continued determination to do so (the CD of the ABCD framework, presented in OECD (2019[1]). As a result of this, implementation intentions can be productively combined with behavioural insights that focus more on the attention (A) and belief (B) of regulated organisations and their constituent staff members.
As far as possible, implementation intentions should be as specific as possible and should be linked to times, places, and routines of existing importance for the entities in order to be most effective. Due to the effectiveness of implementation intentions, it is particularly important that the prompted safety action be informed by evidence.
6. Personalisation
The personalisation of safety directives or recommendations can be an effective way to increase the likelihood that organisations pay attention to the message. Personalisation can range from ensuring that a message uses the name of each recipient organisation, to messages embedded with specific safety-related data, requirements, and recommendations that are unique to the organisation receiving the message.
This study finds that such personalisation can be productively deployed for small-scale regulated firms and previous research has found personalisation to be effective among individuals with regard to different contexts and topics. However, the results do not support personalisation being used for self-employed individuals who are classified as regulated entities due to the negative effects found during our Phase 2 study. More generally, the potential negative reaction of recipients who have privacy concerns should be weighed against personalisation.
Table 1.6. Six behavioural insights and implications for policy and practice
Note: P1 = Phase 1, P2 = Phase 2.
Acknowledge the limitations
These findings and implications should be tempered by the limitations of our studies. On the one hand, this work was frontier research into an area of regulatory policy that is relatively understudied. On the other hand, due to the nature of the experimentation, the sample sizes and results varied significantly. Taken together, additional research would be needed to research further how many of the behavioural insights function in relation to safety and confirm details of the findings presented above.
Concretely, some limitations of this study include:
The complexity of the concepts, contributing factors, and effects of safety culture and safety behaviours.
Behavioural insights do not constitute a “silver bullet” – or cure-all – and need to be used as part of a holistic approach to enhancing safety culture and improving safety. Regulators should consider the relative role of behavioural insights vis-à-vis traditional regulatory tools and structural changes to sectors and organisations to improve safety.
Further research is needed into actual behaviours, as opposed to the hypothetical behaviours captured as part of the survey experiments conducted in both phases of this study. In fact, this study has yet to move to the final stage of testing observed behaviour through focus groups (which are commonly used in the study of safety culture) or field experiments, as recommended above as part of a multi-staged process for applying BI. As such, these studies should be followed by field experiments.
Feedback, norms, and personalisation used within the studies were all fabricated to ensure consistency, and future research should also try to develop such behavioural insights using real data from the relevant organisations and sectors.
Small sample sizes in some countries limited the ability to generate findings for specific subgroups of respondents. This particularly limited our ability to draw out inferences for Canada in Phase 1 and from analysing differences for newly registered entities (less than one year since registration) in Phase 2.
Findings from Phase 2 are specific to the context of Ireland and therefore should be treated with caution when extrapolating from the findings to other contexts and cultures.
While direct causal inferences can be generated due to the randomised controlled trial design, overall implications drawn from across the studies and subgroups are more speculative. This is inevitably the case when trying to draw out policy and practice implications from a controlled study.
Build on the results with future research
We suggest the following avenues for future research to be explored jointly by regulators and researchers. Such research will help advance our understanding of this new and important tool for advancing safety among regulated entities. The areas for future research are divided into research on the direct effects of behavioural insights and research into the mechanisms of those effects.
1. Improving our understanding of direct effects
Conducting survey experiments in new country-contexts that attempt to replicate and extend the findings presented here.
Conducting survey experiments on the effects of using norms, feedback, and personalisation that is linked to actual safety data across the sector and within each targeted regulated entity.
Conducting survey experiments that combine the most effective behavioural insights identified in this work into new combinations, for example combining feedback and implementation intentions.
Conducting field experiments to test if the impacts identified in our survey experiments translate to observed safety-related behaviours in the real world.
2. Improving our understanding of the mechanism behind effects
To better understand how relative status and social group changes the effectiveness of messenger effects, various combinations of these relative positions should be tested to determine the best safety messengers. This could be done through a survey experiment.
To better understand how perceptions of safety culture interact with behavioural insights, a survey experiment should be designed in which perceptions of safety culture are first measured, and then those with high, average, and low safety culture should be stratified before randomising each strata into subgroups for testing the effectives of different behavioural insights among groups with different perceptions of safety culture.
References
[34] Bicchieri, C. (2005), The grammar of society: The nature and dynamics of social norms, Cambridge University Press, Cambridge, http://dx.doi.org/10.1017/CBO9780511616037.
[7] Cameron, K. and R. Quinn (2011), Diagnosing and Changing Organizational Culture : Based on the Competing Values Framework, Jossey-Bass, San Francisco.
[8] Cialdini, R., P. Petrova and N. Goldstein (2004), “The Hidden Costs of Organizational Dishonesty”, MIT Sloan Management Review, Vol. 45/3, pp. 67-73, https://sloanreview.mit.edu/article/the-hidden-costs-of-organizational-dishonesty/ (accessed on 26 February 2020).
[32] Clark, J. et al. (2013), “Evaluating the message or the messenger? Implications for self-validation in persuasion.”, Personality & social psychology bulletin, Vol. 39/12, pp. 1571-84, http://dx.doi.org/10.1177/0146167213499238.
[25] Corkindale, G. (2010), Five Leadership Lessons from the BP Oil Spill, https://hbr.org/2010/06/five-lessons-in-leadership-fro (accessed on 19 March 2020).
[31] DeJoy, D. (2005), “Behavior change versus culture change: Divergent approaches to managing workplace safety”, Safety Science, Vol. 43/2, pp. 105-129, http://dx.doi.org/10.1016/j.ssci.2005.02.001.
[20] Duflo, E. and R. Hanna (2005), “Monitoring Works: Getting Teachers to Come to School” November, pp. 1-47, http://dx.doi.org/10.3386/w11880.
[33] Eckel, C. and H. Gintis (2010), “Blaming the messenger: Notes on the current state of experimental economics”, Journal of Economic Behavior and Organization, Vol. 73/1, pp. 109-119, http://dx.doi.org/10.1016/j.jebo.2009.03.026.
[10] Eisenberger, R. et al. (2002), “Perceived supervisor support: Contributions to perceived organizational support and employee retention.”, Journal of Applied Psychology, Vol. 87/3, pp. 565-573, http://dx.doi.org/10.1037//0021-9010.87.3.565.
[21] Flyvbjerg, B. (2006), “From Nobel Prize to project management: getting risks right”, Project Management Journal, Vol. 37/3, pp. 5-15, https://www.pmi.org/learning/library/nobel-prize-project-management-risks-2545.
[16] Gavetti, G. et al. (2012), The Behavioral Theory of the Firm: Assessment and Prospects, http://dx.doi.org/10.1080/19416520.2012.656841.
[35] Goldstein, N., R. Cialdini and V. Griskevicius (2008), “A Room with a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels”, Journal of Consumer Research, Vol. 35/3, pp. 472-482, http://dx.doi.org/10.1086/586910.
[18] Hallsworth, M. et al. (2016), “Provision of social norm feedback to high prescribers of antibiotics in general practice: A pragmatic national randomised controlled trial”, The Lancet, Vol. 387/10029, pp. 1743-1752, http://dx.doi.org/10.1016/S0140-6736(16)00215-4.
[3] Halpern, D. (2015), Inside the nudge unit: how small changes can make a big difference, Penguin Random House UK, London.
[30] Krause, T., G. Sellers and C. Horn (2001), “Moving to the 2nd generation in behaviour-based safety”, IChemE Symposium Series, Vol. 148, https://www.icheme.org/media/10202/xvi-paper-57.pdf (accessed on 2 March 2020).
[6] Lunn, P. (2014), Regulatory Policy and Behavioural Economics, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264207851-en.
[29] OECD (2019), Delivering Better Policies Through Behavioural Insights: New Approaches, OECD Publishing, Paris, https://dx.doi.org/10.1787/6c9291e2-en.
[1] OECD (2019), Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit, OECD Publishing, Paris, https://dx.doi.org/10.1787/9ea76a8f-en.
[4] OECD (2018), OECD Regulatory Policy Outlook 2018, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264303072-en.
[2] OECD (2017), Behavioural Insights and Public Policy: Lessons from Around the World, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264270480-en.
[26] OECD (2017), Driving Performance of Mexico’s Energy Regulators, The Governance of Regulators, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264267848-en.
[5] OECD (2016), Protecting Consumers through Behavioural Insights: Regulating the Communications Market in Colombia, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264255463-en.
[36] OECD (2014), The Governance of Regulators, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264209015-en.
[23] OECD/NEA (2013), The Fukushima Daiichi Nuclear Power Plant Accident OECD/NEA Nuclear Safety Response and Lessons Learnt, OECD/Nuclear Energy Agency (NEA), Paris, https://www.oecd-nea.org/pub/2013/7161-fukushima2013.pdf (accessed on 19 March 2020).
[24] Reader, T. and P. O’Connor (2014), “The Deepwater Horizon explosion: non-technical skills, safety culture, and system complexity”, Journal of Risk Research, Vol. 17/3, pp. 405-424, http://dx.doi.org/10.1080/13669877.2013.815652.
[12] Schoemaker, P. (1993), “Strategic Decisions in Organizations: Rational and Behavioural Views”, Journal of Management Studies, Vol. 30/1, pp. 107-129, http://dx.doi.org/10.1111/j.1467-6486.1993.tb00297.x.
[17] Shephard, D. (2017), Applying behavioural insights to organisations: Global case studies, OECD; European Commission, https://www.oecd.org/cfe/regional-policy/Shepard_Applying-Behavioural-Insights-to-Organisations_Case-Studies.pdf.
[28] Smith, A., D. Emma and J. Wadsworth (2009), Safety culture, advice and performance, Report for IOSH Research Committee, http://www.iosh.co.uk/researchanddevelopmentfund, (accessed on 5 November 2019).
[22] Social and Behavioral Sciences Team (2015), Annual Report, Executive Office of the President National Science and Technology Council, Washington, D.C., https://sbst.gov/download/2015%20SBST%20Annual%20Report.pdf.
[15] Stingl, V. and J. Geraldi (2017), “Errors, lies and misunderstandings: Systematic review on behavioural decision making in projects”, International Journal of Project Management, Vol. 35/2, pp. 121-135, http://dx.doi.org/10.1016/j.ijproman.2016.10.009.
[14] Sunstein, C. and R. Hastie (2015), Wiser: Getting Beyond Groupthink to Make Groups Smarter, Harvard Business Press, https://store.hbr.org/product/wiser-getting-beyond-groupthink-to-make-groups-smarter/2299 (accessed on 5 November 2019).
[13] Sunstein, C. and R. Hastie (2014), “Making Dumb Groups Smarter”, Harvard Business Review 12(2014), pp. 90-98, https://hbr.org/2014/12/making-dumb-groups-smarter (accessed on 5 November 2019).
[9] Thau, S., M. Pitesa and M. Pillutla (2014), “Experiments in Organizational Behavior”, in Webster, M. and J. Sell (eds.), Laboratory Experiments in the Social Sciences: Second Edition, Elsevier Inc., http://dx.doi.org/10.1016/B978-0-12-404681-8.00019-4.
[19] Thürmer, J., F. Wieber and P. Gollwitzer (2014), “A Self-regulation Perspective on Hidden-profile Problems: If-Then Planning to Review Information Improves Group Decisions”, Journal of Behavioral Decision Making, Vol. 28/2, pp. 101-113, http://dx.doi.org/10.1002/bdm.1832.
[27] Transportation Research Board (ed.) (2016), Strengthening the Safety Culture of the Offshore Oil and Gas Industry, The National Academies Press, Washington DC, https://www.nap.edu/catalog/23524/strengthening-the-safety-culture-of-the-offshore-oil-and-gas-industry.
[11] Wieber, Thürmer and Gollwitzer (2012), “Collective Action Control by Goals and Plans: Applying a Self-Regulation Perspective to Group Performance”, The American Journal of Psychology, Vol. 125/3, p. 275, http://dx.doi.org/10.5406/amerjpsyc.125.3.0275.
Notes
← 1. OECD (2014[36]) defines a regulator as an entity authorised by statute to use legal tools to achieve policy objectives, imposing obligations or burdens through functions such as licensing, permitting, accrediting, approvals, inspections and enforcement, as well as complementary tools such as information campaigns. There are many types of regulators, including economic, financial, competition, consumer protection, technical and/or some that mix these roles. Economic regulators often focus on economic and competition/consumer protection functions in key economic sectors – such as energy, transport, telecommunications, and water – where the market is usually dominated by natural monopolies often under the responsibility of large state-owned or private corporations.
← 2. Formerly the National Energy Board of Canada (NEB), which was renamed to CER in August 2019.
← 3. Vignettes are often used in psychology experiments to describe a certain situation/context, ensuring control across study participants.