Delivering Better Policies Through Behavioural Insights

Chapter 5. Fostering a safety culture in the energy sector

Abstract

This chapter examines how regulatory policymakers can foster elements of a strong culture of safety in the energy sector. It presents the results of an online experiment with regulators and regulated entities in Canada, Ireland, Mexico and Oman that tests the effects of messenger, feedback and social norms on changing behaviours around safety.

Introduction

Individual-level errors, such as inattention, forgetfulness and procedural violations, have long been regarded as the principal factor behind safety incidents and disasters (Reason, 1990). However, the origins of safety risks can often be organisational rather than individual in nature. The conditions under which individuals work and how individuals perceive these conditions are fundamental drivers of safety performance.

Safety culture, defined as the set of “shared values, beliefs, attitudes, norms and practices related to safety within an organisation” (TRB, 2016; Cooper, 2000), is a key aspect of the larger organisational culture and a crucial element for the prevention of organisational accidents. There is evidence from an analysis of global incidents that a poor safety culture contributes to many high-consequence accidents, such as the nuclear safety system failure at the Fukushima Daiichi plant in Japan in 2011 and significant organisational and cultural lapses that contributed to the BP Deepwater Horizon oil spill in 2010.

Prevention of such incidents strongly supports further research on safety culture and action for regulators to better serve the public interest. Regulators have a role to play in working with regulated entities and sharing responsibility to advance safety culture across the industries that they oversee. A key aspect of this duty requires them to lead the way by understanding their own organisational cultures and behaviours, their cultural strengths and vulnerabilities, and how these factors can influence the broader safety and regulatory system. Equally important is to understand the cultural and organisational changes and behaviours of regulated entities and industry to ensure that a safety culture is effectively implemented and impact the sector as a whole.

The use of behavioural insights can provide a powerful tool to understand barriers and opportunities to develop a safety culture within regulators and in regulated entities in high-risk sociotechnical systems. There is growing interest amongst governments and policy institutions to utilise behavioural science to enhance organisational behaviour – from government institutions themselves to external organisations that interact with and are regulated by governments. The increased application of behavioural insights (BI) can help countries across the world to regulate better based on actual and not assumed behaviour.

The OECD has developed extensive knowledge in the application of BI, including in the work of regulatory agencies across sectors and countries (OECD, 2017). As part of the OECD cross-sector work on applying BI to public policy, a project was developed on BI and safety culture, under the auspices of the OECD Network of Economic Regulators and with the support of the Government of Canada (Natural Resources Canada and National Energy Board of Canada); Ireland’s Commission for Regulation of Utilities; Mexico’s Agency for Safety, Energy and Environment; and Oman’s Authority for Electricity Regulation.

The aim of the project was to conduct computer-based experiments with representatives from both regulators and regulated entities in high-risk industries, in order to test the application of BI to strengthen different dimensions of safety culture. The experiment was designed to capture participants’ perception of:

1. The perceptions of workers from regulators and regulated entities regarding safety culture in their respective areas (safety culture).

2. The extent to which different actors would respond to the potential application of behavioural insights to common safety problems (scenarios/vignettes).

This second element constitutes one of the main contributions of the study, as it is one of the first instances of the application of behavioural scenarios and vignettes (a common tool in behavioural and psychological research) to study safety improvement. This allowed us to address a number of novel research questions, including: Do employees of high-risk industries respond differently depending on whether a new safety guideline is introduced by a manager vs. a peer (messenger effect)?; When it comes to safe behaviour, do regulators and regulated entities workers pay more attention to what people do or the norm for what they ought to do (social norms)?; Do participants react more strongly when feedback about their organisations’ safety performance is provided in comparison with the performance of other organisations (social benchmarking/feedback)?

In addition, the study presents a uniquely international sample, which included regulators and regulated entities from diverse contexts: Canada, Ireland, Mexico and Oman. This allowed us to systematically examine whether there was heterogeneity in how different cultures respond to the same behavioural scenarios. The results indicated that there exist important international differences in perceptions of safety culture and not all behaviourally informed initiatives aimed at improving safety performance can work as effectively in all country contexts. Potential explanations are discussed in the results section of this chapter.

Equally, there is an asymmetry in the perception of safety culture by regulators and regulated entities; and this perception also differs at different levels of the organisational structure – with frontline workers generally perceiving the organisational culture to be less safe than managers consider it to be. Finally, the study suggests that not all behavioural principles are equally powerful in strengthening elements of safety culture; and notably, feedback on safety performance was perceived as the most impactful tool for behaviour change among the ones tested.

Overall, the study draws important implications on how context can enhance safety in high-risk industries and points at novel methodologies to study safety culture in different countries. We hope that the results of the experiments can inform guidance on fostering strong safety culture by:

• Describing practical approaches to assessing and addressing behavioural barriers and enablers to strengthening elements of safety culture.

• Understanding key decision points within regulators and regulated entities where action can be taken to strengthen elements of safety culture.

In the sections that follow, we summarise the context, methodology and findings of the study and provide a discussion of the challenges and opportunities for applying BI to safety culture.

Context and problem setting

What is safety culture?

There is no internationally agreed upon definition of “safety culture”, however, at its core safety culture is an aspect of the larger organisational culture, including the organisation’s values, beliefs, attitudes, norms, practices, competencies and behaviours related to safety (TRB, 2016; Cooper, 2000). In the literature, there is a clear understanding that safety culture impacts safety performance (Smith et al., 2009). For instance, one study analysed 15 major petrochemical accidents between 1980 and 2010, and noted that poor safety culture contributed to 12 of the 15 accidents (Fleming, 2012).

The measurement of safety culture is often conducted through employee surveys (Choudhry et al., 2007; Flin et al., 2000; Guldenmund, 2000) and is assumed to reflect employee perceptions of the organisation’s activities and policies regarding safety. Safety culture perceptions are then believed to predict safety-related behaviours, for example, reporting safety concerns and adhering to safety rules. While there is some debate in the literature regarding the degree to which safety culture perceptions map onto organisational accidents (Antonsen, 2009; Kvalheim, Antonsen and Haugen, 2016), it is generally assumed that where safety beliefs and behaviours are shared and positive, then the safety culture is considered strong (Christian et al., 2009; Clarke, 2000; Guldenmund, 2000; Health and Safety Commission, 1993; Singer et al., 2009).

It is widely acknowledged that regulators have an important role in promoting safety culture, however, a number of reviews and commissions of accidents have highlighted the importance of the responsibility of the industries in combination with regulators to promote a safety culture, acknowledging limits of regulation and that regulators cannot create a safety culture on their own (TRB, 2016).

Applying behavioural insights to foster strong safety culture

Safety culture is related to behavioural insights in many ways. Guldenmund (2000) describes a safety culture as a framework that has unconscious assumptions for safety at its core. Surrounding the core assumptions are beliefs and values which are, in turn, surrounded by observable safety culture elements (e.g. safety behaviour). Therefore, behaviours for safety are an essential and, importantly, a visible component of safety culture.

The safety culture paradigm itself is a recent development in a series of safety innovations going back 70 years. The first such innovation comprises a number of technological innovations, safety standards and compliance frameworks going back to the 1950s. Further innovation occurred with a focus on risk assessment and mitigation, and safety management systems in the 1980s. Industrial safety practice became focused on safety culture in the 2000s (Cox and Flin, 1998; Langford et al., 2000). Many tools have been developed in the safety culture space, including awareness drives, regular discussions on safety (e.g. “safety moments”), and of course rules that provide clear boundaries on behaviour. These tools and strategies may take many forms, but at their core, they attempt to apply health and safety rules to moderate behaviour group attitudes and resultant behaviours.

While many hazards have been eliminated, systems put in place to predict risk and efforts made to improve culture regarding safety, further attempts to maintain safety (“Safety 2”) require consideration of safety-critical behaviour (Krause et al., 2001). Furthermore, there remains the issue of the “person-centred problem”, which is not addressed by other methods (Reason, 1990; Talabi et al., 2015a; 2015b; Dekker, 2001; Holden, 2009). Where conscious behaviour according to unwritten cultural rules and written instructions are not enough for safety, additional manipulation of unconsciously made choices might be useful (Lindhout and Reniers, 2017). This is an opportunity to turn to a BI approach. BI would add to concepts of safety culture and behaviour-based safety by creating safety systems that enable safe behaviour and decision-making (Krause et al., 2001). The future of safety culture is likely to absorb BI, including concepts from psychology, behavioural economics, decision-making and influence.

Given the position that behaviours occupy within a wider safety culture conceptualisation – as visible artefacts of several various psychological and cultural processes – it is important to not overstate BI as a solution for all safety issues. BI interventions ought to form one component of a larger safety management system. Yet, some argue that change in culture and change in behaviour are complementary processes for improving workplace safety (DeJoy, 2005). Indeed, developing safety procedures in the absence of safety awareness/knowledge might be dangerous. Notwithstanding, health and safety practitioners must remember that unsafe behaviour and decision-making is not the only cause of accidents, though it may be the last link in a causal chain.

Noting this limitation of BI, the approach has typically been applied to existing policy levers (including regulation, incentives and information) to enhance their effectiveness for shaping behaviour. Past safety culture strategies have focused on the system, BI focuses on the individual. BI interventions would amplify efforts such as making safety behaviour easy, providing timely information to supplement safety behaviour, highlighting the social nature of safe behaviour and making safe behaviour decisions attractive.

Key BI principles in safety culture literature

Messenger

There exists a wealth of research demonstrating that we process the same information differently depending on who we received it from (Clark et al. 2013; Eckel and Gintis, 2010). In behavioural sciences, this is commonly referred to as “the messenger effect”. For example, individuals are more likely to believe a message when it comes from an expert or authority figure. They are also more likely to conform to the behavioural aspects of the message, decreasing violations overall. People also appreciate information more from people they have a positive feeling for or who are a bit like themselves, like in peer-to-peer sharing of knowledge or ingroup-outgroup dynamics.

Social influences

Humans are social creatures and look to the behaviour of others for information on how they themselves should behave (Bicchieri, 2006; Goldstein et al., 2008). There are several psychological mechanisms by which this occurs and we investigate two.

Social benchmarking

People pay attention to feedback in almost everything they do and often cannot adjust their behaviour without it. Providing a benchmark can reduce mistakes and make the consequences of decisions more salient. For example, providing pre- and post-shift hearing test results to workers can increase the use of hearing protection in subsequent shifts, overcoming what is known as the “present bias” (Zohar et al., 1980). However, we do not always get personal feedback on what we do and often look to the behaviour of others and the feedback they receive. This kind of social benchmarking can be useful in positioning our own behaviour.

Social norms (speaking up)

Evidence also suggests that people tend to survey their social and physical environment for attitudinal and behavioural cues and they care deeply about what their neighbours do. This is especially true when their neighbours belong to their same social in-group. Social norms act as a standard, informing individuals of what others think and do. We can be strongly influenced by our group memberships and there are many examples where individuals will automatically follow the behaviour of their peers to comply with social norms (Dolan et al., 2012). There are a number of implementations that exist to take advantage of this human trait, which have been applied to domain as diverse as public health, environmental behaviour, international development (Selinger and Whyte, 2011; Nolan et al., 2008; Thaler and Sunstein, 2008; de Groot-Mesken and Vlakveld, 2014; Goldstein et al., 2008a; 2008b; Haines, 1996; Hansen and Jespersen, 2013; Branson et al., 2012; Sunstein, 2006; Ariely et al., 2003; Oullier et al., 2010; Cialdini, 2005*; Avineri, 2014; Linkenbach and Perkins, 2003; 2005; Perkins et al., 2010). In the context of our research, we are particularly interested in applying social norms to create a norm of speaking up about unsafe practices at the workplace.

Reciprocity

The power of “reciprocity” for inducing co‑operation is also a well-replicated effect in the behavioural literature (Fehr et al. 2002; Rand et al., 2014). As social beings, people like to keep promises and reciprocate. Therefore, when people observe that others are taking the time to do things for them, they are more likely to continue that engagement. For example, behaviour change can be achieved by writing down a promise or commitment to do something (e.g. meeting a deadline). The examples of implementations that take advantage of commitments and reciprocity are numerous (Thaler and Sunstein, 2008; Breman, 2006; Karlan and Zinman, 2007; Hansen and Jespersen, 2013; Oullier et al., 2010).

Safety culture context of countries in the analysis

Each of the countries selected for the study possesses peculiar characteristics with regards to how they regulate the energy sector in their country. Below, we provide relevant information on the country context and a description of the basic features of each of the national regulator entities included in the analysis. These countries were selected based on convenient sampling and a more detailed description of the work and characteristics of each regulator can be found in ‎Annex 5.A.

Canada

The National Energy Board (NEB) is Canada’s energy and safety regulator. It makes regulatory decisions and recommendations that represent the interests and concerns of Canadians. In doing so, the NEB factors in economic, environmental and social considerations. The NEB oversees safety and environmental protection for the full life cycle of a project – from approval to construction, operation, abandonment and works with communities, sharing the goal of making energy infrastructure as safe as it can be. The NEB also monitors aspects of energy supply, demand, production, development and trade which the federal government controls. The NEB reports to parliament through the Minister of Natural Resources.

Mexico

Created in 2015, the Agency for Safety, Energy and Environment (Agencia de Seguridad, Energía y Ambiente, ASEA) is a technical regulator responsible for industrial and operational safety and environmental protection in Mexico’s hydrocarbons sector. It oversees activities throughout the hydrocarbons value chain, from exploration and extraction to midstream and downstream transformation, production and storage as well as distribution and retail at the petrol station level. ASEA’s aims are mapped under five dimensions (clients; industry; process; organisation and learning; and financial resources) and within each of these dimensions, there are medium- to long-term visions.

Ireland

The Commission for Regulation of Utilities (CRU) has responsibility for safety in the energy sector in three broad sectors:

• Regulating the activities of natural gas and liquid petroleum gas (LPG) undertakings with respect to safety under the Energy (Miscellaneous Provisions) Acts 2006 and 2012. This is carried out under the Gas Safety Framework, which covers shipping, supply, storage, transmission, distribution and use of natural gas, as well as certain specified LPG undertakings.

• Regulating upstream petroleum safety, including offshore safety under the Petroleum (Exploration and Extraction) Safety Acts, 2010 and 2015. This is carried out under the Petroleum Safety Framework (PSF) Requirement of the Petroleum Safety Framework (CER/16/023).

• Designation and oversight of the safety supervisory bodies charged with monitoring natural and liquid petroleum gas installers and electrical contractors doing domestic gas and electrical works respectively, with respect to safety under the Energy (Miscellaneous Provisions) Acts 2006 and 2012.

Oman

The Authority for Electricity Regulation (AER) is responsible for regulating the electricity sector and some aspects of the water sector. It was established by Article 19 of the Law for the Regulation and Privatisation of the Electricity and Related Water Sector promulgated by Royal Decree 78/2004 on 1 August 2004 and Amended by Royal Decree 59/2009 and 47/2013 (“the Sector Law”). The authority is a financially and administratively independent organisation and reports directly to the Council of Ministers. The authority’s duties under the Sector Law are to protect the interests of its three main stakeholders: electricity customers, electricity sector companies, and the Government.

Understanding the system – Shared responsibility, awareness of safety culture, complacency

When discussing the scope of the research project with members of the committee from the countries of study, a few key concepts were highlighted as most important. We define them below and describe the approach with which the study addresses them.

Shared understanding of responsibility between regulators and regulated entities

It is important for regulators and regulated entities to be on the same page regarding shared responsibilities for safety culture. Indeed, collaborative contexts (rather than adversarial ones) are likely to lead to safer environments. Safety culture is deemed “strong” when safety attitudes and perspectives are positive but also shared among staff. Thus, it is important that the regulator and regulated entities share a similar perspective on their shared responsibilities as well as the state of safety culture in their sector. We measure the extent to which safety culture perspectives are shared by asking both regulator workers and regulated entity workers to provide their perspective on safety culture in their sector. We describe the perspective as “divergent” when we are able to demonstrate a significant difference between these perspectives and “shared” when we cannot.

In practice, regulators and regulated entities have different perspectives on safety. Regulated entities, for example, have a very detailed understanding of safety in their organisation and relatively little understanding of safety in other organisations, whereas regulators have less of an understanding of any specific organisation’s safety (presumably less than the organisation itself does) but a good understanding of safety in across the sector. Differing perspectives, such as these, may impede a shared understanding of the situation and potentially cause conflict regarding shared responsibility.

Awareness of safety

Although the field of behavioural insights is more about shaping contexts than raising awareness, regulators have identified the importance of awareness of safety practices and the need to address it in their contexts. While it may be difficult to know all the ways in which safety can be compromised (this is the reason that prescriptive safety policies often fail), it is important for workers in safety-critical industries to exercise caution in their work and decision-making. Workers need to understand that there are many “known unknowns” as well as “unknown unknowns”, which are sometimes referred to as “black swans” (Taleb, 2007). This frame of thinking is an essential element of a strong safety culture. For example, safety management systems that are less prescriptive rely on the workforce having appropriate safety and risk awareness. We measure respondents’ awareness of safety culture with questions like “people in regulated entities understand how others’ jobs contribute to safety” and “voicing concerns about safety is encouraged”. While these questions are not usually considered psychometrically valid measures of safety culture awareness, they provide valuable insights into workers’ perception of their environment.

Complacency

Maintaining safe working environments is an ongoing challenge – employees need to always be vigilant of situations where safety might be compromised. But maintaining vigilance itself is a difficult task and can lead to fatigue. It is in situations like these that vigilance is most important. When operators have not faced a major incident in a number of years or when they are using what they understand to be safe technologies for the first time, complacency can negatively impact safety culture. We measure the presence and magnitude of complacency in participant entities with items like “everyone perceives that safety is their personal responsibility” and “people are committed to safety”.

Organisational structure, management and workers

There are studies documenting how understanding national cultural dimensions is important for safety culture (Mearns and Yule, 2009). There are several national cultural dimensions that have been documented to vary between nations: power distance, uncertainty avoidance, collectivism, masculinity-femininity and short-term orientation (Hofstede et al., 2010). These are understood to occur through national, cultural and educational institutions shaping shared values and behaviours with respect to the way nation-members think of and approach different cultural concepts.

Power distance, for example, refers to the way nation-members think of and approach hierarchies and power in interpersonal contexts. Uncertainty avoidance is the degree to which nation-members approach or avoid situations with uncertain social consequences. Collectivism is the tendency and acceptance for nation-members to act predominantly as members of a group or as individuals. Masculinity-femininity refers to a societal balance of “masculine” values (e.g. competitiveness, power) over “feminine” values (e.g. relationships, quality of life). Short-term orientation refers to the societal importance placed on the future (e.g. saving, adaptation) or past and present (e.g. respect for tradition, fulfilling social obligations).

There are documented relationships between these national cultural dimensions and elements of safety culture. For example, power distance has a negative relationship with safety culture: high power distances discourage speaking out and correction of superiors, create an unwillingness to challenge authorities and create asymmetric communication streams between management and frontline staff. Equally, uncertainty avoidance has a negative relationship with safety culture: high uncertainty avoidance restricts innovation, leads to more rigid rules and regulations, and leads to an over-reliance on procedures that cannot be applied to all contexts.

Collectivism also has a negative relationship with safety culture, as high collectivism means that group cohesion is prioritised over speaking up (or challenging group norms), thus increasing embarrassment for errors and self- or career-defensive behaviours. Masculinity is another cultural factor with a negative relation to safety culture: it can create competitive environments that obstruct collaboration. Finally, short-term orientation has a negative relationship with safety culture: high short-term orientation creates pressure for short-term gains at the expense of long-term planning and inhibits the development of safety from a holistic systems perspective.

From a policy standpoint, it may be important to acknowledge these differences amongst countries to understand the related differences in safety culture.

Methodology

Understanding Context and Fine-tuning the Design

A key step to applying BI is a detailed understanding of the context in each of the countries studied. This process began through the OECD Network of Economic Regulators (NER) where the country representatives involved in the project as well as the broader community of economic regulators could provide their inputs into the initial stages of the research agenda. Following these discussions, this project and themes were discussed by representatives from participant countries in a small setting to gain a detailed understanding of each in each country context. These themes were then explored in a scoping literature review and discussed with academics at the London School of Economics (LSE).

Following this initial scoping, contact points within each regulator were established and numerous informal discussions were conducted to explore the practical elements of the project as well as an understanding of safety culture, the potential application of BI and these three themes in a more nuanced way. These discussions helped to inform the selection of the BI principles studied (i.e. messenger, social benchmarking, reciprocity and social norms/speaking up). This was followed by a survey to each contact point to gain a more detailed understanding of specific behaviours that could be addressed in the research.

Following discussions with the contact points and the responses from the behaviour survey, informal focus groups were conducted with two or three representatives from each regulator and regulated entities. Overall, there were a total of eight informal focus groups. The aim of these discussions was to gain a detailed understanding of the safety culture and behavioural science principles we were planning to test, as well as detailed feedback on the safety culture and experiment questions themselves (e.g. suggestions on language and terminology, etc). Feedback was provided on various iterations of the questionnaires, often at multiple times. Hypotheses about the most impactful interventions in each experiment were also collected from representatives of regulators and regulated entities.

A number of academic experts and practitioners from the wider safety culture and BI communities were also engaged to gain feedback on the experiment design and application of BI principles. Ethical approval was obtained from the LSE and the researchers followed OECD principles of confidentiality and ethics. The study was also pre‑registered after data collection, but before data analysis, on the Open Sciences Framework (Tear and MacLennan, 2018).

Experimental design: Questionnaire on safety culture and behavioural scenarios

Once identified the main research questions and hypotheses, we designed a computer-based questionnaire and distributed it as a link to respondents, who completed it in their own time.

Administering the questionnaire

Emails were sent to respondents in regulators and to contact points within regulated entities, who then passed the questionnaire to frontline staff, managers and senior management, including contractors. The questionnaire was not sent to everyone who works in the regulator and certain individuals were excluded from the experimental sample (‎Annex 5.A). As we did not want to collect any personal/identifiable information, we wanted to ensure that the questionnaire was only sent to people who would be able to provide responses as we intended.

The email with the link to the questionnaire came from either a regulator email address or an OECD email address, with an initial email provided by the regulator to ensure that the message would not be lost as it would be received from an unfamiliar email address. In addition, the email was designed in line with the relevant literature on how to increase response to electronic questionnaires.

Language

For Canada, Ireland and Oman, the email and experiments were in English. For Mexico, they were both translated into Spanish. However, respondents had the option of responding to the questionnaire in English or Spanish. Feedback was provided from the contact points in Mexico (Mexico’s Agency for Safety, Energy and Environment and the Mexican Association of Hydrocarbons) who read through the translated documents and agreed with the translation.

Timeframe

The questionnaire was sent in an email in August 2018. Responses were collected for six weeks and a reminder message was sent in early September. This was particularly important as it was highlighted that some staff work on five-week rotations and likely would not receive the initial email.

Design

The experimental questionnaire that was sent to subjects was divided into four main sections: 1) demographics; 2) safety culture; 3) behavioural Scenarios; and 4) qualitative questions. The complete questionnaire can be found in ‎Annex 5.A.

1. Demographics

The aim of this section was to collect basic, unidentifiable data which could inform our analysis in the following sections. This section collected only the regulated entity/regulator of the respondent and what level they were in the organisation (i.e. frontline staff, manager, senior manager). In regulated entities, we additionally asked participants to specify whether they were a contractor. In initial discussions with senior contacts within study country regulators, it was discussed that regulated entity size, level of safety culture maturity and the level of contractors would likely impact safety culture. The information in this demographic section enables us to keep these areas under consideration when conducting the data analysis. We purposefully asked minimal questions in this section so that individuals could not be identified and their privacy would be protected. This was also crucial to ensure that participants felt they were not giving away a substantial amount of personal data and could, therefore, be honest in their responses.

2. Safety culture/climate questions

In discussions at the April 2018 NER as well as in scoping discussions with safety culture academics and contact points in the regulators of study countries, an emphasis was placed on understanding the extent to which views on safety culture differ between individuals in regulated entities and in regulators. Driven by this practical question and an understanding of the literature in this area, we opted to ask a number of related questions to respondents in both regulators and regulated entities.

We first asked participants to report how much they agreed with a number of questions on various dimensions of safety culture. The safety culture questions were derived from the existing literature (Reader et al., 2015) and slightly adapted to include a few novel questions on reciprocity between regulated entitles and the regulator. Examples of questions include: “On average, in regulated entities, information about safety-related changes is clearly communicated to staff” and “On average, in regulated entities, people are committed to safety”. Participants then responded through a 7-steps Likert scale ranging from Strongly Agree to Strongly Disagree. The safety culture questions were asked in the same order for all respondents.

3. Behavioural scenarios

After gauging the general perception of safety culture in different entities, we asked individuals from regulators and regulated entities to respond to questions testing the application of BI principles through vignettes or scenarios.

Although placing individuals in vignettes/scenarios is common in the behavioural insights literature, this is not often done in the safety culture literature. Furthermore, in discussions with the regulator contact points, we ascertained that safety culture surveys using vignettes are very uncommon. Therefore, this project not only contributes value-added in terms of applying BI in an area which is not commonly studied – dimension of strong safety culture – but also, in terms of the methods used.

As described above, the experiments (i.e. vignettes/scenarios testing different behavioural science principles) were designed based upon an initial literature review, consultations with academics, discussions with contact points within regulators as well as informal focus groups with representatives from regulators and regulated entities. Ultimately, the behavioural principles chosen for the analysis were: messenger, benchmarking, reciprocity and social norms (speaking up).

It was important to ensure that the scenarios would resonate within each country context, but we also wanted them to be specific enough such that the respondents would be able to picture/understand each as it relates to their work. The specific examples in the vignettes were chosen through discussions with contact points and then discussed with members of the informal focus groups (we received iterative feedback from two to three people in regulators as well as at least one regulated entity, piloting the vignettes and associated questions). The criteria for the more specific vignettes were that they were generalisable across countries and sectors (from oil and gas to electricity) but also provide some level of specificity.

After careful examination, we determined that we were most interested in observing how participants would react to the following three scenarios:

1. The introduction of a new guideline regarding Personal Protective Equipment (PPE).

2. Reports of bad lost-time injury rate.

3. A situation where a supervisor asks a worker to carry out a task in an unsafe manner.

These three scenarios were then manipulated to test the application of the four behavioural principles mentioned above. For example, we tested whether workers would behave differently as a function of whether the PPE regulation was introduced by a manager or a peer (messenger effect). We acknowledge that these examples are not reflective of safety culture as a whole and have drawbacks in terms of PPE not reflecting wider aspects (e.g. near misses, etc) and lost-time workers often gaming because the indicator gets too much attention, however individuals from frontline workers to senior management and those working for regulators are familiar with these concepts and find them important to some degree. Careful vignette design and bolding were used to place emphasis on the behavioural insights’ principle and not on the context of the specific vignette chosen.

In terms of designing the questions following each scenario, we asked about three main aspects. First, we asked about attention and salience to the information in the vignette, as this is an important concept in BI in terms of understanding effectiveness. We hypothesised that different formulations of the same scenario would have a different saliency in the eyes of participants and that behaviourally-informed information would be perceived as more salient. Second, the questions asked about what the respondent in the survey (regulated entity) would do themselves or what a person in an entity (regulator) would do in a given scenario, which allowed us to get an understanding of how they viewed their behaviour/the behaviour of someone in an entity.

The third section of questions was focused on the organisational management specifically and asked participants how their managers would react to the scenario (regulated entities) or how the entity as a whole would react (regulators). The intention behind this question is linked to safety culture being about what you think of others/not yourself; asking about managers’ behaviour also allowed us to investigate participants’ perceptions of authority. This question was of particular interest as behavioural science literature demonstrates that people are better at predicting how others will behave as opposed to how they themselves will behave.

For the process described above, we also ask what respondents think should/would be impactful. This enabled us to compare a prediction about a factual matter with a judgement about the implications for what is ultimately the right thing to do. In other words, to gauge the perception of both descriptive and injunctive social norms.

Importantly, each respondent was asked all of the vignettes and the experiment did not involve random assignment to treatment. However, the scenarios were presented in a randomised order and we ensured that the same vignettes were not asked together as a group.

4. Qualitative

At the end of the questionnaire, we provided a place for respondents to write in the reasons why they selected what they did for each experiment (e.g. Why did you think a certain messenger would be more impactful compared to the others?). People are often poor predictors of why they make certain decisions and it may be due to rules of thumb/heuristics, but this opportunity for respondents to tell the narrative back to themselves may provide useful additional information which can help to interpret the results of the experiments.

Results and discussion

Results

The following section describes the results of the study. For both the safety culture question as well as the BI experiments, three main hypotheses were tested:

1. Regulator vs. regulated entity: Are there differences between the regulator and regulated entities in terms of the effectiveness of the scenarios? This is an exploratory hypothesis.

2. Country differences: Are differences between regulator and entities driven by national context? Can national culture account for the differences?

3. Frontline vs. manager differences: In nations where power distance is high (Oman, Mexico to a lesser extent), are there differences in responses between frontline staff and management staff? Is this related to the safety culture of the organisation?

Sample size

We conducted a number of data quality checks before completing analyses. First, we took the original dataset ($N=1 366$) and removed the data marked as potential spam (a feature of the online survey software). We then conducted a missing data analysis and created new samples based on the number of safety culture items and vignettes items participants responded to. Those who responded to <50% of the safety culture items were removed from the safety culture sample and those who responded to <50% of the vignettes items were removed from the vignettes sample. This left us with $N=1 033$ for the safety culture sample and $N=885$for the behavioural vignettes sample. Note, that these were not independent samples – it was possible (and likely) that participants would appear in both samples.

It is worth noting here that we do not have a statistically robust sample from Canada ($N=28$). We can only guess at the overall response rate here. Given the number of Canadian organisations that had the opportunity to participate ($N=96$) and the approximate number of workers who could have participated, we must conclude that a sample of 28 is very low and potentially susceptible to selection bias. We encourage caution in interpreting conclusions made about the Canadian sample as inferring conclusions about safety culture is difficult with a small sample size (Pronovost and Sexton, 2005).

As for the other nationalities, we had a total of 92 Irish respondents; 409 Mexican respondents and 504 from Oman.

Safety culture questions

The following section describes the results of the safety culture assessment. In many of the countries studied, this type of safety culture assessment had not previously been completed by regulators. The safety culture survey comprised several dimensions: perceived management commitment to safety, perceived regulator commitment to safety, collaboration for safety, reporting culture, communication for safety, colleague commitment to safety, safety support and perceived relationship between the regulator and regulated entities. These dimensions were then collated into a summary safety culture scores, which was the target outcome variable for these analyses. The target for the safety culture items was the safety culture in regulated entities. Thus, our data reflect the regulator and entity workers’ perceptions of the safety culture in the average sector entity. Safety culture items were rated on a scale from 1 (strongly disagree) to 7 (strongly agree). Respondents were excluded from analyses if they answered less 50% of the safety culture items.

Before presenting the results, we must discuss a caveat for interpreting safety culture survey scores. It is difficult to determine exactly what a safety culture score represents. An element of strong safety culture is the ability for workers to criticise and question the decisions of senior management. Of course, if workers are criticising the decisions of senior management, then that may actually manifest itself as poorer safety culture scores (e.g. “senior management do not take action on safety issues when raised”). Another example concerns organisations whose workers have little experience of safety culture – if they have not seen it before, then can they really know what strong or weak safety culture actually looks like?

For this reason, it is difficult to compare safety culture across organisations because the organisational context will inevitably vary. A comparison that makes more sense, however, is against earlier measurements within the same organisation (e.g. Organisation X at Time 1 and then again at Time 2).

National differences

When comparing how different countries responded to the safety culture questions, we detect significant national differences in the perception of safety culture. Figure ‎5.1 summarises the cross-national findings and their relative statistical significance.

Figure ‎5.1. National differences in safety culture perception

Note: For the Canadian data, due to the small sample size inferring conclusions about safety culture cannot be made.

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

As can be seen from the graph, the Omani sample displayed the most positive perception of safety culture ($M=4.99$, $SD=1.15$), followed by Ireland ($M=5.47$, $SD=0.78$) and Mexico ($M=5.10$, $SD=1.00$). Canada ranked the least positive ($M=4.99$, $SD=1.15$), perhaps due to the small number of respondents we were able to gather from Canada ($N=28$). We also observed that the Canadian sample was overrepresented by regulator workers by a factor of 6 to 1. Thus, the mechanism driving the differences in safety culture perceptions between workers from regulators and regulated entities may be driving the result for the Canadian sample.

Three of the cross-national relationships were found to have statistical significance at the $p<.001$ level: respondents from Oman perceived the safety culture of the average regulated entity as more positive than respondents from Canada ($p<.001$) and Mexico ($p<.001$); and respondents from Ireland ranked significantly higher than the Mexican sample ($p=.001$).

All other cross-national comparisons were statistically similar or only approaching statistical significance, as in the comparison of Ireland and Oman ($p=.076$).

Role differences

Our data on professional role differences replicated a common finding in the literature – frontline staff have a more negative perception of safety culture than management do. Here we observed data from 472 frontline staff, 217 respondents who identified themselves as managers, 107 senior managers and 237 other members of staff.

As displayed in Figure ‎5.2, senior managers displayed the most positive vision of safety culture in their organisation ($M=5.67$, $SD=0.76$), followed by managers ($M=5.57$, $SD=0.72$), other staff ($M=5.46$, $SD=0.90$) and frontline ($M=5.29$, $SD=1.01$). Additionally, we find that both managers and senior managers have a significantly higher perception of safety than frontline ($p=.001$).

Figure ‎5.2. Role differences in safety culture perception

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Behavioural insights vignettes

The following section describes the results of the experiments. They are provided according to the BI principles tested in the experiments.

Overall

While we were able to demonstrate several significant differences in safety culture perception, it should be noted that the average scores were always positive (above 4) and the differences were never in excess of half a Likert point.

Regulator vs. regulated entity

Workers from regulated entities were more favourable to the vignettes than were the regulator worker. This is interesting when taken with the safety culture data, where regulator workers have more negative perceptions of safety culture and here, they are less receptive to behavioural interventions to address safety culture.

Behavioural principle

In general, respondents responded most favourably to the feedback vignettes compared to the messenger and norm vignettes. Vignettes informed by the messenger effect were perceived as the second most impactful. Interestingly, respondents felt like the norm vignettes would be the least effective at attracting the attention of workers or themselves. Figure ‎5.3 unpacks the differences in perceived effectiveness between the principles.

Figure ‎5.3. Comparing behavioural principles

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Feedback effects

While overall, feedback was the most effective behavioural principles, responses to the five items after reading the different feedback vignettes (e.g. social benchmarking vs. feedback, vs. control) were statistically similar. That is, respondents believed that there was no difference between the levels of the feedback variable.

Norm effects

Norm conditions were roughly equivalent across each of the five effectiveness items, with the key descriptive and injunctive levels being statistically indistinguishable from the control.

There were some differences, however, with respondents rating the descriptive norm vignette more positively than the control vignette for two items: i) “this safety information would attract the attention of workers/my attention”; and ii) “this safety information would affect how workers in entities do their job/I do my job”.

Messenger effects

Respondents believed that messenger effects for regulators and management were statistically similar. They felt, however, that the peer messenger effect would be the least effective. This is interesting given there is ample evidence for the influence of peer messengers. Qualitative feedback indicated that peer-initiated safety direction would likely only have influence if it was picked up by the senior management, in which case the messenger becomes the organisation’s senior management.

Figure ‎5.4. Messenger effects

n.s. = no significance

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

International lens and understanding the results

We conducted several analyses of the behavioural principles data to see how their effectiveness might be affected by national cultural contexts.

Messenger

The messenger vignettes were responded to most positively by respondents from Ireland, followed by Omani respondents, and then Canadian and Mexican respondents approximately equal.

Feedback

The feedback vignettes were responded to most favourably by respondents from Oman, followed by Irish respondents, with Canadian and Mexican respondents alternating for least favourable responses.

Social norms

Again, Irish respondents were the most favourable to the norm vignettes, although Mexico and Oman responses were often quite favourable. Canadian responses were the least favourable.

To understand what might be driving these differences, we next looked at responses to each level of the principles by country. We observed that there are statistically significant effects of social norms in Mexico – indicating that the Mexican sample perceives descriptive norms as more effective than control and injunctive norms. In addition, we found that, in both Oman and Ireland, the peer messenger was deemed to be less effective than the management messenger on some items.

For the Canadian sample, no differences were found in how respondents perceived different behavioural cues. There were no differences in the perceived effectiveness of the different messenger types (regulator, management, peers); nor for different feedback types (control, simple control, social benchmarking, reciprocity); nor different norm types (control, descriptive, injunctive). This may be due to the small sample size.

Lens of an occupational role for understanding the results

We conducted further analyses of the behavioural principles data to see how their effectiveness might be affected by the occupational role of respondents.

Messenger

The messenger vignettes were responded to most positively by senior managers, followed by managers, and then frontline staff and other staff respondents responded approximately equally. As can be seen in Figure ‎5.5, the pattern is consistent across different items.

Figure ‎5.5. Occupational response to the messenger effect

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Feedback

The feedback vignettes followed the same general pattern as messenger vignettes, where senior managers responded most favourably, followed by managers, and then frontline staff and other staff respondents responded approximately equally.

Social norms

Responses to the social norm vignettes were slightly different. On Item 1, senior managers and managers were indistinguishable and significantly more positive than frontline staff and other staff (who were indistinguishable). For Item 2, only the difference between frontline staff and managers was statistically distinguishable (managers more positive). For Item 3, the only statistical difference was between other staff and managers (managers more positive). There were no differences between the occupational roles on Item 4. On Item 5, managers and senior managers were significantly more positive than other staff but only managers were more positive than frontline staff.

To understand what might be driving these differences we next look responses to each level of the principles by occupational role. By doing so, we found no differences in the perceived effectiveness of feedback type (control, simple control, social benchmarking, reciprocity) or norm type (control, descriptive, injunctive). However, the data revealed that for frontline staff, managers and other members of staff (excluding senior managers), the peer messenger was deemed to be less effective than messages from regulators/managers.

Lens of an organisational background for understanding the results

We conducted several analyses of the behavioural principles data to see if there were how their effectiveness might be affected by whether the respondents are from regulators or regulated entities.

Across all principles, respondents from regulated entities responded more favourably than respondents from regulators. This is exemplified by Figure ‎5.6, which displays how regulators and regulated entities responded to the messenger formulation of the vignettes. As can be seen, the perception of regulated entities (lighter shade) is consistently higher than that of regulators (darker shade) across all items.

We conducted further analyses into the principles to see if there were differences in the perceived effectiveness of the vignettes.

While there were no differences in the perceived effectiveness of the different feedback types or norm types, we did find that, for both regulators and regulated entities, the peer messenger was deemed to be less effective than manager and regulator messengers on several items, thus confirming the presence of a messenger effect.

Figure ‎5.6. Regulator vs. entity responses to messenger vignettes

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Summary of behavioural results

Messenger

Overall, the messenger of safety instructions seems to mostly only matter in Ireland and Oman. In those countries, the peer messenger was perceived to be the least effective messenger vehicle. From a Hofstedian perspective, Ireland and Oman do not share many cultural similarities. Where they do share similarity is in the rate of regulator worker to entity worker responses (Ireland 1:3; Oman 1:4). Perhaps this majority of entity worker responses explains why the peer messenger was perceived least favourably. Where there is a messenger effect, its direction is such that messages from managers and regulators are deemed more effective than messages from peers.

Feedback

While feedback was overall the most impactful behavioural principle among the ones tested, A deeper investigation into the responses to the feedback vignettes failed to reveal any differences at the country, occupational or organisational level.

Social norms

In general, norms were found to be the least effective behavioural principle overall. However, cross-national comparisons revealed some interesting trends. In particular, the Mexican sample was the only nationality for which there were statistically significant results with regards to norm type. Mexican respondents were found to react more strongly to descriptive norms than control or injunctive messages.

Descriptive norms differ from injunctive norms in that they describe what people actually do, whereas injunctive norms describe what people ought to do (motivation may be unclear). From a Hofstedian perspective, Mexican samples score high on uncertainty avoidance, meaning that the clear signal from descriptive norms may be preferred over the motivationally unclear nature of injunctive norms. Mexican samples also score low on individualism, meaning that they may be more susceptible to group norms in general.

Limitations of the study

There are a number of limitations to this study such as varying degrees of English language abilities (particularly in Oman) and that we relied on contact points within the regulated entities to pass along the information. Where possible, efforts were made to counteract these limitations.

While it was possible that regulator workers have a more accurate perception of the state of safety culture in the sector, our data cannot support such a conclusion. We do not provide any kind of objective benchmark against which to compare workers’ safety culture perceptions. Nor do we test whether regulator workers: i) do have access to more data about the state of the sector’s safety culture; or ii) whether their perceptions are subsequently more accurate.

While we tested the perceived effectiveness of three distinct BI principles, there is likely a degree of overlap between them. For example, the feedback vignettes often incorporate a particular messenger (i.e. the regulator), which we have found elsewhere is an effective messenger. Our feedback vignettes also incorporate elements of social norms (e.g. being in the “bottom/worst 25%” is social information that respondents could compare against). Part of the difficulty in interpreting the feedback vignettes is that they all share these common overlaps with other principles.

Conclusions

Policy lessons

Overall, the project constitutes one of the first applications of behavioural insights through online experiments to the study of safety improvement and elements of safety culture. It is intended to serve as a stepping stone towards a more frequent integration of the field of BI and safety. A number of key policy lessons emerge from the research.

• To avoid unintended negative consequences, it is important for regulators to take into consideration differences in perception within and between actors when designing new safety regulations or policies. The study found that the closer one is to the front line, the lower one’s perception of safety culture. From a system perspective, the study showed that regulators have a more negative perception of safety culture in the regulated entities than the entities themselves, perhaps due to their position overseeing the sector. Moreover, results show that senior managers reacted most favourably to the behavioural principles (i.e. feedback, messenger effects and social norming) than other occupations, indicating that there are differences of perceptions within entities (not only between entities and the regulator) regarding safety culture. When developing policy, it is important to take these differences in perception into consideration to ensure policies are targeted for different audiences.

• The study found that some feedback is better than no feedback but the results are inconclusive as to which type of feedback or benchmarking is best. Results show that respondents reacted most favourably to feedback vignettes, compared to messenger and norm vignettes, generally speaking. From a policy perspective, this highlights the importance of considering providing workers with some form of feedback. However, which form of feedback is most effective and whether feedback is beneficial in every context needs to be studied in further detail.

• The source of safety messages (messenger) still matters, which highlights the need to ensure regulators and senior managers in regulated entities are working together to encourage a culture of safety. Results showed that respondents reacted similarly to messages on safety coming from a regulator as well as senior management of the regulated entity. However, messages from peers were considered the least effective, which runs counter to conventional thinking about the use of norms in nudging.

• Social norming was perceived as the least effective across the sample, which requires more research to determine the benefit for policymakers of using social norms to encourage a culture of safety. Results for all social norm vignettes were statistically indistinguishable from the control. However, some differences were noted for respondents rating the descriptive norm vignette more positively than the control vignette, giving some possible avenues for future research.

• Differences between countries highlight the need for policymakers to take a location-based approach to strengthening elements of safety in their own contexts. While the above notes the trends for each behavioural insight tested, between-country differences were notable. For feedback, there were no statistically significant differences at the country level; however, results did show respondents from Oman reacted most favourably, followed by Irish respondents, and then Canadian and Mexican respondents alternating for least favourable responses. Caution should, however, be taken inferring results from the Canadian results due to small sample size. For the messenger effect, it seems that this really mattered most in Ireland and Oman, perhaps due to a similar regulator worker to entity worker response rate. For social norming, the Mexican sample was somewhat responsive to descriptive norms, though Irish and Omani responses were also favourable. Canadian responses were least favourable.

Potential next steps/areas for further work

Although the results of this study point toward useful areas of potential attention, there is a substantial amount of merit in further research. Context is very important and, in order to have any policy recommendations/a toolkit, it is necessary that further work be carried out. This work may include the following:

• Qualitative follow-up: A key component of creating a toolkit would be a deeper understanding of the nuances in the context in each country setting. In order to obtain this level of understanding, it would be important to carry out focus groups and/or individual interviews in each country context.

• Additional survey experiments: There is scope for additional online experiments which could dive deeper into these behavioural principles, test different principles, focus on certain types of respondents, etc.

• Randomised controlled trials: Field experiments can be conducted to test the application of behavioural science principles in real-world contexts.

• Behavioural lens on existing policies: Taking a closer look at each country context and providing more tailored recommendations.

Complementary field work

Behavioural insights as a field can provide a great deal of understanding and can be complemented by data analytics and other academic disciplines that take a deep dive perspective. For example, qualitative research and design thinking can play an important role in facilitating, augmenting or strengthening and evaluating the impact of BI.

References

Antonsen, S. (2009), “Safety culture assessment: A mission impossible?”, Journal of Contingencies and Crisis Management, Vol. 17(4), pp. 242-254.

Ariely, D., G. Loewenstein and D. Prelec (2003), “Coherent arbitrariness: Stable demand curves without stable preferences”, The Quarterly Journal of Economics, Vol. 118(1), pp. 73-105.

Avineri, E. (2014), “Nudging safer road behaviours”, Tel Aviv Academic College of Engineering Afeka/Acitral https://pdfs.semanticscholar.org/346b/dd0b382aa0bc15530d4d8b1d0411bcfcf1e1.pdf (accessed on 28 October 2018).

Branson, C. et al. (2012), Acceptable Behaviour: Public Opinion on Behaviour Change Policy, Ipsos MORI, https://www.ipsos.com/ipsos-mori/en-uk/acceptable-behaviour (accessed on 28 October 2018).

Bicchieri, C. (2006), The Grammar of Society, Cambridge University Pres, Cambridge.

Breman, A. (2011), “Give more tomorrow: Two field experiments on altruism and intertemporal choice”, Journal of Public Economics, Vol. 95(11-12), pp. 1349-1357.

Choudhry, R.M., D. Fang and S. Mohamed (2007), “The nature of safety culture: A survey of the state-of-the-art”, Safety Science, Vol. 45(10), pp. 993-1012.

Christian, M.S. et al. (2009), “Workplace safety: A meta-analysis of the roles of person and situation factors”, Journal of Applied Psychology, Vol. 94(5), pp. 1103-1127.

Cialdini, R. (2005), “Don’t throw in the towel, use social influence research”, Observer, Vol. 18(4-4), www.psychologicalscience.org/observer/getArticle.cfm?id=1762.

Clark J. et al. (2013), “Evaluating the message or the messenger? Implications for self-validation”, Persuasion, Personality and Social Psychology Bulletin, Vol. 39(12), pp. 1571-84.

Clarke, S. (2000), “Safety culture: Under-specified and overrated?”, International Journal of Management Reviews, Vol. 2(1), pp. 65-90.

Cooper, M.D. (2000), “Towards a model of safety culture”, International Journal of Disaster Risk Reduction. Vol. 36(2), pp. 111-136, https://doi.org/10.1016/S0925-7535(00)00035-7 (accessed on 21 March 2019)

Cox, S. and R. Flin (1998), “Safety culture: Philosopher’s stone or man of straw?”, Work and Stress, Vol. 12(3), pp. 189-201.

de Groot-Mesken, J. and W.P. Vlakveld (2014), “A nudge in the right direction: Safe traffic behaviour”, R-2014-13, SWOV Institute for Road Safety Research, Den Haag.

DeJoy, D.M. (2005), “Behavior change versus culture change: Divergent approaches to managing workplace safety”, Safety Science, Vol. 43(2), pp. 105-129.

Dekker, S. (2001), “Disinheriting Fitts and Jones ’47”, International Journal of Aviation Research and Development, Vol. 1(1), pp. 7-18.

Dolan, P. et al. (2012), “Influencing behaviour: The mindspace way”, Journal of Economic Psychology, Vol. 33(1), pp. 264-277.

Eckel C. and H. Gintis (2010), “Blaming the messenger: Notes on the current state of experimental economics”, Journal of Economic Behavior & Organization, Vol 73(1), pp. 109-119.

Fehr, E. (2002), “Strong reciprocity, human cooperation, and the enforcement of social norms”, Human Nature, Vol. 13(1), pp. 1-25.

Fleming, M. et al. (2012), “Cultural disasters: Learning from yesterday to be safe tomorrow”, Oil and Gas Facilities, Vol. 1(3), Society of Petroleum Engineers, Houston, Texas.

Flin, R. et al. (2000), “Measuring safety climate: Identifying the common features”, Safety Science, Vol. 34(1), pp. 177-192.

Goldstein, N.J., R.B. Cialdini and V. Griskevicius (2008a), “A room with a viewpoint: Using social norms to motivate environmental conservation in hotels”, Journal of Consumer Research, Vol. 35(3), pp. 472-482.

Goldstein, D.G. et al. (2008b), “Nudge your customers toward better choices”, Harvard Business Review, pp. 99-105.

Guldenmund, F.W. (2000), “The nature of safety culture: A review of theory and research”, Safety Science, Vol. 34(1-3), pp. 215-257.

Haines, M.P. (1996), A Social Norms Approach to Preventing Binge Drinking at Colleges and Universities, Higher Education Center for Alcohol and Other Drug Prevention, Newton, MA.

Hansen, P.G. and A.M. Jespersen (2013), “Nudge and the manipulation of choice: A framework for the responsible use of the nudge approach to behaviour change in public policy”, European Journal of Risk Regulation, Vol. 4(1), pp. 3-28.

Health and Safety Commission (1993), Third Report: Organising For Safety, ACSNI Study Group on Human Factors, London.

Hofstede, G., G.J. Hofstede and M. Minkov (2010), Cultures and Organizations: Software of the Mind (Third), McGraw-Hill.

Holden, R.J. (2009), “People or systems? To blame is human. The fix is to engineer”, Professional Safety, Vol. 54(12), pp. 34-41.

Karlan, D.S. and J. Zinman (2009), “Expanding credit access: Using randomized supply decisions to estimate the impacts”, The Review of Financial Studies, Vol. 23(1), pp. 433-464.

Krause, T.R., G. Sellers and C. Horn (2001), “Moving to the 2nd generation in behaviour-based safety”, Symposium Series, No. 148, IChemE XVI-Paper-57, pp. 731–738.

Kvalheim, S.A., S. Antonsen and S. Haugen (2016), “Safety climate as an indicator for major accident risk: Can we use safety climate as an indicator on the plant level?”, International Journal of Disaster Risk Reduction, Vol. 18, pp. 23-31.

Langford, D., S. Rowlinson and E. Sawacha (2000), “Safety behaviour and safety management: Its influence on the attitudes of workers in the UK construction industry”, Engineering, Construction and Architectural Management, Vol. 7(2), pp. 133-140.

Lindhout, P. and G. Reniers (2017), “What about nudges in the process industry? Exploring a new safety management tool”, Journal of Loss Prevention in the Process Industries, Vol. 50, pp. 243-256.

Linkenbach, J. and H. Perkins (2005), “MOST of us don’t drink and drive campaign. A social norms strategy to reduce impaired driving among 21-34 year-olds”, U.S. Department of Transportation, National Highway Traffic Safety Administration.

Linkenbach, J. and H. Perkins (2003), “MOST of us wear seatbelts: The process and outcomes of a 3‑year state wide adult seatbelt campaign in Montana”, presented at the National Conference on the Social Norms Model, Boston, MA, July 17.

Mearns, K. and S. Yule (2009), “The role of national culture in determining safety performance: Challenges for the global oil and gas industry”, Safety Science, Vol. 47(6), pp. 777-785.

Nolan, J. et al. (2008), “Normative social influence is underdetected”, Personality and Social Psychology Bulletin, Vol. 34(7), pp. 913-923.

OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

OECD (2017), Behavioural Insights and Public Policy: Lessons from Around the World, OECD Publishing, Paris, http://dx.doi.org/10.1787/9789264270480-en.

Oullier, O. et al. (2009) “Improving public health prevention with a nudge”, Improving Public Health Prevention with Behavioural, Cognitive and Neuroscience, Centre for Strategic Analysis, Paris, pp. 38-46.

Perkins, H.W. et al. (2010), “Effectiveness of social norms media marketing in reducing drinking and driving: A statewide campaign”, Addictive Behaviour, Vol. 35(10), pp. 866–874.

Pronovost, P. and B. Sexton (2005), “Assessing safety culture: Guidelines and recommendations,” BMJ Quality and Safety, Vol. 14(4), pp. 231-233.

Rand, D.G., E. Yoeli and M. Hoffman (2014), “Harnessing reciprocity to promote cooperation and the provisioning of public goods”, Policy Insights from the Behavioral and Brain Sciences, Vol. 1(1), pp. 263-269.

Reader, T.W. et al. (2015), “Safety sans Frontières: An international safety culture model”, Risk Analysis, Vol 35(5), pp. 770-789, https://doi.org/10. 1111/risa.12327.

Reason, J. (1990), Human Error, Cambridge University Press.

Selinger, E. and K. Whyte (2011), “Is there a right way to nudge? Practice and ethics of choice architecture”, Sociology Compass, Vol. 5(10), pp. 923-935.

Singer, S. et al. (2009), “Relationship of safety climate and safety performance in hospitals”, Health Services Research, Vol. 44(2p1), pp. 399-421.

Smith, A. et al. (2009), “Safety culture advice and performance”, Report for IOSH Research Committee, http://www.behavioral-safety.com/articles/cardiff_safety_culture_report.pdf.

Sunstein, C.R. (2006), Infotopia: How Many Minds Produce Knowledge, Oxford University Press.

Talabi, B.O., F.T. Edum-Fotwe and A.G.F. Gibb (2015a). “Construction actor safety behaviour: Antecedents, current thinking and directions”, in S.D. Smith and F. Sherratt (eds.), Proceeding of ARCOM Doctoral Workshop: Health, Safety and Wellbeing, 11 February 2015, University of Edinburgh, pp. 9-20.

Talabi, B.O., F.T. Edum-Fotwe and A.G.F. Gibb (2015b). “Behaviour-based safety (BBS): A construction industry’s perspective”, in M. Behm and C. McAleenan (eds.), Proceedings of the CIB W099 Benefitting Workers and Society through Inherently Safe(r) Construction, 9-11 September 2015, Belfast, pp. 181–190.

Taleb, N.N. (2007), The Black Swan: The Impact of the Highly Improbable, Random House, New York.

Tear, M.J. and M. MacLennan (2018),” OECD behavioural insights for safety culture”, Open Science Framework, https://osf.io/r482d/.

Thaler, R.H. and C.R. Sunstein (2008). Nudge: Improving Decisions about Health, Wealth, and Happiness, Yale University, New Haven.

TRB, (2016), “Strengthening the safety culture of the offshore oil and gas industry”, Transportation Research Board, Washington, DC.

Zohar, D. (1980), “Safety climate in industrial organizations: Theoretical and applied implications”, Journal of Applied Psychology, Vol. 65(1), pp. 96-102.

Annex 5.A. Additional information and sample survey

Additional details on sample and participants

Annex Table ‎5.A.1. Additional information on regulators by country

	Canada	Mexico	Ireland	Oman
Sector	Oil and Gas	Oil and Gas	Oil and Gas Exploration and Extraction Petroleum Safety Framework (PSF) Gas Transmission and distribution, and LPG distribution networks Gas Safety Framework (GSF) Gas and electrical installers Safety Supervisory Bodies (SSB)	Electricity
Year regulator established	1959	2015	Economic regulation 1999 Safety Function 2007 GSF 2011 PSF	2004
Number of regulated entities/companies	99	18 200	5	28
About the regulator	We regulate pipelines, energy development and trade in the Canadian public interest.	Mission: To guarantee an individual’s safety and environmental integrity with legal, procedural and cost certainty in the oil and gas sector. Vision: To be the agency that takes the Mexican oil and gas sector to be the cleanest and the safest worldwide. Values: 1) Professionalism - We are ethical, knowledgeable and experienced. 2) Transparency - What we do is public and accessible. 3) Impartiality - Decisions are made based upon objective criteria.	Performs three major functions: regulates, gas and electricity consumer and wholesale markets; regulates gas/petroleum industry with respect to safety, including upstream activities, transportation and downstream activities. The CRU also regulates the Irish Water utility provider.	The authority is responsible for regulation of the electricity and related water sector and has a statutory duty to secure the provision of electricity and related water services in all parts of Oman, including rural customers and a duty to protect the interests of customers.
Duties and responsibilities	The National Energy Board (NEB) regulates: - the construction, operation, and abandonment of pipelines that cross international borders or provincial boundaries, as well as the related pipeline tolls and tariffs - the construction and operation of international power lines and designated inter-provincial power lines - imports of natural gas and exports of crude oil, natural gas liquids, natural gas, refined petroleum products, and electricity, oil and gas exploration and production activities in specified areas that are not regulated under joint federal/provincial accords.	Legal mandate: regulate; authorise; supervise. Responsibilities: safety; environmental protection. ASEA has responsibilities throughout the hydrocarbons value chain: from upstream exploration and extraction to midstream and downstream transformation, production and storage, as well as distribution and retail at petrol station level, making it a globally unique technical regulator.	Regulate gas network safety including transmission and distribution systems. Regulate petroleum (oil and gas) safety, including exploration, extraction and decommissioning (onshore and offshore) safety. Design and oversee safety supervisory schemes for electrical contractors and natural gas and liquid petroleum gas (LPG) installers. Set programme of audit and inspection of regulated entities. Issue safety permits for petroleum activities and safety licences to LPG undertakings. Promotion and public awareness of electrical and gas safety issues.	Regulate electricity and some aspects of the water sector. Secure the provision of electricity and protect the interests of customers particularly customers who have limited income, the sick and elderly. Secure and develop the safe, effective and economic operation of the electricity sector and to enhance the safety of the public
Behavioural assets and behavioural needs
Identified main safety culture behavioural asset	Personal accountability	Respectful work environment (Gas); Safety leadership commitment (GSF)	Respectful work environment (Petroleum); Environment for raising concerns (Gas)	Safety leadership commitment (Transmission); Work processes (Generation)
Identified main safety culture behavioural need	Enquiring attitude	Personal accountability (Gas); Work processes (GSF)	Continuous improvement (Petroleum); Safety leadership commitment (Gas)	Enquiring attitude (Transmission and Generation)

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Annex Table ‎5.A.2. Canada National Energy Board

Invited to participate	NOT invited to participate
All Operations Staff (Field Operations and System Operations)	Various Support Staff (Administrative Assistants)
All Energy Adjudication Staff	Communications and Engagement Staff (media relations, webmaster and staff, graphic artists, printing shop staff, translation)
All Executives and Board Members	Legal Services
All Regulatory Policy Staff	People and Knowledge (Human Resources and IT)
	Corporate Performance and Results
	Integrated Energy Information and Analysis (monitoring of energy trade date, supply, imports, etc.) (other than Reg Policy staff who are included)

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Annex Table ‎5.A.3. Mexico: Agency for Safety, Energy and Environment (ASEA)

	Total no. of people
Invited to participate	350
Directive Group	7
Supervision, Inspection, and Surveillance Unit	90
Permits and Authorisations Unit	130
Regulation and Legal Standards Unit	83
Planning, Processes and Strategic Stakeholder Engagement Unit	33
Executive Direction	7
NOT invited to participate	115
Finance and Administration Unit
Legal Affairs Unit
Administrative Assistants
Press and Communication Office

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Annex Table ‎5.A.4. Ireland: Commission for Regulation of Utilities

	Total no. of people
Invited to participate	42
Senior management	6
Frontline staff	16
Managers	4
Others (includes staff who previously worked in Safety Division, Legal advisor on safety and human resources staff with responsibility for inhouse safety)	8
External consultants who participate in inspections	8
NOT invited to participate	58
Economic regulatory managers/analysts
Finance, IT, Communications, and administrative staff

Note: The contact points in the companies are the safety managers. We included some economic regulatory managers and analysts in the group we planned to send surveys to. The ones left out are those with responsibility for water and energy markets.

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Survey on behavioural assets and needs

Using numbers 1-9, can you please rank the following safety culture dimensions with respect to your regulated entities (not your own organisations)? A one (1) corresponds to what you perceive as the strongest safety culture dimension in your regulated entities; a nine (9) corresponds to what you perceive as the weakest safety culture dimension in your regulated entities.

 

	Safety leadership commitment
	Respectful work environment
	Environment for raising concerns
	Effective safety and environmental communication
	Personal accountability
	Enquiring attitude
	Hazard identification and risk management
	Work processes
	Continuous improvement

For the strongest and weakest safety culture dimension (denoted by 1 and 9 above), can you please identify what you believe to be behavioural assets and needs. By “behavioural asset” we mean things that people do to keep their organisation safe – behaviours that need to be protected and promoted. If you placed a 1 next to “environment for raising concerns”, then an example behavioural asset could be “workers challenge their colleagues when they see colleagues’ unsafe behaviour”.

Use this box to describe 2-3 behavioural assets in line with what you perceive to be the strongest safety culture dimensions in your regulated entities…

By “behavioural need” we mean behaviours that must occur in order for the organisation to become safe. If you placed a 9 next to “work processes”, then an example behavioural need could be “workers need to let management know when the written processes do not match how work is done”.

Use this box to describe 2-3 behavioural needs in line with what you perceive to be the weakest safety culture dimensions in your regulated entities…

Finally, please use the space below to provide any other context you think is important for us to understand these behavioural assets and needs? Can you think of a reason why these behaviours exist (or don’t)? Do these behaviours have flow-on effects to larger problems in the organisations?

Safety culture items

Box ‎5.1. Safety culture survey

A manager has people under them in the hierarchy and is responsible for directing their subordinates’ work.

On average, managers in regulated entities...

...are committed to safety.

...take action on safety issues when raised.

...would always provide support if there is a concern about safety.

On average, the regulator...

...is committed to safety.

...has a positive influence on safety.

...takes action on safety issues when raised.

On average, in regulated entities...

...people understand how others’ jobs contribute to safety.

...people who raise safety issues are seen as problematic.

...there are people whom others do not want to work with because of their negative attitude to safety.

...involvement of staff in safety activities is sufficient.

On average, in regulated entities...

...people who report safety-related incidents, near misses, hazardous conditions or occurrences are treated in a just and fair manner.

...voicing concerns about safety is encouraged.

...timely feedback is provided on the safety issues raised.

On average, in regulated entities...

...information about safety-related changes is clearly communicated to staff.

...lessons are learned from safety-related incident or occurrence investigations.

...there is good access to information regarding safety incidents or occurrences.

...there is good communication up and down the organisation about safety.

On average, in regulated entities...

...everyone feels that safety is their personal responsibility.

...there is confidence in other people.

...people are committed to safety.

On average, in regulated entities...

...there is sufficient staff to work safely.

...people share safety-related information.

There is a good relationship between the regulator and the regulated entities.

In regulated entities, good safety behaviour is acknowledged.

Behavioural vignettes items

Annex Table ‎5.A.5. Behavioural vignette items

Behavioural science principle	Level	Vignette
Messenger	Regulator	You hear that employees in the entities are learning about a new Personal Protective Equipment (PPE) guideline introduced by the regulator.
	Management	You hear that the workers in the entities are learning about a new Personal Protective Equipment (PPE) directive introduced by their senior management team.
	Peer	You hear that workers in the entities are learning about a new Personal Protective Equipment (PPE) direction introduced by their peers and colleagues, through word of mouth.
Feedback	Simple benchmarking	A regulated entity in your sector scored in the bottom/worst performing 25%* for lost-time injury rate (LTI) performance.
	Social benchmarking	A regulated entity in your sector scored in the bottom/worst performing 25%* of organisations in similar business areas for lost-time injury rate (LTI) performance. An entity in your sector scored in the bottom/worst performing 25%* of lost-time injury rates (LTI) performance. However, the entity exerted effort to improve LTI rates over the previous period and the regulator acknowledged this progress/work toward progress with a letter from the Director of the regulator.
	Control	A regulated entity in your sector scored in the bottom/worst performing 25%* for lost-time injury rate (LTI) performance and emphasises that lost-time injury performance is a known concern in your industry/organisations in your industry acknowledge the importance of the lost-time injury rate.
	Reciprocity	An entity in your sector scored in the bottom/worst performing 25%* of lost-time injury rates (LTI) performance. However, the entity exerted effort to improve LTI rates over the previous period and the regulator acknowledged this progress/work toward progress with a letter from the Director of the regulator.
Social norms	Control	A supervisor** in an entity instructs a worker to take a short-cut in procedures in order to speed up the completion of an important task. You know the entity’s rule is to report risky behaviour.
	Injunctive	A supervisor** in an entity instructs a worker to take a short-cut in procedures in order to speed up the completion of an important task. You know that the entity’s rule is to report risky behaviour and that 9 out of 10 workers believe that they ought to report their supervisor if they suspect the supervisor’s actions are negatively affecting safety.***
	Descriptive	A supervisor** in an entity instructs a worker to take a short-cut in procedures in order to speed up the completion of an important task. You know the entity’s rule is to report risky behaviour and that 9 out of 10 workers in regulated entities report their supervisors if they suspect the supervisor’s actions are negatively affecting safety.***

* This 25% is used for illustrative purposes.

** A supervisor is in charge of the process and around 10-20 workers.

*** The “9 out of 10” figure is used for illustrative purposes.

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.

Sample of behavioural vignettes

Your organisation’s senior management team notices a safety issue and introduces a new directive around Personal Protective Equipment (PPE).

Annex Table ‎5.A.6. Sample behavioural vignette

	Strongly disagree	Disagree	Somewhat disagree	Neither agree nor disagree	Somewhat agree	Agree	Strongly agree
This safety information from this source attracts my attention.
This safety information from this source should affect how I do my job.
This safety information from this source would affect how I do my job.
This safety information from this source should affect how managers do their job.
This safety information from this source would affect how managers do their job.

Source: OECD (2018), “Behavioural insights and safety improvement in the energy sector: Experimental evidence from Canada, Ireland, Mexico and Oman on strengthening dimensions of safety culture”, Unpublished, Prepared by M. MacLennan and M. Tear for discussion at the 11^th Meeting of the Network of Economic Regulators, 26 November 2018, Paris.