iaeas approach to safety culture assessments · 2019-10-27 · iaeas approach to safety culture...

IAEA International Atomic Energy Agency

IAEAs Approach to Safety Culture Assessments

Monica Haage – [email protected] Expert on Safety Culture and Human & Organizational Factors

IAEA

The IAEA Safety Culture Framework

Safety Culture is that assembly of characteristics and attitudes in organizations and individuals which establishes that, as an overriding priority, protection and safety issues receives the attention warranted by their significance. The 2007 IAEA glossary

The internationally agreed IAEA normative framework defines strong safety culture into: -  5 characteristics -  36 attributes

Recognized that safety culture is an essential component of the defence in depth and need to be addressed proactively

IAEA

Safety is a clearly recognized value Attributes

•  High priority to safety: shown in documentation, communications and decision- making

•  Safety is a primary consideration in the allocation of resources

•  The strategic business importance of safety is reflected in business plan

•  Individuals are convinced that safety and production go ‘hand in hand’

•  A proactive and long-term approach to safety issues is shown in decision-making

•  Safety conscious behavior is socially accepted and supported (both formally and informally)

GS-G-3.1

IAEA

Accountability for safety is clear Attributes

•  Appropriate relationship with the regulatory body exists, which ensures that the accountability for safety remains with the licensee

•  Roles and responsibilities are clearly defined and understood •  There is a high level of compliance with regulations and procedures •  Management delegates responsibilities with appropriate authority to

enable accountabilities •  Ownership for safety is evident at all organizational levels and by all

individuals

GS-G-3.1

IAEA

Safety is learning driven Attributes •  A questioning attitude prevails at all organizational levels •  An open reporting of deviations and errors is encouraged •  Internal and external assessments, including self-assessments are used •  Organizational and operating experience (both internal and external to

the facility) is used •  Learning is enabled through the ability to recognize and diagnose

deviations, formulate and implement solutions and monitor the effects of corrective actions

•  Safety performance indicators are tracked, trended, evaluated and acted upon

•  There is a systematic development of staff competencies

GS-G-3.1

IAEA

Safety is integrated into all activities Attributes

•  Trust permeates the organization •  Consideration for all types of safety, including industrial and

environmental safety and security, is evident •  Quality of documentation and procedures is good •  Quality of processes, from planning to implementation and review, is

good •  Individuals have the necessary knowledge and understanding of the

work processes •  Factors affecting work motivation and job satisfaction are considered •  Good working conditions exist with regards to time pressures, work

load and stress •  Cross-functional and interdisciplinary cooperation and teamwork are

present •  Housekeeping and material condition reflect commitment to

excellence

GS-G-3.1

IAEA

Leadership for safety is clear Attributes

•  Senior management is clearly committed to safety •  Commitment to safety is evident at all management levels •  Visible leadership showing involvement of management in safety related

activities •  Leadership skills are systematically developed •  Management assures that there is sufficient and competent staff •  Management seeks the active involvement of staff in improving safety •  Safety implications are considered in the change management process •  Management shows a continuous effort to strive for openness and good

communications throughout the organization •  Management has the ability to resolve conflicts as necessary •  Relationships between management and staff are built on trust

GS-G-3.1

IAEA

IAEA Safety Culture Publications http://www.iaea.org

Document Title

Safety Fundamentals No. SF-1 Fundamental Safety Principles

Safety Requirements No. GS-R-3 The Management System for Facilities and Activities

Safety Requirements GS-R-Part 2 (supersedes Safety Requirements No. GS-R-3)

The Management System for Facilities and Activities – draft DS-456 – to be published 2015

Safety Guide No. GS-G-3.1 Application of the Management System for Facilities and Activities

Safety Guide No. GS-G-3.5 The Management System for Nuclear Installations

Safety Series No. 75-INSAG-4 Safety Culture

Safety Series No. 75-INSAG-15 Key Practical Issues in Strengthening Safety Culture

Safety Report Series No. 11 Developing Safety Culture in Nuclear Activities

Safety Report Series No. 42 Safety Culture in the Maintenance of Nuclear Power Plants

Safety Report Series: No 74 Safety Culture during Pre-Operational Phases – Published Sept 2012

Safety Report Series: Performing Safety Culture Self-Assessments –– approved and expected to be published 2014/2015

Safety Report Series: How to Continuously Improve Safety Culture – draft – to be published 2015

TECDOC-1321 Self-assessment of safety culture in nuclear installations

TECDOC-1329 Safety culture in nuclear installations

TECDOC-1707 Regulatory Oversight Of Safety Culture In Nuclear Installations

IAEA

•  Culture is seen as something we can influence, rather than something we can control

•  Culture work needs to encompass the whole organization – not only as a top-down process

•  Edgar Schein’s well established iceberg metaphor helps to understand and how to continuously improve safety culture •  observe visible aspects (artefacts and

behaviour, “above surface”) and; •  interpret this information to reveal the cultural

reasons behind (found in attitudes, values and basic assumptions, “below surface”)

•  For safety culture improvement, the IAEA emphasises human interactions (shared space) including trust, mindful communication, learning attitude, inquiring attitude, self-accountability, diversity, self-reflection etc.

IAEAs Approach to Safety Culture Continuous Improvement

Artefacts, Behaviour

Values

Attitudes

Basic Assumptions

IAEA

Shared Space •  Safe performance depends on the ability and

willingness of individuals to continuously think, engage, and demonstrate safe behaviours. It is shaped by •  Personal motivation •  Knowledge and understanding •  Shared space

•  A good shared space is characterized by •  Working relationships that support trust •  Decrease of power dynamics •  Mutual respect •  Openness – free flow in sharing of

thoughts and ideas •  Enables individuals to express views related

to their inner thoughts and feelings about a particular issue without fear of recrimination or exclusion

•  Shared space goes deeper than sharing facts •  Dialogue instead of discussion/argumentation

Me

Shared space

External space

IAEA

IAEA Safety Standards

The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.

IAEA

Safety Standard GS-G-3.5: Assessment of Safety Culture

Safety Culture Self-Assessement should: •  Include the entire organization •  Several different self-assessment tools should be used (e.g. interviews, focus

groups, questionnaires, observations and document reviews) •  A designated team representing all organizational levels and functions at the

installation should carry out the self-assessment •  A specialist in safety culture should be included in the team •  The self-assessment team should receive training •  The self-assessment team should summarize the results and identify areas for

improvement and may suggest actions to be taken •  The results should be reported to the management at an appropriate level •  A follow-up assessment should be performed The independent assessment of safety culture should follow a similar approach

IAEA

•  Member States requests to IAEA to provide practical guidance •  IAEA Safety Standards •  Behaviour and social science •  Past experiences

Basis of the safety culture assessment methodology

IAEA

Overall characteristics of the IAEA safety culture methodology

•  Multiple-methods approach •  Explorative, open approach •  Raw material for interpretation •  Data in itself say little about culture (tip of

the iceberg)

IAEA

•  Using several assessment methods

The IAEA Safety Culture Assessment Methodology

Document Review Interviews

Surveys

Focus Groups

Observations


Interviews

IAEA

Interview technique

S t r u c t u r e d S e m i - s t r u c t u r e d N o n - s t r u c t u r e d

•  Open-ended questions •  Striving towards a non-structured interview technique •  Explorative – more like a conversation


Focus Groups

IAEA

Focus Groups

•  The purpose of focus groups is to develop a broad and deep understanding rather than a quantitative summary

•  7 – 10 participants (2 Facilitators) •  Various cross-section of the organisation

•  Job, Grade, Age, Department, etc. •  A highly effective method for “listening” to others’ views

and gain insight into group dynamics •  Used to draw out attitudes, feelings, beliefs, experiences

and reactions in a way that is not feasible using other methods


Questionnaires/Surveys

IAEA

Why surveys?

•  To capture attitudes and impressions of a large population

•  To make sure that everyone in a organization has had an opportunity to make his/her voice heard

•  To establish a baseline and be able to track changes over time

IAEA

Disadvantages of surveys

•  Surveys identify symptoms rather than causes

•  The information collected is about what employees think they think – this is not the same as how they really act!

•  Surveys are subject to response bias, e.g. respondents may feel that they should respond in certain ways

•  Questionnaires say more about what the person asking questions thinks is important than what the respondent feels is important!

Safety culture only make sense in a qualitative perspective and caution should be made when using quantitative methods

IAEA

IAEAs Safety Culture Perception Questionnaire


Observations

IAEA

Why observations?

•  Visible manifestations of culture •  What you see is factual – whether it should have

happened or not! •  Should involve observations in different areas and

with different people across the organization to ensure the validity of the findings


Document Review

IAEA

Why document review?

•  Documents communicate the organizations values and expectations

•  Reveal approaches/beliefs related to ensuring compliance, e.g. how positional power authority is distributed, degree of formality, approaches to corrective actions, etc.

•  May reveal actual work practices, e.g. event reports.

•  Can show how the organization thinks, e.g. in documents with analytical content.

•  A document says a lot about how the organization presents itself – to itself.

IAEA

•  Separation of descriptive and normative


Descriptive

‘is’ Based on data and a theory of culture

Normative

‘should’ Based on data, a theory of culture and a norm

IAEA


•  Performed in silos – each assessment method treated separate

Survey Interviews Focus groups Observations

IAEA

The IAEA Safety Culture Analysis Process e. g. self-assessment

Final Issues; Normative, evaluative analysis

Overarching themes: Image(s) of culture

Interview data

Cultural expressions

Cultural themes

Survey data


Cultural themes

Focus group data


Cultural themes

Document data


Cultural themes

Observation data


Cultural themes

IAEA

Example of the Descriptive Analysis Process

Focus group data


Cultural themes

Observation data

Cultural

expressions

Cultural themes

Theme “Relaxed attitude towards radiological risks ” Cultural expressions •  Several incident of

body contamination and one internal

•  People express they did not go to RP re-training for last five years

•  People say that RP-techs are sitting mostly in their office

•  Managers have identified problems and taken decisions but nothing have changed

Theme “Contamination risk is not considered” Cultural expressions •  Skipping

contamination area borders

•  Not properly marked boarders

•  Chewing gum in RCA

•  Not wearing gloves when working in the RCA where its required

Overarching theme; Risk awareness related to actions for preventing contamination spread

IAEA

Normative Analysis

Overarching theme; Risk awareness related to actions for preventing contamination spread



IAEAs Approach to

ISCA Independent Safety Culture Assessment

within Operational Safety Review Team (OSART)

IAEA

Safety Culture Independent Assessment should: The independent assessment of safety culture should follow a similar approach as

self-assessment •  The independence and qualification of the members of the assessment team

should be considered crucial for the success of the assessment •  The team should be staffed with sufficient diversity of experience and should

include specialists in behavioural science, with knowledge of statistical methods of analysis

•  The independent assessment team should aim at identifying strengths and areas for improvement

Safety Standard GS-G-3.5: Assessment of Safety Culture

IAEA

The ISCA Teams Area of Expertise •  Psychology •  Cognitive science •  Sociology •  Social Psychology •  Organizational theory •  Cultural theory •  Leadership and management theory •  Human Factor / Human Factor Engineering •  Organizational Factors •  Resilience Engineering •  ITO (interaction between Individuals, Technology and

Organizations) Basic knowledge; Nuclear technology, nuclear organizations,

regulatory framework

IAEA

OSART Findings and Safety Culture

IAEA

Sources of information:

The IAEA ISCA Assessment Methods

Document Review Interviews

Surveys

Focus Groups

Observations

Team Findings

IAEA

Core of IAEA Safety Culture Analysis Process e. g. independent assessment within OSART

Interview data


Cultural themes

Survey data


Cultural themes

Focus group data


Cultural themes

Document data

Cultural

expressions

Cultural themes

Observation data

Cultural

expressions

Cultural themes

Overarching themes: Image(s) of culture


Team findings

data

Cultural

expressions

Cultural themes

IAEA

Example of amount of safety culture facts

•  Team findings: Circa 220 facts •  25 interviews: Circa 150 facts •  7 observations: Circa 30 facts •  7 focus groups: Circa 80 facts •  Survey 389 participated 97 questions In total about 480 fact excluding the survey material

IAEA

Reporting of the results

•  Recommendations for areas in need of attention

•  Three senior managment workshops to reach a shared understanding

•  Exit meeting •  Report

…Thank you for your attention