edison electric institute (eei) leading indicator industry practices s... · 2014-05-15 ·...
TRANSCRIPT
sneed
Edison Electric Institute
(EEI)
Leading Indicator
Industry Practices
Whitepaper
EEI Leading Indicator Project Team
April 30th, 2014
Information Technology Solutions SSaaffeettyy LLeeaaddiinngg IInnddiiccaattoorrss Supporting Employees Along The Road To An Injury Free Workplace
1
Table of Contents
1. Introduction .............................................................................................................. 2 2. Method ..................................................................................................................... 2
3. Trends and Findings ................................................................................................. 2 4. Prospective Solutions ............................................................................................... 4
5. Conclusions and Recommendations ........................................................................ 6 6. References ................................................................................................................ 8 7. Glossary ................................................................................................................... 9 8. Appendix A: Additional Analysis Tables .............................................................. 11
9. Appendix B: Survey Items with Summary of Survey Data ................................... 14
2
1. Introduction There are two functions to an organization’s safety performance: effort and
achievement (Kehoe, Claybaugh, Naglosky, Korczak, & Sauer, 2011). In order to
effectively manage hazards, safety metrics must account for both the effort made and the
results (i.e., achievement). Traditionally, companies have focused on the results or
lagging aspect of safety metrics such as Days Away, Restricted, and Transfers (DART)
and Occupational Safety and Health Administration (OSHA) recordables. However, it is
also necessary to include measures of the effort which lead to the results (i.e., leading
indicators).
This white paper was developed under the direction of the Edison Electric
Institute (EEI) Occupational Safety & Health Committee and is intended to share Leading
Indicator practices used within the utility industry. For years safety professionals from
across the globe have been touting the importance of incorporating the pro-active
approach of identifying, measuring and evaluating the information received from leading
indicators as the best practice to assure a safe work environment. Leading indicators are
proactive inputs or predictors of future safety performance based on selected criteria
(Fitzgerald, 2011). They place focus on the safety process and must be tied to specific
hazards (Kehoe et. al., 2012). When developed properly, leading indicators may have a
direct relationship to performance and thus provide valid mitigation actions. With the
institution of leading indicators, modifications to operations can be made before injuries
occur.
2. Method An electronic survey was sent to the membership of the Edison Electric Institute
(EEI) Occupational Safety & Health Committee members regarding leading indicators.
Approximately half of the membership companies responded, which allowed for a quality
sample size for verification. The survey included 26 questions covering demographics,
leading indicator types and effectiveness. The focus of the study was best practices in the
industry. Data on OSHA, DART, Lost Time, and Severity rate was also collected.
Traditionally, lagging safety indicators are used by companies to measure injury
data after the fact. Programs, policies, and procedures are developed primarily based on
this reactive type of data. However, an evolving trend in the electric utility industry, is
the incorporation of leading indicators as a measure of safety performance so they can
intervene before an accident happens.
3. Trends and Findings The primary outcome variables of interest in the survey were OSHA and DART
rates. Therefore, trends are reported in terms of their apparent relationship to OSHA and
DART rates.
Information was collected simply on whether or not participants track leading
indicators. Companies who track leading indicators had significantly lower OSHA and
DART rates than those who indicated that they do not track leading indicators. Therefore,
it can be reasoned that the mere presence of a leading indicator program, regardless of its
3
parameters, is associated with lower injury rates. Furthermore, participants were asked to
indicate what they believe the effects of their leading indicator program to be.
Approximately half of the companies’ who use leading indicators report that they have
resulted in a meaningful reduction of their DART rate.
Participants were also asked to indicate their communication strategies for leading
indicator metrics. One of the areas of research which was believed to be important and
have an effect on outcomes was how the leading indicator information is shared with
employees. Companies indicated their strategies were as follows:
50% report out to upper management,
66.7% report to all management
60% report the metrics to all employees
The data were assessed to evaluate how the companies in the top quartile of DART
(i.e., the lowest DART rates) communicated their leading indicators; this information was
different than the percentages above. All the top quartile companies had structured their
delivery of leading indicator results. The top quartile companies also communicated to all
of their employees, and not (only) to selected group.
Below is a chart to give the readers additional information on some of the leading
indicators that were communicated by the companies in the top quartile of DART:
Most Frequent Leading
Indicators Communication
Report
Level
Additional Leading
Indicators associated
with better
performance
DART for
Top
Quartile
Companies
1.Safety Observations
2. Training,
3. Improvement plans/
Corrective Action Plans
All employees
Top down –Bottom
up
Executive level to
Operational level to
local area work
centers
BBS, Safety Team
Effectiveness, Near
Misses, Audits, SWOs
Ninety percent of respondents are tracking safety leading indicators and the
majority of companies that use leading indicators as a metric are reporting only the
quantitative data (just the numbers). Although only producing quantitative data does not
appear to be the best form of communication, it is the opinion of the research team (based
on the outcome of the data) that the level of heightened awareness and openness of
communication influences an environment of caring and commitment to improving safety
which ultimately affects a strong safety culture.
4
The old adage of “if it’s not measured, it’s not managed” was very apparent in the
research as the top safety performing companies tracked their leading indicators and
incorporated them into organizational goals and objectives.
Based on the results of the survey, many of the companies indicated that their
attention to focus on leading indicators is in its early stages. Breaking the survey down
we find that:
10% of participants have been measuring leading indicators for less than a year
27% of participants have been measuring leading indicators for 1-3 years
17% of participants have been measuring leading indicators for 3-5 years
23% of participants have been measuring leading indicators for 5-8 years
10% of participants have been measuring leading indicators for 8- 10 years
13% of participants have been measuring leading indicators for 10 or more years
Most of the companies with a more mature leading indicator program were in the
top quartile of the study. The survey respondent’s DART rate ranged from .36 to 2.75
while DART for the 2012 EEI industrywide safety survey was .10 to 6.61.
4. Prospective Solutions This study found that behavioral safety programs were used in ALL top decile
companies. The implementation of Behavior Based Safety (BBS) programs was shown to
be a universal best practice for all top decile companies.
Safety Observations are the most frequently used measure associated with injury
prevention. Top decile companies incorporate Safety Observations into their safety
metrics and measured their results in both quantitative and qualitative formats.
All top decile companies had incorporated software solutions for capturing safety
observation data and training data. Systems and software solutions allow companies to be
consistent in identifying trends throughout their organizations, pro-actively, while
communicating improvement plans throughout their enterprise. Doing so will often times
mitigate a potential injury. An example of this could be as simple as an observation of an
employee using outdated PPE which had been recalled was identified in one area of the
organization but for some reason the initial communication of the recall was never
received by everyone and a re-issue of the recall was part of the improvement plan.
The survey identified that a safety observation program was the number one leading
indicator for injury prevention. Safety Observations are inclusive of BBS metrics. BBS
metrics have a stronger correlation in reducing injury rates indicating that the structure of
a BBS program is what is needed to drive injuries down.
The success of a safety observation program is enhanced by a BBS approach for the
following reasons:
Behavior focused
Interactions required in the field
Recording of observed behavior and interactions
5
Data trending and reporting
Trend focused and preventative
A successful training program which focuses on more than meeting the minimal
compliance requirements and incorporating identified or observed trends was noted by all
top quartile companies as a piece of the puzzle to a successful safety culture. A solid near
miss program and an improvement plan program or corrective action program (CAP)
were also identified as key performance metrics required to support a sustainable safety
culture.
Inclusion of BBS initiatives as a component of performance indication can help build a
case for change resulting in:
Improved injury rates
Reduction in associated Workers Compensation costs
Produces a measured performance across a consistent and common set of metrics
Incorporates rollup status across programs and divisions
Ensures entire organization knows what requirements need to be met
Establishes an enhanced work environment
Creates a new way of working – employee engagement
The goal and objective of any organization and safety professional is to provide
an environment free of recognized hazards. Our data supports the notion that a pro-active
approach to safety pays dividends. As the research team focused on rates, we began to
discuss the impact of a fatality on the rates. Below is an opinion based on the authors’
discussion while developing this paper.
An important fact to share is the impact of a fatality on rates if costs were
normalized. Although not scientific, we believe that sharing our thoughts may influence
upper management’s support of a leading indicator program.
Because the outcome below is based on a small group’s opinion, the authors of
this white paper would like to re-iterate that the information below is opinion and based
on assumption but it was believed that this information was not ever discussed in research
papers and may express that the lagging indicator rates that we capture year in and year
out are potentially not weighted to reflect the true impact to an organization:
Assumptions:
A couple of years ago (2007/2008) the average lost time cost was about $35,000
per NSC.
The BLS indicated 7 or 8 days was average time out per lost time in 2005 or so
and this goes up exponentially (doubles) when you have an aged work force
(which we all do).
The average fatality costs an organization anywhere from 4 million to 9 million
(Waerher et.al., 2004)
Therefore: The cost of 1 fatality equates to the cost of 114 lost time events:
6
Logic:
Using 4 million dollars as the baseline of a fatality and 35 thousand dollars as the
base line for a lost time injury; you could equate one fatality to 114 lost time events
which would significantly change the lagging indicator rate. (4,000,000/35,000)
Statement: A DART rate of 0.5 (25 DART cases) for a 5000 employee company
would increase to 2.78 with 1 fatality based on the above methodology.
Result: From an economic position, the economic impact of a fatality would move a
top decile company’s DART rate to fourth quartile.
5. Conclusions and Recommendations After reviewing all the data received it is the recommendation of the executive
committee of EEI recommends the following:
General recommendations:
Develop a long term plan (begin with a five year plan) that can be implemented,
improved and sustained –
o A strategy of a multi-year implementation plan occurs which begins with
management and safety professionals working together to design the goal
of the program, the manner in which you plan to capture the received
information and the process of engaging the workforce.
Communicate, communicate, and communicate. Design, develop and deliver a
consistent approach to reporting safety. Include safety metrics in an easy to
understand manner and communicate your organization’s successes and
opportunities to improve.
Safety Observations
All companies in the electric industry include a safety observation program which
incorporates measured results.
o Observations should be focused on behavior based safety rather than
conditions
o Results should be measured via an automated system.
Training
Include safety training on an annual basis.
o Engaging the work force to identify training improvements is very
powerful and this has been seen as a common theme within the research.
This training will not only include compliance based training but it will
refresh our expectations associated with leading indicators, safety culture
as well as the identified areas of improvement. This safety training shall
7
include the improvement opportunities and any new safety initiatives such
as an observation program, a near miss program as well as the reasoning
behind organizational decisions to implement a safe work practice which
goes above and beyond the minimal standards which OSHA and other
agencies hold the industry to.
Near Miss
Develop a near miss or close call program and recognize successes. The great
element associated with leading indicators is that OSHA encourages recognizing
pro-activity and actually frowns upon incentive programs associated with injury
reporting (lagging).
The EEI Occupational Safety and Health committee will continue to focus on leading
indicators and committees have been or will be developed to provide guidance on the best
way to introduce and implement the following:
1. Safety Observations
2. Safety Training
3. Near Misses/Close Calls
4. Safety Awards and Recognition
5. Area Audits/Safety Walkthroughs
8
6. References
Waehrer GM, Dong XS, Miller T, Haile E, & Men Y. Costs of occupational injuries in
construction in the United States (2004).
Fitzgerald, Bob, Manager—Construction Safety & Health, Southern Company (2012).
Measuring Leading Indicators. Southern Company, Engineering & Construction
Services
Kehoe, Gary, Claybaugh, Don, Naglosky, Bill, Korczak, Nick, & Sauer, Shane (2012).
Leading Indicator Development. Edison Mission Energy.
9
7. Glossary
Leading Indicators A leading indicator is a measure preceding or indicating a future
event used to drive and measure activities carried out to prevent and control injury.
Lagging Indicators Lagging indicators measure a company’s incidents in the form of
past accident statistics.
Safety Work Orders (SWO) Work orders generated and flagged as Safety
Walkthroughs A thorough assessment of a facility based on
program/procedure/policy requisites or aspects along with resident representatives when
dialogue is engaged regarding observations made and recommendations for corrections
are highlighted. In addition, audited employees are free to share constraints/issues
encountered during routine/non-routine activities.
Behavior Bases Safety (BBS) The use of behavioral psychology to promote safe
behavior in the workplace using employee engagement.
Corrective Action Plan (CAP)/Improvement plans The identification of specific,
meaningful resolutions to hazards which are created as a result of an incident and are
tracked to ensure they are completed on time.
OSHA Occupational Safety & Health Administration
DART Days Away Restrictions & Transfers
p Value (i.e., significance level) Indicates the probability that the results of a given
analysis are due to chance.
Standard Deviation Indicates how tightly all the various cases are clustered around
the mean.
10
R-Squared Describes the percentage of variance in the dependent variable (i.e., OSHA
& DART for the current study) which can be explained by the independent variable (i.e.,
number of leading indicators tracked).
N Sample size (i.e., number of respondents)
x Average
f ratio Statistic derived from an Analysis of Variance (ANOVA). It is the variance
between groups divided by the variance within groups. The f indicates if the experimental
treatment (i.e., companies who are tracking leading indicators in the current study) is
significantly different from the control group (i.e., companies who are not tracking
leading indicators in the current study). The f ration is derived from the ANOVA
equations (See ANOVA Table below).
ANOVA Analysis of variance is statistical technique used to measure the difference in
three or more averages.
11
8. Appendix A: Additional Analysis Tables
Tests of Between-Subjects Effects
Table 1: Model Summary for ANOVA, Dependent Variable: DART,
Independent variable: Who receives communication on leading indicators
Source Type III Sum
of Squares
Df Mean
Square
F Sig.
Corrected
Model 3.570
a 5 .714 2.743 .046
Intercept 21.779 1 21.779 83.689 .000
total_prog 3.570 5 .714 2.743 .046
Error 5.465 21 .260
Total 37.926 27
Corrected
Total 9.034 26
a. R Squared = .395 (Adjusted R Squared = .251)
Table 2: Average difference of
DART rate for companies
depending on if leading indicators
are communicated to upper
management, all management, or
all employees.
Number of
Programs
Average
DART
Upper
management
only 1.37
All management 1.44
All employees .604
12
Table 3: Average difference of DART and
OSHA rates for companies who are tracking
behavioral safety observations vs. not tracking
behavioral safety observations.
OSHA DART *
Safety_Observations
Safety_Observations OSHA DART
Not tracking Mean 2.310 1.136
N 5 5
Std.
Deviation .3799 .6602
Tracking Mean 1.784 .9589
N 19 18
Std.
Deviation 1.027 .4116
Table 4: Average difference of DART and
OSHA rates for companies who are tracking
near misses vs. not tracking near misses.
OSHA DART * Near Misses
Near_Misses OSHA DART
Not tracking Mean 2.5098 .998
N 10 6
Std. Deviation 1.93774 .103
Tracking Mean 1.037
N 22 18
Std. Deviation 1.21761 .377
13
Table 5: Average difference of DART and
OSHA rates for companies who are tracking
Training vs. not tracking Training.
OSHA DART * Training
OSHA DART
Not Tracking Mean 2.6031 1.5277
N 13 13
Std. Deviation 1.8643 1.41948
Tracking Mean 1.8643 0.9988
N 20 17
Std. Deviation 1.2486 0.79895
14
9. Appendix B: Survey Items with Summary of Survey Data Item 1:
Name of Parent Company:
Consumers Energy
Florida Power and Light
Iberdrola USA
National Grid
Integrys Energy Group
PPL Corporation
Pepco Holdings, INC
Bonneville Power Administration
Ni Source
American Transmission Company
CenterPoint Energy
Duke Energy
Southern California Edison
Northwestern Energy
DTE Energy
Henkels & McCoy
Note: Only the companies who gave permission to report their name appear in the above
list. Seventeen participants requested that their company name remain confidential and
therefore their contribution to the study is only reflected in the aggregate data below.
15
Item2:
Item 3:
If completing for a subsidiary of your company, please indicate the name:
Consumer’s Energy
Rochester Gas & Electric
New York State Electric & Gas
New Hampshire Gas
Maine Natural Gas
Central Maine Power
Wisconsin Public Service Corp
LGE-KU, PPL Montana
Northern Indiana Public Service Company
(NIPSCO)
Note: Subsidiaries are only listed for companies who gave permission to release the name
of their parent company.
Item 4:
Industry Type (select all that apply)
16
Item 5:
Company type (check all that apply)
Item 6:
17
Item 7:
Item 8:
Personal Information for contact in case clarification on survey responses is needed.
Name:
Position:
Please indicate the best way to contact you:
18
Item 9:
OSHA rate for 2012 – Recordable incidents include all work related deaths, illnesses, and
injuries which results in a loss of consciousness, restrictions of work or motion,
permanent transfer to another job within the company, or the require some type of
medical treatment.
Median OSHA rate: 1.44
Item 10:
DART rate for 2012 -- The total number of OSHA recordable cases which requires days
away from work for the year and the total number of OSHA recordable cases which
included days that employees were on transferred duties or on restricted work duty due to
an injury or illness.
Median DART: .95
Item 11:
Lost Work Cases for 2012 – The sum of deaths and Days Away from Work Cases
described above calculated from the total of the sums from OSHA for Column G and
Column H.
Median Lost Work Cases: .57
Item 12:
Severity Rate -- the number of lost days experienced as compared to the number of
incidents experienced.
Median Severity rate: 16.34
Item 13:
19
Item 14:
ds
Item 15:
20
Item 16:
Item 17:
Please indicate the safety leading indicators your company is using, if it is part of your
safety metrics, and finally if the metric is quality or quantity.
Do you collect data on the leading indicator?
Answer Options Yes No Response
Count
Safety Observations 27 3 30
Behavior Based Safety Observations 17 11 28
Safety Training 25 4 29
Hazard Resolutions 18 8 26
Corrective Action Plans 19 9 28
Safety Awards and Recognition 22 6 28
Safety Team/Committee Effectiveness 10 18 28
Tailboards/Pre Job Briefing 18 10 28
Near Misses/Close Calls 23 5 28
Preventative Health & Wellness Programs 14 13 27
Work Readiness/Pre-Shift Stretching 9 18 27
Area Audits/Safety Walkthroughs 21 8 29
Leadership Engagement 13 15 28
Safety Culture Grassroots Teams 4 22 26
Safety Culture Perception Survey 15 13 28
Employee Participation in Safety
Activities
14 13 27
Safety Work Orders 13 14 27
21
If yes, is it part of your safety metrics?
Answer Options Yes No NA Response
Count
Safety Observations 21 8 0 29
Behavior Based Safety Observations 10 12 4 26
Safety Training 13 12 2 27
Hazard Resolutions 6 15 4 25
Corrective Action Plans 7 15 3 25
Safety Awards and Recognition 5 18 3 26
Safety Team/Committee Effectiveness 6 12 5 23
Tailboards/Pre Job Briefing 7 15 3 25
Near Misses/Close Calls 11 13 1 25
Preventative Health & Wellness Programs 2 19 3 24
Work Readiness/Pre-Shift Stretching 1 18 3 22
Area Audits/Safety Walkthroughs 12 13 2 27
Leadership Engagement 7 12 4 23
Safety Culture Grassroots Teams 1 16 4 21
Safety Culture Perception Survey 9 13 2 24
Employee Participation in Safety Activities 7 12 4 23
Safety Work Orders 2 16 4 22
Do you measure quantity or quality?
Answer Options Quality Quantity Both NA Response
Count
Safety Observations 0 13 13 3 29
Behavior Based Safety Observations 2 7 8 9 26
Safety Training 1 7 15 4 27
Hazard Resolutions 0 4 8 13 25
Corrective Action Plans 0 5 8 11 24
Safety Awards and Recognition 0 7 5 12 24
Safety Team/Committee Effectiveness 1 4 3 15 23
Tailboards/Pre Job Briefing 0 5 10 10 25
Near Misses/Close Calls 3 9 9 5 26
Preventative Health & Wellness Programs 0 4 6 12 22
Work Readiness/Pre-Shift Stretching 0 3 2 18 23
Area Audits/Safety Walkthroughs 1 5 9 13 28
Leadership Engagement 0 7 4 12 23
Safety Culture Grassroots Teams 0 0 1 20 21
Safety Culture Perception Survey 2 0 10 12 24
Employee Participation in Safety
Activities 1 6 3 13 23
Safety Work Orders 0 8 1 14 23
Item 18:
Please indicate the target or goal for each leading indicator you are measuring:
Contact EEI if you would like further information on the qualitative responses to this
item.
22
Item 19:
Item 20: Please indicate the definitions for the leading indicators your organization is measuring.
Contact EEI for more information on the qualitative responses to this item.
Item 21: Please select the top three leading indicators that have been the most effective in injury prevention and rank
them in order of effectiveness from one to three with one being equivalent to the highest degree of effectiveness.
Only select your top three and mark NA for the remaining indicators on the list below.
Ranking
Answer Options 1 2 3 NA Response
Count
Safety Observations 12 1 5 12 30
Behavior Based Safety Observations 1 4 0 25 30
Safety Training 3 2 3 22 30
Hazard Resolutions 2 0 2 26 30
Corrective Action Plans 3 3 4 20 30
Safety Awards and Recognition 0 1 2 27 30
Safety Team/Committee Effectiveness 1 1 0 28 30
Tailboards/Pre Job Briefing 5 4 5 16 30
Near Misses/Close Calls 3 5 5 17 30
Preventative Health & Wellness Programs 0 2 2 26 30
Work Readiness/Pre-Shift Stretching 1 2 0 27 30
Area Audits/Safety Walkthroughs 2 2 2 24 30
Leadership Engagement 3 4 0 23 30
Safety Culture Grassroots Teams 1 2 0 27 30
Safety Culture Perception Survey 0 3 0 27 30
Employee Participation in Safety Activities 2 0 2 26 30
Safety Work Orders 0 2 0 28 30
Other 0 0 1 29 30
23
Item 22:
Item 23:
24
Item 24:
Item 25: Please provide any additional comments if you would like to share something
we did not ask.
This paper was researched, developed and written on behalf of Edison Electric Institute
and the members of their Occupational Safety and Health committee. The author’s
opinions and recommendations are based on facts gathered through the survey and their
many years of experience.
Contact EEI for more information on the qualitative responses to this question.