MANAGING RISKTHE HUMAN ELEMENT

By

Romney B. Duffey BSc, PhD, FASME

John W. Saull CEng, FRAeS, DAE

A John Wiley & Sons, Ltd., Publication

m6022.2072_cover_page.jpg

MANAGING RISK

Frontispiece

The risk of the unknown in exploring a new homo-technological system produced the fi rst known prior outcome or fatality caused by humans fl ying. In the classical Greek myth illus-trated here on this platter, Daedalus can be seen helplessly watching Icarus fall to his death below him. Daedalus, the designer of the Minotaur’s Labyrinth and imprisoned there, in order to escape had fi tted himself and his son, Icarus, with these innovative wings. But during the escape, Icarus became intoxicated by this new power of fl ight and, despite Daedalus’s repeated warnings and his lack of experience, took the risk of fl ying so high that the sun melted the wax holding his feathered wings. (Photo © John W. Saull.)

MANAGING RISKTHE HUMAN ELEMENT

By

Romney B. Duffey BSc, PhD, FASME

John W. Saull CEng, FRAeS, DAE

A John Wiley & Sons, Ltd., Publication

This edition fi rst published 2008Copyright © 2005 and 2008 Romney B. Duffey and John W. Saull, Published by John Wiley & Sons, Ltd

Registered offi ceJohn Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offi ces, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identifi ed as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

ISBN: 978-0-470-69976-8

Library of Congress Cataloging-in-Publication Data

Duffey, R. B. (Romney B.) Managing risk : the human element / by Romney B. Duffey, John W. Saull. p. cm. Includes bibliographical references and index. ISBN 978-0-470-69976-8 (cloth) 1. Industrial safety. 2. Industrial accidents. I. Saull, John Walton, 1935– II. Title. T55.D816 2008 363.11–dc22 2008033263

A catalogue record for this book is available from the British Library.

Set in 10 on 12 pt Times by SNP Best-set Typesetter Ltd., Hong KongPrinted in Singapore by Markono

www.wiley.com

Contents

About the Authors xiii

Preface xv

Acknowledgements xix

Defi nitions of Risk and Risk Management xxi

Introduction: The Art of Prediction and the Creation of Order 1Risk and Risk Management 1Defi ning Risk 2Managing Risk: Our Purpose, Plan and Goals 4Recent Tragic Outcomes 6Power Blackouts, Space Shuttle Losses, Concorde Crashes, Chernobyl, Three Mile

Island and More . . . 6How Events and Disasters Evolve in a Phased Development: The Human Element 8Our Values at Risk: The Probable Improvement 10Probably or Improbably Not 11How this Book is Organised 12References 14

Technical Summary 15Defi ning the Past Probability 15Predicting Future Risk: Sampling from the Jar of Life 16A Possible Future: Defi ning the Posterior Probability 21The Engineers Have an Answer: Reliability 22Drawing from the Jar of Life: The Hazard Function and Species Extinction 23Experiencing Failure: Engineering and Human Risk and Reliability 25Experience Space 27Managing Safely: Creating Order out of Disorder Using Safety Management Systems 29Describing the Indescribable: Top-Down and Bottom-Up 30What an Observer will Observe and the Depth of our Experience 31References 33

1 The Universal Learning Curve 35Predicting Tragedies, Accidents and Failures: Using the Learning Hypothesis 35The Learning Hypothesis: The Market Place of Life 37Learning in Homo-Technological Systems (HTSs): The Way a Human Learns 39

vi Contents

Evidence of Risk Reduction by Learning 41Evidence of Learning from Experience: Case Studies 42Evidence of Learning in Economics 43Evidence of Learning in Engineering and Architecture: The Costs of Mistakes 44Learning in Technology: the Economics of Reducing Costs 46Evidence of Learning Skill and Risk Reduction in the Medical Profession:

Practice Makes Almost Perfect 48Learning in HTSs: The Recent Data Still Agree 50The Equations That Describe the Learning Curve 52Zero Defects and Reality 54Predicting Failures: The Human Bathtub 55Experience Space: The Statistics of Managing Safety and of Observing Accidents 55Predicting the Future Based on Past Experience: The Prior Ignorance 57Future Events: the Way Forward Using Learning Probabilities 58The Wisdom of Experience and Inevitability 59The Last, First or Rare Event 59Conclusions and Observations: Predicting Accidents 60References 61

2 The Four Echoes 63Power Blackouts, Space Shuttle Losses, Concorde Crashes, and the Chernobyl

and Three Mile Island Accidents 63The Combination of Events 64The Problem Is the Human Element 65The Four Echoes Share the Same Four Phases 66The First Echo: Blackout of the Power Grid 67Management’s Role 69The First Echo: Findings 71Error State Elimination 73The Second Echo: Columbia/Challenger 75The Results of the Inquiry: Prior Knowledge 76The Second Echo: The Four Phases 79Management’s Responsibility 80Error State Elimination 82The Third Echo: Concorde Tires and SUVs 83Tire Failures: the Prior Knowledge 84The Third Echo: The Four Phases 87Management’s Responsibility 87Error State Elimination 87The Fourth Echo: Chernobyl 88An Echo of Three Mile Island 88The Consequences 92Echoes of Three Mile Island 92The Causes 93Error State Elimination 94The Fourth Echo: The Four Phases 95Regulatory Environment and Practices 95Case study: Regulation in Commercial Aviation 96

a) Regulations Development 96b) Compliance Standards 97c) Accident Investigation 97

Contents vii

Addressing Human Error 98Management Responsibilities 99Designing to Reduce Risk and the Role of Standards 99Conclusion and Echoes: Predicting the Unpredictable 101References 103

3 Predicting Rocket Risks and Refi nery Explosions: Near Misses, Shuttle Safety and Anti-Missile Defence Systems Effectiveness 105

Learning from Near Misses and Prior Knowledge 105Problems in Quantifying Risk: Predicting the Risk for the Next Shuttle Mission 107Estimating a Possible Range of Likelihoods 112Learning from Experience: Maturity Models for Future Space Mission Risk 114Technology versus Technology 120Missiles Risks over London: The German Doodlebug 121Launching Missile Risk 124The Number of Tests Required 126Estimating the Risk of a Successful Attack and How Many Missiles We Must Fire 128Uncertainty in the Risk of Failing to Intercept 128What Risk Is There of a Missile Getting Through: Missing the Missile 131Predicting the Risk of Industrial Accidents: The Texas City Refi nery Explosion 132From Lagging to Leading: Safety Analysis and Safety Culture 134Missing Near Misses 137What these Risk Estimates Tell Us: The Common Sense Echo 137References 138

4 The Probability of Human Error: Learning in Technological Systems 141What We Must Predict 141The Probability Linked to the Rate of Errors 144The Defi nition of Risk Exposure and the Level of Attainable Perfection 146Comparison to Conventional Social Science and Engineering Failure and

Outcome Rate Formulations 147The Learning Probabilities and the PDFs 150The Initial Failure Rate and its Variation with Experience 150The ‘Best’ MERE Risk Values 153Maximum and Minimum Likely Outcome Rates 155Standard Engineering Reliability Models Compared to the MERE Result 155Future Event Estimates: The Past Predicts the Future 157Statistical Bayesian-Type Estimates: The Impact of Learning 158Maximum and Minimum Likelihood 161Comparison to Data: The Probability of Failure and Human Error 161Comparison of the MERE Result to Human Reliability Analysis 164Implications for Generalised Risk Prediction 168Conclusions: The Probable Human Risk 170References 171

5 Eliminating Mistakes: The Concept of Error States 173A General Accident Theory: Error States and Safety Management 173The Physics of Errors 174The Learning Hypothesis and the General Accident Theory 176Observing Outcomes 178A Homage to Boltzmann: Information from the Grave 181

viii Contents

The Concept of Depth of Experience and the Theory of Error States 184The Fundamental Postulates of Error State Theory 188The Information in Error States: Establishing the Risk Distribution 189The Exponential Distribution of Outcomes, Risk and Error States 192The Total Number of Outcomes 193The Observed Rate and the Minimum Number of Outcomes 195Accumulated Experience Measures and Learning Rates 198The Average Rate 200Analogy and Predictions: Statistical Error Theory and Learning Model Equivalence 201The Infl uence of Safety Management and Regulations: Imposing Order

on Disorder 201The Risk of Losing a Ship 203Distribution Functions 205The Most Probable and Minimum Error Rate 208Learning Rates and Experience Intervals: The Universal Learning Curve 209Reducing the Risk of a Fatal Aircraft Accident: the Infl uence of Skill

and Experience 212Conclusions: A New Approach 215References 216

6 Risk Assessment: Dynamic Events and Financial Risks 219Future Loss Rate Prediction: Ships and Tsunamis 221Predicted Insurance Rates for Shipping Losses: Historical Losses 224The Premium Equations 225Financial Risk: Dynamic Loss and Premium Investments 226Numerical Example 227Overall Estimates of Shipping Loss Fraction and Insurance Inspections 228The Loss Ratio: Deriving the Industrial Damage Curves 229Making Investment Decisions: Information Drawing from the Jar of Life 231Information Entropy and Minimum Risk 232Progress and Learning in Manufacturing 233Innovation in Technology for the Least Product Price and Cost: Reductions During

Technological Learning 234Cost Reduction in Manufacturing and Production: Empirical Elasticity ‘Power Laws’

and Learning Rates 235A New General Formulation for Unit Cost Reduction in Competitive Markets:

the Minimum Cost According to a Black-Scholes Formulation 237Universal Learning Curve: Comparison to the Usual Economic Power Laws 240The Learning Rate b-Value ‘Elasticity’ Exponent Evaluated 242Equivalent Average Total Cost b-Value Elasticity 244Profi t Optimisation to Exceed Development Cost 246The Data Validate the Learning Theory 247

a) Aircraft Manufacturing Costs Estimate Case 247b) Photovoltaic Case 248c) Air Conditioners Case 250d) Ethanol Prices Case 251e) Windpower Case 252f) Gas Turbine Power Case 253g) The Progress Curve for Manufacturing 254

Non-Dimensional UPC and Market Share 256

Contents ix

Conclusions: Learning to Improve and Turning Risks into Profi ts 259References 260

7 Safety and Risk Management Systems: the Fifth Echoes 263Safety Management Systems: Creating Order Out of Disorder 263Workplace Safety: The Four Rights, Four Wrongs and Four Musts 264Acceptable Risk: Designing for Failure and Managing for Success 265Managing and Risk Matrices 269Organisational Factors and Learning 272A Practical ‘Safety Culture’ Example: The Fifth Echo 273Safety Culture and Safety Surveys: The Learning Paradox 278Never Happening Again: Perfect Learning 280Half a World Apart: Copying the Same Factors 281Using a Bucket: Errors in Mixing at the JCO Plant 283Using a Bucket: Errors in Mixing at the Kean Canyon Explosives Plant 284The Prediction and Management of Major Hazards: Learning from SMS Failures 286Learning Environments and Safety Cultures: The Desiderata of Desires 289Safety Performance Measures: Indicators and Balanced Scorecards 291Safety and Performance Indicators: Measuring the Good 292Human Error Rates Passing Red Lights, Runway Incursions and Near Misses 293Risk Informed Regulation and Degrees of Goodness: How Green is Green? 294Modelling and Predicting Event Rates and Learning Curves Using Accumulated

Experience 297Using the Past to Predict the Future: How Good is Good? 299Reportable Events 300Scrams and Unplanned Shutdowns 301Common-Cause Events and Latent Errors 303Performance Improvement: Case-by-Case 304Lack of Risk Reduction: Medical Adverse Events and Deaths 305New Data: Sentinel Events, Deaths and Blood Work 308Medication Errors in Health Care 313Organisational Learning and Safety Culture: the ‘H-Factor’ 316Risk Indicator Data Analysis: A Case Study 319Meeting the Need to Measure Safety Culture: the Hard and the Soft Elements 321Creating Order from Disorder 324References 324

8 Risk Perception: Searching for the Truth Among all the Numbers 329Perceptions and Predicting the Future: Risk Acceptance and Risk Avoidance 329Fear of the Unknown: The Success Journey into What We Do or Do Not Accept 333A Possible Explanation of Risk Perception: Comparisons of Road and Rail Transport 334How Do We Judge the Risk? 337Linking Complexity, Order, Information Entropy and Human Actions 338Response Times, Learning Data and the Universal Laws of Practice 341The Number and Distribution of Outcomes: Comparison to Data 343Risk Perception: Railways 345Risk Perception: Coal Mining 348Risk Perception: Nuclear Power in Japan 349Risk Perception: Rare Events and Risk Rankings 352Predicting the Future Number of Outcomes 354

x Contents

A Worked Example: Searching out and Analysing Data for Oil Spills 354Typical Worksheet 358Plotting the Data 358Fitting a Learning Curve 358Challenging Zero Defects 359Comparison of Oil Spills to Other Industries 362Predicting the Future: the Probability and Number of Spills 364Observations on this Oil Spill Case 365Knowing What We Do Not Know: Fear and Managing the Risk of the Unknown 365White and Black Paradoxes: Known Knowns and Unknown Unknowns 367The Probability of the Unknowns: Learning from What We Know 368The Existence of the Unknown: Failures in High Reliability Systems 370The Power of Experience: Facing Down the Fear of the Unknown 371Terrorism, Disasters and Pandemics: Real, Acceptable and Imaginary Risks 373Estimating Personal Risk of Death: Pandemics and Infectious Diseases 374Sabotage: Vulnerabilities, Critical Systems and the Reliability of Security Systems 377What Is the Risk? 378The Four Quadrants: Implications of Risk for Safety Management Systems 378References 380

9 I Must Be Learning 383Where We Have Come From 383What We Have Learned 384What We Have Shown 388Legal, Professional and Corporate Implications for the Individual 389Just Give Me the Facts 391Where We are Going 392Reference 393

Nomenclature 395

Appendices: 401Appendix A: The ‘Human Bathtub’: Predicting the Future Risk 403

The Differential Formulation for the Number of Outcomes 405The Future Probability 406Insuffi cient Learning 408

Appendix B: The Most Risk, or Maximum Likelihood, for the Outcome (Failure or Error) Rate while Learning 411

The Most or Least Likely Outcome Rate 411The Maximum and Minimum Risk: The Two Solutions 412Low Rates and Rare Events 413The Limits of Maximum and Minimum Risk: The Two Solutions 414Common Sense: The Most Risk at the Least Experience and the Least Risk as

the First Outcome Decreases with Experience 414Typical Trends in Our Most Likely Risk 415The Distribution with Depth of Experience 417References 418

Appendix C: Transcripts of the Four Echoes 419

Contents xi

Power Blackout, Columbia Space Shuttle loss, Concorde Crash and Chernobyl Accident 419The Combination of Events 419The Four Echoes Share the Same Four Phases 420Appendix. Blackout Chronology and the Dialog from Midday 14 August 2003 420The Second Echo: Columbia/Challenger 432Appendix: Shuttle Dialog and Transcripts 433The Third Echo: Concorde Tires and SUVs 435Appendix: Dialog for the Concorde Crash 436The Fourth Echo: TMI/Chernobyl 439Appendix: Chronology and Transcripts of the Chernobyl Reactor Unit 4 Accident 439Conclusion and Echoes: Predicting the Unpredictable 444

Appendix D: The Four Phases: Fuel Leak Leading to Gliding a Jet in to Land without any Engine Power 447

The Bare Facts and the Sequence 447The Four Phases 449Flight Crew Actions 455

Initial Recognition of the Fuel Loss (04:38–05:33) 455Crew Reaction to the Fuel Imbalance Advisory (05:33–05:45) 456Crew Reaction to the Continued Fuel Loss (05:45–06:10) 458Crew Reaction to the (Two) Engine Failures 460

References 463

Appendix E: The Four Phases of a Midair Collision 465The Bare Facts 465The Four Phases 465References 469

Appendix F: Risk From the Number of Outcomes We Observe: How Many are There? 471The Number of Outcomes: The Hypergeometric Distribution 472Few Outcomes and many Non-Outcomes: The Binomial and Poisson Distributions 475The Number of Outcomes: In the Limit 478The Perfect Learning Limit: Learning from Non-Outcomes 479The Relative Change in Risk When Operating Multiple Sites 481References 482

Appendix G: Mixing in a Tank: The D.D. Williamson Vessel Explosion 483Errors in Mixing in a Tank at the Caramel Factory: The Facts 483The Prior Knowledge 484Another Echo 488References 490

Appendix H: Never Happening Again 491The Risk of an Echo, or of a Repeat Event 491The Matching Probability for an Echo 493The Impact of Learning and Experience on Managing the Risk of Repeat Events 494The Theory of Evidence: Belief and Risk Equivalence 496References 497

xii Contents

Appendix I: A Heuristic Organisational Risk Stability Criterion 499Order and Disorder in Physical and Management Systems 499Stability Criterion 500References 502

Appendix J: New Laws of Practice for Learning and Error Correction 505Individual Learning and Practice 505Comparison to Error Reduction Data 506Comparison to Response Time Data and the Consistent Law of Practice 509Reconciling the Laws 511Conclusions 512References 513

Appendix K: Predicting Rocket Launch Reliability – Case Study 515Summary 515Theory of Rocket Reliability 515

a) Unknown Total Number of Launches and Failures 516b) Known Total Number of Launches and Failures 517

Results 518Measures of Experience 519Comparsion to World Data 520Predicting the Probability of Failure 521Statistical Estimates of the Failure Probability for the Very ‘Next’ Launch 523Independent Validation of the MERE Launch Failure Curve 525Observations 526References 526

Index 527

Romney Beecher Duffey

Romney Duffey, B.Sc. (Hons), Ph.D., FASME, is an internationally recognised scientist, manager, speaker and author, having written more than 200 papers and articles on the risk, safety and design of modern energy systems. Dr. Duffey has also co - authored the book Know the Risk concerning the safety of modern technological systems and the role of human error.

He has a distinguished 30 - year career examining the safety and performance of nuclear systems in Europe and the USA. He is presently the Principal Scientist for Atomic Energy of Canada (AECL) with a wide range of responsibilities, including advanced and future concepts, advanced product development, advice on overall R & D directions, international col-laborations, analysis of global energy and environ-ment scenarios and energy policy, and senior - level reviews.

He is an ASME Fellow, a past Chair of the ASME Nuclear Engineering Division, an active member of

the American, Canadian and British Nuclear Societies, and a past Chair of the American Nuclear Society Thermal - Hydraulics Division.

Photo © RB

About the Authors