common threads among catastrophic mishaps · common threads among catastrophic mishaps lessons not...
Post on 30-May-2018
223 Views
Preview:
TRANSCRIPT
Common Threads Among Catastrophic Mishaps
LessonsNotLearnedVulnerableDesign
WorkmanshipShortcomingsProcessControlFailures
FailuretoControlCri=calMaterialItemsFraud
BrianHughiB OfficeofSafetyandMissionAssurance
PrecursorEventIneffec/veCorrec/veAc/on
VulnerableDesign
Fraud Workmanship/ProcessControlShortcomings
MaterialControlInadequacies
BigDigTunnelCollapse
X X X X
TurkishAirlinesFlight981
X X X
USSThresher
X X X X
Apollo1
X X X X
USSIwoJima
X
Orbi=ngCarbonObservatory/Glory
X X X X
AntaresOrb-3
X X
Falcon9-20
X2
• 7.5milecorridor• 161lanemiles• 5milesoftunnel• 6interchanges• 200bridges• 541,000truckloadsofdirt• 3.8millioncubicyardsofconcrete
Originallyscheduledtobecompletedin1998atacostof$2.6billion, theprojectwasfinallycompletedin2006atacostof$14billion.
3
The Big Dig
OnSeptember9,1999,aconstruc=oncontractoremployeeinstallingven=la=onductworkoverthetunnelceilingno=cedthatseveraloftheanchorshadbeguntopullout.
The Smoking Gun
OnNovember12,1999,aproofloadtestwasperformedononeoftheanchorsthathadshownsignificantnine-sixteenths-inchdisplacement.Theengineernotedthat“theboltheldforafewseconds,thenbegantopulloutwithalmostnoresistance”.
4
The Gi;
5
The Supplier’s Response
Whenthesupplierwascalledtoexaminetheanchordisplacements,theyseemedsurprisedthattheanchorsthathadbeensuccessfullyprooftestedonlyafewmonthsbeforecouldbefailing.Installa=onproblems(e.g.,excessivepreload)werepostulatedasthecause.
Noevidencewasfoundthatthesuppliertookanyfollow-upac=onadertheexamina=on.
-Nofurthertes=ng
-Nofurtherresearch
“At least some supplier officials were aware that their Fast Set epoxy was subject to creep, but this information was apparently not considered or was not known by the representatives who evaluated the failed anchors. Even if the information about poor creep resistance was not common knowledge, a reasonable amount of research would likely have revealed it. The Safety Board would have expected the supplier of a safety critical component to have been more proactive in determining why its product was failing.”
The Builder, Design Agent, and Project Manager Reply
Increasedproofloadtes=ng
Therootcauseforthehangerdisplacementwasneveriden=fied……andsurveillancemonitoringinspec=onswereneverimplemented.
20,000#
15,000#
10,000#
5,000#
0
LbF
Design Service Load Post Installation
Proof Test Finite Element Analysis
2,600 #
Calculated design service load
(2,600 Lb-Force)
20,000#
15,000#
10,000#
5,000#
0
LbF
Design Service Load Post Installation
Proof Test
Finite Element Analysis
2,600 # 3,250 #
After each bolt was installed, a proof test was conducted at 25% higher than design service load
(3,250 Lb.- Force)
20,000#
15,000#
10,000#
5,000#
0
LbF
Design Service Load Post Installation
Proof Test Finite Element Analysis
2,600 #
6,350 #
3,250 #
Later, after slippage was noted, bolts were proof tested to the maximum allowable load
(6,350 Lb-Force)
15,000#
10,000#
5,000#
0
LbF
Design Service Load Post Installation
Proof Test Finite Element Analysis
2,600 # 2,371 # 2,823 #
6,350 #
3,250 #
A finite element analyses determined that the load would be between 2,371 and 2,823 lb force
"You’ve noted the key piece of information that is missing. That is the
cause of the anchor failure and how the repair procedure will overcome that… We are not trying to hold up construction, we are trying to make a determination that the installation is safe…” Design Manager e-mail concerning response to Deficiency Report
“Glaringly absent from the Deficiency Report is any explanation why the anchors failed and what steps are proposed to ensure that this problem does not reoccur.” Structural Engineer e-mail reply
The Gi; (Part 2)
OnDecember17,2001,aqualitycontrolinspectorsubmiBedaNoncomplianceReport,whichstated:
“Severalanchorsappeartobepullingawayfromtheconcrete.Thesubjectanchorswerepreviouslytestedtotherevisedvalueof6,350pounds,allofwhichpassed.[…]Reasonforfailureisunknown.”
“Atthispoint,itshouldhavebeenobvious[…]thattheremedythathadbeendevelopedinresponsetotheanchordisplacementinthe[HighOccupancyVehicle]tunnelin1999hadnotbeeneffec/ve,asanchorsthathadpassedprooftes/ngathighervaluesweres/lldisplacing.Thiswasanotheropportunityto[…]inspectalltheinstalledanchorstodeterminetheextentand,moreimportantly,thecauseoftheanchordisplacement.Instead,thecompaniesapparentlyconsideredthecon/nuingfailuresasisolatedinstancesandtooknoac/ontoaddresstheprobleminasystemicway.”
NTSBAccidentReport
13
Accident Synopsis
At11p.m.onJuly10,2006,a1991Buickpassengercaroccupiedbya46-year-oldmaledriverandhis38-year-oldwifewastravelingeastboundintheI-90connectortunnelinBoston,MA,enroutetoLoganInterna=onalAirport.Asthecarapproachedtheendoftheconnectortunnel,asec=onofthetunnel’ssuspendedconcreteceiling(26tons)detachedfromthetunnelroofandfellontothevehicle,crushingitsrightside.
14
ProximateCause
Useofanepoxyanchoradhesivewithpoorcreepresistance
15
NTSB Accident Report
Creep
ASTMD2990-01 Sincetheproper=esofviscoelas=cmaterialsaredependenton=me…aninstantaneoustestresultcannotbeexpectedtoshowhowamaterialwillbehavewhensubjectedtostressordeforma=onforanextendedperiodof=me.
Epoxyisapolymeranditss=ffnessis=meandtemperaturedependent.Ifaloadisappliedsuddenly,theepoxyrespondslikeahardsolid.Butifthatloadisthenheldconstant,themoleculeswithinthepolymermaybegintorearrangeandslidepastoneanother,causingtheepoxytograduallydeform.Asthedeforma=onincreases,itbecomesirreversible.
Epoxy Secured Bolts
CreepTheUnknownKnown
20
“At least some supplier officials were aware that their Fast Set epoxy was subject to creep, but this information was apparently not considered or was not known by the representatives who evaluated the failed anchors. Even if the information about poor creep resistance was not common knowledge, a reasonable amount of research would likely have revealed it. The Safety Board would have expected the supplier of a safety critical component to have been more proactive in determining why its product was failing.”
NTSB Accident Report
“Thisaccidentinves=ga=onrevealedastrikinglackofawarenessamongdesigners,contractors,managers,andoverseersaboutthenatureandperformanceofpolymeradhesives,evenasthoseadhesiveswerebeingapprovedforuseanapplica=onswhereafailurewouldpresentanimmediatethreattothepublic.
Evenaderbeingpresentedwithevidenceofanchorcreep,projectmanagersandoverseersfailedtorecognizetheinherentweaknessintheepoxyadhesive–aweaknessthatcouldnotbeovercomeevenwiththebestinstalla=onprac=cesorthemostrigorousshort-termprooftes=ng.”
NTSBAccidentReport
TheUnknownKnown(cont)
CogniGve Dissonance
From Wikipedia:
• A psychological term describing the uncomfortable tension that may result from having two conflicting thoughts at the same time, or from engaging in behavior that conflicts with one's beliefs, or from experiencing apparently conflicting phenomena.
• In simple terms, it can be the filtering of information that conflicts with what
you already believe, in an effort to ignore that information and reinforce your beliefs.
Wikipedia
Accidents related failures of imagination
Failure of imagination has been invoked in regards to the Apollo 1 fire by astronaut Frank Borman in 1967 when he spoke at the Apollo 1 investigation hearings. It has also been mentioned in reference to design flaws in the RMS Titanic and … the failure of the United States to anticipate the attack on Pearl Harbor.
Failure of imagination From Wikipedia: A failure of imagination is a circumstance wherein something seemingly predictable (particularly from hindsight) and undesirable was not planned for.Failure of imagination is related to unknown unknowns.
ContribuGng Causal Factor
- Inadequate Industry Standards -
ICCAC58:
Eitheradesignsafetyfactorof5.33ora120-daycreeptestisrequiredforFastSetepoxy.
“Given that the ability to sustain a load over a period of time is a typical requirement for almost any type of fastener, the Safety Board is concerned that the ICC has allowed creep testing of epoxy adhesives to be optional. A design engineer should be provided with all of the relevant information about a product before it is used in a safety critical application.” NTSB Accident Report
Consequently…
Tosupportproductqualifica=on,thesupplierprovidedanEvalua=onReport(ER)whichincludedbondstrengthtablesspecifyingasafetyfactorof5.33forFastSetepoxy-nottheresultsofcreeptests*.
*TheSafetyBoardlearnedduringtheinves=ga=onthatFastSetepoxyhadbeentestedforcreep
performancein1995and1996andhadfailedtomeetthestandard
26
27
DesGnaGon Disaster
OnMarch3,1974,TurkishAirlinesFlight981,onarou=neflightfromParistoLondon,crashedinadenseforestinFrance,resul=nginthelossofall346personsaboard.At11,500feet,thedifferen=alpressureinthecabincausedtheadcargodoortoopenandbeblownoff.Thelargeholesuddenlyappearinginapressurehullcreatedanoutwardaccelera=onofairsorapidastoresembleabombexplosion.
Theexplosiondestroyedtheflooringabovethecargohold,severingthecontrolcablesfortherudder,theelevators,andthenumbertwoengine.
28
The proximate cause of the accident was determined to be a faulty latch.
29
30
LatchDesign
“Thereweremul=plecomplexlinkagesbetweentheexternalhandleandthelockingpinbarwhich,inaggregate,werefartooweakandflexible.Ratherthanencounteringanirresis=bleforceifthelockingpinshitthelugsofanunclosedlatch,abaggagehandlerofnormalstrengthcouldpushthehandlefullydown,thinkingthathehadthusinsuredtheclosingofthedoorwhenallhehaddonewasbendtheinternalbarsandrodsoutofshape.”
31
“Itwas,byanysenseofsafetyengineering,agimcrackpieceofdesign.Yet,becauseofdecisionstakenaboutfloorstrengthandcontrol-cableroutes,thesafetyofeveryman,womanandchildwhowentaboardtheDC-10wasdependentupontheefficacyofthelinkagesfromthemomenttheplanewentintoservice.”
PaulEddy
Des=na=onDisaster
32
Contribu=ngCausalFactors DesignChoices
• DoorConfigura=on• Rou=ngofcables,hydrauliclines&wire• Floorstrength• Latchdesign• Cockpitindicatorlight• Ventdoordesign
33
FMEA
In the summer of 1969, Douglas asked Convair to draft a FMEA for the lower cargo door system of the DC-10. Convair produced a document which accurately foresaw the deadly consequences of a cargo-door latch failure. But neither Convair’s draft FMEA, nor anything closely resembling it, was ever shown to the FAA.
The Gi;
OnMay29,1970,duringgroundtes=ngofShip1toprepareitforitsupcomingmaidenflight,theaircondi=oningsystemwasbeingexercisedtobuildupapressuredifferen=alof4to5poundspersquareinch.Suddenly,theforwardlowercargodoorblewopencausingalargesec=onofthecabinfloortocollapseintothehold.
McDonnellDouglasaBributedtheincidentalmosten=relytohumanfailureonthepartofthebaggagehandler.
34
35
TheApplegateMemorandum:
“Thepoten=alforlongtermConvairliabilityhasbeencausingmeincreasingconcernforseveralreasons…theairplanedemonstratedaninherentsuscep=bilitytocatastrophicfailurewhenexposedtoexplosivedecompressionofthecargocompartmentin1970groundtests…Itseemstomeinevitablethatinthetwentyyearsaheadofus,DC-10cargodoorswillcomeopenandcargocompartmentswillexperiencedecompressionandIwouldexpectthistousuallyresultinthelossoftheairplane.”
F.D.Applegate DirectorofProductEngineering
Convair
The Gi; (Part 2)
OnJune12,1972,AmericanFlight96departedDetroit,MIandwasclimbingthrough11,750feetwhentherearcargodoorblewoutcausinganexplosivedecompressionandlossofflightcontrols.ThecrewmanagedtoregaincontroloftheplaneandreturntoDetroit.
“Thedesigncharacteris=csofthelatchingmechanismpermiBedthedoortobeapparentlyclosed,when,infact,thelatcheswerenotfullyengagedandthelockpinswerenotinplace.” Na=onalTransporta=on
SafetyBoardAccidentReport(NTSB) February28,1973
36
TheMidnightGentlemen’sAgreement:
ThepresidentofDouglasspersuadedtheFAAAdministratorthatcorrec=vemeasurescouldbeundertakenasaresultofagentleman’sagreement,therebynotrequiringtheissuanceofanFAAAirworthinessDirec=ve.
“ When you have a well–constructed state with a
well-framed legal code, to put incompetent officials in charge of administering the code is a waste of good laws, and the whole business degenerates into farce.”
Plato Laws (Book IV)
PertheGentlemen’sAgreement,Douglasissuedtwoairlineservicebulle=ns:
1. Installapeepholeandadecalshowingdiagramma=callywhatthehandlerwouldseeifthelockingpinwassafelyhome.IssuedasaSafetyAlert.
2.Installasupportplatetoholdupthetorquetubejustinsidethehandle.Issuedasarou=neservicebulle=n.
PlanningDepartmentrecordsclearlyshowthatonJuly18,1972,threeinspectorsseeminglyappliedstampsindica=ngthatthesupportplatehadbeeninstalledandthelocktubehadbeenmodified.Thesethreemenwerebroughtforwardandexaminedunderoath.ItemergedthatnotoneofthesethreecouldrecallhavingworkedonthecargodoorofanyDC-10atany=me.Norcouldtheyrecallonanyoccasionwhateveronwhichtheyhadworkedtogether.
Douglasmaintainedtotheendthathumanerrormustaccountforthefalsityoftherecords.
40
A Clear Case of Fraud
“Becausehistoryisanunrepeatableexperiment,wecannotprovethattheextraurgency,legalweight,andpublicitywhichgowithAirworthinessDirec=veswouldnecessarilyhavemadethedifference.Butthecrucialpointisthedetermina=onontheFAAAdministrator’spartthattheDouglascompanyitselfcouldbeledtohandlethemaBerinitsownway.”
PaulEddy
Des=na=onDisaster
Apollo1CommandModule
FirstinItsClass
Itwasmuchlargerandmorecomplexthananypreviousdesign:theleadingedgeofU.S.
spacecradtechnology.42
The Loss of Two Technological Marvels with All Crew Aboard
USSThresher
FirstinItsClassItwasfast,quietanddeepdiving:theleading
edgeofU.S.submarinetechnology.
OnApril10,1963,whileengagedinadeeptestdive220mileseast
ofCapeCod,MA,theUSSThresher
waslostatsea,seBlingatadepth
of8,400feetwithallaboard.
Intheend,112navalofficersand
enlistedpersonneland7civiliansperished.
43
Loss of the USS Thresher
Loss of the Apollo 1 Command Module
OnJanuary27,1967,theApollo1crewenteredthespacecradtoperformalaunchcountdownrehearsaltest.Thetestcommencedwithinstallingthehatchdoorandpurgingthecabinwitha100percentoxygenatmosphere.
Pad34:SiteoftheApollo1Fire
Hourslater,asparkfromfaultywiringinsula=onignitedafireconsuminganabundanceofflammablematerialsinthecockpit.Thefirecreatedanoverwhelmingpressureagainstthehatchdoor,sealingthecrewinside.
Grissom,White,andChaffee
44
USSThresher
Improperlybrazedpipejoint
Apollo1
“Theboardfoundnumerousexamplesinthewiringofpoorinstalla=onand
poorworkmanship.”
PoorlybrazedpipesledtotheelectricalshortagethatledtothelossoftheUSSThresher
Wireswherethefirewassuspectedtohavestarted
45
ContribuGng Causal Factors Inadequate Workmanship
Apollo1KennedySpaceCenterinspectorscitedmul=pleinstancesofdeficientparts,equipment,andworkmanship.
46
ContribuGng Causal Factors IneffecGve CorrecGve AcGon
USSThresherPortsmouthNavalShipyardinspectorsusingnewlydevelopedultrasonictes=ngtechniquesiden=fiednumerousinstancesoffaultybrazedjoints;however,manybrazedjointsontheUSSThresherwerenevertestedusingthenewtechnique.
Apollo1• Pureoxygenatmosphere
• Combus=blematerials
• Inwardopeninghatch• Inadequateescapeprovisions
47
ContribuGng Causal Factors
Vulnerable Design Inadequate Emergency Recovery
Unforeseen Failure Mode
USSThresher• Reactorshutdown• Impairedaccesstovitalequipment
• Compromisedballasttankblow
WreckagefromtheUSSThresher’ssonardomecanbeseenontheoceanfloor
Fallen Astronauts
Thereasoningbehindtheuseofpureoxygenseemedsoundenough.Inthevacuumofspacethecabinpressureneededtobemaintainedatlessthan6psi.Testshadalreadyprovedthatanyfireatthispressure,eveninapureoxygenenvironment,couldbeeasilycontainedandex=nguished.Duringgroundtests,however,asea-levelpressureof14.7psiwouldenvelopthespacecrad.Iftheoutsidepressureexceededthatinsidethespacecradbymorethan2psi,therewasachancethatthepressurehullcouldrupture,soitwasimpossibletotestthespacecradonthepadusingthe5.2to5.6psithatwouldbestandardoncethespacecradhadachievedorbit.Insteadtheengineerscrankeduptheinteriortomorethan16psitoexceedsea-levelpressure.Itwouldprovetobeafatalerrorofreasoning:thateasilycontainedfireinspaceatlessthan6psiwouldbecomeanexplosiveinfernoat16psi.
ColinBurgess
49
When VulnerabiliGes Line Up, Consequences Can Be DevastaGng
SapaProfilesmayhavealteredmechanicalproper=estestresultsperformedonaluminumextrusionsproducedfromatleast1996to2007,andmayhavereportedalteredtestreportsinmaterialcer=fica=onsgiventoitscustomersduringthat=meframe.
50
Fraud
51
Counterfeit Parts
CombaXng Fraud Heightened Awareness and Understanding
Supplier Oversight and TesGng of Incoming Product
Western Titanium
M&M Metals A&P Alloys
52
“OrbitalATKandNASAdiscoveredthepresenceofadefectintroducedduringmachiningofthebearingborehousing…Forensicinves/ga/onofEngineE17,whichfailedduringATPinMay2014,discoveredthepresenceofasimilarnon-conformingdefectinthehousing”
53
Workmanship/CorrecGve AcGon
JULY 20, 2015
CRS-7 INVESTIGATION UPDATE “Preliminaryanalysissuggeststheoverpressureeventintheupper-stageliquidoxygentankwasini=atedbyaflawedpieceofsupporthardware(a“strut”)insidethesecondstage.SeveralhundredstrutsflyoneveryFalcon9vehicle,withacumula=veflighthistoryofseveralthousand.Thestrutthatwebelievefailedwasdesignedandmaterialcer=fiedtohandle10,000poundsofforce,butfailedat2,000pounds,afive-folddifference.Detailedclose-outphotosofstageconstruc=onshownovisibleflawsordamageofanykind.”
54
Material Control
Muskalsoblamesoverconfidencefortheaccident…“Whenyou’veonlyeverseensuccess,youdon’tfearfailurequiteasmuch.
Highperformingorganiza/onsarepreoccupiedwiththeprospectoffailure.
"What we really learned from the Apollo fire, in the words of [former astronaut] Frank Borman, was the failure of imagination," said William H. Gerstenmaier, NASA's associate administrator for space operations. "We couldn't imagine a simple test on the pad being that catastrophic. "The message to the team is to remember how difficult our business is, the importance of staying focused and using our imaginations to envision what can go wrong."
40 years later, recalling the lessons of Apollo 1 January 28, 2007|Michael Cabbage | Orlando Sentinel
AddiGonal Stuff
58
DemonstratedReliabilityvsInsight/Oversight
USS IWO JIMA Mishap
A Set-up for Failure
BryanO’ConnorChief,OfficeofSafetyandMissionAssuranceNASAHeadquarters,Washington,DCOSMAPOC,BrianHughiB(202)358-1572
(61)
What Happened
-October1990:USSIWOJIMAAmphibiousAssaultShipDeployedtoPersianGulf,Opera=onDESERTSHIELD
-DockedatBahrainshipyardforemergentrepairs
-Asshipwasleavingport-onehouraderpropulsionplantbroughtonline-bonnetfastenersfora4”valvesupplyingsteamtoShipsServiceTurbineGeneratorfailedcatastrophically
- 850degreesuperheatedsteamat600psiescapedintomannedcompartment
- Ninesailorskilledinstantly,onemorefatallyinjured
(62)
Proximate Cause
Unauthorizedsubs=tu=onofblackoxidecoatedbrassfastenersforhighstrengthsteelfasteners
AlloySteelNutsBlackOxideCoatedNuts
(63)
The Set-Up
- BOCBF’svirtuallyiden=calinappearancetohighstrengthsteelfasteners.Coa=ngservednofunc=onalpurpose-appliedinordertostandardizecommonparts
- Fastenersreadilyavailableandeasilyinterchangeable
- Brassexhibitssignificantlydiminishedstrengthproper=esfromsteelunderelevatedtemperatures
- ManufacturerLogoonBOCBFbox:“FastenersforHighStrengthApplica=ons”
(64)
ContribuGng Causal Factors
- Repairspecifica=onsdidnotiden=fyfastenerpartnumbers
- NoevidencethatGovernmentholdpointinspec=onswereperformed
- InadequateknowledgeofLevelImaterialcontrolrequirements
-Segrega=onfromnon-LevelI-Segrega=onoflook-alikeparts-Markings-Colorcoding-Documenta=on
top related