the impact of safety probability interpretation on risk ... · faacaam 0.1 events and a final fix...

Reliability Maintenance and Managing RiskConference 2019

David McDermottPratt & Whitney

[email protected]‐565‐5204

The Impact of Safety Probability Interpretationon Risk Management

RMMR 2019, Oct. 15 ‐ 16, San Antonio, TX 1

This document contains no export-regulated technical data .This document has been publically released. © 2019 United Technologies Corporation


Outline of Abstract

• Risk combines probability and consequence severity leading to management decisions

• MIL‐STD‐882E defines probability as the likelihood of occurrence

• Probability often applied as a rate per hour• There is variation in probability interpretation• Summary and conclusions of the impact



Warm Up #1

• Police officer: I’m pulling you over because you were going 80 mph in a 65 mph zone.

• You: But officer I left my house an hour ago and I have traveled 60 miles so my speed is 60 mph.

• Police officer: I’m going to need your license and registration.



Warm Up #2

• Car Salesman: This car shows you gas mileage throughout your trip. Take a left ahead.

• You on test drive: *takes left turn to go downhill*• Car Salesman: Look at the gas mileage on this thing, it’s got to be >50 mpg!



Important Terms

• Probability– Value between 0 and 1, inclusive, as likelihood of an event

• Cumulative Rate– Total events divided by total hours

• Instantaneous Rate– Expected events in an increment divided by increment hours

• Fleet Rate– Inclusive of all individual units

• Individual Rate– Per unit rate



MIL‐STD‐882E Hazard Risk Matrix

RMMR 2019, Oct. 15 ‐ 16, San Antonio, TX

Probability focus on likelihood of events.

6


USAF Airworthiness Bulletin AWB‐150A


Probability focus on event rate.

7


FAA Continued Airworthiness “Trapezoid”


CAAM = Continued Airworthiness Assessment MethodologiesL4 Event = Fatalities, Loss of Aircraft, Forced Landing

Probability focus on both event rate and number of events.

Acceptable

Immediate Action Required

Action Required to not Exceed Event Limit

Action Required

Guidelines specify use of fleet cumulative (average) risk for rate calculation.

8


Application with Example Fleet ‐ Population


Hypothetical Case Study

• 500 aircraft fleet, dual engine• Crack in Life Limited Part (LLP)• 10,000 cycle part life• Service life is 20,000 cycles• 2 hours per flight (cycle)

LLP cycle distribution

Fracture Weibull =3.8=36,000 cyclesB0.1 = 5800 cycles

• 10% of fractures result in an loss of aircraft (LOA) event

• Loss of aircraft = CAT 1 severity• Loss of aircraft = CAAM Level 4 event

Population at Risk

Frequency of OccurrenceConsequences

Background

Case study is a hypothetical scenario to illustrate differences in criteria.

9


Case Study – Baseline Risk Results

Baseline Risk ‐ UncorrectedExpectedEvents

Fleet Cumulative Rate Max Inst. Rate per ACFH

Per ACFH Per AC Flt Fleet Individual

Fracture Events 15.2

Loss of Aircraft (LOA) Events= CAT 1 events= CAAM Level 4 events

1.5 8.6E‐08 1.7E‐07 1.3E‐07 2.9E‐07



Extremely Large Variation Possible in Risk Management Conclusions


MIL-STD-882EEvent Probability

AWB-150AEvent Rates

FAA CAAMEvent & Rate Limit

Expected fleet LOA events = 1.5

HRI Level 1C or 1D ?

High or Serious Risk Level

Mode of Poisson distribution is 1 so 1D may be selected

Expected LOA/ACFH= 8.6E-08 (fleet cumulative)= 1.3E-07 (fleet inst. max)= 2.9E-07 (indv. inst. max)

HRI Level 1E

Medium Risk Level

0 < Probability <10-6

Expected LOA/AC Flight= 1.7E-07 (fleet cumulative)

Expected LOA events = 1.5

Corrective action required; must reduce risk during CA period to

<0.1 events and final fix < 1E-09

Action Required to not Exceed Event Limit

11


Required Part Life Limit to Achieve Acceptable (1E) Risk Level


Source Limiting Criteria RequiredLife Limit

Expected LOA Events with Limit

MIL‐STD‐882E “Unlikely to occur, but possible”; Poisson suggests 0.7 expected events at 50/50 chance boundary 8075 cycles 0.7

AWB‐150A None, all rates meet <10‐6 criteria 10000 cycles 1.5

FAA CAAM 0.1 events and a final fix required 4400 cycles and fix at mid‐life 0.1

Customized HRI with <10‐7criteria

Cumulative rate 10000 cycles 1.5

Max fleet instantaneous rate 9150 cycles 1.1

Max individual unit instantaneous rate 6825 cycles 0.5

Case study illustrates the variation that can be encountered based on the source used for acceptable risk criteria and the event rate calculation interpretation

Baseline part life limit = 10000 cycles

12


Summary of Observations


• Three reputable sources for risk management decisions resulted in much different corrective action requirements

• Rate calculation method selected resulted in much different corrective action requirements

• Thresholds based exclusively on event rates may yield a high number of expected events (and vice versa)

• Other than in cases utilizing an exponential distribution (constant failure rate), calculating instantaneous rates adds complexity in modeling – Most propulsion system failure modes are NOT best modeled with constant

rate models when accounting for the physics of the mode

13


Conclusions and Recommendations


• When quantitative risk assessments are utilized, programs should establish criteria that is bounded by both event and rate thresholds

• Applying an event threshold will create motivation for final fix• With an absolute event threshold and expected remaining

exposure, a cumulative rate threshold can be derived thus may be redundant in nature

• Individual unit instantaneous rate threshold protects against an aircraft flying with excessive risk

• If applied, an individual unit instantaneous rate threshold should be much larger than the corresponding cumulative rate, perhaps 10x

14


Final Thoughts


• Many current risk management guidelines could benefit from more specifics on how to apply the risk criteria

• Since many quantitative risk assessments utilize constant rate models, the nuances are not surfaced

• FAA CAAM guidelines are best‐in‐class for addressing rate calculation nuances and acceptable risk criteria– See FAA Advisory Circular 39‐8– Does not address or require instantaneous risk which may be an opportunity

in future updates

15

the impact of safety probability interpretation on risk ... · faacaam 0.1 events and a final fix...

Documents