High-Confidence, Low-Probability-of-Failure Screening

January 11, 2014

0 2 4 6 8 10 12 14 16 18 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

RLGM?

P[P[Failure|Eq]<0.05]95%

Magnitude

HCLPF depends on whom you believe

• Which components need seismic safety improvements? – If HCLPF < RLGM (Review Level Ground Motion), improve them– “HCLPF is defined as the earthquake [ground] motion level at which there is a high

(95 percent) confidence of a low (at most 5 percent) probability of failure” Interim Staff Guidance, DC/COL-ISG-020

• HCLPF: SoV vs. CDFM – AmExp[-2.32bC] “CDFM” (Conservative Deterministic Failure Margin, “because

engineers understand it”) [Kennedy et al.] – AmExp[-1.65(bR+bU)] [“SoV” (Separation of Variables), (EPRI TR-103959, Ravindra,

and others]• Am should be the a “High-Confidence” limit on the natural logarithm of median strength• -1.65 and -2.32 are the number of 5% and 1% standard deviations below the normal distribution

[log] mean Am

• The b-parameters represent randomness or aleatory variability (R), epistemic uncertainty (U), or combined uncertainty (C)

• Notice any sample uncertainty?

Objective of Seismic Screening: Component HCLPF < RLGM?

• Guidance unspecific, encourages industry initiative, abhors statistics?– ASCE/SEI 41-06 and -13, FEMA E-74 Chapter 4.1, and IEAE NS-G-1.6– RG 1.208, “A Performance-based Approach to Define the Site-

specific Earthquake Ground Motion” (NRC, 2007b) “The desired performance being expressed as the target value of 10-5 for the mean annual probability of exceedance (frequency) of the onset of significant inelastic deformation “

• Plants with higher seismic risk are already shut down– Vallecitos, Humboldt Bay, Zion, Fukushima Daiichi, San Onofre– Older plants have little time left, except for 20-or-30-year

extensions!

Alternative Uncertainty Model

Q = P[f < f’ | a]; i.e., the subjective probability (confidence) that the conditional probability of failure, f, is less than f’ for a peak ground acceleration a (e.g., RLGM)-1(.)= the inverse of the standard Gaussian cumulative distribution function.

R

Um

QAa

In

f

1-

How about some Rational Economic Resource Allocation?

• Cost-benefit analysis and budget constrained resource allocation is not under current consideration– Cost of alternatives? Fix some? Shut down plant?

• How to allocate resources to aleatory (random) or epistemic (unknown) uncertainty?– Is reducing epistemic uncertainty [unknown unknowns] a scam?

Quants use scenarios. Elizabeth Paté-Cornell says insure or hedge (Allin’s wife)• Price-Anderson act provides insurance to nuclear utility industry• NRC insures industry for costs incurred by delays caused by NRC

– How to allocate to high confidence vs. low probability of failure• “The public appears to heavily value confidence and places a much

smaller, although still positive, emphasis on accuracy.” [Smith and Wooten]

Legal, Precautionary Principle, and Social Equity

• “…if an action or policy has a suspected risk of causing harm to the public or to the environment, in the absence of scientific consensus that the action or policy is harmful, the burden of proof that it is not harmful falls on those taking an action.” http://en.wikipedia.org/wiki/Precautionary_principle

• Search for “Kip Viscusi and Risk Equity” for guidance • ISO 26000 Social Responsibility, ISO 16269-6

“Determination of Statistical Tolerance Intervals”

HCLPF Borrows Credibility• To statisticians, HCLPF is a tolerance limit or a confidence limit on a sample

percentile (ISO 16269-6 and http://en.wikipedia.org/wiki/Tolerance_limit– The “High-Confidence” part of HCLPF corresponds to the statistical confidence

limit, typically 95%– The “Low-Probability-of-Failure” part of HCLPF corresponds to the percentile,

typically 5-th percentile of the fragility function– Tolerance limits are supposed to be estimated from data, perhaps censored,

perhaps extrapolated• To reliability engineers, HCLPF is a R95C95 reliability demonstration test:

95% confidence that reliability > 95%– RnnCmm demonstration uses sample test or field (observed) data

• HCLPF uses subjective opinions or response analysis to relate component fragility to RLGM instead of local component response– Awkwardly related to material strength distributions [ASME et al.]

Methods: Economics

• Minimize seismic risk subject to budget constraint– Allocate resources to biggest bang-per-buck = Seismic Risk/$

$$(component fragility parameter)• Seismic risk = E[Discounted future costs due to earthquakes]

– = exp(-dt)*P[Eq of magnitude m at age t]*E[Cost|Eq of magnitude m at age t]dF(m,t)

– integrate from now to license expiration– Conditional on components to be improved

• $$$(component fragility parameter) = cost per unit change in component fragility parameter

– Use chain rule (next slide) – Use law of diminishing marginal returns: the more you spend

on one component, the less you get

Chain Rule

• Seismic Risk / $$$(component fragility parameter) = – Seismic Risk / Subsystem Risk* Subsystem Risk / $$$

(component fragility parameter)– Subsystem Risk / $$$(component fragility parameter) =

Subsystem Risk / Component type risk * Component Type Risk / $$$(component fragility parameter)

– Component Type Risk / $$$(component fragility parameter) = P[Component Failure|Eq] / $$$(component fragility parameter)

– Where component parameters include fragility correlations!

Why is HCLPF = AmExp[-1.65b]?

• Assume strength at failure RV is AmeReU

– Where Am is lower 5-th confidence limit on median component strength at failure and

– ln(eR) and ln(eU) are normally distributed, independent random variables with means 0 and (logarithmic) standard deviations bR and bU

• Then P[Stress > Strength] = [ln(Stress/Am)/(bR2+bU

2)] and– Screen is P[RLGM > HCLPF?] = 5% – HCLPF? = -1[0.05, ln(Am), (bR

2 + bU2)] or

– AmExp[1.64485(bR2+bU

2)] where standard normal z(0.05) = 1.64485

– i.e., P[Stress/(AmeReU) > 1] = [ln(Stress/Am) > 0] = 0.05

Methods: Data and Subjective Opinions to Obtain HCLPF

• Data: ASME and other material strength distributions, component test data, and post-earthquake failure observations– Maximum likelihood, Bayes, and method of moments– “The seismic capacity of such equipment in regard of their

functionality during and after an earthquake is impossible, difficult or unreliable to evaluate by other methods” [Other than seismic test, Tengler]

• Subjective opinions on median, percentiles, (logarithmic) standard deviations, and Y1|Y2 [NUREG/CR-3558 and others]– Least squares and weighted least squares– Output Am and b from “High-Confidence” fragility function, and r

Excel Workbook Implementations

• HCLPF….xlsx– Computes HCLPFs and P[RLGM > HCLPF] for various components

from various input parameters• SubjFrag.xlsx

– Computes High-Confidence fragility functions from test and seismic observation data, subjective opinions, and multiple-failure-mode fragility functions

– Estimates fragility correlations from subjective opinions of P[X1|X2] • NoFail.xlsm

– Estimates lognormal fragility function parameters, including correlation, from earthquake responses and component no-failure observations

HCLPF…xlsx SpreadsheetsSpreadsheet Name Contents

IP2ESEL Jim Moody’s list of components

BoM FLEX PWR list of components

Inherent ESEL Summary table list of components

HCLPF Compute HCLPF and P[RLGM > HCLPF] from component lists and fragility parameters

HELParameters Bandyopadhyay fragility parameters

Structures Structure CDFM fragility parameters

Equipment Equipment CDFM fragility parameters

ToleranceInterval Estimate correct tolerance limit from sample data 95% confidence on 5-th percentile

HCLPF Inputs

• List of component candidates for screening, QPA, structure function (series, parallel, RBD, fault tree)

• Design basis and RLGM and their units • Method(s) (CDFM or SoV?) • Component “fragility” parameters Am and b

– Strength-at-failure distributions for materials and some components or “High-Confidence” Am (median) and bR (logarithmic) standard deviation assuming lognormal strength

– Uncertainties due to components’ strength and responses to seismic ground motions, bU

– Safety and other fudge factors for CDFM computation of structure and equipment fragility parameters

HCLPF…xlsx:HCLPF Spreadsheet

• HCLPF spreadsheet does the HCLPF and P[RLGM>HCLPF]? and other computations– From component lists, parameter lists, and parameter

computations in other spreadsheets– Table 1 contains notes– Table 2 originated from AREVA proposal form

• Added QPA, Units, Method, Factors, Am, b, and computation columns• Computations include: z(p)Amb, RLGM>HCLPF?, (ln(RLGM/Am)/b), ({ln(RLGM/Am)+ bU-1(Q(RLGM)))/bR), and P[All QPA components survive]

• Could use VLOOKUP() function or links to enter parameters from other spreadsheets

HCLPF Spreadsheet Table 2

ESEL Item # ID Description

QPA assumed in series

Reference Ground Motion (Design)

Review Level Ground Motion RLGM Units

Method SoV? CDFM?

TRS(C)Test Response Spectrum

RRS(C) Required Response Spectrum

F(D) Device capacity factor

F(RS) Response factor for structure

S(A) Avg. spectral acceleration

Scale Factor SF for reference ground Am Units R U C

Am*exp(-1.65*R)

RLGM > HCLPF?

Am*exp(-1.65*(R+U))

RLGM > HCLPF?

Am*exp(-2.326*C)

RLGM > HCLPF?

(ln(RLGM/Am)/R)

(ln(RLGM/Am)/Sqrt(R^2+bU^2))

((ln(RLGM/Am)+U*^(-1)(Q(a)))/R)

P[min > RLGM]

MechanicalTurbine driven AFW pump 1 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Turbine driven AFW valves 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00SG PORVs 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Condensate storage tank 1 0.1 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Tanks, Ch. 7 TR-103959 2 0.1 0.2 pga, g SoV 1.14 0.537736 0.053774 pga, g 0.22 0.17 0.278029 0.037404 TRUE 0.028255 TRUE 0.028165 TRUE 0.999999999 0.999998846 0.999999999 1.00E+00Tanks, Table 7-11 and NP-6041 2 0.1 0.2 pga, g CDFM 0.52 pga, g 0.229 0.244 0.33463 0.356373 FALSE 0.238264 FALSE 0.238765 FALSE 1.50612E-05 0.002148932 1.50612E-05 3.01E-05SG injection valves 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00RCS injection valves 2 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 0.00E+00Other mechanical equipment 10 0.2 pga, g SoV? CDFM? 1 pga, g 0.2 0.2 0.282843 0.718924 FALSE 0.516851 FALSE 0.517942 FALSE 4.23585E-16 6.3437E-09 4.23585E-16 4.44E-15Electrical 0.2 pga, g SoV? CDFM?Batteries 2 0.2 pga, g SoV? CDFM? 0.5 pga, g 0.2 0.2 0.282843 0.359462 FALSE 0.258426 FALSE 0.258971 FALSE 2.30877E-06 0.000598536 2.30877E-06 4.62E-06DC distribution panels 2 0.1 0.2 pga, g SoV 3.3 0.33 pga, g 0.11 0.27 0.291548 0.275226 FALSE 0.176283 TRUE 0.167495 TRUE 2.65058E-06 0.042930687 2.65058E-06 5.30E-06DC MCCs 2 0.1 0.2 pga, g SoV 2.8 0.28 pga, g 0.1 0.23 0.250799 0.23741 FALSE 0.162437 TRUE 0.156246 TRUE 0.000383104 0.089862431 0.000383104 7.66E-04DC switchgear 2 0.1 0.2 pga, g SoV 4.4 0.44 pga, g 0.13 0.37 0.392173 0.355056 FALSE 0.192823 TRUE 0.176722 TRUE 6.59536E-10 0.022190135 6.59536E-10 1.32E-09Vital AC distribution panels 2 0.1 0.2 pga, g SoV 3.3 0.33 pga, g 0.11 0.27 0.291548 0.275226 FALSE 0.176283 TRUE 0.167495 TRUE 2.65058E-06 0.042930687 2.65058E-06 5.30E-06Battery chargers 1 0.2 pga, g SoV? CDFM? 0.5 pga, g 0.2 0.2 0.282843 0.359462 FALSE 0.258426 FALSE 0.258971 FALSE 2.30877E-06 0.000598536 2.30877E-06 2.31E-06Inverters 1 0.2 pga, g SoV? CDFM? 0.5 pga, g 0.2 0.2 0.282843 0.359462 FALSE 0.258426 FALSE 0.258971 FALSE 2.30877E-06 0.000598536 2.30877E-06 2.31E-06Instrument racks 4 0.1 0.2 pga, g SoV 1 1 1 1 1 0.1 pga, g 0.2 0.2 0.282843 0.071892 TRUE 0.051685 TRUE 0.051794 TRUE 0.999735609 0.992869981 0.999735609 1.00E+00

If C is not SQRT(R^2+U 2), please enter your value

HCLPF Computations Include…

• AmExp[z(p) ]b : HCLPF = AmExp[z(p)b] etc. for alternative z(p) and b (p = “Low-Probability-of-Failure” 5% or 1%)

• RLGM>HCLPF?: TRUE or FALSE • (ln(RLGM/Am)/b) = P[RLGM > HCLPF] for alternative b

• ({ln(RLGM/Am)+bU-1(Q(RLGM))}/bR) [Ravindra, Kennedy, et al.]

• P[min>RLGM] = P[All QPA components survive] – Assumes QPA co-located, series components with iid fragilities– Change the formula for parallel components or other

configuration (FTA) or correlated fragilities

SubjFrag.xlsx SpreadsheetsSpreadsheet Name Contains

ToleranceInterval Same as HCLPF…xlsx:ToleranceInterval

19experts Compute least-squares lognormal fit to subjective lower 5th percentiles

19expertsWeighted Ditto with weighted least squares

10expertsBootstrap Bootstrap 95% “High-Confidence” lognormal fit with fewer than 19 experts

SubjCorr Estimate r from subjective distribution of Y1|Y2 by least squares

Tolerance Limit on Lognormal RV Fragility Function

• Estimate correct tolerance limit on a lognormal distribution from a sample of means m (= ln(Am)) and (logarithmic) standard deviations b

• Input desired “High-Confidence” and “Low-Probability-of-Failure) and a sample of means and standard deviations from tests or subjective opinions– "Exposure Assessment: Tolerance Limits, Confidence Intervals, and Power

Calculation…" K. Krisnamoorthy et al.• Confidence limit for m + z(p)*b is constructed of the form…• m +Q(p,alpha)* b from estimates m and b are the MLEs and Q(p,alpha) is the

tolerance factor determined so that …• P[ m +Q(p,alpha)* b < something] > 1 alpha…• where (somethingm)/b is approximately ~ (z(p)m)/b

• Output is tolerance limit or HCLPF (table 2 is simulated)

“High-Confidence” Subjective Fragility Function Estimation

• Suppose 19 experts give 19 opinions on fragility median Am and (logarithmic) standard deviation

• For each (discrete) value of strength y, find maximum of 19 cdfs and connect with a curve Fmax(y)– P[Strength < y|Expert Am and b], (or Am and some percentile) and – Assume each represents the upper 95% confidence limit (“High-

Confidence”)– Fit a lognormal distribution to minimum curve to represent a 95%

“High-Confidence” fragility function using (weighted) least squares

• Note that P[F(y) ≤ Fmax(y) for all y] 0.95

Lognormal Fit to Lower 5-th Percentiles

0 1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

19 opinions, maxima, and 5th percentile fragility function

lower 5th %ileExpert 1Expert 2Expert 3Expert 4Expert 5Expert 6Expert 7Expert 8Expert 9Expert 10Expert 11Expert 12Expert 13Expert 14Expert 15Expert 16Expert 17Expert 18Expert 19Max

ln[strength at failure]

By Weighted Least Squares

0 1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

19 opinions, maxima, and 5th percentile fragility function

lower 5th %ileExpert 1Expert 2Expert 3Expert 4Expert 5Expert 6Expert 7Expert 8Expert 9Expert 10Expert 11Expert 12Expert 13Expert 14Expert 15Expert 16Expert 17Expert 18Expert 19Max

ln[strength at failure]

What if there aren’t 19 experts?

• Bootstrap• Correlation estimate requires at least one

subjective opinion of distribution of X1|X2• Use least squares to combine experts’

subjective distribution information– Sum of squared errors indicted magnitude of

experts’ deviation from lognormal distribution

Bootstrap 10 experts

0 2 4 6 8 10 12 14 16 18 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

lower 5th %ileExpert 1Expert 2Expert 3Expert 4Expert 5Expert 6Expert 7Expert 8Expert 9Expert 10Max

NoFail.xlsm Spreadsheets

Spreadsheet Name Contents

Freq Mle of lognormal fragility parameters from 19 non-failure responses, assuming 20-th would be failure

Bayes MoM estimates of lognormal fragility parameters of a-posteriori distribution of P[Failure|Eq] from 19 non-failure responses assuming non-informative prior

BayesCorr Same as Bayes, including estimate of fragility correlations

• Imagine inspections after earthquakes indicate component responses and failure or non-failure

Freq Spreadsheet

• Input iid responses for which no failures occurred– 19 responses were simulated for example and

convenient interpretation of mean and standard deviation estimates as “High-Confidence”

– Assume ln[Stress]-ln[Strength} ~ N[muX-muY, Sqrt(sigmaX^2+sigmaY^2)]

• Use Solver to maximize log likelihood of PP[Non-failure|Response] subject to constraint– Either constrain CV or P[Failure]

• Output is ln(Am) and b

Bayes Spreadsheets

• Bayes estimate of reliability r = P{ln[Stress]-ln{strength] > 0]– Non-informative prior distribution of r– Same inputs as Freq: responses and non-failures– Use MoM to find ln(Am) and b to match posterior E[r]

= n/n+1 and Var[r] = n/((n+1)^2*(n+2))– Ditto to find correlation r from third moment of a-

posteriori distribution of r• Bayes posterior P[ESEL component life > 72

hours|Eq and plant test data]

Parameter Estimates from 19 Non-Failure Responses

• Given 19 earthquake responses with ln(Median) = 0.5 and b = 0.1 and reliability P[ln(Stress) < ln(strength)] ~ 95% • Bayes non-informative prior on reliability P[Response < strength] => posterior

distribution • Use Method of Moments to estimate parameters for a-posteriori distribution of

reliability

Parameter Assume 20-th is failure

Bayes Bayes Correlation

m(ln(strength)) 0.781404 0.781389 0.950047891 b ln(strength) 0.148132 0.148123 0.226336118 R ln(strength) N/A N/A 0.752266708

What is the correlation of fragilities?

• See SubjFrag.xlsx:SubjCorr and NoFail.xlsm:BayesCorr spreadsheets to estimate correlations from subjective opinions on Y1|Y2 or from no-failure response observation

• HCLPF ignores fragility correlation• Risk doesn’t ignore it

What if multiple, co-located components?

• Responses are same (Refer to work for Howard last year)

• In series? Parallel? RBD? Fault tree?• Using event trees, Jim Moody argues that

HCLPF for one component is representative of all like, co-located components

What if like-components are dependent?

• Fragilities could be dependent too!• But not necessarily all fail if one fails – True, P[Response > strength] may be same for all

like, co-located components– But what is P[g(Stress, strength) = failure] for

system structure function g(.,.)?

More References• NAP, “Review of Recommendations for Probabilistic Seismic Hazard Analysis:

Guidance on Uncertainty and the Use of Experts (1997) / Treatment of Uncertainty,” National Academies Press, http://en.wikipedia.org/wiki/Quantification_of_margins_and_uncertainties

• Viscusi, W. Kip, “Risk Equity,” ISSN 1045-6333, (2000) http://www.law.harvard.edu/programs/olin_center/papers/pdf/294.pdf

• Der Khiureghian, Armen and Ove Ditlevsen, “Aleatory or Epistemic? Does It Matter?” (2007), Risk Acceptance and Communication Workshop, Stanford

• Mannes, Albert and Don Moore, “I Know I’m Right, A Behaviourial View of Overconfidence,” Significance, vol. 10, issue 4, August 2013

• Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol. 10, issue 4, August 2013

• Tengler, Marek, “Seismic Qualification of NPP Structures, Systems and Other Components,” Seminar, Nov. 2011

• Smith, Ben and Jadrian Wooten, “Pundits: The Confidence Trick,” Significance, vol. 10, issue 4, August 2013