11

Industry Trends ProgramWorkshop

Indicator Development Team

Paul Harris, NRR

Tom Boyce, NRR

Dale Rasmuson, RES

Steve Eide, INEEL

Cory Atwood, Statwood Consulting

Bob Youngblood, ISLftbtfw

" wmAwM*

July 29. 2003 P Publc Workshop I Jh29,2003 1TP Public Wonkthop

Agenda> Background> Industry Trends Program (ITP)> Initiating Event (IE) Trends and Assessment> Baseline Risk Indicator (BRIE) [previously called

Integrated Industry Initiating Event Indicator (IIIEI)]* Technical basis. Example results. Uncertainty/variability and sensitivity runs. Thresholds for the BRIE

Technical Discussion> Implementation Discussion> Summary and wrap-up

Background> NRC published 7 industry performance

indicators (PIs) in the annual P reportthrough 1999

> ASP results published in annual reportthrough 1998

> ROP process started in FY 1999> Performance report to Congress (GPRA)> Responsibility for industry PIs transferred

from RES to NRR late 2000

July 29, 2003 ITP Public Workshop 3 Juy29, 2003 UTP Pub Wrkho 4

Background (cont.)

> NRR formally organized the IndustryTrends Program in 2001

> ITP currently uses the 7 Pls and trendsfrom the Accident Sequence Precursor(ASP) Program

> ITP reports annually to the Commission. SECY-01-0111 (6/2001). SECY-02-0058 (4/2002). SECY-03-0057 (4/2003)

Performance Goals & Reporting

> NRC has 4 Performance Goals> For Maintain Safety Goal, 5 measures

* No statistically significant adverse industrytrends in safety performance

. Significant events ( 1 10-3 ASP Event)

. Additional measures for exposures, releases,security

> NRC uses qualified set of ITP indicators toreport against measure to Congress

July 29,2003 rP Publlc Wo*shWp 5 July29,2003 rrP Putbic Wokshop 6

4.

ITP Policy

> Identify and evaluate indicators of industryperformance for adverse trends and tocommunicate this trend information in apredictable and measured manner toenhance commercial reactor safety andNRC oversight processes.

Industry Trends Program

> The ITP is designed, in part, tocomplement the Reactor OversightProcess (ROP)

> The ITP focuses on multi-plant/ multi-siteperformance, while the ROP focuses onplant-specific performance

> ITP complements generic communicationsand generic safety issues processes

July 29,2003 ITP Public Worimhop 7 *Jul2g.2003 mT Public Workshop

ITP Purposes

> Provide one means to assure that thenuclear industry is maintaining the safetyof nuclear reactors

> Enhance stakeholder confidence in theefficacy of the NRC regulatory processes

ITP Objectives

> Collect and monitor industry-wide data toassess whether the nuclear industry ismaintaining safety performance of operatingplants and to provide feedback to the ROP andother NRC processes

> Assess the safety significance and causes ofany statistically significant adverse trends

> Communicate industry-level information toCongress and other stakeholders

Jy 29. 2003 r Pus Workstp 9 J*29,23 rr PUbfl Wodksba 10

.

ITP Process Industry Performance IndicatorsIdentiy Adverse Trends

Sign of Mlope m dvrse or not

Analyses ot Issues,* .Few pnt aNo erv e texf* P" c en V~* Emme egeno detlbame* AlWtewpo ss I ces* NRqA etdcallmndi revlew*Mmine as"yiniec

Rold to Comaess*NRC Pertanvun &

* Gma% bode

[ Identify Short TeM istmExceed -rdcto 1knf

Collect Indicator Data

.Lou SEMt(2):MOR ROPPII1l) LER

ASp -0) h.OaPeeie*Wwdt*yaftwdft o v " SA

* OPAR mb.h

- AP

co"MMMIlelons* NRC web pope for fTP* AnuWi report toMR1-10

* NRC W Lip

a -xif okrne

flOP Comemtone Current Pis FlOP Pis (Industry) Other

~~~ ~AukanmilSCrarn urgarmed Soram ASP~~invsnts~~~~ ,~~ Scran Wfth Low at hodt rmmals 15 WM"ltfr

~~~ ~~~ ~~~ ~U~-W -dWag S EvwntSOMeY SYsteM AMUhMo Un-Wd~bit of ASPBe"ei System Fekmu. HMC. HPCG. RCIC EP, RHRt WWR)Equlpinff Forced Outes HPS1. AFW. EP. Re (PMi

RCS Ac"t ASPBarder kiegrly Re~~~~~tIs LUM"eg

DO5txeroedom-MQW Preparedree ERO DOf Partl~a~m

MAte and Nol~ee lon Sydtem ____

OMVWWWon p*a" Saety Cade#-v RateS. Exposur 0Corupals Exposur Conrtc

P~~ flaeo Prat~c"ri Rndonoal EMlkvonts

Personnel SOON"*~Saleguesdo ~~~~~~~Peravoel Reblty

Aggrocy Respons

* No aCfmwcmiku tomm

* Genet seyipco

iiII~~~z Senior

ReviewApenry Aotlre

RevGe mees

MYg 29, 2003 ITP Pwft Wofthop 11 Adq29, 2003 ITP Public Worshop 12

BRIE Development> Indicators are only counts without significance

Risk significance of Es can be calculated (i.e., BRIE)from risk model weighting factors

Some initiating events (Es) too infrequent toregulate plant-level performance using the ROP. However, these Es can be used in the ITP to monitor

industry-level performance. Monitorino 10-15 risk-significant Es can provide

better insights than current set of industry indicators> Numerous indicators may not be understandable

. Roll up indicator (i.e., BRIE) can simplifycommunications to Congress/stakeholcers

BRIE Concept

> Two-tiered approachIE-level: Monitor 10-15 most risk-significantIlEs across all plants

* Monitor Individual Es using statistical techniques• NRC investigates when prediction limit is

exceeded or a potential trend starts to appear

. Comerstone-level* Roll up indicator to simplify report of Es to

Congress* Threshold derived from the NRC's safety goal

Jh 29,203 ri PutbflWodshop 13 M 29,2003 ITP Ptbfo Wokshop 14

.4

IE-level Monitoring

> Potential ways to monitor the individuallEs:. Quarterly: Monitor for changes in IE

occurrence using predictive distributions andprediction limits (focus is on currentperformance)

. Annually: Monitor for trends and predictionlimits (focus is on multi-year performance)

Cornerstone-level Monitoring

> Use the BRIE to monitor the total risk from,the individual Es. Assess annually. Use thresholds set by an expert panel to

judge performance. Use in the report to Congress

Ju 29. 2003 1P PdIlc Woksh1 I5 J*29, 20 FTPewkO PWC o"1

Development Schedule

> Review of the draft report - June 2003> Public Workshop - July 30, 2003> Comment Resolution - August 2003> Final Report - September 2003> ACRS Briefing - December 2003> Commission Paper - Jan./Feb. 2004

Initiating Event Trendsand Prediction Limits

July29. 2003 ITP PubliO WoSh 17 JtOy29,2003 mTP Pub lc Woaksp 18

Initiating Events for BWRs> Loss of Offsite Power> Loss of Vital AC Bus> Loss of Vital DC Bus> Small / Very Small LOCA> Loss of Feedwater> BWR General Transients> BWR Loss of Instrument Air> BWR Loss of Heat Sink> BWR Stuck Open Relief/Safety Valve

Initiating Events for PWRs> Loss of Offsite Power> Loss of Vital AC Bus> Loss of Vital DC Bus> Small / Very Small LOCA> Loss of Feedwater> PWR General Transients> PWR Loss of Instrument Air> PWR Loss of Heat Sink> PWR Stuck Open Relief/Safety Valve> Steam Generator Tube Rupture

July29, 2003 ITP Pubic Wukshap 19 July 29, 2003 mT Publi eWothop 20

BWR General Transients Loss of Vital DC Bus

0.04

0.035'

0.03

0.025

0.02

0.015

0.01

0.00w

0. 1968 19 1990 991 1992 90M 1994 996 1996 1997 1998 10 2 200 22 2003 1986 1989 1w9 1991 192 1993 994 1996 1996 197 1999 2000 200 2002 22

rE=Yeald Rate-96% Pmcdoni -BasahIe 99% P lJ1 M3Yndy Rat. -95% PmkimLt D-Bumim -9%PretonLkil

Jul 29,2003 UP PubPhWokshop 21 Jtdy 29,2003 22

Example Scenarios> Loss of offsite power

L arge increase in LOSP events in one year* Events occurred because of unexpected increase In severe storms

on east coast. ITP could do the following:

* Provide Information to inspectors for affected plants* Examine the inspection procedure for Preparation for Adverse

Weather* Might issue an Information Notice to all licensees giving lessons

learned> Increase in general transients

* ITP could do the following:* Review the LERs to ascertain causes* Create a Temporary Instruction directing Inspectors to look at what

was found from the LER review* Possibly Issue an Information Notice to all licensees

Baseline Values

> Obtained from operating experience over aninterval on which the trend is basically constant(trend parameter is not statistically significant)

> For initiating events with few occurrences, theinterval is 1988-2001.

> For .E.'s with more frequent occurrences, theinterval is shorter, but includes at least 4 yearsUsed for obtaining predictive distribution(individual Es) and constrained noninformativeprior distribution for Bayesian updates(integrated indicator)

July,29. 20 Publk Wowhhop 23 J*29, 20 ITP Pu*bc Womksho 24

Predictive Inference

> Most statistical methods focus on estimating orlearning about relevant properties of modelparameters

> Predictive inference focuses on calculatinginferences for the unseen part of the population(called future observations) given the sampledata from that populationPredictive inference often has a clearerinterpretation than parametric inference

> A predictive distribution is used for this purpose

Predictive Distribution(Events in Time)

> For events in time, a predictive distributionis the negative binomial or the gamma-Poisson distribution, a generalization. Observe number of events in past exposure

time. Predict number of events in future exposure

time

J* 29, 2= m PWIC WokWhWp 25 uy 29, 200a rrM PubWz Workmcp 26

Predictive Distribution ExampleLoss of Offsite Power

IE Prediction Limits(for one year)

0.i

OAd

0.1 -

0.16 '- -

ma f S

1.00

0.3

LSD \/ --- 0.7 0_!-

I: \a AO I

an

0<0 . . . .

Initiati ng Expected 95 99%Event Events PL PL

Loss of AC Bus 2.6 7 8Loss of DC Bus 0.3 2 3LOSP 1.2 4 6Small LOCA 0.4 3 4Loss of Feedwater 9.5 16 19

0 1 2 3 4 5 I E INo. of EVns

e X 2 4 5 a

NetlE0 1 2 4 5 6

a

July29,2003 P W29, 2003 ITP Pub W 27 JWY2s.2003 ITP Poflk WorkshT 28

BWR IE Prediction Limits(for one year)

PWR IE Prediction Limits(for one year)

Initiating Expected 95% 99%Event No. of PL PL

Loss of Inst. Air 0.3 3 3General Transients 28.4 39 44Loss of Heat Sink 6.0 12 14Suck Open SRV 0.7 3 4

Initiating Expected 95% 99%Event No. of PL LEvent Events P1 P

Loss of Inst. Air 0.8 3 5General Transients 50.0 61 67Loss of Heat Sink 6.0 12 14Stuck Open SRV 0.2 2 3SGTR 0.3 2 3

30Jy 29, 2003 rTPPbfWorkshop 29 J*y29, 2003 rrP PWiC Workuhop

4-

Industry Initiating EventPerformance Indicator

(11EPI)

Indicator Characteristics> Used as performance measures in the annual

performance report to Congress> Complementary to the plant-specific ROP> Provides Industry information for an ROP

cornerstone> Uses industry data available from current NRC

programs> Related to or tied closely to risk (e.g., CDF or

ACDF)> Utilizes risk-informed measures for assessing

their significance (e.g., safety goal, RG 1 .174)

July29. 2003 ITP Pwft Woilchop 31 A2,2003 ITP Pub k Workshop

Philosophy

> Trending individual initiating events does notcapture their risk importance

> Mitigating systems performance indicator (MSPI)has provided a way of combining riskinformation with operating experience

> MSPI approach can be used for initiating events

' Operating ExperienceEw

PRA InformationEved Tm I

D@<z; Industry

Initiating Event

Performance

t��Indicator

*--

Juy 29.2003 P PuW Woudhop 33 Jy 29.2003 ITP Pubic Wokshop 34

Industry Initiating EventPerformance Indicator

> Addresses the first cornerstone of safety(initiating events) from an industry perspective

> Is complementary to the plant-specific ROP> Is related to core damage frequency> Allows combined trending of frequent and

infrequent initiating events with different riskimportances

> Does not require any new data be submitted bylicensees

IIEPI Scope

> Focuses on the most risk-significantinternal initiating events

> Does not include fire, flood, seismicinitiating events, or other external orshutdown initiating events

> Does not address containment failure

July 29,2003 ITP Pubic Wowshop 35 AbY29.2003 ITP Pubes Workshop 38

What is the HIEPI?

> The IIEPI is the product of the industry initiatingevent frequency times the associated industryaverage conditional core damage probability(Bimbaum importance measure) summed overthe set of risk Important initiating events

> One integrated indicator for BWRs and anotherfor PWRs

IIEPI Calculations

g g i X E II

-la N.

iff, tj � � �: -D I ,." . ... � .!,:: J-1'!-T�j

: ,

Juy 29,2003 37 July 29. 2003 rr Pubel Wokshop 38

IIEPI Equation

where

B, = industry average risk weighting factor(Bimbaum importance measure) forinitiating event i

Al = industry initiating event frequency

m = number of initiating events

HIEPI Data Sources

BQ an

atj

J* 29. 20 m PIc Wo39w. 39 J29. MM0 " PLtft WO*M" Q

Indicator Hierarchy Current Performance

> Current performance is estimated using. A constrained non-informative prior

distribution based on the baseline value. One or more years of data (events and

reactor critical years)> The difference between the current value

and the baseline can be positive ornegative since the current value can beless than or greater than the baselinevalue.

-Suvoortina dlots and tables

* Unks to the underling data

4- * Events of nterest

July 29, 203 41 July29,2003 P PuMblclwosop 42

BWR IIEPI Summary

Example Results

Initiating Average Baseline Baseline PercentEvent Importance Frequency CDF of Mean NMkum N95

IE-SLOCA 5.62E-05 0.0047 2.62E-07 2.5% 0.4 21.3IE-TRANS 1.36E-06 0.8950 1.22E-06 11.68% 28.6 167.01E-LOHS 8.44E-06 0.1900 1.60E-06 15.3% 6.1 29.9IE-LOFW 1.45E-05 0.1020 1.49E-06 14.2% 9.7 49.5IE-LOSP 3.22E.04 0.0125 4.03E-06 38.4% 1.2 3.6IE-LODC 2.70E-04 0.0030 8.00E-07 7.6% 0.3 5.9IE-VAC 0.OOE+00 0.0275 0.OOE+00 0.0% 2.6 _IE-LOIA 8.20E-06 0.0108 8.86E-08 0.8% 0.3 55.7IE-SORV 4.71E-05 0.0213 1.OOE-06 9.5% 0.7 7.5

Total - - 1.05E-05 100.0% - -

July 29, 2003 ITP Ptlc Wotkshop 43 Jty29, 2003 mr PubiWokshop 44

PWR HEPI SummaryInitiating Average Baseline Baseline Percent

Event Importance Frequency CDF of Mean _ _

IE-SLOCA 2.52E-03 0.0047 1.17E-05 32.2% 0.4 2.2IE-TRANS 2.0tE-06 0.7640 1.54E-06 4.2% 47.9 739.8IE-LOHS 1.88E-0B 0.0974 1.84E-06 5.1% 6.1 85.2IE-SGTR 7.90E-04 0.0044 3.452-06 9.5% 0.3 5.2IE-LOFW 1.88E-05 0.1025 1.93E-06 5.3% 9.7 125.8IE-LOSP 3.25E-04 0.0125 4.07E-06 11.2% 1.2 10.3IE-LODC 2.99E-03 0.0030 8.84E-06 24.3% 0.3 2.2IE-VAC 0.OOE+00 0.0275 0.OOE+00 0.0% 2.6 -

IE-LOIA 8.35E-05 0.0122 1.02E-06 2.8% 0.8 28.1IE-SORV 6.36E-04 0.0031 1.972-06 5.4% 0.3 9.1

Total _ 3.642-05 100.0% - _

BWR IIEPI (CDF)(3-Year Bayesian Update)

12EM5

- 1.OE-05

o B.OE.06

I 6.OE-06

4.OE-06

ZOE-06

O.OE+00

FY 1997 FY 19S FY 1999 FY 2000 FY 2001 FY 2002

Fiscal Year

July29, 2003 mW Pblfc Workshop 45 July 29. 2003 ITP Pubic Workshop 46

4

PWR IIEPI (CDF)(3-Year Bayesian Update)

BWR IIEPI (CDF)(3-Year Bayesian Update)

3.5E-05

SR 3.0rEa05

> 2.5E-05

9 2.0E05

'c 1.5E-45

1.OE-05

a

'2

FY1997 FY1998 FY1999 FY2000 FY2001 FY2002

Fiscal Year

FY1997 FY1998 FY1999 FY22000 FY2001 FY2002

Fiscal Year

July 29, 2003 ITP Pub!Wonshop 47 JAdy29, 2003 Jm7 Pubic Workshop 48

-~~~~~~~~~~~~~~~~~~ -

PWR HIEPI (CDF)(3-Year Bayesian Update)

BWR IEPI (ACDF)(3-Year Bayesian Update)

0

IFY1997 FYI99 FY1999 FY2000 FY200 FY2002

Fiscal YearFY 1997 FY 199 FY IBO FY20 F 2001 FY 22

F19sa Yew

Ju 29, 2003 rrP Pulic Wwkshop 49 JuY 29, 2003 ITP PbDi Woulhop 50

PWR IIEPI (ACDF)(3-Year Bayesian Update)

6.OE00

4.0E-6

2.06-06

> -2.0E08

I -4.0E-06

a 4G.0E46*8.O-8.0E-06

-1.0E-05

-1.2E-05

UncertaintyNariability andSensitivity Assessment

FY 1997 FY IO FY1999 FY2000 FY2001 FY2002

Fiscal Year

July29, 2003 TP Publfc W tshop0 51 .luy29,2003 FmPublicwsop 52

IIEPI Characteristics

> IIEPI is. A function of counts of events. Used to estimate CDF or ACDF

9 Two sources of variability/uncertainty* Random variation of the counts around their true

frequencies (aleatory"). Uncertainty in the underlying frequencies

("epistemic")

> Uncertainty in the Bimbaum importances hasbeen considered

PWR Baseline Distribution

~ 1 6 aE-A SIE- 4Eb c" -6 .Bp* CDF, PF

MY 29.2003 rr Ptb Wohop 53 JA 29, 200n rTP Ptift Wokswp 54

PWR IIEPI Predictive Distribution Previous Two DistributionsCompared

Ened CDFCDF

65J*u2, 200 I Pubc W oSh5 J*29.2w3 I PIk Wotkshop M

Sensitivity Studies> Sensitivity studies are needed to show behavior of the

IIEPI when conditions are not at baseline.> Two kinds of deviation from baseline have been

considered:* All parameters Increase by the same factor (1.5 or 2) over the

baseline* Frequencies of the most important Es increase

* LOSP for BWAs and SLOCA and LDCBus for PWRs* Increase by a factor of 2 or 3* A other frequencies remain at baseline

> Technical detail• In senstivity studies, underlying frequencies are assumed* Only random variation of the counts Is present

PWR IIEPI Sensitivity Studies*(using 1-year Bayesian update)

asO 1564 E *4 4 SE4 04 76 SE-I S- -4EsWid cDf-

JU* 29, 2003 UP PutlWofkl p 57 Jl* 29, 2003 ITP PuiC Wodcshtp 58

PWR IIEPI Sensitivity Distributions(using 3-year Bayesian update)

BWR IIEPI Sensitivity Studies(using 3-year Bayesian update)

I0.6

*5 0O -:EC-

MOM 5*54 10E-5 155 204 25E45 306-Esltfrwde CDF

MYl~ 29, 2003 so JtIY29, 2003 ITP Pbif Wmkshop 60

PWR IIEPI Sensitivity Studies(SLOCA IE and 3-year Bayesian update)

BWR IIEPI Sensitivity Studies(LOSP and 3-year Bayesian update)

1"5 264 355 464 565 654 754 554 E6 1-4

bfimls cO1.04 1" 2.06

Esplied CF&.06

Ju29. 203 UPM Pubic Wdmop 61 Jy29. ax03 rrP Pubic Womshp w2

Baseline IE Performance Baseline BWR IE PerformanceInitiating Starting Critical No. of Baseline

Event Year Years Events MeanLoss of DC Bus 1988 1182.3 32 0.0275Loss of AC Bus 1988 1182.3 3 0.003LOSP 1997 439.4 5 0.0125Small LOCA 1988 1182.3 5 0.0047Loss of 1993 785.4 80 0.012Feedwater I_______

Initiating Starting Critical No. of BaselineEvent Year Years Events Mean

Loss of Inst. Air 1994 231.5 2 0.0108General Transients 1997 146.9 131 0.895Loss of Heat Sink 1996 176.2 33 0.0190Suck Open SRV 1993 258.2 5 0.0213

July 29, 2003 ITP Pubic Wotkshop 63 JLI29.2003 ITPPblcWhp 64

Baseline PWR IE Performance BWR Baseline CDF Breakdown

Initiating Starting Critical No. of BaselineEvent Year Years Events Mean

Loss of Inst. Air 1990 696.1 8 0.0122General Transients 1998 239.0 182 0.764Loss of Heat Sink 1991 641.9 62 0.0974Suck Open SRV 1988 800.6 2 0.0031SGTR 1988 800.6 3 0.0044

Initiating Event Mean Percent BimbaumLOSP 4.03E-6 38.4% 3.22E-4Loss of Heat Sink 1.60E-6 15.3% 8.44E-6Loss of Feedwater 1.49E-6 14.2% 1.45E-5General Transient 1.22E-6 11.6% 1.36E-6Suck Open SRV 1.OOE-6 9.6% 4.71 E-5Loss of DC Bus 7.99E-7 7.6% 2.70E-4Small LOCA 2.61 E-7 2.5% 5.62E-5Loss of Inst. Air 8.85E-8 0.8% 8.20E-6IIEPI Totdl (CDF) 1.05E-5

J* 29.2003 rrP Pubc Woqkshop e5 Jdy29,2003 fTP PtI& Workshop 66

PWR Baseline CDF Breakdown BWR Prediction Limits (CDF)Initiating Event Mean Percent Birnbaum

Small LOCA 1.17E-5 32.2% 2.52E-3Loss of DC Bus 8.85E-6 24.3% 2.99E-3LOSP 4.07E-6 11.2% 3.25E-4SGTR 3.45E-6 9.5% 7.89E-4Suck Open SRV 1.99E-6 5.5% 6.36E-4Loss of Feedwater 1.93E-6 5.3% 1.89E-5Loss of Heat Sink 1.84E-6 5.0% 1.89E-5General Transients 1.54E-6 4.2% 2.02E-6Loss of Inst. Air 1.02E-6 2.8% 8.35E-5IIEPI Total (CDF) 3.64E-5 100.0%

InItiatfing Event

FM 5 L Pinuite =99% PL Resut, - 8&ae4fn

July 29,2003 rTP Pubkc Workshop er Jul2g. 2W ITP Pubk Workshop 68

PWR Prediction Limits (CDF)

IAE-04

1 2E4 f

41.0154N

2AE4)S

Birnbaum Importance Measures

\s S \\ .1,-N '1 "I#, 0 0 4

InItiatlng Event

la95%wi PL R.ult 1399% Pl. sa -1aelilnI

ITP Public WorkshopJ*29. 2003 69 J*29,2003 P Pubic Wakstp 70

BWR LOSP BirnbaumImportance

PWR Loss of DC BusBirnbaum Importance

12

U.

3D

20

10

i.5-S 2.0-3 2.-3 30.S 35.

LOOP w *N hws

rTP Public Wwkap

O.O. 2.Oe-34A"6.0.j-S3.0.31AS-21.2.-2 t..-2 t.6-2 UM52Z0s-22.2-40-2on"t= Wio o o LOs c DC Bus

WY 29, 2003 ITP Public wokshopJuly 29, 2003 71 72

PWR SLOCABirnbaum Importance

40-

Thresholds for the IIEPI

20

I0. L U^

O.OeO

J* 29, MMz

1.06-2 2.0*2 &O" 4.06-2SLOCA HaM lpMM

rP PjWafwo

5.0W2

73 J*29,2003 ITP PublIc WmkdMhop 74

Thresholds for the IIEPI> Thresholds may be established using the

following information. Safety Goal and/or Regulatory Guide 1.174. Behavior of the integrated indicator

* Simulations* Maximum value* Major contributors* Distributions of the Bimbaum importance measures* Past operating experience trends for nitiating event

> An expert panel would be established topropose threshold values that satisfy policy andoperational needs and objectives

July29.2003 m Pittc Wothop 75

Threshold Characteristics> Thresholds should have a rational basis that is well

documented.> Thresholds should be practical, that is, possible to

determine and compare from data or other meanswith modest effort.

> Thresholds should be conceptually simple.> Thresholds should be consistent with the existing

regulatory framework.> Thresholds should reflect risk (including associated

uncertainties), safety, and regulatory perspectives.

J* 29, 20 TP Pthldc Wodcshop 78

Kinds of Thresholds> Thresholds for the IIEPI (BWR and PWR)

. Used to measure degrading industry performance,similar to thresholds used in ROP process

. Related to CDF or ACDF

. Anchored to the safety goal or Reg. Guide 1.174

> Prediction Limits for Individual IE Trends. Used to alert NRC to an change in Individual industry

trends that may indicate a degradation In Industrysafety performance

Inputs for Expert Panel

> Start with values for the ROP indicators, values fromrisk-based performance indicator report, and/or riskinsights from PRAs

> Assess current industry performance (e.g., trend,average)

> Estimate trend statistical characteristics (e.g.,prediction intervals, Bayesian predictive distribution)

> Using these inputs pick a feasible value for thethreshold

> Evaluate the threshold's risk implications> Suggest threshold values based on principles from

the threshold characteristics

77Jt 29, 20 " Pk Wafthp Ju 292003 78

Protocol for Setting Thresholds

> Develop risk and statistical informationrelated to trends for input to an expertpanel

> Provide associated safety and regulatoryinformation for expert panel

> Expert panel sets thresholds based uponinput and expert judgment

Plant-Specific Calculations

MY 29, 2003 ITP Publi W oupl 279 Juy29,0 ITP PcW~OP so

Plant-Specific IE. Indicator

> The same concepts used to develop theIIEPI can also be used to develop plant-specific initiating event indicators

> This can be done without licenseessubmitting any data

Plant-Specific Results for ITP

> One alternative is to use the average ofthe plant-specific initiating eventperformance indicators for the ITP

> The plant-specific IEPI has some issuesthat need to be resolved. What IE frequency do we use for the rare Es?

*29, 2M fP PuLc Wofkhq 81 J*29, 20 nP Pub Wokshc 82

4-

Observations

> The average value differs from the IIEPI. The two values can be of opposite sign in the

ACDF formulation. The two values can vary by an order of

magnitude or more> Plant-specific values are driven by the

Bimbaum importances> Plant-specific values are strongly

influenced by rare lEs

Conclusions

> Using plant-specific calculations can biasthe industry perspective for the ITP

> Plant-specific calculations can be done forthe more frequent Es

July 29,003 m Pbic Wwkop M AA2 20 ITP Phbc Wotlhow 84

BWR IIEPI vs. Averageof Plant-Specific IEPI

PWR IIEPI vs. Average ofPlant-Specific IEPI

Sg

I4

a

C31

G.E-M

4.E-06

T 2.E06

0 E0

o-2.E-063 4.E-06-

-6.E-06

-.E-OS

-i.E-S0197 1998 19 2000 2001 2002

Fiscal Year

USPlant-Specelf EHIEPI|

1997 1998 1999 2000 2001 2002Fscal Year

INlant-Spelfo IIEPI

July29,2003 ITP Public WowshM 85 JUlY29, 2003 ITP Public Worslhop 88

:BWR Plant-Specific IEPI Results PWR Plant-Specific IPEI Results

July29. 2003 ITP Public Worichop 87 JuI29,2003 UTP Publi Wso ea

Plant-Specific Thresholds vs.Industry Thresholds

> Combining plant-specific thresholds may not bemeaningful for industry thresholds• Unplanned scram green/white threshold = 3 unplanned scrams

per reactor* This implies an industry threshold of 300 unplanned scrams per

year based on 100 reactors

> Industry thresholds must consider the industryperformance as well as other factors

Industry unplanned scram average is about 0.6 scrams perreactor per year.

Draft Report Review Comments

J l 20 riT PuM Wodkhop 90July 29,2003 PuF Wokshop 89

Draft Report Reviewers

> Draft report reviewed by. Internal NRC staff

* Headquarters* Regions

. Extemal reviewers* NEI* Union of Concerned Scientists* EPRI* INPO* Owners Groups

Technical Questions> Is Equation 5 rather than Equation 2 or 3 most

appropriate for quantifying the IIEPI?> Is the method for determining baseline performance

adequate?> Is the proposed method for calculating the current

frequencies for the Es appropriate?> Should CDF or ACDF form of the indicator be used?> Given the characteristics of the IIEPI and the simulation

results, what might be appropriate CDF and ACDFthresholds for the IIEPI?

> How often should the initiating event baselineperformance be updated?

July 29,2003 rTP Pu a Woimop9 91 J*29. MM0 92

Technical Questions (cont.)

> Should the industry-average Bimbaum importances beobtained from the SPAR models or from Industry riskmodels?

> If the Bimbaum importance measures are obtained fromthe industry, how will the differences between the twomodels (industry and SPAR) be addressed?How often should the Bimbaum importance measures beupdated?Is the treatment of uncertainties adequate?

> Should the thresholds be set so that no one event In athree-year period would cause the threshold to beexceeded?

Summary of NRC Comments

> It is inappropriate to use industry-level-averageBimbaum importances. Plant-specificinformation exists and can be used

> The rolled-up indicator is not useful because anyrational response to it requires looking at lowerlevel constituent indicators anyway

> The several purposes for the indicator are noteasily addressed by a single rolled up quantity.The proposed formulation serves one objectivebut not others

J*29,203 rr Pua Wotm 93 J* 29.200 rr Publi WO*lShcp 94

4

NRC Summary (cont.)

> Why not factor in changes under theMitigating Systems cornerstone as well?

> The formulation needs to discriminateagainst tripping industry indicators in only-a-few-plant scenarios

> What about LERF? An SDP-type processcould furnish a LERF perspective basedon this development

> What about external events?

External Comments

)UCS. ITP is important & good to improve. IIIEI not valuable since it does not monitor the

right data - Instead, develop indicator forissues that resulted in plant shutdowns

. If used, may mask plant-specific events

. "Too Ouija-boardish"

. Recommend NRC abandon IIIEI

J* 29,2003 m PWIc Wahop9 s5 J*29.2=C rm Pt"lk Woftkhop 96

External Comments (cont.)

> NEI. Continue to risk-inform ROP & regulations. Concept of IIIEI used to replace plant-level

Scrams with Loss of Normal Heat Removal Pi. Use of IIIEI for industry-level is inappropriate

* Existing indicators sufficient* Unclear what CDF means for actions* Mix of indicators better than roll up indicator

Summary

JujyI29.2003 ITP Public Workshop 97 Iuy 29,2003 mTP Public Wodkshop 98

I

Interactions with Stakeholders

> Briefed ACRS joint subcommittee in May 2002,Nov. 2002, and May 2003

> RES sent out draft report on the IIEPI for internaland external review

> Briefed ITP and IE indicator to Industry at publicmeetings on ROP

> Third annual ITP Commission paper issued inApril 2003

> Briefed NRC management, AARM, andCommission on concept

Summary> The ITP is developing a protocol for looking at

the initiating events cornerstone of safety froman industry perspective

> Must involve multiple plants and multiple sites> Protocol is a two-tier process

* First tier monitors the individual initiating eventoccurrences and trends and focuses on performance

. The second tier monitors the initiating events in anIntegrated way and focuses on risk

July29, 2003 ITP Public Wodkshop ICJu2, 203ITP Plic Workshop go 00

Two-Tiered Process

> Two-tiered approach addresses safety andperformance• Individual IE trends and predictive

distributions* Use prediction limits for the Individual trends

(performance)* Prediction limits help us assess f there has been a

change in performance. Integrated indicator with thresholds

* Use thresholds, anchored to the safety goal for theintegrated Indicator (safety)

Tier 1

> Monitor the individual initiating events intwo ways:

. Monitor the IE occurrence using predictivedistributions and prediction limits quarterly(focus is on current performance)

• Monitor the trends on an annual basis (focusis on multi-year performance)

Jh29. 2003 TP Pub Wohop 101 Juy 29, 2003 rTP P Ubc Woftshop 102

4.

Tier 2

> Use the IIEPI to monitor the risk-significantinternal initiating events on an annualbasis. Use thresholds set by an expert panel to

judge performance

. Use in the report to Congress

Development Schedule

> Review of the draft report - June 2003> Public Workshop - July 30, 2003> Comment Resolution - August 2003> Final Report - September 2003> ACRS Briefing - December 2003> Commission Paper - Jan./Feb. 2004

J*29,M 203 PM Www"ho 103 *29, 2003 rrP PubH Wofthop 104