SACADAStatus and the Data Available to Public

Y. James Chang, Ph.D.

Human Factors and Reliability BranchDivision of Risk Analysis

Office of Nuclear Regulatory ResearchU.S. Nuclear Regulatory Commission

Presented at the HRA Data WorkshopRockville, MD

March 14-15, 2019

1

Cicada(from Google images)

Current SACADA Work Emphases• Outreach to more plants to use SACADA• Develop SACADA-2• Analyze SACADA data for human reliability

analysis (HRA)• Make SACADA data available to HRA

community

2

SACADA: Scenario Authoring, Characterization, and Debriefing Application

Outreach

• STP continues to use SACADA-1– Will migrate to SACADA-2 when ready

• Votgle 3&4 is piloting SACADA-2 since 1/2019– Evaluate SACADA-2 for three training cycles

• The NRC continues to outreach to domestic and international NPPs to use SACADA

3

Develop SACADA-2• The development was motivated by the industry’s feedback on

SACADA-1– To have all operator performance information in one place, i.e.,

simulator training, job performance measure, written tests, and actual operations

– Able to track the performance of an individual operator• Key other improvements

– Significantly reduced IT demand – Do not need SQL– Comprehensive incorporation of INPO taxonomy for debriefing– Improving debriefing taxonomy. May shades light on modeling error

recovery.• STP and Vogtle 3&4 instructors support the development• Aim to have all SACADA-2 functions implemented in summer 2019

4

SACADA-2 Authoring Screen Shot

17

SACADA-2 Characterization Screen Shot

22

Debriefing Screen ShotsEvaluate the Performance

30

Link to INPO Operator Fundamentals

Debriefing Screen ShotUnderstand the Deficiencies

• Seven classes– Monitoring/Detection– Diagnosis/Understanding– Procedure Following/Decisionmaking– Manipulation– Supervision– Teamwork– Communication

• Stimulate crew to think• Can choose more than one class

32

ReportsExportable formats: PDF, MS Word, and Excel

44

SACADA-1 vs. SACADA-2STP and Vogtle Electric Generating Plant, Units 3 and 4, Provide Technical

Support for SACADA-2 Development

SACADA-1 SACADA-2

Collect Simulator Data √ √

Collect Job Performance Measure (JPM) Data √

Collect Written Exam Results √

Directly Support Crew Notebook √

Provide Simulator Guide √

Track Crew Performance √(Simulator only)

√(All areas)

Track Individual Performance √(All areas)

10

All areas: Simulator, JPM, and written exams.

Analyze SACADA Data for HRA• HRA data of interest for HEP estimates

– Basic human error probabilities (HEPs)– Performance influencing factors (PIFs) effect– Error recovery effect– Task dependency effect

• Types of data and preliminary assessment– Basic HEPs – Okay to Good

• Methods are demonstrated by Drs. Pamela Nelson and Ali Azarm in 2018• A key uncertainty is the success criteria difference between UNSAT and human err in HRA

– PIFs effect – Good• Demonstrated by Drs. Jing Xing and Song-Hua Shen in this workshop

– Error recovery effect – Maybe• Haven’t analyzed the data for error recovery

– Task dependency effect – Maybe• Training data may not help. Exam data and experiment data could help.

• Compare with HuREX results– Still to be done

11

Make SACADA Data Available to HRA Community

• Abide the NRC agreements with the data providers on SACADA data publications– The NRC is sensitive to the data providers’ concern about

releasing data to the HRA community– Only release system-neutral statistics for HRA

• Alarm detection example:

12

Context PerformanceStatistics

Planned Data/Information to be Available

• Statistics of context and performance in various macro-cognitive functions– Detecting alarms, reading parameters, diagnosis,

deciding, actions, and external communication• Statistics of context and failure modes (and the

system-neutral performance information of the failure modes)

• Do not plan to release the system-specific statistics, e.g., crew performance of training objective elements and of malfunctions

13

Data Distribution

14

High Percentage of Non-characterized Data Points

15

15 50 80 100 120 Time

InitialCondition

Elements(TrainingObjective Elements)

Malfunctions/Conditions

16

Identified Some Reasons but Not All Reasons

Fire in No. 12 Condensate Pump Motor

US Enter POP04-ZO-0008, Fire/Explosion

CREWActivate the fire alarm, make the announcement, and call out the Fire Brigade

CREW Other item to discussNot-characterized

Conclusion

• SACADA has made progress in– Reaching out to more potential users– Improving usability for operator training– Demonstrating the use of data for HEP estimates– Expanding the scope of data collection– Establishing a long-term data collection program

• Tasks ahead– Continue the existing activities– Ensure data quality and make data available to public– Compare human performance indications with the

other empirical data

17