Safety Cases: Purpose, Process and Prospects

John McDermid, OBE FREngUniversity of York UK

2

» Safety casesBasic concepts

Purpose(s)

» ProcessUsed for system acceptance

Used for argument construction

» ProspectsBetter safety cases

Integration of approaches

» Conclusions

Outline

3

» A safety case is:A structured argument, supported by evidence,

which provides a comprehensive and compelling case that a system is safe to operate, in a given scenario

» Compared to a safety assessment report (SAR)Big difference is the argument (in the sense of a

justification)

But what might we argue?

Safety Case Concept

4

» Examples might beCompleteness and quality of hazard identification

» Including use of skilled people

Appropriateness of risk reduction

» Including proper use of (MilStd 882) priorities

Tolerability of risk

» More than just acceptance by authority, e.g. ALARP or cost-benefit analysis

In general, things which are often implicit in a SAR

Possible Arguments

5

» Safety cases can be used for many purposesSub-systems rather than systems (like SSAR)

Through the process, e.g. preliminary safety case

» Initially just the argument, to see if it would be acceptable if it could be supported by evidence at the end

Different roles

» Overall system, e.g. aircraft, safety case

» Integrated view, e.g. system of systems

» Operational, e.g. for a mission

Focus, for now, on system acceptance

Purpose(s)

6

» A safety case is too big to deliverNo aircraft could lift its own (paper) safety case

» A safety case report isA document which summarises the arguments and

evidence of the safety, and documents progress against the safety programme

Really two roles

» Deliverable summarising (final) safety case

» Progress reports, including evidence generation

Safety Case and Reports

7

» Safety casesBasic concepts

Purpose(s)

» ProcessUsed for system acceptance

Used for argument construction

» ProspectsBetter safety cases

Integration of approaches

» Conclusions

Outline

8

» The MoD process is focused on acceptanceUsed as an illustration as it is probably the closest

approach to US DoD practices

» Focuses on safety case report at the end

» In practice, earlier drafts issued

Could also support uses in other domains

References to SMP are to Safety Management System Procedures out of MoD’s POSMS (Project Oriented Safety Management System)

MoD Process

9

Role of (Final) Safety Case

10

Safety Cases and Reports

Detail depends o

n the

regulatory

structu

re, e

tc.

11

Argument Construction Process (1)

12

» The “process” is quite judgmentalNot unusual in safety engineering

Hence easy to do it wrong

Not very much guidance on “good practice’

» Available guidanceSome published “argument patterns”

» Typical approaches, e.g. argument over hazards

Tim Kelly’s thesis

And see later

Argument Construction Process (2)

13

» Following are key elements of most standards: Scope System Description System Hazards Safety Requirements Risk Assessment Hazard Control / Risk Reduction Measures Safety Analysis / Test Safety Management System Development Process Justification Conclusions

Typical Safety Case Contents

14

» Purpose of a Goal Structure Diagrammatic notation to make argument clear

To show how goals are broken down into sub-goals,

and eventually supported by evidence (solutions)

whilst making clear the strategies adopted,

the rationale for the approach (assumptions, justifications)

and the context in which goals are stated

Goal Structuring Notation

A/J

15

Simple Example

» Example based on a hypothetical factory situation Assumed to be at a town

called “Whatford” in the UK

» The factory contains a metal press Presses sheet steel to

make car body parts

Has a single operator who inserts metal sheets and removes parts

Interlock to protect operator

16

G1

Press is acceptably safe to operate within Whatford Plant

C1

Press specification

C2

Press operation

C3

Whatford Plant

Sn1

FTA analysis

Sn2

Formal verification

Sn3

SIL3 certificate

Sn4

Audit report

Sn5

Compliance sheet

A Simple Goal Structure

17

G1

Press is acceptably safe to operate within Whatford Plant

C1

Press specification

C2

Press operation

C3

Whatford Plant

S1

Argument by addressing all identified operating hazards

S2Argument of compliance with all applicable safety standards and regulations

C4

All identified operating hazards

C5

All applicable safety standards and regulations

G2

Hazard of 'Operator Hands Trapped by Press Plunger' sufficiently mitigated

G3

Hazard of 'Operator Upper Body trapped by Press Plunger' sufficiently mitigated

G4

Hazard of 'Operator Hands Caught in Press Drive Machinery' sufficiently mitigated

G5

Press compliant with UK HSE Provision and Use of Work Equipment Regulations

G6

Press compliant with UK enactment of EU Machinery Directive

G7

PES element of press design compliant with IEC1508

Sn1

FTA analysis

Sn2

Formal verification

Sn3

SIL3 certificate

Sn4

Audit report

Sn5

Compliance sheet

A Simple Goal Structure

18

G1

Press is acceptably safe to operate within Whatford Plant

C1

Press specification

C2

Press operation

C3

Whatford Plant

S1

Argument by addressing all identified operating hazards

S2Argument of compliance with all applicable safety standards and regulations

C4

All identified operating hazards

C5

All applicable safety standards and regulations

G2

Hazard of 'Operator Hands Trapped by Press Plunger' sufficiently mitigated

G3

Hazard of 'Operator Upper Body trapped by Press Plunger' sufficiently mitigated

G4

Hazard of 'Operator Hands Caught in Press Drive Machinery' sufficiently mitigated

G5

Press compliant with UK HSE Provision and Use of Work Equipment Regulations

G6

Press compliant with UK enactment of EU Machinery Directive

G7

PES element of press design compliant with IEC1508

Sn1

FTA analysis

Sn2

Formal verification

Sn3

SIL3 certificate

Sn4

Audit report

Sn5

Compliance sheet

Simple Goal Structure

Safety Requirements & Objectives

Safety Evidence

Safety Argument

19

» Safety casesBasic concepts

Purpose(s)

» ProcessUsed for system acceptance

Used for argument construction

» ProspectsBetter safety cases

Integration of approaches

» Conclusions

Outline

20

» Learning from experienceNimrod XV230 is salutary

» PragmatismUnderstanding when

» Arguments add value, and when they don’t

Understanding the nature of arguments

» See next slide

Better reviewing

» Make safety case report basis for “challenge”

Better Safety Cases

21

The “McDermid Square”

22

» ANSI, MilStd 882, ARP Familiar-Familiar – evidence standard documents,

possibly only “argue” confidence in evidence

» UASFamiliar-Familiar for “standard aspects”

Unfamiliar-Unfamiliar – e.g. sense and avoid

» Argument that problem well enough characterised that solution will be adequate (safe)

» Argument that solution works across all scenarios

Integration of Approaches

23

» Safety casesBasic concepts

Purpose(s)

» ProcessUsed for system acceptance

Used for argument construction

» ProspectsBetter safety cases

Integration of approaches

» Conclusions

Outline

24

» Safety cases/reports can add valuePrimarily arguments to articulate rationale in

novel/complex systems/situations

Secondarily confidence (even in standard bits)

» Safety cases hard to construct wellNeed to avoid them where they don’t add value

Need better guidance on development/review

» Safety case (argument) patterns helpful but insufficient

» A good starting point would be a systematic review

Conclusions

25

» For the definition of the notation see:http://www.goalstructuringnotation.info/documents/GSN_Standard.pdf

This is a “community standard” but it is quite stable

There are also support tools, some of which are linked from:

http://www.goalstructuringnotation.info/

Goal Structuring Notation