safety analysis methodology for ads-b based surveillance ... · safety analysis methodology for...

Safety Analysis Methodology for ADS-B Based Surveillance Applications

Jonathan Hammer, The MITRE corporation

Gilles Caligaris, EUROCONTROL

Marta Llobet, EGIS Avia (Sofréavia)

ContentContent

1. ADS-B background and safety

2. Joint US/Europe Safety Assessment Methodology for ADS-B

3. Case Study: ADS-B-NRA Safety Assessment

4. Conclusion

ADSADS--B Applications & Safety BackgroundB Applications & Safety Background1

2

3

4

Applications

Air-to-Air:� Parallel runways� Spacing & Merging� Air-to-air / Airport Surface

Situational AwarenessAir-to-ground:� Non-Radar Areas� Radar Areas� Airport AreasPackage 1: � A set of ADS-B

applications,� Internationally agreed

Safety

� Internationally agreed frameworks� TCAS

� data link communications services.

� Techniques not directly applicable to ADS-B applications

� ADS-B safety work done in US and Europe independently

� Need for joint safety effort

RFG: OSEDS, Safety and Performance Requirements (SPR)FAA, EUROCONTROL, RTCA, EUROCAE, AirServices Australia, Japan

RFG Safety Assessment MethodologyRFG Safety Assessment Methodology

� Joint US/Europe

� Focuses on ADS-B applications.

� Applies to surveillance components but also to other CNS/ATM system elements

� Mainly based on:� ED78A/DO264

� EUROCONTROL SAM

� FAA SMS (Safety Management System) Process

� Aimed at delivering safety requirements for the standard

1

2

3

4

Main Steps1

2

3

4

OSED: Operational Services & Environment Definition

Safety Requirements

Safety Process:

�OHA: Operational Hazard Identification and Assessment

�ASOR: Allocation of Safety Objectives and Requirements

SPR document

Performance requirements

Operational requirements

Interoperability requirements

ASOR OHA

Safety Obj.

Safety Assessment Overview

Safety Target

OH

detected

OH undetected

Internal Mitigation

Means

OSEDEnvironmental

Conditions

Application

Description

Basic

Causes

Basic

Causes

External

Mitigation Means

Op.Effect

Sev.1

Op.Effect

Sev.5

1

2

3

4

OHA Process

� Brainstorming sessions with operational experts: � air-traffic controllers� Pilots� Safety experts

� Hazards identified between causes and operational effects level.

� Detected and Undetected hazards are identified

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

SO=min(STi/Nmax,i/Pei )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe

Step 1 Step 2 Step 3 Step 4

AssignSafety Objective

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

( )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe Safety Objective

1. Hazard Identification

1

2

3

4

OHA Process

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

SO=min(STi/Nmax,i/Pei )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe

Step 1 Step 2 Step 3 Step 4

AssignSafety Objective

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

( )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe Safety Objective

2. Severity Class Allocation

1

2

3

4

OHA Process

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

SO=min(STi/Nmax,i/Pei )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe

Step 1 Step 2 Step 3 Step 4

AssignSafety Objective

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

( )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe Safety Objective

3. Determine Probability Pe

� Equivalent probability Pe: conditional probability which expresses the probability that the occurrence of a hazard will results in a specific operational effect.

OH1

OE – SC1

OE – SC2

OE – SC4

OE – SC3

PePePePe2,12,12,12,1

Mitigation MeansExternal Mitigation Means

1

2

3

4

OHA Process

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

SO=min(STi/Nmax,i/Pei )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe

Step 1 Step 2 Step 3 Step 4

AssignSafety Objective

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

( )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe Safety Objective

ATM Risk budget apportionment

1

2

3

4

ATM Operational Effects

Safety Targets

ADS-B Application

ADS-B Application

ADS-B Application

CPDLC Application

ADS-B Application

ADS-B Application

Budgeting Safety Targets

Safety Targets

Safety Targets

Safety Targets

Safety Targets

Safety Targets

OHA Process

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

SO=min(STi/Nmax,i/Pei )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe

Step 1 Step 2 Step 3 Step 4

AssignSafety Objective

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

( )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe Safety Objective

4. Safety Objective Assignment

� Calculation of the quantitative Safety Objectivefor each hazard = Specify the maximum acceptable frequency of the hazard.

� Formula applied: SOj = mini (STi / Nmax,i / Peij), i.e. it takes SO from the most demanding pair (effect, frequency).

1

2

3

4

ASOR Process

OHundetected

Technical Environmental Human Procedural

Mitigation means related causes

Basic Causes

Causes related to actions

and functions

Internal Mitigation Means Failures

Failure Failure

Failure

Failure

OHdetected

FailureFailure

Failure FailureFailure Failure Failure

1

2

3

4

OH

BC1

BC3

BC2

IMM

Step 5

SO(from OHA

Step 4 )

Step 6

(BC1, SOaportion1)

(BC3, SOaportion3)

(BC2, SOaportion2)

(IMM, SOaportionIMM)

Step 7

Fault Trees Development

Safety Objective Allocation

Safety Requirements Derivation

SR list

ASOR Process

6. Safety Objective Allocation

OH

BC1

BC3

BC2

IMM

Step 5

SO(from OHA

Step 4 )

Step 6

(BC1, SOaportion1)

(BC3, SOaportion3)

(BC2, SOaportion2)

(IMM, SOaportionIMM)

Step 7

Fault Trees Development

Safety Objective Allocation

Safety Requirements Derivation

SR list

OHundetected

Internal Mitigation Means Failures

Failure Failure

Basic Cause Basic Cause Basic Cause Basic Cause

FailureFailure FailureFailure

OHdetected

Failure

FailureFailure Failure Failure

Safety Objective

Apportion SO

Apportion SO

Apportion SO

Apportion SO

1

2

3

4

ASOR Process

7. Derive Safety Requirements

OH

BC1

BC3

BC2

IMM

Step 5

SO(from OHA

Step 4 )

Step 6

(BC1, SOaportion1)

(BC3, SOaportion3)

(BC2, SOaportion2)

(IMM, SOaportionIMM)

Step 7

Fault Trees Development

Safety Objective Allocation

Safety Requirements Derivation

SR list

OHundetected

Internal Mitigation Means Failures

Failure Failure

FailureFailure FailureFailure

OHdetected

Failure

FailureFailure Failure Failure

Safety Objective

Basic Cause Basic Cause Basic Cause Basic Cause

Apportion SO

Apportion SO

Apportion SO

Apportion SO

1

2

3

4

ASOR Process

7. Derive Safety Requirements

OH

BC1

BC3

BC2

IMM

Step 5

SO(from OHA

Step 4 )

Step 6

(BC1, SOaportion1)

(BC3, SOaportion3)

(BC2, SOaportion2)

(IMM, SOaportionIMM)

Step 7

Fault Trees Development

Safety Objective Allocation

Safety Requirements Derivation

SR list

Basic Cause Basic Cause Basic Cause Basic Cause

Apportion SO

Apportion SO

Apportion SO

Apportion SO

List of Safety requirements

The system has to included detection means

Training must be provided

The availability of the system must be x%

The probability that An incorrect information is provided by the system shall be no more than 1E-05fh.

List of Safety requirements

The system has to included detection means

Training must be provided

The availability of the system must be x%

The probability that An incorrect information is provided by the system shall be no more than 1E-05fh.

List of Safety requirements

The system has to included detection means

Training must be provided

The availability of the system must be x%

The probability that An incorrect information is provided by the system shall be no more than 1E-05fh.

List of Safety requirements

The system has to included detection means

Training must be provided

The availability of the system must be x%

The probability that An incorrect information is provided by the system shall be no more than 1E-05fh.

Safety Requirements

1

2

3

4

People

Procedures

Equipment

ASOR OHA

Safety Obj.

Safety Assessment Overview

Safety Target

OH

detected

OH undetected

Internal Mitigation

Means

OSED Environmental

Conditions

Application

Description

To mitigate the effects of the

hazard

To meet the SO assigned to

the hazardSafety

Requirements

Safety

Requirements

Basic

Causes

Basic

Causes

External

Mitigation Means

Op.Effect

Sev.1

Op.Effect

Sev.5

1

2

3

4

1

2

3

4

Case Study

ED126/DO303 ED126/DO303

Enhanced Air Enhanced Air Traffic Services in Traffic Services in NonNon--Radar Areas Radar Areas using ADSusing ADS--B B surveillancesurveillance

1

2

3

4

Functional description of the system

ADS-B NRA identified Hazards 1

2

3

4

Hazards identified at this level

Examples of Hazards

• Controller loses position for one AC

• Incorrect position information for multiple AC is displayed to controller

…

Hazard and Basic Causes: example 1

2

3

4

Undetected Incorrect Position (corruption) for

one AC is provided to ATCO

Undetected Incorrect Position (position source failure) for one AC is provided to

ATCO

Corrupted Position Information1

2

3

4

Undetected Incorrect Position (corruption) for

one AC is provided to ATCO

Pe

Mid-air Collision

SO

1Q=5.00e-9

Corruptedinf ormation

prov ided to ATC

3Q=2.00e-5

Corruptedinf ormation

prov ided to ATC

2

Corruption is <50 NM

Q=0.00025Q=2.50e-4

4Q=1.00e-5

Ground systemcorruptsposition

5

AC domaincorruptsposition

r=1e-005Q=1.00e-5

6

ATC Processingor display

corrupts position

r=5e-006Q=5.00e-6

7

ADS-B receiv esubsy stem

corrupts position

r=5e-006Q=5.00e-6

Fault Tree1

2

3

4

Undetected Incorrect Position (corruption) for

one AC is provided to ATCO

SO

1Q=5.00e-9

Corruptedinf ormation

prov ided to ATC

3Q=2.00e-5

Corruptedinf ormation

prov ided to ATC

2

Corruption is <50 NM

Q=0.00025Q=2.50e-4

4Q=1.00e-5

Ground systemcorruptsposition

5

AC domaincorruptsposition

r=1e-005Q=1.00e-5

6

ATC Processingor display

corrupts position

r=5e-006Q=5.00e-6

7

ADS-B receiv esubsy stem

corrupts position

r=5e-006Q=5.00e-6

Basic Causes1

2

3

4

1Q=5.00e-9

Corruptedinf ormation

prov ided to ATC

3Q=2.00e-5

Corruptedinf ormation

prov ided to ATC

2

Corruption is <50 NM

Q=0.00025Q=2.50e-4

4Q=1.00e-5

Ground systemcorruptsposition

5

AC domaincorruptsposition

r=1e-005Q=1.00e-5

6

ATC Processingor display

corrupts position

r=5e-006Q=5.00e-6

7

ADS-B receiv esubsy stem

corrupts position

r=5e-006Q=5.00e-6

Basic Causes1

2

3

4

1Q=5.00e-9

Corruptedinf ormation

prov ided to ATC

3Q=2.00e-5

Corruptedinf ormation

prov ided to ATC

2

Corruption is <50 NM

Q=0.00025Q=2.50e-4

4Q=1.00e-5

Ground systemcorruptsposition

5

AC domaincorruptsposition

r=1e-005Q=1.00e-5

6

ATC Processingor display

corrupts position

r=5e-006Q=5.00e-6

7

ADS-B receiv esubsy stem

corrupts position

r=5e-006Q=5.00e-6

Requirements1

2

3

4

Safety Requirements on airborne AND ground elements, as an input to (for local implementation):

• design assurance level for equipment

• design configuration

• etc.

� Joint process between US and Europe

� End to end safety process covering airborne and ground domains, operational and technical part

� Used on NRA DO 303/ED 126 and will ultimately contribute to aircraft certification and deployment for ADS-B

� The approach is expected to be re-used in local implementations

� Will be used (and refined) for next to come ADS-B standards to be delivered by RFG

Conclusions1

2

3

4

ApplicationATM

OHA Process

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

SO=min(STi/Nmax,i/Pei )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe

Step 1 Step 2 Step 3 Step 4

AssignSafety Objective

OH

SC1

SC3

SC2

SC4

(ST1, Pe1)

(ST3, Pe3)

(ST2, Pe2)

(ST4, Pe4)

( )

IdentifyHazard

AllocateSeverity Classes

Determine Probability Pe Safety Objective

ATM Risk budget apportionment

1E-024

1E-043

1E-052

1E-081

[fh] or [flight]

ST ATM

1E-021E-01

1E-041E-03

1E-051E-04

1E-081E-07

[ATSUh] TMA

[ATSUh] en route

Units

45

35

25

20

Nmaxper SC

Risk Apportionment

2E-042E-03

3E-063E-05

4E-074E-06

5E-0105E-09

[ATSUh] TMA

[ATSUh] en route