safety case development in atm r&d safety feedback for decision-makers and concept developers...

Safety case development in ATM R&D

Safety feedback for decision-makersand concept developers

Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

Jelmer J. ScholteNLR-ATSICAATS II

Brussels, 13 & 14 Oct 2009


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

2

Contents

• Motivation

• Safety case contents

• Practical development of safety case

• Concluding remarks


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

3

History (1/2)

Accident statistics of Large Aeroplane flights in commercial aviation

Accidents Fatal Accidents Fatalities

1980-1999 period 2340 613 15,554Average per year 117 30.7 777.7Average per flight 5.57 E-6 1.46 E-6 37.0 E-6Separation related 7.9% 3.75% 5.0%

Source: NLR-ATSI’s Air Safety Data Base


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

4

History (2/2)

64

42

39

22

7

8

3

1

1

6

14

1

0

0

0

11

1

4

147

620

0

0 10 20 30 40 50 60 70

Collision with Vehicle

Collision with standingaircraft on ground

Collision with moving aircrafton ground

Collision with aircraft - bothairborne

Collision with aircraft - oneairborne

Aircraft encountedvortex/wake turbulence

Near collision with aircraft -both airborne

Accidents Fatal accidents Fatalities


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

5

Current picture

• It is good practice for an ANSP to develop a safety case for implementation of changes to its ATM system to fulfill its own objectives and responsibilities to satisfy safety regulations

• Several safety regulations and methods are in use that were developed for use by an ANSP for changes to its ATM system ESARR 4 EC regulation 2096/ 2005 EATMP ANS Safety Assessment Methodology (SAM) Eurocontrol Safety Case Development Manual (SCDM)


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

6

A practical example (1/2)

Independent parallel departures on SIDs


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

7

A practical example (2/2)

• Key hazards in cockpit and at ATC Crew makes error in entering the SID in FMS ATC fails to communicate a late SID change to aircraft ATC-published SID design entered wrongly in database

• Resolution of conflicts involves ATCo and pilots ATCo cannot solve the conflict without pilot Pilot may correct SID errors independently Timing of pilot’s R/T frequency change from TWR to APP

• Challenge: The role of the airline and the pilots is crucial Focusing on ANSP is not desired!


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

8

Future challenges (1/2)

AIR TRANSPORT

MANUFACT-

URERS

OTHER

REGU-

LATORS

POLICY

MAKERS

ANSPs

AIRPORTS

AIRSPACE

USERS

HUMAN

SOCIETY

OTHER SERVICE

PROVIDERS

HUMAN

OPERATORS

ASSOCIATIONS


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

9

Future challenges (2/2)


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

10

Example solutions proposed

• Reference business trajectories

• Functional airspace blocks

• Flexible use of airspace

• ASAS applications

• Reduced separation criteria

• ...AIR TRANSPORT

MANUFACT-

URERS

OTHER

REGU-

LATORS

POLICY

MAKERS

ANSPs

AIRPORTS

AIRSPACE

USERS

HUMAN

SOCIETY

OTHER SERVICE

PROVIDERS

HUMAN

OPERATORS

ASSOCIATIONS

R&D required to tackle the major design hurdle faced!


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

11

E-OCVM (1/2)

• E-OCVM to support effective R&D: “... the process whereby the many stakeholders eventually should come to a decision to either: Continue development to ... or stop or substantially modify developments...”

V1Scope

V2Feasibility

V3Integration

V4Pre-operation

V5 Operation

V0ATM Needs

IdeaImplemented

Concept

Identify ATM performance

needs & constraints

Scope operational concepts and create validation strategy

Iteratively develop and

evaluate concept

Integrate concept in wider contextAnd confirm performance

Industrialisation and procedure

approvalImplementation


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

12

E-OCVM (2/2)

• E-OCVM poses specific, new requirements to safety case development

• Feedback to stakeholders!


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

13

Summary of motivation

• Most safety regulations & methods were developed for use by ANSP for changes to its ATM system

• Major changes to air traffic operations are needed to maintain an acceptable level of safety ambitious targets in multiple KPAs large number of stakeholders involved

• Major changes require R&D supported by safety analysis

• E-OCVM is the framework for validation of these major changes

• E-OCVM poses specific, new requirements to safety case development


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

14

Contents

• Motivation





Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

15

Safety analysis feedback to design

Design Analysis


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

16

Safety analysis tailoredto maturity

The aim of safety analysis changes from V1 to V5

Safety feedback to

design

Safety assurance

V1 V5


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

17

Safety analysis objectives per phase

V0:ATM Need

• Identify ATM need w.r.t. safety• Identify barriers

V1:Scope

• Plan & scope, based on evidence• Feedback to design

V2:Feasibility

• Determine feasibility• Feedback to design

V3: Integration

• Determine system level performance • Feedback to design


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

18

Safety analysis methods

Safety case development in R&D has been subject of a lot of recent researchExperiences with developing a safety case in E-

OCVM are just building upLarge design challenges pose several new

needs to safety case development in R&DSeveral complementary approaches are

emerging that aim to address the SESAR-identified emerging needs

Integration so far limited


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

19

SESAR-identified emerging needs

A. The need for a ‘macro’ safety case B. The need to address safety regulations C. The need to address the multi-stakeholder nature of

advancing air traffic operations D. The need to address the success side of a change

alsoE. The need to cover human operators in the ATM

system F. The need to identify unknown ‘emergent’ risks G. The need to address E-OCVM requirements H. The need to assess concept maturity I. The need for managing relations between cases


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

20

A: The need for a ‘macro’ safety case

• Motivation: SESAR consists of multiple local

changes by various stakeholders.

• Example: Functional Airspace Blocks Includes many smaller changes

• Identified approaches: Connect to an overall incident-accident model Apportioned safety criteria based on statistics ‘Joint safety analysis’


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

21

B: The need to address safety regulation

• Motivation: “Developing the ATM safety regulatory framework is essential to the success of SESAR”

• Example: ASAS applications Responsibilities transfer

from ground to cockpit ESARR 4 applied to airline?

• Identified approaches: Early scanning of concepts on fundamental safety issues

including existing safety regulations Address impact of changed regulations in early safety analysis Safety assessment assuming current regulations, while keeping

track of needs for changes


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

22

C: The need to address the multi-stakeholder nature

• Motivation SESAR will fundamentally

change stakeholder roles

• Example: FABs Who manages traffic? Who is responsible? Who decides on

acceptability of risk?

• Identified approach: Safety validation framework with active roles to be played

by all stakeholders- joint goal oriented approach- joint safety validation


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

23

D: The need to address the success side of a change also

• Motivation: Safety assessments have often

focused on failure ICAO has always asked to

address the success side also

• Example: TCAS RA downlink Focus on failure of downlink? What if downlink successful?

• Identified approaches: Integrated safety analysis covering both failures and

successes Complement traditional ‘failure approach’ with dedicated

‘success approach’


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

24

H: The need to assess concept maturity

• Motivation: How to decidewhether a concept is readyfor next E-OCVM phase?

• Example: individual SESAR development projects

• Identified approaches: Generic SARD criteria (Strategic Assessment of ATM R&D) Safety case specific set by CAATS II in SARD update Safety case specific set by EEC (for ‘SAME’)


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

25

I: The need for managing relations between cases

• Motivation: effectiveness and efficiency

• Example: use of real-time simulations Can multiple cases

benefit?

• Identified approaches: Safety & HF: share info where useful, disjoint where needed Safety & environment: disjoint analyses Safety providing input to business

Framework for managing relations between cases


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

26

Contents

• Motivation





Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

27

Basic steps

I. Select the phase of E-OCVM’s Concept Lifecycle Model to be tackled

II. Determine objective and scope of safety analysis in line with the selected phase

III. Determine methods and techniques to be used

IV. Document the results


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

28

Selection of methods/ techniques

• Develop expertise and practical experience with emerging methods

• Work on integration of emerging methods to combine their strong points

• There are complementary needs of advanced safety courses and hands-on safety learning

• Get an expert aware of these emerging needs, and with experience with emerging approaches!


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

29

Documentation

• ‘Negative’ analysis results have great value as feedback to design

• In R&D, the value is in the explanation why a concept is not yet valid or safe

Validation is most of the time invalidation Only the last cycle is validation!


Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

30

Contents

• Motivation





Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

31

Concluding remarks

• Experiences with developing a safety case in E-OCVM are just building up

• Several needs are emerging for safety case development for large design challenges, as traditional approaches fall short

• Several complementary approaches have been identified that aim to address the SESAR-identified emerging needs

• Key focus points: Gain experience with emerging complementary approaches Integration of emerging complementary approaches

Questions?

Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent


safety case development in atm r&d safety feedback for decision-makers and concept developers...

Documents

safety regulations

sids slide

concept developers episode

atm rd safety feedback

scholte nlratsi caats

practical example

atm system esarr

sid errors