Agent-based Dynamic Risk Modelling for ATM A White Paper

January 2014

EUROCONTROL / FAA AP15: Safety Research

EUROCONTROL / FAA Action Plan 15 on Safety Research is aimed at advancing safety concepts and practices in air traffic management, via the sharing of expertise from its membership. It has three main axes: understanding system safety, developing new approaches to assess and improve safety, and disseminating its results into the industry. AP15 came into existence in 2003 and its current terms of reference run until end of 2013. AP15 Membership EUROCONTROL – Barry Kirwan [Co-chair], Tony Licu, Eric Perrin, Andrew Kilner, and Dave

Young FAA – Paul Krois [Co-chair], Sherry Borener, Pradip Som, Dino Piccione, and Warren

Randolph. NASA – Michael Feary NATS (UK) – Neil May DFS (Germany) – Joerg Leonhardt AVINOR (Norway) – Anne-ki Chavez LFV (Sweden) – Billy Josefsson Austrocontrol – Michaela Heese NavCanada – Heather Henderson NLR (ALSO representing the Dutch CAA) – Michel Piers and Henk Blom For further information: barry.kirwan@eurocontrol.int paul.krois@faa.gov

Agent-based Dynamic Risk Modelling for ATM 1

FOREWORD

The FAA-EUROCONTROL Action Plan 15 on Safety Research (known simply as AP15) has been in existence since 2003. The FAA and EUROCONTROL, together with a number of European Air Navigation Service Providers (ANSPs) and NASA, have worked to evaluate and progress promising safety research areas that can aid real operational Air Traffic Management (ATM) safety and safety assurance. The first main output of AP15 was the ATM Safety Techniques Toolbox [5], which selected the most suitable safety assessment techniques for ATM from a ‘landscape’ of more than 500 safety techniques developed in nine industries. Since then, complementary AP15 outputs have included development of a ‘macro-model’ of ATM safety both in Europe and the US, referred to as ’Integrated Risk Picture’, as well as a series of ‘White Papers’ attempting to de-mystify key advances in safety, for example in the areas of safety culture, human performance and resilience, as well as safety during degraded mode operations. Motivated by the increased safety risk assessment challenges posed by NEXTGEN and SESAR developments, the current White Paper elaborates on the more advanced techniques already identified in the ATM safety techniques toolbox, under the title of agent-based Dynamic Risk Modelling (agent-based DRM). The key phrase in this White Paper is ‘Dynamic Interactions’. Whereas most conventional safety techniques such as fault and event trees consider a limited set of failures, events and outcomes, often in a binary fashion, agent-based DRM considers many more possibilities. For example, instead of saying the controller can do either ‘A’ (correct) or ‘B’ (failure), agent-based DRM will consider a broader range of human-system interactions. Such modelling is useful when investigating scenarios in which an analyst, when asked what will happen next, keeps on saying “well, it depends.” If there are many possible combinations or permutations in a risk scenario, agent-based DRM offers a more comprehensive way to assess risk. Moreover, agent-based DRM also assesses the success side of novel ATM design. Such modelling capability is ‘advanced’ in safety terms and requires particular safety and modelling competencies as well as taking more time to develop the model before it can be evaluated. Agent-based DRM is therefore not for everything, and one of the aims of this White Paper is to identify when it should be applied. However, agent-based DRM has shown to lead to novel and important insights, with a sharper and more accurate assessment of risk and successes in complex time-dependent scenarios. Such benefits are exemplified in the White Paper via short case studies from real applications. The uptake of agent-based DRM to date in ATM safety has been less than is probably warranted, given the ‘dynamicity’ and complexity of both contemporary and future ATM operations. It is therefore hoped this White Paper will help ‘de-mystify’ agent-based Dynamic Risk Modelling, so that its use can be considered both for large infrastructure development programmes such as NEXTGEN and SESAR, as well as more local safety cases concerning new developments and improvements.

2

INTRODUCTION FAA-EUROCONTROL Action Plan 15 on Safety Research identified the need to promote a wider use within the Air Traffic Management (ATM) community of the more advanced techniques in AP15’s toolbox of safety assessment techniques [5]. These more advanced techniques are summarized under the term agent-based Dynamic Risk Modelling (agent-based DRM). The aim of this white paper is to provide a high level overview of agent-based DRM techniques together with insight on when and how to apply them to ATM. The white paper aims to answer the following questions: Why do we need more advanced techniques to address safety risk assessment in ATM? What do we mean by the term ‘agent’, and why do we need to think of ATM in terms of

multiple human and technical agents that work together to provide a safe operation? What do we mean by the terms ‘hazards’ and ‘dynamic interactions’? What is the principal difference between agent-based DRM and logic tree-based approaches? How does one conduct agent-based DRM? What types of output does agent-based DRM provide and what are some specific examples of

the feedback that it delivers to operational design? What are some examples of agent-based DRM applications? When do you need to use agent-based DRM, and when could you do with other approaches? What are the resource requirements for agent-based DRM? How can agent-based DRM be used together with other approaches?

The white paper is completed with a list of references to literature.

TCAS:

Climb!

Climb!

TCAS:

DESCEND!

DESCEND!

ATC:

BK 7886, I told

you to

Descend!

Looks like a

dynamic situation,

here, captain!

Agent-based Dynamic Risk Modelling for ATM 3

WHY AGENT-BASED DRM If there is one phrase that captures the difference between ATM and other high risk industries, it is the phrase ‘dynamic interactions’. ATM operations are characterised by dynamic interactions between a broad variety of distributed human decision makers, a multitude of technical systems and a wide variety of environmental conditions. Some examples of such interactions are a navigation system supporting pilots, a surveillance system supporting controllers, changes in weather causing outdated flight plans, and communication between pilots and controllers. In ATM things are always moving, and the controllers and pilots are working successfully with real risk in real-time. Over decades, this complex socio-technical ATM system has evolved to its current way of operation, while providing a very high level of safety. Growing demands in commercial air transport form the driver for NEXTGEN and SESAR to work on the design and implementation of conceptual changes in this complex socio-technical ATM system. Because of the complex dynamic interactions, such changes may not only lead to the intended improvements, but also lead to unforeseen safety-critical behaviours. If these unforeseen behaviours are not identified and understood during the SESAR or NEXTGEN design stages, they potentially undermine ATM’s future safety level. At the same time, the complex dynamic interactions may lead to foreseen or unforeseen behaviour that is in fact positive for safety. The result is a socio-technical ATM system that exposes a complex dynamic mix of positive and negative safety contributions. It should come as no surprise that as a system increases in complexity, the evaluation of its safety poses novel challenges to safety risk assessment. Moreover, the safety evaluation of this socio-technical system design should start as early as possible, in order to improve it before major investments in infrastructure and technical systems are beyond a point of no return.

NEXTGEN-identified features that contribute to dynamic complexity [14]

Air transportation system of systems featuresü Emergent propertiesü Autonomous operationsü Interconnected consituentsü Ambiguous/changing boundariesü Multiple contexts and influencesü Dynamic stakeholder relationships

NEXTGEN compared to current ATMü Higher traffic density ü Increased use of automation and

automation at higher levels ü Operations that are more tightly coupled ü Decentralized operations vs. centralized

operations ü Introducing multiple new elements within

a short time frame

SESAR-identified safety evaluation needs [11]

Needs regarding safety assessmentü Producing a macro safety caseü Addressing the success sideü Covering performance of human operatorsü Identifying unknown emergent risksü Covering organizational safety

Needs regarding organizing safety validationü Addressing E-OCVM requirementsü Managing relations of the safety case with

other casesü Addressing the multi-stakeholder natureü Addressing future safety regulations

4

In order to cope with the challenge of safety risk assessment for a complex socio-technical system, [15] identifies the need for a Dynamic Risk Modelling (DRM) approach that integrates stochastic dynamic behaviour, and generates various scenarios through dynamic simulation. Such approach uses models of controlled process dynamics and human operator behaviour, and simulation techniques such as Monte Carlo simulation to evaluate safety in various socio-technical systems. However, as is depicted in the figure below, there also are significant differences between the socio-technical systems in the different industries. In nuclear or chemical industries a catastrophic event may involve a much larger number of fatalities than in ATM. On the other hand, in ATM the different actors are more highly distributed: each aircraft has its own crew which interacts with several air traffic controllers on the ground. This implies a highly distributed network of interactions between many humans and technical systems. These highly distributed interactions pose challenges to safety risk analysis for ATM that are complementary to what DRM alone can address.

The evaluation of highly distributed interactions in a socio-technical system asks for an agent-based approach. Agent-based modelling and simulation has been applied in various areas, such as ecology, political science, social science, economics, evolutionary biology, biomedical science and computer science. In all these areas, it has shown to be a powerful approach in learning to understand the effect of dynamically interacting agent situations. For ATM safety risk analysis this means that there is a need for an integration of agent-based modelling and simulation and DRM. This finding has motivated NLR researchers to focus on the development and integration of agent-based and DRM techniques in e.g. [16], [17], [18], [26], [27], [32] with challenging applications to ATM in mind. Recently, these advanced agent-based DRM techniques have also been compared [48] to an established event sequence based risk assessment for a specific ATM application. This comparison showed significant advantages of agent-based DRM techniques. The only challenge, when it comes to applying these agent-based DRM techniques, is a need for safety analysts to learn novel kinds of expertise.

Agent-based Dynamic Risk Modelling for ATM 5

AGENT-BASED THINKING Agent-based thinking is already extensively applied to various socio-technical systems such as in politicology, biology, and economics, e.g. [13]. Although various definitions are used, an agent is basically an autonomous entity, which may interact with other agents to exchange information of various kinds. Examples of interactions are observations by controllers, pilots and surveillance systems of the air traffic situation; communications between controller and pilots, including read back; failures and alerts of technical systems having an impact on operator behaviour, etc. The interactions do not have a predefined order and timing and can be of various types, as is illustrated below.

The modelling of individual agents may require complementary expertise. For example, the modelling of a human agent such as a pilot or an air traffic controller may require expertise from pilots and/or controllers, as well as of psychologists or other human factors experts. The modelling of a technical system agent such as a surveillance support system or an aircraft system may require expertise from system developers or maintenance technicians. A drawback of this need is that the input from many diverse experts may be required to build a good agent-based model. A key advantage is that during the modelling phase, each expert only needs to consider the agent within their field of expertise; this is in contrast with safety analysts using non-agent-based modelling, who need to be able to understand the risks involved in the whole operation under all conditions during the modelling phase.

Various interactions: An ATM example

Consider a runway with an aircraft about to land and a taxiing aircraft arriving at the runway entry point wishing to take off. A tower controller is monitoring this situation by visually observing both aircraft, also supported by ground radar and a conflict alerting system. The pilot of the taxiing aircraft reports to be at the runway stopbar and requests clearance to enter the runway. After a reaction time, the controller (erroneously) issues the clearance, after which an audio conflict alert is triggered.

In this situation, we see various types of interactions occurring. The visual observation of the aircraft position and velocity by the controller is an example of information that is continuously passed on and is actively obtained; this information is also an element from a continuous set: it is not countable. The surveillance system, using radar sweeps, collects the position information at discrete and regular time intervals of, say, a few seconds. The reaction time of the controller is an example of a stochastic time interval: there is natural variation, which may additionally be affected by current workload and skills. The alert by the conflict detection system, triggered by the expected position of the two aircraft being too close together, is an example of information triggered by a particular event. The controller does not actively obtain this audio alert, which makes it an example of passively obtained information. Examples of discrete states are an aircraft being at a stopbar or not, or an audio alert being on or off.

The information may be passed on continuously, albeit sometimes during a finite period of time, or at discrete time instants.

These discrete time instants may be after regular time intervals, or after stochastic (random) time intervals, or be triggered by a particular event.

The information may be regarding a discrete state or be a value from a continuous set such as an interval.

The information may be obtained passively or actively.

6

HAZARDS AND DYNAMIC INTERACTIONS

Hazards Hazards are any conditions, events or circumstances that could induce an accident. Often, it cannot be known in advance if an identified circumstance has potential to actually induce an accident. Therefore, for proper safety risk assessment of complex sociotechnical operations in ATM, the term “hazard” should be interpreted in a sufficiently broad way, such that all aspects that may somehow influence safety are addressed. As such, the term includes a wide variety of events/conditions/circumstances that may contribute to the development of safety-relevant air traffic situations (root hazards) or that may hamper the resolution of these air traffic situations (resolution hazards), or both.

Scenarios with dynamic interactions In safety risk assessment of air transport scenarios there are many potential orderings by which hazards and other events may occur, there are many temporal relations between the events, and there are many ways by which such dynamically and stochastically varying event sequences may lead to an accident type. To explain this, consider the pictures to the left, which show two examples of event sequences that may occur in a single ATM operation. The operation considers

one aircraft taking off on a runway, and one aircraft taxiing near the runway. The air traffic controller has a Runway Incursion Alerting System that may warn them in case the taxiing aircraft makes a runway incursion. In the pictures, vertically, the various agents in the operation are shown; horizontally there is the time axis, with events happening for the various agents at various time instants. The events and their timing for different agents may also be interdependent and be dependent on the context and hazards occurring. The event sequences may lead to various outcomes. There are many more potential event sequences like these two, with many more potential outcomes. Safety risk assessment must account for such large varieties of event sequences and appropriately estimate the probability of event sequences leading to accidents, given realistic operational contexts.

Examples of hazards in ATM R/T communication system is not working. False track in surveillance data. Delay in alert by ATC system. Controller misinterprets aircraft location on radar screen. Tower controller is not regularly monitoring out of the window. Pilot misunderstands clearance of controller. Pilot does not know location on aerodrome. Poor visibility condition. Slippery runway condition.

Time

Agent-based Dynamic Risk Modelling for ATM 7

DYNAMIC SIMULATION vs EVENT TREES Consider the example event tree below. It starts with an aircraft crossing while it should not, and with another aircraft in take-off. Next, horizontally, it shows a sequence of events that may fail or be successful. The result is 21 possible consequences, clustered in 5 results: no conflict, early resolution, medium resolution, late resolution, or accident. Quantification is by estimating the probability of the success or failure of each intermediate event, conditional on all previous events. Since all interactions between and behaviour of agents and variation in circumstances need to be taken into account in these probabilities somehow, this is not a trivial task. Of even more concern, many potential event sequences and alternative consequences are not included because they do not fit in this layout. Note that it is important to realise that these observations are not specific for this example tree, but are typical for the approach of modelling through finite and static event sequences.

Agent-based DRM takes a different approach. Using a stochastic dynamic model implemented in a software environment, it dynamically simulates what may happen in the operation, taking into account that agents interact with each other and react on the information they are aware of, and taking into account that most events occur at stochastic times and possibly in various orders, just like in the two figures earlier. Millions of simulations are run like this, and the event sequences are recorded. Compared to the number of event sequences in the event tree above, this provides a far more comprehensive overview of the variety of event sequences occurring, including several event sequences or behaviours that emerge unexpectedly. Importantly, one can trace back the simulated event sequences leading up to the safety relevant consequences, in order to understand where the risk is coming from. One does not have to ‘guesstimate’ the probabilities of all conditional event combinations; these are an output of the model rather than an input. Another advantage is that one can ‘explore the model’. For example, one could consider the situation where the controller does not have a conflict alerting system, simply by choosing a specific set of initialising parameter values that set the occurrence of an alarm to zero. Since all interactions are already modelled, there is no need to reassess all conditional event sequences; there is only the need to re-run the simulations.

Aircraft crossing while it

should not

Pilots recogn early

ATCo recogn early

RIASS early

Comm early

. RIASS medium

Comm medium

Comm late

Result

Early resolution

Early resolution

Early resolution

Medium resolution

Medium resolution

Medium resolution

Medium resolution

Medium resolution

Late resolution

Late resolution

Late resolution

Late resolution

Late resolution

Late resolution

Accident

Accident

Accident

Accident

Accident

Accident

No aircraft in take off

No conflict

Pilots

recogn

medium

ATCo

recogn

medium

Pilots

recogn

late

ATCo

recogn

late

How would you quantify

the probability of an early

alert by the Runway

Incursion Alerting System

(RIASS), conditional on

the event that neither the

pilots nor the controller

(ATCo) recognised the

conflict early? How about a conflict that is recognised but the resolution

is ineffective, due to the pilot already implementing a

resolution, upon which the controller gives an instruction

that is not in line with the pilot resolution? The evolution

of such a situation would depend on the physical position

and velocity of both aircraft and the precise timing at

which the various actions are initiated.

8

NO

ANALYSIS ALLOWED

!

AGENT-BASED DRM – HOW IS IT DONE?

Preparatory step: Setting the scope and collecting input information The agent-based DRM activity starts with setting the scope, goal and objectives of the dynamic risk modelling, and collecting all necessary input information. Setting and clearly formulating the scope is essential, to avoid having to chase a moving target, and to clarify expectations. The input information to be collected includes a description of the operation be assessed, and a list of hazards. A description of the operation is usually available from the designers of the operation, e.g. by means of an operational environment description. However it is important to collect and edit this material into a concise description that ensures that no information relevant for dynamic risk modelling is missing.

The list of hazards includes a wide and comprehensive list of ‘anything that may affect safety’. Anything satisfying this definition can be put on the list; overlapping or similar hazards, or issues that may affect safety only in an indirect way are not removed at this stage. The hazards are not analysed yet; an identification of causes, consequences or likelihoods of hazards is of no concern yet. The reason for this is that at this point, before any dynamic risk modelling and assessment has been done, it cannot be known which hazards are negligible, and which hazards, in combination with other hazards or conditions, could lead to emergent effects. A list of hazards is often available from preceding risk assessments, or from risk assessments of similar operations. Typically, however, it is necessary to organise a complementary hazard brainstorm session with experts who are able to play the devil’s advocate, and who are able to also identify hazards that are unimaginable by classical approaches.

Scope, Goal and Objectives

The very first step of any safety risk assessment exercise is the specification of the scope, the goal, and the safety design objectives of the safety risk assessment:

Scope. The scope specifies the boundaries of the operation to be modelled. This includes the boundaries of the operational area considered, the types of functions or the types of equipment/procedures/people that are included, and the types of risk that are considered in the safety assessment (e.g. collision risk, runway incursion risk, etc.). The scope is best presented as a bulleted list of issues to be included and not included in the assessment.

Goal. The goal specifies what the safety risk assessment aims to achieve; for instance, to provide feedback to operational concept designers on the level of risk associated with the operation and on the main contributors to this risk.

Objectives. Safety criteria or safety objectives refer to target or acceptable levels of risk that are aimed to be achieved.

Description of the operation

After the scope, goal and objectives are set, a description of the operation is produced, which typically includes:

Context. This covers the timeframe, the traffic characteristics, and the geometric aspects of the operation.

Human roles. This covers a clear description of the roles and responsibilities of the humans involved in the operation.

Procedures. The operational procedures, both on the ground and on board are described, including procedures in dealing with non-nominal situations.

Technical systems. Finally, a logical description of the technical systems used in the operation is provided, covering their overall performance and how they are operated.

Agent-based Dynamic Risk Modelling for ATM 9

Agent-based DRM steps After this preparatory step, the agent-based DRM steps follow: 1. Deciding who are the agents 2. Modelling hazards 3. Developing a stochastic dynamic model for each agent, and modelling their interactions 4. Rare event Monte Carlo simulation 5. Evaluating the differences between the model and the real operation

Step 1: Deciding who are the agents

The first major step in agent-based DRM is identifying the agents in the operation. Each agent is an autonomous entity in the operation that is able to perceive its environment, including other agents, and that is able to act upon and interact with this environment. Agents can be human operators, technical systems, or complete organisations, but the weather is sometimes also referred to as an agent.

Each agent may consist of various sub-agents or entities. For example, the ATC system may consist of various subsystems such as radar display, conflict alert system, communication system, etc. A human agent such as an air traffic controller may also be seen as composed of various ‘entities’, e.g. goal, tasks, situation awareness, current level of workload, etc. In addition, for one operation there may be more emphasis and more detail required on a particular agent than in another operation, and several entities could better be ‘clustered’ or ‘split up’. Typically, several iterations are required before an appropriate set of agents and agent entities is obtained.

10

Step 2: Modelling hazards in agent-based DRM The second step is to identify the agent-based hazard model constructs required to capture the hazards applicable to the operation. Each such model construct is an abstraction of one aspect in an agent. Through a recent study, a rather complete library of 38 model constructs has been developed for application in ATM [21], [25], [33]. The study has also shown that this library addresses some 95% of all potential hazards in ATM. Most of the model constructs in the library (i.e. about two-thirds) are human performance related, and include model constructs for Human Information Processing [6], [12], Human Error [9], Cognitive Control Mode [7], Multi-Agent Situation Awareness [32], Extended Mind [1], Operator Functional State [22], Group Emotion [23], etc. A few example model constructs are depicted below.

Identity of LState of LMode of LIntent of L

E.g. Call-sign, Alert type

E.g. Aircraft position, speed

E.g. Stopbar status, Flight mode

E.g. Taxiing route, Take-off time

Situation awareness of agent K at time t

about agent L=

“Multi-Agent Situation Awareness” writes the awareness of one agent about the situation of another agent as a vector

This situation awareness can be updated through Observation, Communication, or Reasoning

“Discrete Mode” distinguishes various modes of a technical system, as well as the mode switches.

These modes have particular durations.

Mode 1 Mode 2

E.g. System is failing E.g. System is functioning without failure

“Cognitive Control Mode” considers that humans can function in a number of cognitive control modes as depicted below. This mode may depend on human

performance aspects such as range of tasks to be done.

“Dynamic variability” describes the variability of agent states due

to dynamic processes.E.g. Differential equation for the Position and velocity of an aircraft

“Group Emotion” describes the dynamics of the spread of emotion over a group of

individuals, based on personal characteristics of the individuals and

relations between individuals“Operator Functional State” describes a person’s functional

state as a dynamical state, which is a function of task properties and personal characteristics.

In general, not all 38 model constructs from the library will apply to the specific ATM operation considered. In order to find out which do, first for each applicable hazard it is determined by which of the model constructs from the library it is addressed. Next, the applicable model constructs are collected for each agent identified in Step 1 of agent-based DRM. Finally, these model constructs are integrated in a multi-agent model [25]. In practice, it may even be an advantage to restrict the number of model constructs to the ones assessed as being most important, and to evaluate the effect of the choices during Step 5.

Agent-based Dynamic Risk Modelling for ATM 11

Step 3: Developing a stochastic dynamic model for each agent, and modelling their interactions The next step is to develop a stochastic dynamic model for each agent in the operation, and to model their interactions. For this, the formalism of stochastically and dynamically coloured Petri net (SDCPN) is used, since it has many important strengths. The SDCPN formalism was obtained through extensions [28], [29], [31] of ordinary Petri Nets, developed in 1962 by C.A. Petri [8]. These extensions maintain the graphical elements and key properties of ordinary Petri nets, and add the notions of time, continuous-valued processes, various types of stochastics, and hierarchical modelling.

The process to develop an SDCPN-based model proceeds in several substeps. The first substep is to develop a local SDCPN-based model for each agent entity identified for the ATM operation, by specifying all SDCPN elements, and how they work together. This uses the results of Step 2 (previous page) Next, the entities within one agent are coupled by modelling the interactions within each agent. Subsequently, all agent models are coupled by modelling the interactions between agents. Normally, there are iterations and loops between all substeps. Finally, all parameters are given a value.

G

D

I

- Place, representing discrete mode or state.- Token, which is an object residing in a place. Each token has a value (colour).- Delay transition, will fire (i.e. remove and produce tokens) after a stochastic delay which is exponentially distributed.- Guard transition, will fire if its input tokens reach particular colours.- Immediate transition, will fire without delay.- Arc, connects transitions with places. - Enabling arc, is a special arc along which tokens are not removed when a transition fires.

SDCPN Elements

I

I

I

I

Turn Straight Descend

G Surveillance info

Aircraft evolution

Surveillance radar updates

STCA

STCA availability

In the Aircraft Evolution local SDCPN, there is one aircraft (one token) that can be in one of three modes (three places): Turn, Straight, or Descend; mode switches are controlled by the pilots’ Flight control local SDCPN. The token in Aircraft evolution has a multi-dimensionl value (a colour) that corresponds with the current aircraft position and velocity. This colour evolves

through time according to a stochastic differential equation corresponding to the mode (place) the token resides in.

In the Surveillance radar updates local SDCPN,

every few seconds, the aircraft positions are

determined by surveillance radar, which uses as input the actual

aircraft positions modelled in Aircraft

evolution. The surveillance information

is documented in the colour of the token in

place Surveillance info.

The STCA availability local SDCPN models that

the Short Term Conflict Alert system may be On or

Off due to maintenance

The STCA local SDCPN uses as input the aircraft positions received from Surveillance

radar, and compares these with each other to predict conflicts between aircraft. This only

works if STCA availability is in mode STCA On.

SDCPN-based example of a Short Term Conflict Alert system

Flight control

G

DD

STCA Off

STCAOn

G

GConflictalert

No conflict alert

Main strengths of SDCPN SDCPN have a graphical representation, making sure that the models are readable and verifiable. Their modelling power admits all types of stochastic dynamic processes and interactions occurring in ATM

operations, including causal dependencies, concurrent processes, synchronisation of events, continuous processes, discrete events, random events, etc.

The models can be built in a hierarchical way, starting from models for local agent entities, and building up to multi-agent models including all interactions.

Each agent entity model maintains its own state throughout the modelling and safety risk analysis process. This improves readability and allows recycling of agent entity models from previous modelling exercises.

The parameters in an SDCPN-based model are largely of a physical nature, and are relatively easily quantified compared to the parameters of approaches that rely on probabilities of conditional events.

SDCPN generated processes have the strong Markov property, which is a pre-requisite for speeding up the analysis (see Step 4 on next page).

12

Step 4: Rare event Monte Carlo simulation Once the SDCPN-based model for the operation is completed, it is implemented in a software environment for Monte Carlo simulation. This way, the operation can be played out millions of

times to see how various event sequences may evolve under all conditions. Selected safety-critical events can be recorded, and their frequency of occurrence can be determined by counting how many times they happen in these millions of runs. However, air traffic is a very safe means of transport and the probability of a collision between two aircraft is extremely low. The assess-ment of such low collision risk values through straightforward Monte Carlo simulation would require an unrealistic number of simulation runs. Therefore, acceleration methods are required. For collision risk assessment in ATM, several main means of achieving such acceleration have been developed (described further below). They all rely on a certain mathematical property of SDCPN-based models, which is the ‘strong Markov property’. This property allows pausing a simulation run at a well-defined random moment in time called a ‘stopping time’, recording the current multi-

agent state, and starting it up again without needing to keep track of everything that led to this state: the knowledge of the current state is sufficient to re-start the simulation. If the paused state does not suggest interesting event sequences from a safety risk point of view, we can skip the remainder of the simulation and save time. It also allows starting up from a given conditional state, e.g. a state in which a particular initiating event is about to happen, and skipping the first part of the simulation. If we collect many Monte Carlo runs that have all been paused at a particular stopping time (which does not have to be the same time on the clock for all runs), we can line them up and restart them all from one time instant on the clock. Means of achieving acceleration mentioned above include Risk decomposition, and Particle Swarm approaches. Risk decomposition consists of decomposing accident risk simulations in a sequence of conditional Monte Carlo simulations and combining the results of these conditional simulations into the assessed collision risk value. The conditional runs are started from a conditional state, which is allowed due to the strong Markov property of SDCPN. Particle Swarm approaches [19], [20] introduce a sequence of intermediate aircraft encounter conditions that are always preceding a collision. The collision probability is determined as the product of

conditional probabilities of reaching these intermediate encounter conditions. Such a conditional probability is estimated by simulating in parallel a large number of copies of the process, where each simulated copy forms a particle that follows a trajectory of the process dynamics. Again, the strong Markov property guarantees that the Particle Swarm converges well.

A process has the ‘Markov property’ if the probability of occurrence of any future state only requires knowledge about the current state; hence no knowledge about past states is needed.

A ‘stopping time’ is a random moment in time at which ‘something detectable happens’, in the sense that we are able to decide on the basis of the information known about the process, whether the time moment happens.

Strong Markov PropertyA process has the ‘strong Markov property’ if it has the ‘Markov property’ at any ‘stopping time’. In mathematics, these notions have precise meanings.

Math Alert!

Monte Carlo simulationConsider a given model, representing a safety critical process such as an air traffic operation. The model uses parameters that take on quantitative values, e.g.: reaction time of controller, wind speed, aircraft width. Parameters may be used by the model multiple times, depending on how the modelled process evolves. Usually, some parameters are stochastic; they are represented by probability density functions (pdf), which show that they may take on a range of values, each with a certain probability of occurrence.

In Monte Carlo simulation of this given model, the model is simulated in a software environment. Each time a parameter is used, it is given a single random value that is selected from its respective pdf. One complete run of the model generates an output result. A large number (e.g. millions) of runs is conducted; the outcome is a large number of separate and independent output results. These results are assembled into a pdf or expected value for the output, e.g. probability of occurrence of a safety-critical event.

Frequent events ~ 10-100 times per flight hour, e.g. operator actions

Medium frequency events ~ once per 1E4 flight hours, e.g. incidents

Rare events ~ once per 1E9 flight hours, e.g. aircraft collisions

The safety pyramid is very high

Agent-based Dynamic Risk Modelling for ATM 13

Step 5: Evaluating differences between the model and the real operation By the very nature of any model, there are differences between a real operation and a model of the operation. It is simply not possible, but also unnecessary, to model reality in complete detail. Hence the advantages of modelling (i.e. one is able to evaluate numerous situations that cannot be evaluated in real life) come at the expense of having to take into account the effects of differences between model and reality. In an agent-based DRM-based risk assessment, this is pursued by identifying all potential differences between SDCPN based model and reality, and by systematically assessing the ‘bias and uncertainty’ in the risk that is expected to be inferred by those differences [30].

The bias and uncertainty assessment not only provides a value for the model-based safety risk compensated for all differences between model and reality, it also has other important results: It provides a complete list of differences between model and reality, including all

assumptions adopted in the modelling. This list is provided in words, hence is readable also to those who are not willing or able to read the SDCPN-based model. This way, the model is made transparent to a large group of experts.

The bias and uncertainty assessment process helps to provide further insight into the model and the safety risk results. The sensitivity analysis helps to better understand how the output reacts to changes in the input, and helps to better understand where the risk is coming from.

It identifies those differences that have the highest impact on risk, in terms of bias, uncertainty, sensitivity or combinations of these. These differences can be used to find elements in the operation that are most efficiently improved, hence provide effective feedback to the designers of the ATM operational concept.

Steps in the bias and uncertainty assessment:ü Identify potential differences between model and reality, including differences between

(i) the values assumed in the SDCPN simulation and the real parameter values; (ii) differences between SDCPN structure assumed and structure in reality; (iii) differences due to hazards that are not modelled by the SDCPN; and (iv) differences between the operational concept assumed in the SDCPN and the real one.

ü For each parameter value, assess a bias factor (A) and a corresponding uncertainty interval (B). For other types of differences, assess the probability that the difference applies (C).

ü Assess the sensitivity to risk for changes in parameter values (D). Additional Monte Carlo simulations with the SDCPN are conducted for this.

ü Combine the bias (A) and uncertainty interval (B) of each parameter value with the risk sensitivities (D), to assess the effect of each potential parameter value difference on the risk outcome (E).

ü For the non-parameter types of differences, assess a conditional risk bias given the difference exists (F) and combine this with the probability that the difference exists (C) into a risk bias for the difference (G).

ü The joint effect of all differences on the bias and uncertainty interval of the risk is determined by combining the results (E, G).

... so this is why the safety risk of this design is so high

14

AGENT-BASED DRM OUTPUT The variety in types of results that can be attained by agent-based DRM goes beyond the types of results than are typically achieved in traditional safety risk assessment approaches. Agent-based DRM output includes the following types of results. Probability estimates of safety-relevant events, such as aircraft collisions. This is a main

safety risk assessment result, which is also achieved by traditional approaches. Level of uncertainty in the risk estimates. This provides insight in the achieved accuracy of

the risk assessment, which is sometimes also achieved by traditional approaches. Risk uncertainty and risk sensitivity for individual parameters. This provides insight in the

aspects that contribute most to uncertainty in risk results. (See previous page) Contributions to risk reduction by various agents (e.g. pilots, controllers, alerting systems)

interacting in a safety-relevant scenario. As result of the risk decomposition and the flexibility of changing model settings in agent-based DRM, it is straightforward to obtain Monte Carlo simulation-based risk results for many conditions and variations. By enabling/disabling roles of agents in the Monte Carlo simulations, insight is attained in the risk reduction contributions. Such results depend considerably on the interactions in the multi-agent system, which are harder to predict by traditional safety assessment approaches. (Figure top)

Risk results for variations of physical parameters in the operational concept. Due to the direct representation of physical processes in the agent-based model, Monte Carlo simulation-based risk results can be obtained as a function of relevant inputs, e.g. the distance of a stop-bar to a runway. (Figure bottom-left)

Probability density functions of safety-relevant outputs in the scenario considered. In the Monte Carlo simulations of a scenario a selection of safety-relevant outputs can be tracked, e.g. aircraft positions at the time of conflict recognition or at the time of initiating collision avoiding action. This provides further insight in dynamics of the performance of the agents. (Figure bottom-right).

PF-TX refers to Pilot-Flying Taxiing Aircraft. PF-TO refers to Pilot-Flying Taking-Off Aircraft. ATCo-R stands for Runway Controller. Alerts are ATC runway incursion alerts. The value on top of each bar represents the risk increase factor with respect to case C1 (nominal case, performance of agents is not affected).

Agent-based Dynamic Risk Modelling for ATM 15

FEEDBACK TO DESIGN - EXAMPLES The development of the agent-based DRM method described in this white paper started in the 1990s, and from the beginning the method has been applied to safety risk assessment of numerous ATM operations. In all these applications, the outcome of the agent-based DRM study has been fed back to the operation development through communication with the associated design teams. Two example applications of this feedback are described below.

Case of Active Runway Crossing This case addresses an operation to be deployed by LVNL, the ANSP at Amsterdam airport. In 2000, to reduce noise impacts of air traffic, the physical construction of an additional runway 18R/36L was started. As all terminals and parking positions were around the centre of the airport, all aircraft using the new runway had to pass runway 18C/36C that was often in use as well. In order to limit taxi time to and from runway 18R/36L, the preferred option was to cross runway 18C/36C, even under active departure/landing conditions. On the basis of a functional hazard assessment it was concluded that active runway crossings would be acceptably safe if a dedicated runway controller would be supported by a reliable runway incursion alert system. Upon this, LVNL decided to proceed with the further development of this active runway crossing, and asked NLR to perform a safety risk analysis. For this active runway crossing case, NLR conducted two subsequent safety risk assessments: a fault/event tree based study [3], and an agent-based DRM study [32]. Both studies showed that it could not be assured that the operational safety risks would be acceptably low. In addition, the agent-based DRM study provided some key new insights regarding the effectiveness of the various operational actors in identifying and responding to conflicts, [45]. Based on this feedback, LVNL decided to avoid crossing active runway 18C/36C, and instead to taxi around it. Another agent-based DRM showed that this taxiing around operation would be acceptably safe, [45]; this operation has been deployed.

Case of Airborne self-separation In this case, agent-based DRM has been used to assess two well-developed airborne self-separation designs on the capability of safely accommodating very high en-route traffic demands. The first airborne self-separation concept was the so called Autonomous Mediterranean Free Flight (AMFF) concept. The outcomes of an agent-based DRM study of AMFF were twofold [36]. First, for low traffic demands the study showed how safety is realized, and how reliable the various technical systems should be. Second, the study showed that the AMFF concept falls short in safely accommodating high en-route traffic demands. Triggered by these findings, and NASA developments, a more advanced airborne self-separation design was developed under the name Advanced Autonomous Aircraft (A3) concept of operations [2]. Through a subsequent agent-based DRM evaluation of this A3 design, positive emergent behaviours were identified under very high en-route traffic demands [37], [38], and it has been concluded that advanced airborne self-separation is capable of safely accommodating these high traffic demands. For advanced ATM designers in SESAR and NEXTGEN this opens novel ways at looking toward future ATM designs as well as the safety/capacity evaluation of these designs through agent-based DRM [38].

Airport

Centre

Runway

18C/36C

Runway

18R/36L

16

AGENT-BASED DRM APPLICATIONS

The figure below shows a selection of agent-based DRM applications to various operations at different lifecycle stages of the European Operational Concept Validation Methodology (E-OCVM), [4]. The lifecycle stage varies from a V1/V2 design of a potential future ATM operation to a V5/V6 design by an ANSP organisation. The overview also indicates which party performed the agent-based DRM application. Notice that these parties are NLR, the University of Sao Paulo (Brazil), ENRI (Japan) and the University of Belgrade (Serbia).

Airborne self separation.This application has been addressed in a sequence of studies, on various airborne self separation operational concepts in high density en route airspace. Of key interest was the assessment of conflicts being resolved by the pilots themselves.

Application by: Lifecycle stage: Client: Reference:

NLR V1 (Scope), V2 (Feasibility)EUe.g. [36], [37], [38]

ASAS-based Interval Management.This application considered aircraft on approach supported by ASAS-based interval management. Of key interest was understanding the behaviour of aircraft during an ASAS surveillance failure.

Application by: Lifecycle stage: Client:

Reference:

Electronic Navigation Research Institute (Japan)V3 (Pre-industrial)Japan Ministry of Land, Infrastruc-

ture, Transportation and Tourism

e.g. [42]

Conventional ATM: Reduction of separation minima.

This application has been addressed in a sequence of studies. Of key interest was the development of good models for human cognition.

NLRV6 (Operations)Eurocontrol and EUe.g. [34]

Application by: Lifecycle stage: Client: Reference:

SESAR 2020+ early design.This application considered novel design for future Terminal Manoeuvring Area. Of key interest was the assessment of collision risk in case of aircraft deviating from their negotiated trajectory.

Application by: Lifecycle stage: Client: Reference:

NLR with PartnersV1 (Scope), V2 (Feasibility)SESAR / EUe.g. [41]

Active runway crossing.This application considered an active runway with stopbars and with a runway incursion alerting tool for the controller. Of key interest was collision risk between a landing aircraft and a taxiing aircraft that wasn’t aware they were entering an active runway.

NLRV2 (Feasibility)ATC The Netherlandse.g. [46], [48]

Application by: Lifecycle stage: Client: Reference:

Airborne Collision Avoidance System.This application considered modelling of TCAS II by means of an agent-based dynamic risk model. Of key interest was determining the most critical elements contributing to the collision risk of the operation.

Univ. Of Belgrade (Serbia)V6 (Operations)EUe.g. [44]

Application by: Lifecycle stage: Client: Reference:

ASAS time-based spacing.This application considered aircraft on approach supported by ASAS time-based spacing, but with a controller in the loop. Of key interest was the distribution of tasks between pilots and ATC.

Univ. Of Sao Paulo (Brazil)V2 (Feasibility)n.a.e.g. [39]

Application by: Lifecycle stage: Client: Reference:

Simultaneous use of converging runways.This application has been addressed in a sequence of studies that considered various combinations of two active converging runways at Schiphol airport. Of key interest was the assessment of collision risk in case of a simultaneous missed approach.

NLRV5 (Deployment)CAA The Netherlandse.g. [35]

Application by: Lifecycle stage: Client: Reference:

Agent-based Dynamic Risk Modelling for ATM 17

WHEN YOU USE AGENT-BASED DRM AND WHEN YOU DON’T This white paper explained agent-based Dynamic Risk Modelling, in terms of why you need it, and how it works. But it did not explain yet when you actually need agent-based DRM, and when you could do with a static risk modelling approach. Agent-based DRM should be used if any of conditions A-E below holds true A. If the operation is such that hazards and other events may occur in many different sequences

or combinations and if different sequences or combinations have different effects on risk. B. If the risk of an event is dependent on whether other events occur earlier or later. C. If the value of continuously changing variables (e.g. position of an aircraft) affects the risk

associated with the behaviour of the operation. D. If the interactions in the proposed ATM design are novel, relative to conventional ATM, as a

result of which unknown positive or negative (rare) emergent behaviour may apply. E. If the values for physical parameters in the design remain to be decided (e.g. position of a

runway crossing, maximum traffic demand).

A possible way to identify whether one or more of these conditions apply is to conduct a qualitative safety analysis of the operation considered. It is also noted that one or more of these conditions will usually hold true to some extent. In such case, the decision to use agent-based DRM in favour of other approaches also depends on the expertise and tools available, and sometimes a decision is made to use agent-based DRM for part of the analysis only (see page on ‘How agent-based DRM can work together with other approaches’).

Other directionsAgent-based DRM

TCAS:

Climb!

Climb!

TCAS:

DESCEND!

DESCEND!

ATC:

BK 7886, I told

you to

Descend!

18

RESOURCE REQUIREMENTS FOR AGENT-BASED DRM

Data requirements The data requirements for agent-based DRM are for a part similar to those for other approaches. Logic tree-based approaches rely on identification of causes and consequences of events and conditions, and rely on probability estimates for such events and conditions, including estimates for events conditional on sequences of other events. Agent-based DRM also includes probabilities of failure of specific behaviour, but these are usually limited to a single agent only and they are independent of occurrences of other events. To a large extent, agent-based DRM uses physical parameters such as runway length, aircraft size, for which data is easier to obtain. In addition, during the bias and uncertainty assessment, any uncertain parameter values can be varied to learn their effect on the output. This provides insight into the operation, even if parameter values are unknown.

Recycling A great advantage of agent-based DRM, and of agent-based modeling in general, is that submodels built for safety risk analysis of other operations can be largely re-used. This is because for a new operation, the individual agents often do not change, even though their interactions with other agents may change. This is in contrast with logic tree-based approaches, which typically require a new structure and re-quantification of all parameter values. If submodels for agents and entities and their software implementations are collected in a database, this is a great resource for future projects. Therefore, investments made in agent-based DRM assessments actually pay off.

Expertise needed The operational expertise required for agent-based DRM is similar to that required for other approaches to assess ATM operations. Common requirements are access to multi-disciplinary knowledge regarding the technical systems, human operators, environmental influences and the interactions between these entities. However, there are also significant differences between approaches. In logic tree-based approaches, the collected information is analysed and subsequently synthesized into a manageable number of potential event sequences, and data is collected for the estimation of the various conditional event probabilities in the tree. For frequent events, real data often is available to estimate these probabilities. For less frequent events, however, typically interviews with controllers and pilots form the main source of data collection. Agent-based DRM requires expertise on capturing all background in an SDCPN model, and in running efficient rare event Monte Carlo simulation, including the use of dedicated acceleration techniques to capture rare events. There is no need to identify event sequences and the order in which the events may happen; these are an output of the analysis rather than an input. Rather, the approach allows developing the model in a compositional way, agent by agent. This requires access to knowledge on the physics and behaviour of the individual agents, and of the interactions between the agents of the operation, rather than of the conditional probabilities of events occurring.

Resources required The level of resources required, in terms of person hours and throughput time, highly depends on the level of recycling applicable. For a completely new operation, with innovative technical systems and procedures and changed roles and responsibilities for human operators, there may not be many re-usable submodels yet, and the resources required can be substantially more than for logic-tree based approaches (provided the logic tree-based approach has access to accurate success and failure data). If the operation to be assessed has elements in common with an operation already assessed using agent-based DRM, agent submodels and software can largely be re-used, and the resources required can be much smaller. If assessing the new operation is a matter of changing physical parameter values, the resources may be even much lower than with other approaches.

Agent-based Dynamic Risk Modelling for ATM 19

HOW AGENT-BASED DRM CAN WORK TOGETHER WITH OTHER APPROACHES Even though the development of a dynamic risk model may provide added value in any risk assessment exercise, provided that adequate analysis and modelling effort is allocated, it is often desirable to investigate what level of sophistication is necessary in the risk modelling. There are various means that may scale down the modelling effort. The effectiveness of each of these means may be dependent on the situation and conditions at hand, and the potential loss of quality/certainty of the results needs to be taken into account in the selection of the best means. −

−

−

−

−

−

Note that also within agent-based DRM, different levels of sophistication can be applied. In the development of the overall model, simple models could be used for the performance of technical systems and human operators rather than full-blown models with a lot of details. The development effort of simple models is smaller and they may be more transparent. There is a potential trade-off with the validity of the models, since they should still capture the basic dynamics and interactions between the entities. In the bias and uncertainty assessment the effect of this trade-off is evaluated with respect to the risk criticality.

Using agent-based DRM for specific conflict scenarios One can usually distinguish different conflict scenarios within one operation. For example, there may be different geographical points at which aircraft can meet (e.g. various runway entry points), or the circumstances at which they meet may be different (e.g. various visibility conditions), or there are scenarios with various levels of severity (e.g. radar separation infringement versus actual collision). In such case, it is possible to treat some of the conflict scenarios with agent-based DRM, and other conflict scenarios with other approaches. For example, agent-based DRM could be applied to only those conflict scenarios that contribute to high levels of uncertainty near the range of unacceptable risk. This decision could be based upon a preceding qualitative risk assessment cycle, using argumentation based approaches, statistical data, or logic trees.

Using agent-based DRM for conditional risks Often, the safety risk of an event can be written as the weighted sum of conditional risks. In a mathematical format, this is done, e.g., as follows: Distinguish a number, say N, of distinctive independent conditions, i.e. Condition-1 through Condition-N. Then the total risk for this event is equal to

Probability(Event given Condition-1) × Probability(Condition-1)

+ … + Probability(Event given Condition-N)

× Probability(Condition-N) In such case, agent-based DRM could be used to determine the probabilities of the event given the conditions, and another approach could be used to determine the probabilities of the conditions (or vice versa).

As an example of combining agent-based DRM with other approaches, one could:

Use agent-based DRM to evaluate conflicts between aircraft merging onto one route, and

Use other approaches to evaluate conflicts between aircraft on the same route

20

ROADMAP FOR AGENT-BASED DRM

Agent-based DRM has been developed over many years, and the approach has been applied in numerous ATM safety risk analysis applications over the past decade. Review comments from users and clients were used to further develop and shape the approach, and new methods and techniques were integrated in support of the methodology. The method and the results have been documented in technical reports, conference papers and journal articles, and training courses have been given on request. This White Paper aimed to have clarified what agent-based DRM is, why it is important, and when to use it. The experience with agent-based DRM applications demonstrates that the approach can already be applied during E-OCVM [4] phase V1, e.g., in order to identify unknown rare emergent behaviour (positive or negative) early on in the design (Scope). In such early phase a complete cycle of the safety risk assessment cycle can already be performed and the agent-based DRM based risk results and risk sensitivity results can provide effective and early feedback to the operational concept developers. In phases V2 (Feasibility) and V3 (Pre-industrial development and integration), in which the concept and related technical systems are developed in more detail, relevant parts of the agent-based model can be extended for the relevant aspects of the more detailed operational concept. The choice of the relevant aspects, requiring more detailed study, is driven by the preceding analysis in phase V1, complemented by input from the industry. In particular, aspects that have been identified as being potentially critical for attaining the required level of safety by the V1 analysis, can be studied in more detail by extended models that focus on the agent interactions for the potentially safety-critical aspects. Such agent-based modelling in phases V2 and V3 includes the advantages with respect to transparency, quantification of parameters, explanation of risk results and feedback to design. Given the more detailed concept in phases V2 and V3, the advantages of the agent-based DRM approach may well be further emphasised with respect to phase V1, because the agent-based DRM approach is closer to the physical reality of the operation considered in comparison with the more conceptual failure and error based focus in traditional event sequence based approaches. In phases V4 (Industrialisation), V5 (Deployment), V6 (Operations), more and more details about the design and operation become available, allowing to further refine and detail the agent-based models. The most obvious way forward for ATM safety is to define critical areas and scenarios wherein agent-based DRM should be applied, and to feed integrate such results into safety cases. This may require a training programme for assessors, and/or effort to develop key sub-models to be used in multiple agent-based DRM assessments. The resulting expanding library of agent-based dynamic risk models for various applications, together with the library of corresponding Monte Carlo simulation toolsets, will form a valuable resource in support of an increasing number of low-cost agent-based DRM applications. Within the SESAR Programme, agent-based DRM is currently being evaluated, and dedicated guidelines are being developed for its application [10]. This opens the door for application of agent-based DRM to key test cases in SESAR where relevant.

Agent-based Dynamic Risk Modelling for ATM 21

REFERENCES

General references

[1] Clark A, Chalmers D (1998). The extended mind. In: Analysis. Vol 58, pp 7-19.

[2] Cuevas G, Echegoyen I, García JG, Cásek P, Keinrath C, Weber R, Gotthard P, Bussink F, Luuk A (2010). Autonomous Aircraft Advanced (A3) ConOps, iFly report D1.3.

[3] De Jong HH, Tump RS, Blom HAP, Van Doorn BA, Karwal AK, Bloem EA (2001). Qualitative safety risk assessment of a RIASS based operation at Schiphol airport – crossing of departures on 01L/19R under good visibility conditions, NLR report CR-2001-157.

[4] E-OCVM (2010), European Operational Concept Validation Methodology, Version 3.0, Volume I, and Volume II - Annexes, Eurocontrol.

[5] FAA/EUROCONTROL (2007). ATM Safety Techniques and Toolbox, Safety Action Plan-15, Version 2.0, http://www.eurocontrol.int/eec/gallery/content/public/documents /EEC_safety_documents/Safety_Techniques_and_Toolbox_2.0.pdf

[6] Foyle DC, Hooey BL (Eds.) (2008). Human performance modeling in aviation, Boca Raton, (FL), USA, CRC Press.

[7] Hollnagel E (1993). Human reliability analysis, context and control. London, UK: Academic Press.

[8] Petri CA (1962). Kommunikation mit Automaten. PhD thesis, Institut für instrumentelle Mathematik, Bonn, Germany. Schriften des IIM, number 2.

[9] Reason J (1990). Human Error, Cambridge University Press.

[10] Romero J, Cano JJ, Almendros P, Rossello S, Krakenes T, Everdij M, Stroeve S (2012). Initial guidelines for Dynamic Risk Modelling (DRM) application, SESAR Project 16.01.03 Develop techniques for Dynamic Risk Modelling.

[11] Scholte JJ, Blom HAP, Pasquini A (2010). Study of SESAR implied safety validation needs, Proc. 4th Int. Conf. on Research in Air Transportation (ICRAT), Budapest, Hungary, June 1-4.

[12] Wickens CD, Hollands JG (2000). Engineering psychology and human performance. Upper Saddle River (NJ), USA: Prentice Hall.

[13] Wikipedia, http://en.wikipedia.org/wiki/Agent-based_model, accessed on 8th October 2013.

[14] Xu X, Ulrey ML, Brown JA, Mast J, Lapis MB (2013). Safety sufficiency for NextGen – Assessment of selected existing safety methods, tools, processes, and regulations, Boeing Research & Technology, Seattle, Washington, NASA/CR-2013-217801.

[15] Zio E (2009). Reliability engineering: Old problems and new challenges. In: Reliability Engineering & System Safety. Vol 94, pp 125-41.

References on agent-based DRM method development

[16] Blom HAP, Bakker GJ, Blanker PJG, Daams J, Everdij MHC, Klompstra MB (2001). Accident risk assessment for advanced air traffic management. In: Donohue GL, Zellweger AG, editors. Air Transport Systems Engineering: AIAA. pp 463-480.

[17] Blom HAP, Daams J, Nijhuis HB (2001). Human cognition modelling in ATM safety assessment. In: Donohue GL, Zellweger AG, editors. Air Transport Systems Engineering: AIAA. pp 481-511.

[18] Blom HAP, Stroeve SH, De Jong HH (2006). Safety risk assessment by Monte Carlo simulation of complex safety critical operations. In: Redmill F, Anderson T, editors. Developments in Risk-based Approaches to Safety, Proc. 14th Safety-critical Systems Symposium, Bristol, UK, 7-9 February 2006: Springer.

22

[19] Blom HAP, Krystul J, Bakker GJ, Klompstra MB, Klein Obbink B (2007). Free flight collision risk estimation by sequential Monte Carlo simulation, In: Cassandras CG and Lygeros J, editors, Stochastic hybrid systems, Taylor & Francis/CRC Press, pp 249-281.

[20] Blom HAP, Bakker GJ, Krystul J (2009). Rare event estimation for a large scale stochastic hybrid system with air traffic application, In: Rubino G and Tuffin B, editors, Rare event simulation using Monte Carlo methods, J.Wiley, pp 193-214.

[21] Blom HAP, Stroeve SH, Bosse T (2013). Modelling of potential hazards in agent-based safety risk analysis, Proc. 10th USA/Europe Air Traffic Management R&D Seminar (ATM2013), June 10-13, 2013, Chicago, Illinois, USA.

[22] Bosse T, Both F, Van Lambalgen R, Treur J (2008). An Agent Model for a Human's Functional State and Performance. Proc. 8th IEEE/WIC/ ACM Int. Conf. on Intelligent Agent Technology (IAT'08).

[23] Bosse T, Duell R, Memon ZA, Treur J, Van der Wal CN (2009). A Multi-Agent Model for Emotion Contagion Spirals Integrated within a Supporting Ambient Agent Model. In: Yang JJ, Yokoo M, Ito T, Jin Z, Scerri P (eds.), Proc. 12th Int. Conf. on Principles of Practice in Multi-Agent Systems, PRIMA'09. Lecture Notes in Artificial Intelligence, vol 5925. Springer Verlag, pp 48–67.

[24] Bosse T, Treur J, Stroeve S, Blom H, Sharpanskykh A (2013). Formal specification of model constructs, Mathematical Approach towards Resilience Engineering in ATM (MAREA), D2.4, Edition 00.01.00, February 2013.

[25] Bosse T, Blom HAP, Stroeve SH, Sharpanskykh A (2013). An Integrated Multi-Agent Model for Modelling Hazards within Air Traffic Management. Proc. 2013 IEEE/WIC/ACM Int. Conf. on Intelligent Agent Technology, Atlanta, Georgia, USA, 17-20 November 2013.

[26] Corker KM, Blom HAP, Stroeve SH (2005), Study on the integration of human performance and accident risk assessment models: Air-MIDAS and TOPAZ, Proc. Int. Seminar on Aviation Psychology, Oklahoma, USA, 18-21, April 2005.

[27] Everdij MHC, Blom HAP (2003). Petri nets and hybrid state Markov processes in a power-hierarchy of dependability models. Proc. IFAC Conference on Analysis and Design of Hybrid Systems. Saint-Malo Brittany, France, 2003. pp 355-360.

[28] Everdij MHC, Blom HAP (2006). Hybrid Petri Nets with diffusion that have into-mappings with generalised stochastic hybrid processes. In: Blom HAP, Lygeros J, editors. Stochastic Hybrid Systems: Theory and Safety Critical Applications. Berlin, Germany: Springer, pp 31-63.

[29] Everdij MHC, Klompstra MB, Blom HAP, Klein Obbink B (2006). Compositional specification of a multi-agent system by stochastically and dynamically coloured Petri nets. In: Blom HAP, Lygeros J, editors. Stochastic Hybrid Systems: Theory and Safety Critical Applications. Berlin, Germany: Springer, pp 325-350.

[30] Everdij MHC, Blom HAP, Stroeve SH (2006), Structured assessment of bias and uncertainty in Monte Carlo simulated accident risk, Proc. 8th Int. Conf. on Probabilistic Safety Assessment and Management (PSAM8), May 2006, New Orleans, USA.

[31] Everdij MHC, Blom HAP (2010). Hybrid state Petri nets which have the analysis power of stochastic hybrid systems and the formal verification power of automata. In: Pawlewski P, editor. Petri Nets. Vienna, Austria: I-Tech Education and Publishing, pp 227-252.

[32] Stroeve SH, Blom HAP, Bakker GJ (2003). Multi-agent situation awareness error evolution in accident risk modelling. Proc. 5th USA/Europe Air Traffic Management R&D Seminar. Budapest, Hungary, 2003.

[33] Stroeve SH, Bosse T, Blom HAP, Sharpanskykh A, Everdij, MHC (2013). Agent-based modelling for analysis of resilience in ATM, Proc. SESAR Innovation Days.

Agent-based Dynamic Risk Modelling for ATM 23

References on agent-based DRM applications

[34] Blom HAP, Stroeve SH, Everdij MHC, Van der Park MNJ (2003). Human cognition performance model to evaluate safe spacing in air traffic, Human Factors and Aerospace Safety, Vol 3, pp 59-82.

[35] Blom HAP, Klompstra MB, Bakker GJ (2003). Accident risk assessment of simultaneous converging instrument approaches, Air Traffic Control Quarterly, Vol 11, pp 123-155.

[36] Blom HAP, Klein Obbink B, Bakker GJ (2009), Simulated safety risk of an uncoordinated airborne self separation concept of operation, ATC Quarterly, Vol 17, pp 63-93.

[37] Blom HAP, Bakker GJ (2012). Can airborne self separation safely accommodate very high en-route traffic demand?, Proc. AIAA ATIO conference, 17-19 September 2012, Indianapolis, Indiana, USA.

[38] Blom HAP, Bakker GJ (2013). In search of positive emergent behaviour in trajectory based operations, Proc. 3rd SESAR Innovation Days, KTH, Stockholm, 26-28 November 2013.

[39] De Oliveira IR, Vismari LF, Cugnasca PS, Camargo Jr. JB, Bakker GJ, Blom HAP (2010). A case study of advanced airborne technology impacting air traffic management, Eds: Li Weigang et al., Computational models, software engineering and advanced technologies in air transportation, Engineering Science Reference, Hershey, pp 177-214.

[40] Everdij MHC, Blom HAP, Bakker GJ (2007). Modelling lateral spacing and separation for airborne separation assurance using Petri nets. In: Simulation: Transactions of the Society for Modelling and Simulation International, Vol 83, pp 401-414.

[41] Everdij MHC, Blom HAP, Bakker GJ, Zmarrou H (2012). Agent-Based Safety Risk Analysis of Time Based Operation in Future TMA, Proc. 3rd Air Transport and Operations Seminar (ATOS2012), Delft, The Netherlands, 18-20 June 2012.

[42] Itoh E, Everdij MHC, Bakker GJ, Blom HAP (2012). Effects of surveillance failure on airborne-based continuous descent approach, Proc. IMechE, Vol 226, Part G: J. Aerospace Engineering, November 2012, pp 1470-1480.

[43] Itoh E, Uejima K (2013). Applying Flight-deck Interval Management based Continuous Descent Operation for Arrival Air Traffic to Tokyo International Airport. Proc. 10th USA/Europe Air Traffic Management R&D Seminar (ATM2013), June 10-13, 2013, Chicago, Illinois, USA.

[44] Netjasov F, Vidosavljevic A, Tosic V, Everdij MHC, Blom HAP (2013). Development, validation and application of Stochastically and Dynamically Coloured Petri Net model of ACAS operations for safety assessment purposes, Transportation Research Part C, Vol 33, pp 167-195.

[45] Scholte JJ, Blom HAP, Van den Bos JC, Jansen RBHJ (2009). Management of ATM performance in operational concept development and validation: a case study, Proc. 8th USA/Europe Air Traffic Management R&D Seminar (ATM 2009), Napa, CA, 29 June-2 July.

[46] Stroeve SH, Blom HAP, Bakker GJ (2008). Systemic accident risk assessment in air traffic by Monte Carlo simulation, Safety Science, Vol 46, pp 238-249.

[47] Stroeve SH, Van Doorn BA, Bakker GJ (2011). Safety assessment of a future taxi into position and hold operation by agent-based dynamic risk modelling. Journal of Aerospace Operations, Vol 1, pp 107-127.

[48] Stroeve SH, Blom HAP, Bakker GJ (2013). Contrasting safety assessments of a runway incursion scenario: event sequence analysis versus multi-agent dynamic risk modelling, Reliability Engineering and System Safety, Vol 109, pp 133-149.

24

FURTHER INFORMATION

For further information on agent-based Dynamic Risk Modelling you are welcome to contact any of the authors.

ACKNOWLEDGEMENTS This White Paper is the result of a team effort, principally from the NLR team (including first three authors below) and the EUROCONTROL Project Officer.

AUTHORS

Mariken Everdij (NLR) everdij@nlr.nl

Henk Blom (NLR) blom@nlr.nl

Sybert Stroeve (NLR) stroeve@nlr.nl

Barry Kirwan (Eurocontrol) barry.kirwan@eurocontrol.int

© European Organisation for the Safety of Air Navigation

EUROCONTROL January 2014

The document is published by EUROCONTROL in the interests of exchange of

information. It may be copied in whole or in part, providing that EUROCONTROL is

acknowledged as a source. The information contained in this document may not be modified without prior written permission from EUROCONTROL.