airborne validation results part a · emma2 airborne validation results part a save date:...

EUROPEAN AIRPORT MOVEMENT MANAGEMENT BY A-SMGCS, Part 2 Contract No. TREN/04/FP6AE/S07.45797/513522

Project Funded by European Commission, DG TREN The Sixth Framework Programme Strengthening the competitiveness

Contract No. TREN/04/FP6AE/S07.45797/513522

Project Manager M. Röder

Deutsches Zentrum für Luft und Raumfahrt Lilienthalplatz 7, D-38108 Braunschweig, Germany

Phone: +49 (0) 531 295 3026, Fax: +49 (0) 531 295 2180 email: [email protected]

Web page: http://www.dlr.de/emma2

© 2010, - All rights reserved - EMMA Project Partners The reproduction, distribution and utilization of this document as well as the communication of its contents to other without explicit authorization is prohibited. This document and the information contained herein is the property of Deutsches Zentrum für Luft- und Raumfahrt and the EMMA project partners. Offenders will be held liable for the payment of damages. All rights reserved in the event of the grant of a patent, utility model or design. The results and findings described in this document have been elaborated under a contract awarded by the European Commission.

Airborne Validation Results Part A

Thomas Wittig

FAV

Document No: 2-D6.6.1a Version No. 1.00

Classification: Public Number of pages: 142

EMMA2


Save date: 2010-02-08 Public File Name: 2-D661a_VO-TR_V1.0.doc Version: 1.00

Distribution List

Member Type No. Name POC Distributed1

Internet http://www.dlr.de/emma2 Web

Intranet https://extsites.dlr.de/fl/emma 1 DLR Joern Jakobi 2 AENA Guilio Maira 3 AIF Marianne Moller 4 SELEX Giuliano D'Auria 5 ANS_CR Jan Kubicek 8 DSNA Philippe Montebello 9 ENAV Antonio Nuzzo 10 NLR Jürgen Teutsch 11 PAS Alan Gilbert 12 TATM Corinne Heinrich 13 THAV Marc Fabreguettes 15 AUEB Konstantinos G. Zografos 16 CSL – Prague Airport Libor Kurzweil 17 DAS Joachim Bader 18 DFS Klaus-Ruediger Täglich 19 EEC Stéphane Dubuisson 20 ERA Jan Hrabanek 21 ETG (FAV) Thomas Wittig 23 SICTA Mariacarmela Supino 24 TUD Carole Urvoy

Contractor

25 SOF Lionel Bernard-Peyre CSA Karel Muendel

Sub-Contractor Körte Max Körte

Customer EC Doris Schroecker

Additional EUROCONTROL Bengt Collin

1 Please insert an X, when the PoC of a company receives this document. Do not use the date of issue!

EMMA2



Document Control Sheet 2-SP6 Project Manager Marcus Biella Responsible Author Thomas Wittig FAV

Marcus Biella DLR Jörn Jakobi DLR Thomas Ludwig DLR Carsten Wehrstedt DLR Christian Drege TUD Carole Urvoy TUD

Additional Authors

Ronny Friebel FAV Subject / Title of Document: Airborne Validation Results Part A Related Task('s): 2-WP6.3 Deliverable No. 2-D6.6.1a Save Date of File: 2010-02-08 Document Version: 1.00 Reference / File Name 2-D661a_VO-TR_V1.0.doc Number of Pages 142 Dissemination Level Public Target Date 2008-12-19

Change Control List (Change Log)

Date Release Changed Items/Chapters Comment 2008-07-24 0.01 Structure by DLR

Initial draft: Results for RTS Pt.1 in GECO (Validation of GTD and TAXI-CPDLC) by DLR

2008-09-24 0.02 Addition of pilot comments

2008-11-03 0.03 Results for RTS Pt.2 in GECO (Validation of GTD and TAXI-CPDLC) by DLR

2009-01-20 0.04 Update of DLR and TUD results 2010-02-08 1.00 No EC comments. Approved Final Version 1.0

EMMA2



Table of Contents Distribution List ...................................................................................................................................... 2 Document Control Sheet ......................................................................................................................... 3 Change Control List (Change Log) ......................................................................................................... 3 Table of Contents .................................................................................................................................... 4 1 Executive Summary ............................................................................................................................. 5 2 Introduction .......................................................................................................................................... 7

2.1 Scope of Document ....................................................................................................................... 7 2.2 Validation Approach ..................................................................................................................... 8 2.3 Schedule of 2-WP6.6 Activities .................................................................................................... 8

3 Ground Traffic Display & TAXI-CPDLC Trials (DLR) ..................................................................... 9 3.1 Real Time Simulation Results ....................................................................................................... 9

3.1.1 Set Up..................................................................................................................................... 9 3.1.2 Results to Operational Feasibility (RTS) ............................................................................. 33 3.1.3 Results to Operational Improvements (RTS) ....................................................................... 51

3.2 Operational Flight Trials Results................................................................................................. 60 3.2.1 Set Up................................................................................................................................... 60 3.2.2 Results to Technical Feasibility (OST) ................................................................................ 62

4 Surface Movement Awareness and Alerting System SMAAS; Traffic Conflict Detection; Ground-Air Data Base Upload (TUD)................................................................................................................ 65

4.1 Real Time Simulation Results ..................................................................................................... 65 4.1.1 Set Up................................................................................................................................... 65 4.1.2 Results to Operational Feasibility (RTS) ............................................................................. 80 4.1.3 Results to Operational Improvements (RTS) ....................................................................... 89

4.2 Operational Flight Trials Results............................................................................................... 105 4.2.1 Set Up................................................................................................................................. 105 4.2.2 Technical and Operational approval (FAV / TUD trials)................................................... 121 4.2.3 Results to Technical Feasibility (FAV Trials).................................................................... 125 4.2.4 Results to Operational Feasibility (OST) ........................................................................... 127

5 Conclusions ...................................................................................................................................... 128 Conclusions to the new A-SMGCS services ................................................................................... 128 Conclusions to Operational Improvements ..................................................................................... 131

6 Annex I............................................................................................................................................. 133 6.1 References ................................................................................................................................. 133 6.2 Abbreviations ............................................................................................................................ 134 6.3 List of Figures ........................................................................................................................... 136 6.4 List of Tables............................................................................................................................. 138

7 Annex II............................................................................................................................................ 139 7.1 Raw Data (DLR RTS) ............................................................................................................... 140

7.1.1 QE-OF ................................................................................................................................ 140 7.1.2 QE-OI (Safety) ................................................................................................................... 141 7.1.3 QE-OI (Capacity/Efficiency).............................................................................................. 142 7.1.4 SUS..................................................................................................................................... 142 7.1.5 ASSA.................................................................................................................................. 142

EMMA2



1 Executive Summary This document reports the results of the A-SMGCS airborne validation activities carried out in EMMA2 WP6.6 by the green team (DLR, TUD, FAV, and DAS). There were four test campaigns:

• Real time simulations (RTS) o at DLR facilities (in June and October 2008) o at TUD facilities (in July 2008)

• Prague Airport on-site trials (OST) o by TUD & FAV (in August 2008) o by DLR & TUD (in November 2008)

During the airborne validation trials the following EMMA2 higher A-SMGCS components were implemented in different platforms and under evaluation.

• Ground Traffic Display (GTD) o RTS (DLR platform, TUD platform) o OST (DLR platform, TUD platform)

• Ground Air Data Base Upload o RTS (TUD platform)

• Surface Movement Alerting (SMA) o RTS (TUD platform) o OST (TUD platform)

• Traffic Conflict Detection (TCD) o RTS (TUD platform) o OST (fed by TIS-B) (TUD platform)

• TAXI-CPDLC o RTS (DLR platform) o OST (DLR platform)

They were operated in different trials in total by twenty different pilots. Additionally, a tower simulation with ANS_CR controllers was linked to the DLR cockpit simulator during the DLR trials, and at the on-site trials, the DLR test aircraft ATTAS enabled a comprehensive and realistic testing of the TAXI-CPDLC service carried out in the test bed room of the Prague Tower in shadow mode. Generally speaking, the overall system provided a medium maturity and could fulfil the high operational expectations. The pilots approved its operational feasibility by accepting most of the operational requirements and new procedures. Operational improvements were measured. Subjectively the pilots rated significantly that

- GTD is regarded as potential means for increased safety which eases pilots work considerably. With its indication of the surrounding traffic the GTD is an additional information source for the pilots to navigate and to manage the aircraft speed more safely and efficiently.

- GTD significantly increases the pilots’ situation awareness (by allowing an intuitive use, providing the appropriate amount of information, and as an extraordinary aid in locating traffic) especially under low visibility conditions.

- Pilots’ workload is not affected negatively when using the GTD. - Furthermore, pilots report that the use of a GTD will improve their performance. - TAXI-CPDLC is regarded as potential means for increased safety which will ease pilots

work considerably with appropriate input devices. The graphical presentation of the cleared

EMMA2



taxi route on the EMM (compared to textual clearances only) will enable pilots to follow an assigned taxi route more safely and efficiently.

- An increase of workload while using the TAXI-CPDLC service could be observed and was explained by the pilots due to the fact that a novel and not-well designed input device was used.

- Situation Awareness was not affected by the use of TAXI-CPDLC. - The Ground to Air Database Upload function is regarded as potential means for increased

safety which will ease pilots work especially in large airports. - The Ground to Air Database Upload function significantly increases the pilots’ situation

awareness by enabling the loading of and the interaction with electronic Preflight Information Bulletin Data (e.g. NOTAM information).

- Pilots’ workload is not affected negatively when using the Ground to Air Database Upload function.

- The SMA and the TCD are regarded as potential means for increased safety as they will diminish the risks of accidents and incidents on the surface movement area especially in low visibility conditions.

EMMA2



2 Introduction

2.1 Scope of Document This document is positioned within the framework of activities for the ‘European airport Movement Management by A-SMGCS, part 2 (EMMA2)’ project. Based on the technical annex, the sub-project 2-SP6 deals with the validation A-SMGCS services implemented at four ground sites (Paris Charles de Gaulle, Prague-Ruzynĕ, Toulouse-Blagnac, and Milano-Malpensa) and the airborne site. Specific validation test plans and test reports were developed at each site of the project, presenting the site-specific validation process and strategy and their respective results. This test report bases on 2-D6.1.6, “Validation Test Plan Airborne” [6] and follows the E-OCVM guidelines [1]. In accordance to the EMMA2 High Level Objectives, which were identified in the 2-D6.1.1a “Validation Plan” [5], it covers following aspects.

1. Operational Feasibility 2. Operational Improvements 3. Additional Results 4. Conclusions.

This document reports the results of the airborne validation activities carried out in WP6.6 by the green team (DLR, TUD, FAV, and DAS). It comprises results from real time simulations (RTS) and on-site trials (OST) at Airport Prague with Pilots from DLH, CSA, and Germania (and ATCOs from ANS CR), by support of the industry DAS, FAV, Airtel ATN, PAS and the R&D enterprises EEC, TUD and DLR. There were two DLR test campaigns: RTS in June and October 2008 at DLR premises, and the on-site trials at Airport Prague in November 2008. RTS1 focussed exclusively on the technical and operational feasibility of the new A-SMGCS services. Results were mainly used to improve the services for the on-site trials. In RTS the operational feedback of the pilots was compiled and is documented in this report. In DLR RTS a Tower simulation environment was linked with cockpit simulator with commercial pilots in order to get more realistic results to the interaction between ATCOs and pilots supported by new A-SMGCS services like TAXI-CPDLC and on-board ground traffic displays (GTD). Results compiled by the tower simulation (incl. feedback of the pilots) can be found in a separate EMMA2 document (2-D6.3.1, [13]); the present document focuses on the pilot perspective exclusively. There were two TUD test campaigns: RTS in July 2008 at TUD premises, and the on-site trials at Airport Prague in August 2008 with FAV and November 2008. Both campaigns focussed on the technical and operational feasibility of the SMA, Ground Air Data Base upload and TCD. On-site trials were used to prove technical and operational feasibility aspects in real life that could not be assessed in a simulated environment. The details to the planning of those test campaigns can be found in the EMMA2 SP6 “Validation Test Plan Airborne” (2-D6.1.6) [6] and “Validation Test Plan – Prague” (2-D6.1.3) [15]. This document describes the results achieved and derived conclusions but also repeats some core aspects of the test plan [6] to make the document readable as a stand alone document. The document delivers important input to the overall EMMA2 2-D6.7.1 “Validation Comparative Analysis Report” [14], which will provide an overall analysis of all EMMA2 validation results.

EMMA2



2.2 Validation Approach During the proposal phase of EMMA2, it was decided to use the ‘Master European Validation Plan (MAEVA)’ project approach to validation as the basis for EMMA2 validation activities. With the start of EMMA2 it was decided to use E-OCVM. The E-OCVM constitutes a widely accepted Validation Methodology which has been based on the experience gained in ATM R&D activities within the framework of EC and EUROCONTROL funded projects [1]. In EMMA2 the E-OCVM approach has been applied and adapted to the EMMA2 needs. The EMMA2 “Validation Plan” [5] and the “Generic Experimental Test Plan” [4] are the outcome documents of this process. They are the valid generic validation plans for all EMMA2 validation activities (except the validation activities by THAV and AIF, the so-call “blue team”). Based on those generic EMMA2 validation documents the specific “Validation Test Plan Airborne” [6] has been produced to plan all validation activities carried out in SP6 WP6.6 by the “green team” and THAV of the “blue team”.

2.3 Schedule of 2-WP6.6 Activities 2-WP6.6 of EMMA2 focuses on airborne A-SMGCS validation activities, implemented by the green team (DLR, TUD, FAV, and DAS) in the sub-project 2-SP2 of EMMA2. Here, two validation platforms will be used:

• Real-time simulation platform, to simulate safety-critical events, validate A-SMGCS procedures, and measure operational improvements in a realistic environment

• On-site trials with a test aircraft or test vehicle, to validate requirements and procedures in the real operational environment and show some potential operational improvements.

The series of tests started with real-time simulations at DLR-Braunschweig’s Cockpit Simulator, concentrating on Ground Traffic Display (GTD), and TAXI CPDLC. Here the whole system was to be validated, by measuring operational feasibility and operational improvements (safety and human factors issues in the first place). These real-time simulations were a preparatory step for the flight trials with the ATTAS at Prague Airport, which are aimed at validating the higher A-SMGCS services under real operational conditions. It should be borne in mind that even though simulation re-creates a realistic environment, and permits the safe testing of safety-critical events and repetitive testing of rare events, at the end operational trials must be performed to prove the operational feasibility of new services and procedures.

EMMA2



3 Ground Traffic Display & TAXI-CPDLC Trials (DLR)

3.1 Real Time Simulation Results

3.1.1 Set Up

3.1.1.1 The validation platform DLR’s Generic Experimental Cockpit (GECO) is designed as a fixed-base flight simulator based on the Airbus A 320 architecture using the flight dynamics of the VFW614 (in accordance to DLR’s test aircraft ATTAS) in a fly by wire version (Figure 3-1). It is capable for testing and evaluating new on-board systems giving a maximum of flexibility.

Figure 3-1: DLR’s Generic Experimental Cockpit (GECO)

The simulator is equipped with a collimated outside view and widespread hardware equipment (Figure 3-2):

• Two seated-flightdeck including Primary Flight Display (PFD) und Navigation Display (NAV) (providing HSI during in flight and EMM during taxiing) at both sides for CM1 and CM2; and Engine Display (ENG)

• Several components of the Flight Management System (FMS; with RAD/NAV page in use during EMMA2 trials) including Flight Control Unit (FCU) and Multipurpose Control and Display Unit (MCDU)

• HMI-Interfaces (e.g. trackball, touchpad, touchscreen) • Head-Up Display (HUD) • High-Resolution Airport Environment

EMMA2



Figure 3-2: PFD, NAV, ENG and FCU in GECO

The GECO is part of an integrated network; it can be used in co-operation with other simulation platforms e.g. Apron & Tower Simulator (ATS) or Air Traffic Management and Operations Simulator (ATMOS). Further equipment can be connected via ISDN or internet. The communication between the GECO and ATS is realised via the TCP/IP protocol. Experimental conditions and weather parameters will be coordinated and controlled by the instructor. The architecture between the cockpit and the tower simulator is shown in Figure 3-3.

EMMA2 Airborne Validation Results Part A


Figure 3-3: Architecture of Cockpit & Tower Simulation, Pseudo Pilots and DMAN

EMMA2



3.1.1.2 Experimental Design The experimental design is based on the use of real experiments. In real experiments, the same scenarios are used for the Baseline System and the A-SMGCS set-up in order to achieve ceteris paribus conditions. In this way, results from the Baseline system can be directly compared with the A-SMGCS within a traffic scenario. For “TAXI-CPDLC” different scenarios will be used. Therefore no direct experimental comparison can be made. Nevertheless comparisons will be drawn and considered as well and the results are used

• to check the operational feasibility initially since the maturity level was not to be expected to be V3 (in accordance to E-OCVM [1], and

• to give first hints (!) for operational improvements.

The independent variable (IV) for on-board activities is therefore defined as follows: [1] IVSYS the system version (EMM vs. higher A-SMGCS services) with four levels of the treatment “A-SMGCS Services”:

Level 1 “Baseline” (EMM), Level 2 “GTD”, (EMM + GTD) [Level 3 “TAXI-CPDLC standard” (EMM + GTD + standard TAXI-CPDLC)] [Level 4 “TAXI-CPDLC advanced” (EMM + GTD + advanced TAXI-CPDLC)] The independent variable is a treatment factor that is operated by the experimenter and is supposed to cause expected effects. It is described and argued in the table 3-1. The second independent variable is the role of the pilot in the cockpit. He will act in half of the scenarios as CM1; these scenarios are later repeated to have more statistical power, and then he will act as CM2 (and vice versa for the other half of the scenarios). [2] IVROLE subjects role with two levels (CM1, resp. CM2). Independent Variables (IV)

Description Levels

IVSYS System Version= Technical System varied by different functions and their provided services

This variable has four different technical system versions corresponding to an A-SMGCS (= EMM) and A-SMGCS higher services:

Level 1 EMM (baseline) Level 2 EMM + GTD Level 3 EMM + GTD + standard TAXI-CPDLC Level 4 EMM + GTD + advanced TAXI-CPDLC

IVROLE Role of Subject This variable has two different levels. Pilots will act as: • CM1 • CM2

Table 3-1: Independent variable and levels in RTS

3.1.1.3 TAXI-CPDLC Procedures

3.1.1.3.1 Standard TAXI-CPDLC: Procedures Outbound Given below is a TAXI-CPDLC example for an outbound flight for RWY06.

EMMA2



• The aircraft is parked at apron north (Gate 34)

• RWY06 has been assigned for departure

• The scheduled taxi route will be H F E, with crossing the RWY 13/31 in between

• A-SMGCS provides expected route information by TAXI-CPDLC on pilot’s request, after the departure clearance has been issued

• CDD issues start up-clearance via TAXI-CPDLC

• CDD transfers the strip to GEC, and the system transmits the CONTACT message by TAXI-CPDLC

• PNF contacts GEC by voice

• GEC issues pushback clearance by TAXI-CPDLC

• GEC issues taxi clearance up to RWY 13/31 by TAXI-CPDLC

• GEC transfers the strip to TEC, and the system transmits the CONTACT message by TAXI-CPDLC.

• PNF contacts TEC by voice

• TEC issues crossing RWY 13/31 clearance, taxi to RWY 06 clearance, line-up and take-off clearance by voice

Table 3-2 shows the communication between ATCO and PNF in detail. (Both “GECO Staff” columns can be neglected here as they refer only to the scenarios which will be used in RTS2.)

DM Pilot non flying GECO staff UM ATC GECO staff

Taxi Route Information: Aircraft Status: Parking at Apron North,

Stand 34, Departure Clearance received, before Start-up

A-SMGCS: TAXI-CPDLC

132 REQUEST TAXI ROUTING INFORMATION

243 EXPECT ROUTING TO HOLDINGPOINT E RWY 06 VIA TWY H F E

Send expected Routing to GECO

3 ROGER

Start-Up: Aircraft Status: Ready for Start-up

CDD:

25 REQUEST START-UP CLEARANCE

239 START-UP APPROVED

0 WILCO

EMMA2




Handover to GEC: Aircraft Status: Performing or having

performed Start-up

CDD:

117 CONTACT LKPR GROUND 121.900

0 WILCO

Push-Back Aircraft Status: Initial call to GEC made by

voice, ready for pushback

GEC:

25 REQUEST PUSHBACK CLEARANCE

239 PUSHBACK APPROVED

0 WILCO

Taxi Clearance: Aircraft Status: Pushback finished,

ready for taxiing

GEC: Note: The Taxi Clearance is a multi-

element clearance transmitted in one go.

25 REQUEST TAXI CLEARANCE

Send Request to ATS

241

TAXI TO HOLDINGPOINT E RWY 06 VIA TWY H F

271 HOLD SHORT OF RWY 13

274 NEXT EXPECT VIA TWY F E

0 WILCO

Handover to TEC: Aircraft Status: Reaching RWY 13/31

GEC:

117 CONTACT LKPR TOWER 118.100

0 WILCO

Crossing of Runway 13/31, Taxiing to Runway 06, Line-up and Take-off: Aircraft Status: Initial call to TEC made by

voice, …

TEC: voice

Request crossing RWY 13/31

Cleared to cross RWY 13/31 and proceed to holding point RWY 06 via Foxtrott and Echo

Cleared to cross RWY 13/31 and

EMMA2



DM Pilot non flying GECO staff UM ATC GECO staff proceed to holding point RWY06 via Foxtrott and Echo

Ready for Departure on Runway 06

Line-Up Runway 06

Lining-Up Runway 06

Cleared for Take-Off Runway 06

Cleared for take-Off Runway 06

Handover to DEP: (optionally) Aircraft Status: Climb-out

A-SMGCS: voice

CONTACT DEPARTURE 120.525


Table 3-2: TAXI-CPDLC standard procedures for an outbound flight

3.1.1.3.2 Standard TAXI-CPDLC: Procedures Inbound • The aircraft approaches Prague Airport on RWY 06

• The assigned stand is on apron north (Gate 34)

• The scheduled taxi route will be L G

• A-SMGCS provides expected route information by TAXI-CPDLC before final approach.

• TEC issues landing clearance by voice.

• TEC issues handover to GEC by voice.

• GEC issues taxi clearance by TAXI-CPDLC

Table 3-3 shows the communication between ATCO and PNF in detail.

EMMA2




Taxi Route Information: Aircraft Status: Before Final Approach for RWY

06

A-SMGCS:


243 EXPECT ROUTING TO STAND 34 VIA TWY L G


3 ROGER

Landing clearance: Aircraft Status: Approaching RWY 24

TEC: voice

TOWER, ILS RWY 06 established, request landing clearance

Wind 240 00

knots, cleared to land RWY 06

Cleared to land RWY 06

Handover to GEC: Aircraft Status: After landing on RWY 06

TEC: voice

Contact LKPR Ground on 121,900

Contact Ground on 121,900

Taxi Clearance: Aircraft Status: Initial call to GEC made by

voice.

GEC: Note: GEC will issue taxi

clearance without pilot’s request.

GROUND, on your frequency

Follow data link

241

TAXI TO STAND 34 VIA TAXIWAY L G

0 WILCO

Leaving Frequency (optionally via data link): Aircraft Status: Parked on stand.

GEC:

136 ON-BLOCKED

No acknowledgement by GEC via TAXI-CPDLC required

EMMA2



Table 3-3: TAXI-CPDLC standard procedures for an inbound flight

3.1.1.3.3 Advanced TAXI-CPDLC: Procedures Outbound • The aircraft is parked at Apron North (Gate 34).

• Runway 24 has been assigned for departure

• The scheduled taxi route will be H Z

• A-SMGCS provides expected route information by TAXI-CPDLC on pilot’s request, after the departure clearance has been issued

• CDD issues start up-clearance via TAXI-CPDLC

• CDD transfers the strip to GEC, and the system transmits the CONTACT message by TAXI-CPDLC.

• PNF contacts GEC by voice.

• GEC issues pushback clearance by TAXI-CPDLC

• GEC decides for a different taxi route via H and A and selects this route at the routing function. GEC issues revised taxi clearance by TAXI-CPDLC.

• GEC transfers the strip to TEC, and the system transmits the CONTACT message by TAXI-CPDLC.

• Pilot contacts TEC by voice

• TEC issues line-up and take-off clearance by voice Table 3-4 shows the communication between ATCO and PNF in detail. (Both “GECO Staff” columns can be neglected here as they refer only to the scenarios which will be used in RTS2.)


Taxi Route Information: Aircraft Status: Parking at Apron North,

Stand 34, Departure Clearance received, before Start-up

A-SMGCS: TAXI-CPDLC


243 EXPECT ROUTING TO HOLDINGPOINT Z RWY 24 VIA TWY H Z


3 ROGER

EMMA2




Start-Up: Aircraft Status: Ready for Start-up

CDD:

25 REQUEST START-UP CLEARANCE

239 START-UP APPROVED

0 WILCO

Handover to GEC: Aircraft Status: Performing or having

performed Start-up

CDD:

117 CONTACT LKPR GROUND 121.900

0 WILCO

Push-Back Aircraft Status: Initial call to GEC made by

voice, ready for pushback

GEC:

25 REQUEST PUSHBACK CLEARANCE

239 PUSHBACK APPROVED

0 WILCO

Taxi Clearance: Aircraft Status: Pushback finished,

ready for taxiing

GEC: Note: The Taxi Clearance is a multi-

element clearance transmitted in one go.

25 REQUEST TAXI CLEARANCE

253

REVISE TAXI TO HOLDINGPOINT A RWY 24 VIA TWY H A

0 WILCO

Handover to TEC: Aircraft Status: Reaching RWY 24 Holding

Point A

GEC:

117 CONTACT LKPR TOWER 118.100

0 WILCO

Line-up and Take-off: Aircraft Status: Initial call to TEC made by

voice, …

TEC: voice

Ready for Departure on Runway 24

EMMA2




Line-Up Runway 24

Lining-Up Runway 24

Cleared for Take-Off Runway 24

Cleared for take-Off Runway 24

Handover to DEP: (optionally) Aircraft Status: Climb-out

A-SMGCS: voice



Table 3-4: TAXI-CPDLC advanced procedures for an outbound flight

3.1.1.3.4 Advanced TAXI-CPDLC: Procedures Inbound • The aircraft approaches Prague Airport on RWY 24

• The assigned stand is on apron north (Gate 31).

• The scheduled taxi route will be D F G with crossing of RWY 13/31 in between

• A-SMGCS provides expected route information by TAXI-CPDLC before final approach

• TEC issues landing clearance by voice

• TEC issues taxi clearance, crossing-RWY13/31-clearance and handover to GEC by voice

• The assigned stand is changed from Gate 31 to Gate 34 and thus a different taxi route via F G has to be cleared.

• GEC issues revised taxi clearance by TAXI-CPDLC

Table 3-5 shows the communication between ATCO and PNF in detail. (Both “GECO Staff” columns can be neglected here as they refer only to the scenarios which will be used in RTS2.)

EMMA2




Taxi Route Information: Aircraft Status: Before Final Approach for RWY

24

A-SMGCS:


243 EXPECT ROUTING TO STAND 31 VIA TWY D F G


3 ROGER

Landing clearance: Aircraft Status: Approaching RWY 24

TEC: voice

TOWER, ILS RWY 24 established, request landing clearance

Wind 060 00

knots, cleared to land RWY 24

Cleared to land RWY 24

Taxi Clearance up to RWY 13/31, Crossing of RWY 13/31 and Handover to GEC: Aircraft Status: After landing on RWY 24.

TEC: voice

Taxi to RWY 13/31 via TWY D F cross RWY 13/31 and contact Ground on 121,900 after crossing

Taxi to RWY 13/31 via TWY D F cleared to cross RWY 13/31 and contact Ground on 121,900 after crossing

Taxi Clearance: Aircraft Status: RWY 13/31 crossed,

initial call to GEC made by voice.

GEC: Note: Assigned stand on flight

strip has been changed from 31 to 34. GEC will issue taxi clearance without pilot’s request.

GROUND, on your frequency

Follow data link

253

REVISETAXI TO STAND 34 VIA TAXIWAY F G

EMMA2



DM Pilot non flying GECO staff UM ATC GECO staff 0 WILCO

Leaving Frequency (optionally via data link): Aircraft Status: Parked on stand.

GEC:

136 ON-BLOCKED

No acknowledgement by GEC via TAXI-CPDLC required

Table 3-5: TAXI-CPDLC advanced procedures for an inbound flight

3.1.1.4 Traffic Scenarios A description of the scenarios for the GECO is listed in table 3-6 and 3-7. EXE04 will be a standard scenario for TAXI-CPDLC; EXE02 will be an advanced TAXI-CPDLC scenario. Different versions of EXE06 will be a baseline scenarios resp. GTD scenarios.

EMMA2



EXE 02 EXE 04 EXE 06

Start Gate 34 13NM before

RWY24 Gate 34

13NM before

RWY06 Gate 34

13NM before

RWY24 Gate S6

Destination RWY24 Gate 34 RWY06 Gate 34 RWY24 Gate S6 RWY24

Taxi Route H B D F G

H F E L G H A D F

L P P L H A

Visibility CAT I Wind Calm

Time of day Day Day Day

TAXI-CPDLC

(standard) n.a.

Expected Taxi Route (congruent to actual clearance)

n.a.

TAXI-CPDLC

(advanced)

Expected taxi-out route TWY Z revised to TWY A

Expected taxi-in route revised after crossing by receiving actual taxi clearance granted by the GEC: preliminary Gate 31 revised to Gate 34

n.a. n.a.

Duration ~ 20’ ~ 20’ ~ 60’

Table 3-6: Scenario Description Overview RTS (Part 1, June 2008)

EMMA2



EXE 02 EXE 04 EXE 06

Start Gate 18 13NM before

RWY24 Gate 18

13NM before

RWY06 Gate 55

13NM before

RWY24 Gate S6

Destination RWY24 Gate 34 RWY06 Gate 34 RWY24 Gate S6 RWY24

Taxi Route H B D F G

H F E L G H1

H A D F

L P R L H A

Visibility CAT I Wind Calm

Time of day Day

TAXI-CPDLC

(standard) n.a.

Expected Taxi Route (congruent to actual clearance)

n.a.

TAXI-CPDLC

(advanced)

Expected taxi-out route TWY Z revised to TWY A

Expected taxi-in route revised after crossing by receiving actual taxi clearance granted by the GEC: preliminary Stand 34 revised to Gate 18

n.a. n.a.

Duration ~ 20’ ~ 20’ ~ 60’

Table 3-7: Scenario Description Overview RTS (Part 2, October 2008)

Taxi routes are depicted in red in Figure 3-4 –Figure 3-10; preliminary taxi routes are coloured in blue.

EMMA2



Figure 3-4: Taxi Route in EXE02, Pt. 1 outbound (Gate 34 RWY24)

Figure 3-5: Taxi Route in EXE02, Pt. 2 inbound (RWY24 Gate 34)

EMMA2




Figure 3-7: Taxi Route in EXE04, Pt. 2 inbound (RWY06 Gate 34)

EMMA2




Figure 3-9: Taxi Route in EXE06, Pt. 2 inbound (RWY24 Gate S6)

EMMA2



Figure 3-10: Taxi Route in EXE06, Pt. 3 outbound (Gate S6 RWY24)

The GECO will appear as an Airbus A320 with a DLH-callsign in the ATS. Callsigns will change from trial to trial to make the GECO less extraordinary for the ATCOs (see table 3-8). Exercise Callsign Action SID/STAR ETA/ETD StandEXE02 DLH72C Departure MEDOV 1A 00:10 34 EXE02 DLH73C Arrival on final 00:28 34 EXE04 DLH471 Departure TABEM 2D 00:12 34 EXE04 DLH472 Arrival on final 00:28 34 EXE06_01 DLH621 Departure MEDOV 1A 00:11 34 EXE06_01 DLH622 Arrival on final 00:28 S6 EXE06_01 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_02 DLH621 Departure MEDOV 1A 00:10 34 EXE06_02 DLH622 Arrival on final 00:28 S6 EXE06_02 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_03 DLH621 Departure MEDOV 1A 00:10 34 EXE06_03 DLH622 Arrival on final 00:28 S6 EXE06_03 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_04 DLH621 Departure MEDOV 1A 00:10 34 EXE06_04 DLH622 Arrival on final 00:28 S6 EXE06_04 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_05 DLH621 Departure MEDOV 1A 00:10 34 EXE06_05 DLH622 Arrival on final 00:28 S6

EMMA2



EXE06_05 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_06 DLH621 Departure MEDOV 1A 00:10 34 EXE06_06 DLH622 Arrival on final 00:28 S6 EXE06_06 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_07 DLH621 Departure MEDOV 1A 00:10 34 EXE06_07 DLH622 Arrival On final 00:28 S6 EXE06_07 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_08 DLH621 Departure MEDOV 1A 00:10 34 EXE06_08 DLH622 Arrival On final 00:28 S6 EXE06_08 DLH623 Departure MEDOV 1A 01:04 S6 EXE06_09 DLH621 Departure MEDOV 1A 00:10 34 EXE06_08 DLH622 Arrival On final 00:28 S6 EXE06_09 DLH623 Departure MEDOV 1A 01:04 S6

Table 3-8: List of GECO call signs for every test run

3.1.1.5 Participants There were eight pilots, seven of them are working currently for the Deutsche Lufthansa, Czech Airlines, and Germania. Their average age is 42,4 years (SD: 15,3 years) and the mean reported flight hours are 7471 (SD: 7731).

Age Sex Nationality Amount of flight hours

Hierarchical Company

A/C currently operated

PRG Ex-

perience Base Airport

33 Male German 1700 F/O Germania Fokker 100 No MUC

39 Male German 6000 SF/O DLH 744 Yes FRA

59 Male German >20000 CPT DLH A330, A340 Yes FRA

28 Male German 3000 F/O DLH A320 Yes MUC

67 Male German 23600 CPT ex DLH / Daimler Mercedes n.a. Yes MUC

28 Male German 1500 F/O DLH A300-600 No FRA

47 Male Czech 10000 CPT CSA A32F Yes LKPR

43 Male Czech 6500 CPT CSA B737 Yes LKPR

Table 3-9: Demographical Description of the pilots participating at the DLR RTS



3.1.1.6 Schedule of Executed Test Runs

ATS GECO Validation Area Traffic Scenario Day No. Treatment TEC GEC CDD CM1 CM2 Treatment

1 Training (EFS/DMAN) 2 1 3 A B Training Training EXE04 RWY06 2 Training(TAXI-CPDLC) 1 3 2 B A Training TAXI-CPDLC Training EXE02 RWY24 3 Training(TAXI-CPDLC) 3 2 1 B A TAXI-CPDLC

standard Op. Feasibility EXE04 RWY06

4 Baseline 1 2 3 A B Baseline Op. Improvement EXE061 RWY24 5 EFS 3 1 2 A B GT Op. Improvement EXE065 RWY24

Mo, 6/23

6 Baseline 2 3 1 B A Baseline Op. Improvement EXE062 RWY24 7 EFS 1 2 3 B A GT Op. Improvement EXE066 RWY24 8 TAXI-CPDLC + Routing 1 2 3 A B TAXI-CPDLC


9 TAXI-CPDLC + Routing 3 1 2 B A TAXI-CPDLC revised expected taxi-out: H Z → H A

Op. Feasibility EXE02 RWY24

Tu, 6/24

10 TAXI-CPDLC + Routing 2 3 1 A B TAXI-CPDLC revised actual taxi-out H Z → H A, taxi-in 31 → 34


11 Without Tower Simulation C D Training Training EXE04 RWY06 12 Baseline 3 1 2 D C Training Op. Improvement EXE063 RWY24 13 EFS 2 3 1 C D Baseline Op. Improvement EXE064 RWY24 14 EFS+DMAN 1 2 3 D C Baseline Op. Improvement EXE067 RWY24 15 EFS+DMAN 3 1 2 C D GT Op. Improvement EXE068 RWY24

We, 6/25

16 EFS+DMAN 2 3 1 D C GT Op. Improvement EXE069 RWY24 17 TAXI-CPDLC + Routing 2 3 1 C D TAXI-CPDLC


18 TAXI-CPDLC + Routing 3 1 2 D C TAXI-CPDLC standard


19 TAXI-CPDLC + Routing 1 2 3 C D TAXI-CPDLC rev. exp. taxi-out: H Z → H A


Th, 6/26

20 TAXI-CPDLC + Routing 2 3 1 D C TAXI-CPDLC revised actual taxi-out H Z




ATS GECO Validation Area Traffic Scenario Day No. Treatment TEC GEC CDD CM1 CM2 Treatment

→ H A, taxi-in 31 → 34 21 Training (EFS/DMAN) 5 4 6 E F Training Training EXE04 RWY06 22 Training(TAXI-CPDLC) 4 6 5 F E Training TAXI-CPDLC Training EXE02 RWY24 23 Training(TAXI-CPDLC) 6 5 4 F E TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 24 Baseline 4 5 6 E F Baseline Op. Improvement EXE061 RWY24 25 EFS 6 4 5 E F GT Op. Improvement EXE065 RWY24

Mo, 10/13

26 Baseline 5 6 4 F E Baseline Op. Improvement EXE062 RWY24 27 EFS 4 5 6 F E GT Op. Improvement EXE066 RWY24 28 TAXI-CPDLC + Routing 4 5 6 E F TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 29 TAXI-CPDLC + Routing 5 6 4 F E TAXI-CPDLC

revised exp. taxi-out route: G A → H A


Tu, 10/14

30 TAXI-CPDLC + Routing 6 4 5 E F TAXI-CPDLC revis. actual taxi-out H1 H Z A → H1 H A, in 50 → 51


31 Without Tower Simulation G H Training Training EXE04 RWY06 32 Baseline 6 4 5 H G Training Op. Improvement EXE063 RWY24 33 EFS 5 6 4 G H Baseline Op. Improvement EXE064 RWY24 34 EFS+DMAN 4 5 6 H G Baseline Op. Improvement EXE067 RWY24 35 EFS+DMAN 6 4 5 G H GT Op. Improvement EXE068 RWY24

We, 10/15

36 EFS+DMAN 5 6 4 H G GT Op. Improvement EXE069 RWY24 37 TAXI-CPDLC + Routing 5 6 4 G H TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 38 TAXI-CPDLC + Routing 6 4 5 H G TAXI-CPDLC stand. Op. Feasibility EXE04 RWY06 39 TAXI-CPDLC + Routing 4 5 6 G H TAXI-CPDLC

revised expected taxi-out route: H1 H Z A→H1 H A


40 TAXI-CPDLC + Routing 5 6 4 H G TAXI-CPDLC revised actual taxi-out H Z A → H A, taxi-in 18→17


Th, 10/16

41 TAXI-CPDLC + Routing 6 4 5 G H TAXI-CPDLC revised actual taxi-out H Z →H A, in 34→18


Table 3-10: Test Schedule including treatment, allocation of Pilots to the Cockpit and ATCOs to the CWP positions, Validation Area and the used traffic scenario



3.1.1.7 Metrics and Indicators In accordance to the EMMA2 high and low level objectives, the following table outlines the respective indicators and metrics that were used in 2-WP6.6 on-board validation activities by the DLR. They were derived from the general EMMA2 validation documents: the “Validation Plan” [5] and the “Generic Experimental Test Plan” [4]. Area HLO LLO Indicator Metric

68 items of the QE-OF Questionnaire [IND_1.1.1]

6 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_1.1.1.1]

Operational Feasibility

Verification of EMMA2 Operational Requirements and Procedures [HLO_1]

Fulfilment of EMMA2 Operational Requirements and Procedures [LLO_1.1]

10 items of the SUS [IND_1.1.2]

5 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_1.1.2.1]

Mid-run questionnaire I.S.A. for Situation Awareness [IND_2.2.1]

Mid-run self assessed situational awareness (after specific events) -11 point Likert scale - ANOVA of mean per test run are compared [M_2.2.1.1]

Increase of Safety [HLO_2]

Situational awareness according to the operator will be preserved or improved [LLO_2.2]

Post-trial self assessed Situation Awareness [IND_2.2.2]

Questionnaire ASSA (15 items) – a 5 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_2.2.2.1]

Appropriate level of user’s workload [LLO_4.1]

Mid-run questionnaire I.S.A. for workload [IND_4.1.1]

Mid-run self assessed workload (after specific events)-5 point Likert scale - ANOVA of mean per test run are compared [M_4.1.1.2]

Operational Improvements

Suitability of Behaviour and Working Performance [HLO_4]

Improved situational awareness [LLO_4.2]

Mid-run questionnaire I.S.A. for Situation Awareness [IND_4.2.1]

Mid-run self assessed situational awareness (after specific events) -11 point Likert scale - ANOVA of mean per test run are compared [M_4.2.1.1]



Area HLO LLO Indicator Metric Post-trial self assessed Situation Awareness [IND_4.2.2]

Questionnaire ASSA (15 items) – a 5 point Likert scale to each item proved for their significance by a non-parametric binominal test [M_4.2.2.1]

Less human errors [LLO_4.3]

Self assessment of human errors [IND_4.3.1]

Debriefing [M_4.2.2.1]

Table 3-11: Metrics and Indicators (RTS)



3.1.2 Results to Operational Feasibility (RTS) This chapter describes the results gained to decide on the operational feasibility of the new A-SMGCS services, which were validated. In particular, it is aimed to decide on the verification of the EMMA2 operational requirements from an operational and also from a technical point of view. The operational part is covered by debriefings and the EMMA2 Operational Feasibility Questionnaire for Pilots (QE-OF) asking the users for their acceptance to service or performance requirements. Additionally, the QE-OF gives pilots room to provide recommendations, how to improve the new procedures. All results can be found in the following two sections. However some operational requirements had also to be verified by the decision of the system designer: the system designer were asked by the EMMA2 checklist whether a specific requirement had been fulfilled or not. Sometimes both technician and user had to decide to exhaustively decide upon an operational requirement.

3.1.2.1 Operational Feasibility Questionnaire (QE-OF) After finishing off all of test runs, the “Operational Feasibility Questionnaire” (QE-OF) was given to the pilots. The questionnaire consisted of 72 items ordered by the EMMA2 services and with reference to their operational requirements and procedures described in the SPOR [3]. Not all items could be answered because not every feature of a service was implemented or could be tested. Sometimes pilots were not even affected by a situation that was asked (for instance a runway conflict) – in this case the Controller could answer “not affected” (n.a.). Additionally they could answer with “not important” (n.i.) when they felt that a requirement is not that important anymore after experiencing the whole service. The following instruction has been given to them before they were requested to fill in the questionnaire:

“Introduction: Below you find questions/statements for which we are interested in your personal opinion how far you can agree to or not (answers from 1 “Strongly disagree” to 6 “Strongly agree”). Your answers will help us to decide if the system design, system performances or new procedures have met your demands. There are two additional columns where you can make a cross: If you feel that the content of a statement is of no value for you “NOT IMPORTANT” or you were not affected by this item during the test: “NOT AFFECTED”, make your cross in those columns. For instance, you are asked to an alert performance but you wasn’t affected by such an alert. Please refer your answers to the experiences you gained while you were using the new A-SMGCS services (GTD, TAXI-CPDLC). If you want to provide additional comments/explanations you can use the whole row. Particularly when you do not agree to a statement we would be interested in the reason why. Your data will be kept confidential. Thank you in advance.”



The following hypotheses pair was applied to each of the item of the QE-OF: Identifier Hypothesis

OF-H0 The users’ opinion does not agree to the “operational feasibility” aspects of a specific item.

OF-H1 The users’ opinion agrees to the “operational feasibility” aspects of a specific item.

3.1.2.1.1 Results In section 7.1.1 all answers (raw data) of the pilots in the QE-OF can be found. Since the sample size is only eight different pilots per item, the binominal test as a non-parametric statistic was used to prove the results for its statistical significance [8]. The binominal distribution can be seen as a rather conventional and robust statistic, which is supporting the H0 hypothesis, meaning that results become harder significant in the expected direction (H1). The advantage is that results, which become significant, are more likely to be really meaningful. By use of a binominal test for a single sample size, each item was proven for its statistical significance by following conditions:

• Binominal Test • Answers from 1 (disagreement) through 6 (agreement) • Expected mean value = 3.5 • Test ratio: .50 • N = 8 • α = 0.05

A star (*) attached to the p-value means that a questionnaire item has been answered significantly because the p-value is equal or less than the critical error probability α, which is 0.05. Additionally, such items were coloured green. When comments were given to an item, they are reported directly below the statement. Rows coloured in

• green indicate a statistical significance • amber indicate a negative attitude to this statement, what means that average answers were

below the expected mean value of 3.5 Table 3-12 shows the respective results.



General requirements

ID Questions / Statements M N SD p EMMA2 OR

When working with the new cockpit services, procedures, responsibilities and functions are clearly defined.

a) Ground Traffic Display 5.38 8 1.06 0.07

2-G

c) TAXI-CPDLC 4.00 8 1.20 0.73

GEN_Serv-06

I think the new cockpit services were well integrated into the existing systems.

a) Ground Traffic Display 5.38 8 0.74 0.01**

3-G

c) TAXI-CPDLC 3.88 8 0.83 0.07

GEN_Serv-10

The new cockpit services are capable of being used appropriately when operating within the movement area.


5-G

c) TAXI-CPDLC Comments: Pilot 5:

• TAXI-CPDLC operation requires too much head down times while using the CDTI. This means either a/c needs to be stopped (time is lost) or redundancy of second crew member operating the CPDLC is lost.

4.25 8 1.28 0.29

GEN_Serv-13

Even in case of a failure of an element of a new cockpit services, the failure effect was such that the status was always in the "safe" condition.


7-G


• for outbound “slightly disagree”; for inbound “disagree”

4.38 8 1.60 0.73

GEN_Serv-16

The new cockpit services enable me to interface and function efficiently.

9-G


GEN_Serv-22




• for outbound “slightly sagree”; for inbound “disagree”

3.25 8 1.28 1.00

The design of the new cockpit services excludes failures that result in erroneous data for operationally significant time periods.


11-G


• for outbound “slightly disagree”; for inbound “disagree”

Pilot 3: • it’s not easy to decide for pilots if this is

really the case

3.50 8 1.07 0.73

GEN_Serv-24

The new cockpit services have the ability to provide continuous validation of data and timely alerts to the user when the system must not be used for the intended operation.


12-G



Pilot 6: • no error messages received in the

scenarios

3.13 8 1.36 1.00

GEN_Serv-25

The new cockpit services provide a continuous service.


14-G



4.38 8 1.19 0.29

GEN_Serv-27

15-G Any unscheduled break in operations was sufficiently short or rare as not to affect the safety of aircraft.

GEN_Serv-28






Pilot 3: • breaks wouldn’t have been necessary if

CDTI hadn’t been so novel

4.13 8 1.36 0.73

I was able to detect significant failures and could initiate remedial action to restore the service or provide a reduced level of service.

a) Ground Traffic Display 5.14 7 0.69 0.02*

16-G


• Failures with the CDTI occurred

4.25 8 1.04 0.07

GEN_Serv-29

The new cockpit services allowed me for a reversion to adequate back-up procedures if failures in excess of the operationally significant period occur.

a) Ground Traffic Display Comments: Pilot 6:

• Yes, by chart

5.63 8 0.74 0.01**

17-G


• Yes, by voice

4.75 8 1.39 0.07

GEN_Serv-32

The new cockpit services are capable of supporting operations of aircraft and vehicles within their minimum and maximum speeds and any heading.


19-G


• Rather disturbs than helps

4.25 8 1.67 0.73

GEN_Perf-01



Surveillance requirements

ID Questions / Statements M N SD P EMMA2 OR

1-S2 The ground traffic display provided accurate position information on all movements within the movement area that I could work safely and efficiently. Comments: Pilot 1:

• very effective

5.75 8 0.46 0.01**

SURV_Serv-01

2-S The ground traffic display provided identification and labelling of the surrounding traffic that I could work safely and efficiently.

5.75 8 0.46 0.01**

SURV_Serv-02

3-S The ground traffic display was coping with moving and static aircraft and vehicles that I could work safely and efficiently.

5.50 8 0.76 0.01**

SURV_Serv-03

4-S The ground traffic display was sufficiently capable of updating surveillance data of the surrounding traffic that I could work safely and efficiently.

5.63 8 0.52 0.01**

SURV_Serv-04

5-S (While I was using it) The ground traffic display was unaffected by operationally significant effects such as adverse weather and topographical conditions.

5.86 7 0.38 0.02*

SURV_Serv-05

Routing/planning requirements


2 In a simulation environment it is not possible to address the operational feasibility of a surveillance service since the performance of the service itself is simulated.



10-R The used terminology or symbology to represent a taxi route was easy to interpret. Comments: Pilot 4:

• RWY Identification markers could have been larger

5.38 8 0.52 0.01** ROUT_Serv-12

Requirements related to HMI


2-H The new display maintained a balance between human and machine tasks.

5.38 8 0.52 0.01** HMI_Serv-02a

3-H The new display permitted me to retain the power to make decisions as to those functions for which I was responsible.

5.25 8 0.46 0.01** HMI_Serv-02b

4-H The new display provided a convenient mix of visual, audio and tactile inputs and responses. 4.63 8 1.19 0.07

HMI_Serv-02c

5-H Input devices were functionally simple - involving me in a minimum number of input actions. Comments: Pilot 6:

• Totally un-ergonomically

3.63 8 1.77 0.73

HMI_Serv-03

7-H The new display allowed me to operate the new services in a safe and efficient manner. Comments: Pilot 4:

• For Ground Traffic Display “agree“; for TAXI-CPLDC “slightly disagree“

5.25 8 1.04 0.07

HMI_Serv-07

8-H The new display was harmonised where possible with existing cockpits components. Comments: Pilot 4:

• For Ground Traffic Display “agree“; for TAXI-CPLDC “slightly disagree“

5.13 8 0.35 0.01**

HMI_Serv-08

9-H The new display design took into account my working environment under various operational conditions, means, the new display was adaptable to the various circumstances. Comments: Pilot 4:

• For Ground Traffic Display “slightly agree“; for TAXI-CPLDC “slightly disagree“

4.50 8 0.93 0.07

HMI_Serv-09



10-H The new display allowed me to utilise the display capabilities (e.g. range scale selection, brightness, map overlays). Comments: Pilot 7:

• Brightness control knob would be appreciated.

5.25 8 0.71 0.01**

HMI_Serv-10

11-H The new display provided me with the complete airport traffic situation, allowing a rapid situation assessment. Comments: Pilot 3:

• This is the main message from the trials!

5.63 8 0.52 0.01**

HMI_Serv-12

14-H I was presented with a clear 'picture' to easily locate and identify aircraft. Comments: Pilot 7:

• Yes, for close proximity (up to 500m).

5.75 8 0.46 0.01**

HMI_Serv-15

15-H Only the minimum traffic information was permanently displayed. 3.63 8 1.60 0.73

HMI_Serv-21

18-H I was provided with clear and visible indication when I was deviating from my cleared taxi route. Comments: Pilot 6:

• Additional hints / alerts would be more appreciated

4.57 7 1.27 0.13

HMI_Serv-29

23-H All relevant surrounding traffic was presented in clear and pre-defined formats that helped me operating my a/c in a safe and efficient manner. Comments: Pilot 5:

• E.g. “follow TWY A B to R [line break] WY…” should be “

follow TWY A B to R WY… “ in ONE line, not to mistake it with TWY R

4.88 8 0.99 0.07

HMI_Serv-39

24-H Depending on my operational needs, the new display was highly configurable with regard to layout, size, shape, fonts, colours and interaction capability.

4.38 8 1.19 0.07

HMI_Serv-40

30-H TAXI-CPDLC assisted me in handover operations. Comments: Pilot 7:

• Low experience with the new device.

3.50 8 1.41 0.73

HMI_Serv-48



35-H I was reliably presented with a means to input TAXI CPDLC messages. Comments: Pilot 7:

• It will need some training and some new listing of messages to operate the on-board system efficiency.

3.38 8 1.30 0.73

HMI_Serv-55

36-H It was possible to easily correct a mistaken action. Comments: Pilot 3:

• Not easy because CDTI is novel for pilots in the trials..

3.25 8 0.71 0.73

HMI_Serv-58

38-H I was provided with a quick and efficient means to modify a previous ATC assigned route after receiving a revised taxi clearance.

3.13 8 0.64 0.29 HMI_Serv-61

39-H I could always easily intervene with the new display to set additional constraints unknown to the new on-board function (e.g. closed taxiway). Comments: Pilot 7:

• Not installed.

2.43 7 1.13 0.13

HMI_Serv-62

40-H The indication of stop bars was well integrated into the new display. Comments: Pilot 6:

• I wasn’t able to delete the stop bar

4.25 8 1.49 0.29

HMI_Serv-67

42-H When the crossing was operated by TAXI-CPDLC, the status of the displayed stop bar was always in accordance with ATCO clearances. Comments: Pilot 6:

• Had to be confirmed manually

3.25 8 1.75 1.00

HMI_Serv-69

47-H The presentation of other surrounding movements was not delayed to an extent where it is no longer operationally acceptable.

4.38 8 1.51 0.07 HMI_Perf-04

48-H The response time of the HMI was adequate to allow making inputs without having to wait unduly for the system to process and validate the input.

4.25 8 1.49 0.29

HMI_Perf-05



TAXI-CPDLC requirements


2-T When needed, it was always possible to switch back from data link communication to voice communications in a safe and efficient manner.

5.50 8 1.07 0.07

TAXI-CPDLC_Serv-02

3-T I was provided with an effective human-machine interface to permit efficient TAXI-CPDLC communication with ATC.

3.13 8 1.55 0.73

TAXI-CPDLC_Serv-06a ATCO

6-T In the event of an unexpected termination of a data link application I was sufficiently notified of the failure.

4.17 6 1.17 0.22 TAXI-CPDLC_Serv-09

8-T I was always provided with the capability to adequately respond to TAXI-CPDLC messages and to issue requests, as appropriate. 3.00 8 0.93 0.73

TAXI-CPDLC_Serv-13

10-T TAXI-CPDLC messages were appropriately displayed and stored in a manner that permitted timely and convenient retrieval when such actions were necessary. Comments: Pilot 3:

• Difficult to find old messages – especially if new message is sent. The listing should be upside down with the newest message appearing on the top of the list.

1.88 8 0.64 0.01**

TAXI-CPDLC_Serv-16

11-T I was always under the control of only one ATC unit at a time. Comments: Pilot 7:

• Sometimes it seemed that CDD and GEC worked simultaneously. .

4.50 8 1.41 0.29

TAXI-CPDLC_Serv-17

13-T Each TAXI-CPDLC message transmission was followed by a positive technical acknowledgement, which informed me that the message has completely been transmitted and is available on the ATCO’s display. Comments: Pilot 2 & 6:

• No feedback in some cases

2.71 7 1.60 1.00

TAXI-CPDLC_Serv-19



14-T The time I needed to look outside was not considerably impaired by operating TAXI-CPDLC. Comments: Pilot 6:

• Consequently, the a/c needs to be stopped while using the CDTI because at least one pilot will look head down.

Pilot 7: • Especially revised instruction on apron!

2.50 8 1.20 0.29

Anticipated constraint, SPOR §3.7

15-T The amount of inputs to operate TAXI-CPDLC was reasonably low. Comments: Pilot 6:

• CDTI menu unnecessarily convoluted, unconventional commands (WILCO)

2.38 8 1.19 0.07


16-T The total time required for selecting a TAXI-CPDLC message, transmission of the message, reading and interpretation of the message was adequate to communicate in a safe and efficient manner. Comments: Pilot 6:

• Safe, but not efficient

2.88 8 1.13 0.29

ICAO Doc9694 PART IV, §1.8

17-T ATCO’s TAXI-CPDLC response time was sufficient short to work in a safe and efficient manner. Comments: Pilot 7:

• For the given status of the system, yes!

3.63 8 1.19 0.73


18-T The handling of other aircraft by TAXI-CPDLC did not lead to loss of important information, which is usually provided by a R/T partyline effect. Comments: Pilot 6:

• Party line effect is missed. Pilot 6:

• No queue info during push-back. Party line for additional situation awareness no longer possible.

3.17 6 1.17 0.69




19-T The mix of TAXI-CPDLC and voice communication with different clearances did not lead to confusion and safety critical communication errors. Comments: Pilot 6:

• Different clearances via voice resp. TAXI-CPDLC were experienced.

2.71 7 1.38 1.00


22-T The TAXI-CPDLC messages were easy to understand and could be handled in a safe and efficient way. If not please write down which one should be improved and how: Comments: Pilot 6:

• Use “ACCEPT” or “READ-BACK/CONFIRM” instead of “WILCO”

• listing should be upside down with the newest message appearing on the top of the list

4.13 8 1.36 0.73

EMMA2 message set

23-T It was easy to recognise an incoming TAXI-CPDLC message. Comments: Pilot 3:

• No, should be improved (more volume). Pilot 6:

• Use “ACCEPT” or “READ-BACK/CONFIRM” instead of “WILCO”

4.38 8 1.69 0.29

EMMA2 SPOR §3.7.3f

25-T Receiving TAXI-CPDLC taxi route information in advance of the real taxi route clearance is an appropriate procedure to provide me with important preliminary information. Comments: Pilot 2:

• But not in 2000ft alt during approach Pilot 6:

• Preliminary route deviates from the actual route too often

4.25 8 1.83 0.73

EMMA2 SPOR §3.7.3j

26-T Receiving TAXI-CPDLC taxi route information in advance of the real taxi route clearance did not exceed my available mental and time resources. Comments: Pilot 7:

• Not in case of message arriving when ILS established.

4.75 8 0.71 0.01**

EMMA2 SPOR §3.7.3j



27-T TAXI-CPDLC communication while the aircraft was taxiing could be performed in a safe and efficient way. Comments: Pilot 3&6:

• Consequently, the a/c needs to be stopped while using the CDTI because at least one pilot will look head down.

3.88 8 1.46 0.73

EMMA2 SPOR §3.7.3m

28-T The frequency of the next control position can be transmitted silently by a TAXI-CPDLC message, but the initial call from the pilot at the next control position should be retained by voice. Comments: Pilot 6:

• Yes. Otherwise it’s not guaranteed if there’s contact with the tower. E.g. in case of emergency if a revised clearance (“Hold position”) is given.

4.38 8 1.92 0.29

EMMA2 SPOR §3.7.3p

Table 3-12: QE-OF - Means, N, SD, and P-Value

Conclusions There were 72 questionnaire items asking the pilots acceptance to specific operational requirements and procedures of the new higher-level A-SMGCS services. Finally,

• 20 operational requirements or procedures could be verified in the DLR RTS test campaign, proven by a non-parametric binominal test

o (highlighted in green in table 3-12), • 36 operational requirements or procedures were accepted but not with a probability it can

doubtlessly relied on o (statements in black in table 3-12), and

• 16 operational requirements or procedures were not answered positively o (highlighted in amber in table 3-12).

The pilots disagreed on 16 of the 72 items, and respectively could not accept the experienced performance. Those 16 negative answers are mainly caused by the use of a novel and unfamiliar input device which served for selecting CPDLC messages to be sent and for the textual displaying of received clearances, the CDTI (cf. chapter 3.1.2.3.2.1)

3.1.2.2 SUS Questionnaire In addition to the tailor-made QE-OF for pilots the standard questionnaire Standard Usability Scale (SUS) [9] was given. While it is difficult to answer the question “is system A more usable than system B”, (due to the fact that measures of effectiveness and efficiency may be very different), it can be argued that a sufficiently high-level definition of subjective assessments of usability, makes comparisons between systems possible. SUS has generally been seen as providing a type of high-level subjective view of usability and is thus often used in carrying out comparisons of usability between systems (here: baseline vs. higher A-SMGCS on-board services). It contains five items in favour of a new system (item 1, 3, 5, 7, and 9) and five items to the disadvantage of a new system (item 2, 4, 6, 8, and 10).



The SUS was given to the pilots after the very last trial. Answers range from 1 “Strongly disagree” to 5 “Strongly agree”. Again, the following hypotheses pair was applied to each of the item of the SUS: Identifier Hypothesis

OF-H0 The users’ opinion does not agree to the “operational feasibility” aspects of a specific item [in terms of usability].

OF-H1 The users’ opinion agrees to the “operational feasibility” aspects of a specific item [in terms of usability].

3.1.2.2.1 Results In section 7.1.4 all answers (raw data) of the pilots in the SUS can be found. Since the sample size is only eight different pilots per item, the binominal test as a non-parametric statistic was used to prove the results for its statistical significance [8]. The binominal distribution can be seen as a rather conventional and robust statistic, which is supporting the H0 hypothesis, meaning that results become harder significant in the expected direction (H1). The advantage is that results, which become significant, are more likely to be really meaningful. By use of a binominal test for a single sample size, each item was proven for its statistical significance by following conditions:

• Binominal Test • Answers from 1 (disagreement) through 5 (agreement) • Expected mean value = 3.0 • Test ratio: .50 • N = 8 • α = 0.05

A star (*) attached to the p-value means that a questionnaire item has been answered significantly because the p-value is equal or less than the critical error probability α, which is 0.05. Additionally, such items were coloured green. When comments were given to an item, they are reported directly below the statement. Rows coloured in green indicate a statistical significance in favour of the new system (p < 0.05). Table 3-13 shows the respective results. Answers range from 1 “Strongly disagree” to 5 “Strongly agree”. ID Questionnaire Item M N SD P 1 I think that I would like to use this system frequently

4.38 8 0.92 0.29

2 I found the system unnecessarily complex. 2.63 8 1.06 0.29

3 I thought the system was easy to use. 3.13 8 0.83 0.73

4 I think that I would need the support of a technical person to 1.63 8 1.06 0.07



ID Questionnaire Item M N SD P be able to use this system.

5 I found the various functions in this system were well integrated. 3.13 8 0.83 0.73

6 I thought there was too much inconsistency in this system . 2.50 8 0.53 0.01**

7 I would imagine that most people would learn to use this system very quickly. 4.00 8 0.93 0.73

8 I found the system very difficult to use. 2.38 8 0.92 0.01**

9 I felt very confident using the system. 3.63 8 0.74 0.29

10 I needed to learn a lot of things before I could get going with the system. 1.88 8 0.83 0.01**

Table 3-13: SUS- Means, N, SD, and P-Value

Conclusions The pilots answered all ten items in favour of the new system. Three of them (highlighted in green in table 3-13) became even statistically significant, proven by a non-parametric binominal test.

3.1.2.3 Debriefing Comments This chapter summarizes statements made by the pilots at the debriefings after the RTS trials. The comments reflect a common opinion of all eight pilots. The comments mainly address aspects of the operational feasibility and proposals for improvements of the new A-SMGCS services. When such proposals could already be implemented within EMMA2 they are coloured in “green”, and “blue” when it could not be solved yet.

3.1.2.3.1 Ground Traffic Display

• The GTD was appreciated and highlighted by all pilots. • The added value of the GTD is seen especially under low visibility conditions.

• The intuitive approach allows the pilots to work in a simple and interactive way.

• According to the pilots, the design fits completely the pilots’ needs.

• In the opinion of the pilots, the GTD worked reliable in the simulation trials (= with an

update rate of 5Hz)

• The different modes (PLAN, ARC and NAV) and the zoom stages are highly appreciated and intuitive to use.

• The GTD allows pilots to utilise the display capabilities (e.g. zoom and mode selection)

appropriately. This was realised by both the mode resp. range selector in the EFIS and soft buttons on the GTD’s touchscreen. Pilots preferred to use the knobs in the EFIS.



• Pilots reported that the head down times while using the GTD display (located on the

navigation display) are acceptable. Nevertheless, the outside view through the cockpit windows remains their main source of information.

Minor additional comments:

• Two pilots propose to display only traffic with transponder “on”. • Two pilots propose to display landing traffic on short final. • Two of eight pilots expressed their wish to have 3D buildings in the display.

Not considered to integrate 3D buildings by now • One pilot wants to integrate EMM on the PFD; and blanking out the NAV display.

Instead of merging the information with the PFD, the use of a HUD is considered.

3.1.2.3.2 Taxi-CPDLC

• TAXI-CPDLC is regarded as potential means which will ease pilots work considerably with appropriate input devices. The graphical presentation of the cleared taxi route on the EMM (compared to textual clearances only) will enable pilots to follow an assigned taxi route in an intuitive way.

• The pilots’ feedback for requesting and receiving clearances by TAXI-CPDLC for start-

up, push-back, taxi-in, and taxi out was predominantly positive. It must be distinguished clearly between

o the positively rated graphical information on the EMM display (represented in this chapter) and

o the rating of the usability of the CDTI, which served for selecting CPDLC messages to be sent and for the textual displaying of received clearances (cf. chapter 3.1.2.3.2.1).

• Pilots highly appreciated and trusted the displayed taxi routes and the clearances. • They agree that the used terminology and symbology to represent a taxi route were easy

to interpret. Paper charts were only held ready as a back-up as expected. • The pilots stated that they were provided with a very effective human-machine interface in

terms of the EMM display to operate a data link communication with ATC.

• The visualisation of taxi routes on the EMM was appealing for them.

• Especially the exocentric perspective view on the EMM was used by the CM2 to gain advance information during taxiing.

• Six pilots appreciate the initial call at the beginning of each TAXI-CPDLC dialogue at each

new ATC position.

• Pilots responded that they would like to use the TAXI-CPDLC in the future and that they agree in general with its concept given and realised in EMMA2. Yet some of the proposed procedures are perceived as being clumsy.

o Pilots recommend that the CDD should issue departure clearance, start up and change to ground frequency in one step, when the situation allows it.

Fulfilled in the RTS in October as far as possible



o In the trials the crossing was performed by voice communication, due to its nature of a safety critical clearance. This led to the situation that the pilots in the cockpit had to switch off the red crossbar (received from the Ground controller as the clearance limit) on their EMM. This procedure is of course criticised by the pilots. They agree that this task must be performed by an ATCO.

Further testing of adapted procedures would be needed here, e.g. crossing clearance by TAXI-CPDLC, or by a voice clearance combined with an automatic deletion of the displayed stop bar by the ATCO’s pressing of the “crossing clearance” button on their EFS.

o In addition to the initial call a dedicated radio frequency for emergency situations is recommended by two pilots.

• Pilots agree that a preliminary taxi route will not bring added value. The assigned route

can change o at short notice during initial approach (inbound) or o preparing the a/c for departure (outbound),

and thus is not a reliable information in the opinion of the pilots. • During arrival

o no preliminary taxi route should be sent after pilots’ preparation of the landing (approx. before 8.000 – 10.000 ft; preliminary information at 2.000 ft is too late)

o resp. a preliminary information about the expected gate would be sufficient. • Nevertheless receiving a TAXI-CPDLC taxi route information in advance did not exceed

the available mental and time resources in the pilots’ opinion.

• The audio signal for an incoming message is choked until the CM1 or CM2 push the button. No other message can be detected before the button is pressed.

o Therefore an additional signal ins necessary and o the CDTI must automatically return into the main page so that the oncoming message

can be seen easily. o Furthermore, if one message button is pressed, both lights must go off automatically.

Solved during the June 2008 trials. • Labelling of the buttons must be integrated in harmonisation with existing cockpits

components. o “ACCEPT” or “READ-BACK/CONFIRM” is a better choice than “WILCO”

Considered for trials in the future. • When receiving a revised taxi route (or the taxi route approved by the ground controller) all

other taxi routes shown before (= first route before revised; or preliminary taxi route) must be deleted to avoid confusions and an information overflow. Pilots reported that the colour

o of an old route remained green and the revised route appeared in yellow (happened twice in the June trials).

Solved during the June 2008 trials by reprogramming. o that the colour of revised route appeared in green on the left, but still in yellow on the

right ND (happened three times in the June trials). Two different displays were in use resulting in different latencies; solved as

far as possible during the June 2008 trials; • Furthermore it happened that the CM1 pushed the cross button on the ND and the red lined

crossbar in his ND disappeared as expected, but not on the CM1 side. Solved during the June 2008 trials by reprogramming.

Four pilots explained that ATCO’s handling of other aircraft by TAXI-CPDLC led to loss of important information, which is usually provided by a R/T partyline effect.



A minority of pilots proposed following changes in the system: • Use of a larger font (especially necessary during turbulences) (two pilots) • Implementation of a printer (one pilot) • Louder aural signal (one pilot)

3.1.2.3.2.1 CDTI

• Using the CDTI as an additional input device for data link communication (instead of using the FMS) is appreciated by the majority of the pilots. In the opinion of most of the pilots (= six) it might be too confusing to integrate the data transfer into the FMS resulting in a difficult handling of the TAXI-CPDLC. Nevertheless the novelty of the menu navigation in the CDTI and the resulting high complexity is not acceptable for the pilots.

• Therefore the handling of the messages could not always be successfully performed.

Pilots’ experiences are reflected here together with their recommendations. • Pilots had difficulties to find new messages in the system as long as they were working on the

previous message. It was not easy or intuitive enough to find the new message in the log page. In some cases, pilots were not able to find the logical acknowledgment from the tower indicating that the pilots’ message was sent successfully to the tower.

o One crew preselected „request taxi“ and was not able to receive any new messages because the system was blocked.

This problem was solved after the June 2008 trials. Therefore they have concerns that new message will get lost especially if the pilot is occupied. • Pilots recommend

o that a new message must be suppressed as long as the last message is not been processed by the pilots.

o A clear screen button for the last message or a o refresh button (the last two recommendations by two pilots)

• Furthermore, they had problems to find old messages on the log page. o Additional problems occurred when old messages remained as not affirmed on the

CDTI display after their clearance. this problem was solved during the June 2008 trials.

• The CDTI was used in the validation trials for data link communication as a test-release resp.

prototype to test a new user menu in the simulation. This new menu concept was not approved by the pilots and led to longer head-down times than acceptable. The CDTI needs important upgrades if a further use of the CDTI is foreseen in simulation and flight trials. As indicated in the pilots’ feedback, additional and crucial recommendations for the CDTI were made: • To provide a menu similar to FANS B • To integrate LLSK (Left Line Select Keys) and RLSK (Right Line Select Keys) • To ensure accessibility of the main menu • To guarantee clear arrangement of the messages received in the CDTI’s log function, e.g.

the listing should be upside down with the newest message appearing on the top of the list. TAXI-CPDLC messages were not appropriately displayed and stored in a manner that permitted timely and convenient retrieval when such actions were necessary.

Nevertheless it must be highlighted that the use of CDTI was necessary to have a flight-worthy and certified interface for preparing and conducting the subsequent flight trials.



3.1.3 Results to Operational Improvements (RTS) This chapter describes the results gained to decide on the operational improvement of the new A-SMGCS services, which were under test. Subjective data were collected by opinions of the pilots, who were asked to estimate the potential of a new service to provide operational improvement. Such are a useful resource to decide on a hypothesis and finally to validate a new service. Subjective data were gained by

• the EMMA2 Operational Improvement Questionnaire for Pilots (QE-OI) [6], • Airport Surface Situation Awareness Questionnaire (ASSA) [11]; [12] • Instantaneous Self Assessment (ISA) [10]

o of Workload o of Situation Awareness

3.1.3.1 Safety (HLO_2)

3.1.3.1.1 Operational Improvements Questionnaire (QE-OI), Sub-Scale Safety After finishing all test runs the “Operational Improvement Questionnaire for Pilots” (QE-OI) was given to the pilots. Five of its eight items address operational improvements in terms of safety that could be expected due to the implementation and operational use of the higher-level A-SMGCS services. The items refer either to “expected benefits” that were identified in the 2-D1.1.1 SPOR document [3] or refer directly to a low level objective (cf. [5]; [4] and §3.1.1.7). Following instruction has been given to them before they were requested to fill in the questionnaire:

“Introduction: Below you find statements for which we are interested in your personal opinion how far you can agree to or not (answers from 1 “Strongly disagree” to 6 “Strongly agree”). Your answers will help us to decide if the new services are able to provide operational improvements in terms of supporting you in aerodrome operations. Please refer your answers to the experiences you gained while you were using the new EMMA2 A-SMGCS services (GTD and TAXI-CPDLC) and compare with the current situation in the cockpit. If you want to provide additional comments/explanations you can use the whole row. Your data will be kept confidential. Thank you in advance.”

The following hypotheses pair was applied to each of the safety items of the QE-OF: Identifier Hypothesis

SF1-H0 An increment of safety while using [higher services of A-SMGCS] is not perceived by the pilots.

SF1-H1 An increment of capacity while using [higher services of A-SMGCS] is perceived by the pilots.



3.1.3.1.1.1 Results In section 7.1.2 all answers (raw data) of the pilots in the QE-OF can be found. Since the sample size is eight different pilots per item, the binominal test as a non-parametric statistic was used to prove the results for their statistical significance [8]. By use of a binominal test for a single sample size, each item was proved for its statistical significance by following conditions: Binominal Test

• Expected mean value = 3.5 • Test ratio: .50 • Answers from 1 (disagreement) through 6 (agreement) • N = 8 • α = 0.05

A star (*) attached to the p-value means that a questionnaire item has been answered significantly because the p-value is equal or less than the critical error probability α, which is 0.05. Additionally, such items are coloured green. When pilots gave comments to an item, they are reported directly below the statement. Table 3-14 shows the respective results. Answers range from 1 (disagreement) through 6 (agreement).



ID Questions / Statements M N SD p EMMA2 OR

1P-OI

The communication via TAXI-CPDLC would reduce potentially safety-critical misunderstandings between ATCO and Flight Crew compared to using radio communication only.

4.63 8 0.92 0.07

2P-OI

The display of graphical taxi clearances on the Electronic Moving Map (EMM) would enable me to follow an assigned taxi route more safely. Comments: Pilot 5:

• The potential danger exists that you follow the EMM which is not certified as “primary means of navigation”.

5.63 8 0.52 0.01**

4-OI

The receipt of only textual taxi clearances (without a graphical display) via TAXI-CPDLC would also enable me to follow an assigned taxi route more safely.

4.63 8 1.30 0.07

6P-OI

The indication of the surrounding traffic is an additional information source to navigate and manage the aircraft speed more safely.

5.75 8 0.46 0.01**

8P-OI

Particularly in high workload situations, a Traffic Conflict Detection (TCD) service would clearly indicate and better draw my attention to a potential safety critical situations. Comments: Pilot 7:

• I strongly agree but had no [real] opportunity to test the system.

5.43 8 0.53 0.02*

Table 3-14: QE-OI - Means, N, SD, and P-Value for safety-related items

3.1.3.2 Capacity / Efficiency (HLO_3)

3.1.3.2.1 Operational Improvements Questionnaire (QE-OF), Sub-Scale Efficiency

As mentioned in section 3.1.3.1.1 the “Operational Improvement Questionnaire for Pilots” (QE-OI) was given after the last trial. Three of eight items address operational improvements in terms of efficiency that could be expected due to the implementation and operational use of the higher-level A-SMGCS services. The items refer either to “expected benefits” that were identified in the 2-D1.1.1 SPOR document [3] or refer directly to a low level objective (cf. [4] and §3.1.1.7). The following hypotheses pair was applied to the capacity/efficiency items of the QE-OI: Hypothesis

CE1-H0 An increment of capacity & efficiency while using [higher services of A-SMGCS] is not perceived by the pilots.



CE1-H1 An increment of capacity while using [higher services of A-SMGCS] is perceived by the pilots.

3.1.3.2.1.1 Results In section 7.1.3 all answers (raw data) of the pilots in the QE-OF can be found. The analysis of data is explained in section 3.1.3.1.1.1. Table 3-15 shows the respective results. Answers range from 1 (disagreement) through 6 (agreement).


3P-OI

The display of graphical taxi clearances on the Electronic Moving Map (EMM) would enable me to follow an assigned taxi route more efficiently. Comments: Pilot 7:

• it may reduce taxi times, allows higher taxi speeds, increases TWY capacity

5.25 8 0.71 0.01**

5P-OI

The receipt of only textual taxi clearances (without a graphical display) via TAXI-CPDLC would also enable me to follow an assigned taxi route more efficiently.

3.50 8 1.41 0.73

7P-OI The indication of the surrounding traffic is an additional information sources to navigate and manage the aircraft speed more efficiently.

5.63 8 0.74 0.01**

Table 3-15: QE-OI - Means, N, SD, and P-Value for capacity/efficiency related items

3.1.3.3 Suitability of Behaviour and Working Performance (HLO_4)

3.1.3.3.1 Workload (LLO4.1)

3.1.3.3.1.1 ISA The ISA [10] measuring workload was applied twice in each segment (both inbound and outbound) of each test run. Pilots were asked approx. in the middle of the segment (e.g. after a runway cross) and at the end of the segment (before reaching the gate resp. before line-up). The scale ranges from “under-utilised” (1) to “excessive” (5). The following hypotheses pair was applied to the ISA results: Identifier Hypothesis

HF1-H0 An increment of workload while using [higher services of A-SMGCS] is perceived by the users.

HF1-H1 An increment of workload while using [higher services of A-SMGCS] is not perceived by the users.



3.1.3.3.1.1.1 Results In table 3.19 all answers (raw data) of the pilots in the workload ISA can be found. Answers range from “under-utilised” (1) to “excessive” (5). The means of each test run were analysed in a 4 x 2 (A-SMGCS treatment x pilot role) two-way repeated measures analysis of variance (ANOVA) [8].

• The ANOVA revealed a highly significant main effect of the A-SMGCS treatment (F(3,21) = 5.418; p = .006) with a mean of M = 2.20 for the (EMM) baseline, respectively M = 2.12 for GTD and M = 2.48 for standard TAXI-CPDLC (incl. GTD) and M = 2.36 for advanced TAXI-CPDLC (incl. GTD) on a scale reaching from “under-utilised” (1) to “excessive” (5).

• No significant main effect could be shown for the pilot role (F(1,7) = 1.932; p = .207), both CM1 and CM2 seem to have a similar level of workload.

• Yet there is a significance for the interaction between treatment and pilot role (F(3,21) = 3.743; p = .027), with the highest mean M = 2.52 in the condition CM2 using advanced TAXI-CPLDC.

Factors df Square

sum F-value Significance

p-value A-SMGCS treatment 3 1.610 5.418 .006**

error 21 2.080 pilot role 1 .181 1.932 .207

error 7 .654 Treat * role interaction 3 2.472 3.743 .027*

error 21 4.623

Table 3-16: Two-way repeated measures ANOVA “ISA Workload”

3.1.3.3.2 Situation Awareness (LLO_4.2)

3.1.3.3.2.1 ASSA The questionnaire for Airport Surface Situation Awareness [11]; [12] was given to the pilots after the very last trial. Answers range from 1 “Strongly disagree” to 5 “Strongly agree”. The following hypotheses pair was applied to each of the item of the ASSA: Identifier Hypothesis

HF3-H0 An increment of situation awareness while using [higher services of A-SMGCS] is not perceived by the users.

HF3-H1 An increment of situation awareness while using [higher services of A-SMGCS] is perceived by the users.

3.1.3.3.2.1.1 Results In section 7.1.5 all answers (raw data) of the pilots in the ASSA can be found. Answers range from 1 (disagreement) through 5 (agreement). Since the sample size is eight different pilots per item, the binominal test as a non-parametric statistic was used to prove the results for their statistical significance [8].



By use of a binominal test for a single sample size, each item was proved for its statistical significance by following conditions: Binominal Test

• Expected mean value = 3 • Test ratio: .50 • Answers from 1 (disagreement) through 5 (agreement) • N = 8 • α = 0.05

A star (*) attached to the p-value means that a questionnaire item has been answered significantly because the p-value is equal or less than the critical error probability α, which is 0.05. Table 3-17 shows the respective results.

ID Questions / Statements M N SD P 1 Display use did not increase head down time 2.75 8 1.49 0.73

2 Display clutter was not a problem 3.63 8 0.52 0.73

3 Aided in surface traffic acquisition 4.25 8 0.46 0.01**

4 Aided in surface visual traffic acquisition 4.25 8 0.46 0.01**

5 Increased time for checklist 2.88 8 0.64 0.07

6 Increased time for crew duties 3.25 8 0.89 0.29

7 Surface map easy to bring up 4.38 8 0.52 0.01**

8 Ownship symbol easy to see 4.50 8 0.53 0.01**

9 Accurately shows my position 4.88 8 0.35 0.01**

10 More info should be displayed 2.00 8 0.53 0.01**

11 Useful info provided 4.63 8 0.52 0.01**

12 Easy to get orientated to MMS Display 4.38 8 0.52 0.01**

13 Aid in determining position of traffic 4.38 8 0.52 0.01**

14 Aid in understanding ATC communication 4.00 8 0.53 0.07

15 Aid in locating traffic 4.63 8 0.52 0.01**

Table 3-17: ASSA - Means, N, SD, and P-Value

3.1.3.3.2.2 ISA (Situation Awareness) The ISA [10] in an adapted version measuring situation awareness was applied twice in each segment (both inbound and outbound) of each test run. Pilots were asked approx. in the middle of the segment (e.g. after a runway cross) and at the end of the segment (before reaching the gate resp. before line-up). The scale ranges from “low” (1) to “high” (10). The following hypotheses pair was applied to the ISA results: Identifier Hypothesis



HF3-H0 An increment of workload while using [higher services of A-SMGCS] is perceived by the users.

HF3-H1 An increment of workload while using [higher services of A-SMGCS] is not perceived by the users.

3.1.3.3.2.2.1 Results In table 3.20 all answers (raw data) of the pilots in the situation awareness ISA can be found. The scale ranges from “low” (1) to “high” (10). The means of each test run were analysed in a 4 x 2 (A-SMGCS treatment x pilot role) two-way repeated measures analysis of variance (ANOVA) [8].

• The ANOVA revealed no significant main effect of the A-SMGCS treatment (F(3,21) = .146; p = .931) with a mean of M = 8.09 for the (EMM) baseline, respectively M = 9.03 for GTD and M = 8.70 for standard and M = 8.74 for advanced TAXI-CPDLC (incl. GTD) on a scale ranging from “low” (1) to “high” (10).

• There is no significant main effect for the pilot role (F(1,7) = 1.337; p = .286), both CM1 and CM2 seem to have a similar level of situation awareness.

• There is no significance for the interaction between treatment and pilot role (F(3,21) = 1.391; p = .273).

Factors df square

sum F-value Significance

p-value A-SMGCS treatment 3 .540 .146 .931

error 21 25.926 pilot role 1 1.025 1.337 .286

error 7 5.369 Treat * role interaction 3 2.732 1.391 .273

error 21 13.750

Table 3-18: Two-way repeated measures ANOVA “ISA Situation Awareness”



Table 3-19: Pilots’ raw data, M, and SD of the ISA (Workload)

Table 3-20: Pilots’ raw data, M, and SD of the ISA (Situation Awareness)



3.1.3.4 Debriefing Comments

3.1.3.4.1 Ground Traffic Display

• In the pilots’ opinion the use of a GTD will improve their performance significantly. • GTD is regarded as potential means for increased safety which eases pilots work

considerably. • GTD significantly increases the pilots’ situation awareness (by allowing an intuitive use,

providing the appropriate amount of information, and as an extraordinary aid in locating traffic) especially under low visibility conditions.

• Pilots’ workload is not affected negatively when using the GTD. • With its indication of the surrounding traffic the GTD is an additional information source for

the pilots to navigate and to manage the aircraft speed more safely and efficiently.

3.1.3.4.2 Taxi-CPDLC

• TAXI-CPDLC is regarded as potential means for increased safety. The graphical presentation of the cleared taxi route on the EMM (compared to textual clearances only) will enable pilots to follow an assigned taxi route more safely and efficiently. An appropriate input device is necessary.

• An increase of workload while using the TAXI-CPDLC service could be observed and was

explained by the pilots due to the fact that a novel and not-well designed input device (CDTI) was used.

• Situation Awareness was not affected by the use of TAXI-CPDLC.



3.2 Operational Flight Trials Results

3.2.1 Set Up

3.2.1.1 The validation platform The on-site validation exercises used the A-SMGCS test-bed components at Prague-Ruzynĕ airport, established under 2-SP3 & 2-SP2 (green team) of the EMMA2 project.

• ATTAS test aircraft: o EMM, o Ground traffic display (fed by TIS-B), o TAXI-CPDLC (CDTI + Graphical presentation on the EMM), o ADS-B out/in, and o Walkie-Talkie (provided by Airport Prague).

The ATCOs could listen to the three real different control radio telecommunication (CDD, GEC, TEC) but worked in shadow mode, that is, they did interact with the ATTAS but not with the other a/c. They moved the EFS in accordance to the real world for all a/c. Identically to the set-up in the ATS, there were three CWPs equipped: TEC, GEC, and CDD. All the EMMA2 systems were linked to the real environment: surveillance and flight plan data.

• Tower Test Bed: o 3 CWPs: TEC, GEC, CDD o TSD, EFS, DMAN, Routing, TAXI-CPDLC, RIMCAS, basic conformance

monitoring o Frequency monitors (black box with loudspeaker and volume control) o Handsets with predefined buttons for communication with operational TWR

Figure 3-11: DLR ATTAS Test Aircraft (D-ADAM) during OST in PRG



For more detailed information see the SP6 “Validation Test Plan Airborne” (2-D6.1.6, [6]).

3.2.1.2 Experimental Design There is no real experimental design possible in field trials since very less can be controlled to perform real experiments. Nevertheless, flight trials provide an excellent opportunity to check aspects of the technical and operation feasibility that cannot be checked in simulation trials, where for instance the traffic and real interfaces are simulated. Therefore, two different “experimental designs” has been chosen: GTD and TAXI-CPDLC trials.

• ATTAS / TUD Van test Ground Traffic Display (GTD via TIS-B) (between 1,5 hour in between of 0900 – 1300)

• TAXI-CPDLC trials

o the ATTAS is controlled by voice via the regular ATCOs o in parallel the ATTAS communicates via TAXI-CPDLC with the EMMA2 ATCOs in

the test bed the respective clearances / messages o Procedure would be, as tested in simulation (cf. chapter 3.1.1.3) o ATTAS performs an outbound / inbound scenario (two aerodrome circling, approx. 80

minutes) (four IFR flight plans (two outbound, two inbound)) o responsibilities:

During the Tests, the regular controllers have the ATTAS (D-ADAM) under control, but

The actual ATCO (CDD/GEC) would only perform the initial call with the ATTAS pilots, they are briefed about the silent START-UP, PUSHBACK, and TAXI clearance given by TAXI-CPDLC from the EMMA2 ATCOs, stay in contact with EMMA2 ATCOs, monitor the control of the ATTAS and intervene by R/T in case of critical situations

3.2.1.3 Traffic Scenarios ATTAS parking position during the trials was S17 on APRON SOUTH.

Figure 3-12: DLR ATTAS Test Aircraft (D-ADAM) parking position during OST in PRG

The two main scenarios for TAXI-CPDLC trials were:



• ATTAS taxiing out from Apron South to RWY 24 performing an outbound scenario. • After aerodrome circling and arrival on RWY 24, ATTAS performed an inbound scenario

back to S17.

3.2.1.4 Participants Three DLR test pilots and a CSA pilot from the RTS performed the trials in the ATTAS cockpit.


Pilot Tu, 18. 11. We, 19. 11. Th, 27. 11. 1100-1300 1400-1500 0800-1000 1300-1500 1000-1230 1300-1500

TIS-B CPDLC TIS-B CPDLC TIS-B CPDLC

A X X X (X) X X B X X X X C X X D X X

Table 3-21: Pilots’ executed Test Runs in the ATTAS Test Aircraft

3.2.1.6 Metrics and Indicators As real experiments with real baseline comparisons cannot be done in the field, the test focussed on the assessment of the technical feasibility of the new A-SMGCS services, concentrating on those aspects that could not be covered yet by the real time simulation trials, mainly the GTD fed by TIS-B and the exchange of real TAXI-CPDLC messages in shadow mode trials in parallel with real traffic. Therefore interviews were carried out at the end of the day to allow the pilots to address aspects of the technical and as far as possible operational feasibility of the new A-SMGCS services.

3.2.2 Results to Technical Feasibility (OST) This chapter describes the main results in terms of “technical feasibility” gained by the on-site trials. Results mainly base on observations by the test team and open discussions with the pilots after the trials. All technical results are documented in the 2-D2.5.2 “Assessment Report for the Ground CPDLC Clearance function PRG/MXP” [17].

3.2.2.1 Ground Traffic Display (GTD) • After tuning of the TIS-B function it was proven that TIS-B is capable to fulfil the needs to

provide the complete traffic situation of the airport for the GTD. DLR test pilots were able to use the display during taxiing on PRG airport in its full functionality resulting in general acceptance of the GTD. The update rate was sufficient enough to provide the pilots with the complete current traffic situation.

o Partially, during the first TIS-B trials the integration of the TIS-B was unreliable – some false targets were displayed and own A/C symbol was skipped on the displayed taxi route.

• The system worked very well and will be a great step ahead in taxi procedures according to the pilots.



• Automatic switching from AERODROME to Flight NAV mode is highly desirable (approx. 100 ft after lift off and vice versa during approach and landing) to improve availability of NAV information.

• According to the pilots, mentioned comments are associated with the experimental installation and could be easily solved more or less with today’s technology.

• One pilot stated o that he is not in favour of the exocentric 3-D-esque mode of the EMM. In his opinion

there is no added value and prefers to use the 2-D mode. o that the use of colours on the EMM should be reconsidered.

• In general it is very appreciated by the pilots and there were no further comments to it.

3.2.2.2 TAXI-CPDLC • In general, the TAXI-CPDLC service worked perfectly. There were some initial problems

with respect to doubled EFS caused by following aerodrome circuits and one link lost due to the ATTAS leaving of the VDLm2 range, but this could be solved easily.

• In detail, there were five aerodrome circuits performed by the ATTAS test aircraft:

o 1st circuit, 2008-11-18, 14:20 – 15:00 unintentionally there were created to departure EFS for the ATTAS (call sign

D-ADAM), which caused trouble with the log on – the log on was done on the former created but already deleted EFS – after retrieving the deleted EFS and binning the actual EFS this problem was solved and the test could started

the transmission of all TAXI-CPDLC clearances run perfectly during the flight phase, when the ATTAS was 15,5 miles the log off was

performed With inbound the ATTAS logged on again (green log on sign was seen on the

D-ADAM inbound EFS) After crossing RWY13/31 and handover to the regular GEC, the regular GEC

instructed “follow data link” and the test bed GEC provided the inbound taxi route to the final stand by TAXI-CPDLC

o 2nd circuit, 2008-11-19, 12:20 – 13:15

all TAXI-CPDLC outbound clearances worked perfectly (see also a screenshot of the EFS display)

during the aerodrome circuit the link was lost as the ATTAS distance to the VDLm2 antenna was too far

with inbound the log on failed

o 3rd circuit, 2008-11-19, 13:15 – 15:00 Call sign ADAM was chosen instead of D-ADAM in order to avoid problems

with the log on All out- and inbound clearances worked perfectly

o 4th circuit, 2008-11-27, 12:35 – 13:25

All out- and inbound clearances worked perfectly

o 5th circuit, 2008-11-27, 13:30 – 14:10 All out- and inbound clearances worked perfectly

Pilots made additional comments:

• The system worked technically very well and will be a great step ahead in taxi procedures. • In general the principle and the concept are appreciated by the pilots.



• Nevertheless, in their opinion, any CDU or keyboard will improve work with listing and sending messages on CPDLC - compared to the use of the CDTI.

• A “CREW MESSAGE” amber light for better crew awareness was missing in the cockpit/on the front panel.

• According to the pilots, mentioned comments are associated with the experimental installation and could be easily solved more or less with today’s technology.



4 Surface Movement Awareness and Alerting System SMAAS; Traffic Conflict Detection; Ground-Air Data Base Upload (TUD)

4.1 Real Time Simulation Results

4.1.1 Set Up

4.1.1.1 The validation platform The fixed-base Research Flight Simulator of TUD’s Institute of Flight Systems and Automatic Control is a highly modular and configurable research simulator, featuring a sophisticated collimated visual system consisting of a three-channel retro-projection with a viewing frustum of 180° horizontally and ±20° vertically. Pilots perceive the resulting image, which seems to be located at infinity, through a mirror. Consequently, refocusing occurs whenever pilots change their view from head-down activities inside the cockpit to the outside visual, and this approach guarantees an excellent approximation of reality for human factors evaluations [16]. The cockpit is not an exact reproduction of the flight deck of a specific aircraft, but deliberately kept at the more generic level of a modern “glass cockpit” with two flight crew members. The inside dimensions of the cockpit correspond to those of a modern widebody aircraft of the Airbus A330/340 family. While the flight simulation employs the flight mechanical model of an Airbus A300 B4, a fly-by-wire flight control system with sidesticks was chosen because it represents the state of the art and allows an unobstructed view of the flight guidance displays.

Figure 4-1: View of the TUD fixed-base Research Flight Simulator The cockpit features all primary and secondary controls; in some cases actual aircraft parts, such as an original A320 Flight Control Unit (FCU) and genuine thrust levers, contribute to enhance the level of immersion and realism. In other cases, as for the MCDU, parts obtained from a commercial supplier of flight simulation equipment have been used, and even some developments of the Institute were installed. As an example, the simulator features active side-sticks, which can also be operated in a standard mode. Flight guidance and system displays are presented on large 15” LCD screens, which can either, be arranged in portrait or landscape mode. Cockpit Configuration for the experiment



The First Officer (CM-2) crew station will be used for evaluation, mainly because this position is already equipped with an MCDU hardware mock-up. The two LCD displays for PFD and ND are arranged in portrait configuration, thus resembling the A380 display arrangement. The outer screen is used to display a standard Airbus style PFD. The inner screen will feature a conventional Airbus style ND supplemented by the Airport Moving Map and additional HMI features as described subsequently. Display control will be achieved via the conventional EFIS Control Panel, which features a workaround for the ranges below 10 nm: When selecting the constraints (CSTR) button, pilots can use the QNH selector to control AMM range.

4.1.1.2 Objectives and Hypotheses Introducing a new display system in the cockpit with all of its information could seduce the pilots to increase their head down time, due to the overload of information presented by the system while the pilots needs undue time to recognize the important information. Objective O1: Verify that the introduction of SMAAS does not tend to result in more Head down time for the pilots. Hypothesis H1-1: The SMAAS does not cause excessive extra head down time for the crew. With the introduction of the SMAA System in the cockpit displaying most of the important information for the crew directly on the ND, the crew could start trusting the system too much. Mainly the display of traffic information is dependent on the availability of corresponding equipment in the other aircraft (ADS-B) or on the airport (TIS-B). The crew must be aware at all times that the SMAAS only gives out the information available to the cockpit – and therefore not trust the system completely. Objective O2: Assess the confidence the pilots have in the SMAAS. Hypothesis H2-1: The crew maintains a good level of confidence in the System – the crew trusts what it sees on the display but stays aware of the limits of the System. The increase in situation awareness comes along with the increase in safety, because crew failures will be minimized. To ensure that the SMAAS increases situation awareness with its presentation of surface restrictions as well the associated alerts, Objective 3 was defined. Objective O3: Verify that the presentation of surface restrictions as well as alerts in addition with the Airport Moving Map will increase situation awareness. Hypothesis H3-1: Presenting closed runways and other pertinent runway or airport restrictions from NOTAM or similar sources is an essential feature to increase situational awareness on an airport moving map. The alerting features associated are operationally relevant. Similarly, to ensure that the SMAAS increases situation awareness with its presentation of traffic information as well as the associated alerts – the Traffic Conflict Detection function, Objective 4 was defined. Objective O4: Verify that the presentation of traffic information as well as the associated alerts in addition with the Airport Moving Map will increase situation awareness. Hypothesis H4-1:: Presenting traffic information is an essential feature to increase situational awareness on an airport moving map. The alerting features associated are operationally relevant. Objective 5 was defined to verify that the overall design principles that should be used in the developing of aircraft systems will be adhered. Objective O5: Verify that the introduction of SMAAS follows the overall design principles used in the cockpit. Hypothesis H5-1: The HMI is consistent with existing standards/implementations. Taxiing along the airport belongs to the phases with the most workload for the pilots. Introducing a new system that will increase their workload in this phase is not desirable. Because it is unlikely to develop an information system that will have no effect on the workload it is desirable to develop a system with just a small increase of workload.



Objective O6: Verify that the introduction of SMAAS does not lead to any undue increase of workload. Hypothesis H6-1: The introduction SMAAS does not lead to any undue increase of workload.

4.1.1.3 Traffic Scenarios For the fixed base research simulator, seven different scenarios were used. The two first scenarios were used mostly to familiarize the pilot with the TUD Simulator and with the SMAAS System. Two locations were chosen for the scenarios. Frankfurt Airport (EDDF) was used for the familiarization scenarios: FAM 01 and FAM 02 as well as the first two SMAAS scenarios: SMA 01 and SMA 02. The following three SMAAS scenarios take place at the Charles-de-Gaulle Airport (LFPG) in Paris: SMA 03 to SMA 05. The scenarios are described in detail in the following paragraphs.

4.1.1.3.1 FAM 01 This scenario was developed, to get the pilot familiar with the simulator and the Airport Moving Map Display. For this reason no traffic was provided, while the pilot had to taxi out from Gate A16 to Runway 25R at Frankfurt Airport (EDDF).

Figure 4-2: Scenario FAM 01

EMMA-FAM-01 Simulator & Airport Moving Map Familiarisation Remarks / ATC

Location Frankfurt/Main Airport (EDDF) Familiarisation scenario in daytime VMC, CAVOK

Objectives Primary: To familiarise the crew with the handling of TUD’s flight simulator and to acquaint them with the airport moving map functionality.

Secondary: To collect first comments on the overall HMI from the crew.

Duration: ca. 20 min

Background Fictitious airline flight from Frankfurt (EDDF) to Paris Charles-de-Gaulle (LFPG).

Crew Task Taxi from Gate A16 to RWY 25R via M, N, G1, A and D.

Display Configuration: Airbus Standard PFD and Airport Moving Map on ND

SMAAS Route (if applicable): M N G1 A D 07L.25R, activate “hold short”

Weather Daytime VMC, CAVOK



Traffic No traffic.

Start of Scenario Aircraft has been pushed back from parking position A16, the engines are running.

Scenario Script ATC instructs the crew to taxi to 25R via M, N, G1, A and D, holding short of E.

RWY 25R is marked as FMS RWY on the airport moving map.

The SID/flight plan is visualized on the ND.

The crew requests taxi instructions from ATC. Upon review/readback of the assigned route, the crew releases the parking brake and taxis to RWY 25R following the assigned route, with intermediate hold-short of E

The crew holds short of RWY 25R and is subsequently cleared for line-up and takeoff.

Alternative taxi clearance: M, A, D.

Controller: Lufthansa Four Tango Uniform, taxi to Holding Position Runway Two-Five Right via Mike, November, Golf One, Alpha and Delta.

Controller: Lufthansa Four Tango Uniform, proceed via Alpha and Delta, hold short of Runway Two-Five Right.

Controller: Lufthansa Four Tango Uniform, line up and wait Runway Two-Five Right.

Controller: Lufthansa Four Tango Uniform, cleared for takeoff Runway Two-Five Right, wind is calm, report airborne.

Subjective Measurements

None for this scenario (familiarisation).

Objective Measurements


4.1.1.3.2 FAM 02 In this scenario a first alert will be provided. The pilot had to taxi from Gate A16 to Runway 07R. At the transit from Taxiway A to Taxiway W, which is the assigned taxi route, the pilot was asked, to stay on Taxiway A and head to Runway 18, which has been marked as a closed Runway, slowly. After that he had to resume the assigned taxi route (Figure 4-3).

Figure 4-3: Scenario FAM 02

For this scenario no traffic movement will occur at the airport. EMMA-FAM-02 Simulator & Airport Moving Map Familiarisation Remarks / ATC

Location Frankfurt/Main Airport (EDDF) Familiarisation scenario in night VMC, CAVOK



Objectives Primary: To familiarise the crew with the handling of TUD’s flight simulator and to acquaint them with the airport moving map functionality.

Secondary: To collect first comments on the overall HMI from the crew.



Crew Task Taxi from Gate A16 to RWY 07L via M, A, W and R.

Divert from assigned route and proceed straight onto closed RWY 18.


SMAAS Route (if applicable): M A W R 07R.25L

Weather Night VMC, CAVOK

Traffic No traffic.


Scenario Script ATC instructs the crew to taxi to 07R via M, A, W and R.

At any rate, the assigned taxi route will be displayed on the airport moving map.



The crew releases the parking brake and follows the assigned route. The crew, however, does not make the turn into W, but continues straight onto the completely closed RWY 18.

Alternative taxi clearance: M, N, L.

Controller: Lufthansa Four Tango Uniform, taxi to Holding Position Runway Zero-Seven Left via Mike, Alpha, Whisky and Romeo.





4.1.1.3.3 SMA 01 The assigned Route is from Gate A16 to Runway 25L (Figure 4-4). Runway 18 is completely closed, while Runway 25R, which must be crossed by the pilot is in use only as Taxiway. Neither a highlighting nor the alerting systems (SMAAS) are enabled.



Figure 4-4: Scenario SMA 01

EMMA-SMA-01 Taxiing to RWY 07R and departure Remarks / ATC

Location Frankfurt/Main Airport (EDDF) Line-oriented scenario

Objectives Primary: Line-oriented scenario without the use of the ePIB/SMA function: no display of NOTAM information, only read at the beginning of the scenario.

Secondary: Assessment of Situational Awareness with respect to NOTAM information.



Crew Task Taxi from Gate A16 to RWY 25L via M, N, F, C and D.


SMAAS Route (if applicable): M N F * F C D 07R.25L


Traffic Up to 10 traffic items


Scenario Script The crew reads through a stack of NOTAM information including the complete closure of RWY 18 and the use of RWY 07L only as taxiway (not take-off/landing).

ATC instructs the crew to taxi to 07R via M, N, F, C and D.

The crew releases the parking brake and follows the assigned route.


Controller: Lufthansa Four Tango Uniform, taxi to Holding Position Runway Twenty-Five Left via Mike, November, Foxtrot, Charlie and Delta.


Post-run and debriefing questionnaires.

Post-run and debriefing pilot comments.


None for this scenario.



4.1.1.3.4 SMA 02 This scenario (Figure 4-5) is the same as the previous scenario (SMA 01), except, that the highlighting of active or closed Runways and the alerting system is enabled.


During the taxiing, the pilot gets the information, that Runway 25R is not longer in use just as a Taxiway but can be used as a regular Runway. The pilot should use this information, to interact with the MCDU to unlock the Taxiway restriction from Runway 25R. After that he should proceed along his assigned route. EMMA-SMA-02 Taxiing to RWY 07R and departure Remarks / ATC

Location Frankfurt/Main Airport (EDDF) Line-oriented scenario

Objectives Primary: Line-oriented scenario with assessment of the ePIB/SMA function on a smaller/more familiar airport.




Crew Task Taxi from Gate A16 to RWY 25L via M, N, F, C and D.


SMAAS Route (if applicable): M N F * F C D 07R.25L




Scenario Script The crew loads NOTAM information including the complete closure of RWY 18 and the use of RWY 07L only as taxiway (not take-off/landing) with the ePIB on the RFMS.


Controller: Lufthansa Four Tango Uniform, taxi to Holding Position Runway Twenty-Five Left via Mike, November,



ATC instructs the crew to taxi to 07R via M, N F, C and D.

At any rate, the assigned taxi route will be displayed on the airport moving map.



The crew releases the parking brake and follows the assigned route.

Foxtrot, Charlie and Delta.





None for this scenario.

4.1.1.3.5 SMA 03 This scenario is related to scenario SMA 01. The crew had to taxi from Gate Z4 to Runway 09L, while Runway 09R and Runway 08L/26L are completely closed. Neither a highlighting nor the alerting system is enabled (Figure 4-6). The pilot can interact with the MCDU but does not get any feedback of his interaction visually on the displays since the highlighting and alerting system – SMAAS is not on.


EMMA-SMA-03 Taxiing to 09L and departure Remarks / ATC

Location Paris Charles-de-Gaulle (LFPG) Line-oriented Scenario

Objectives Primary: Line-oriented scenario without the use of the ePIB/SMA function: no display of NOTAM information, only read at the beginning of the scenario.





Background Fictitious airline flight from Paris Charles-de-Gaulle (LFPG) to Frankfurt (EDDF) or Amsterdam (EHAM).

Crew Task Taxi out to RWY 09L, perform takeoff and fly SID. Display Configuration: Airbus Standard PFD and Airport Moving Map on ND SMAAS Route (if applicable): NA1 N B DB D K2 Z2 09L.27R



Start of Scenario After a fictitious pushback from Gate Z4 at Terminal 1, the aircraft is on taxiway NA1.

Scenario Script The crew reads through a stack of NOTAM information including the complete closure of RWY 09R/27L and the use of RWY 08L/26R only as taxiway (not take-off/landing).

The crew is instructed to taxi to RWY 09L via NA1, N B, DB, D, K2, Z2 and to hold short of RWY 09R.

If the crew notices that RWY 09R is completely closed, ATC excuses itself and says: yes this runway is now used as a taxiway.

The crew is cleared for take-off on RWY 09L.

Alternative taxi clearance: -

Controller: Lufthansa Four Tango Uniform, taxi to Holding Position Runway Zero-Niner Left via November, Bravo, Delta-Bravo, Kilo Two and Zulu Two, hold short of Runway Zero-Niner Right. Controller: Lufthansa Four Tango Uniform, crossing Runway Zero-Niner Right is approved, hold short of Runway Zero-Niner Left. Controller: Lufthansa Four Tango Uniform, line up and wait Runway Zero-Niner Left. Controller: Lufthansa Four Tango Uniform, cleared for takeoff Runway Zero-Niner Left, wind is calm, report airborne.


Post-run and debriefing questionnaires Post-run and debriefing pilot comments


Does the crew notice 09L: taxiway?

4.1.1.3.6 SMA 04




This scenario is similar to scenario SMA 03, except the active state of the highlighting and alerting system. Like in SMA 03 the pilot should interact with the MCDU to unlock the complete closure of Runway 09R (Figure 4-7) EMMA-SMA-04 Taxiing to 09L and departure Remarks / ATC

Location Paris Charles-de-Gaulle (LFPG) Line-oriented Scenario

Objectives Primary: Line-oriented scenario with assessment of the ePIB/SMA function on a larger/less familiar airport.




Crew Task Taxi out to RWY 09L, perform takeoff and fly SID. Display Configuration: Airbus Standard PFD and Airport Moving Map on ND SMAAS Route (if applicable): NA1 N B DB D K2 Z2 09L.27R




Scenario Script The crew loads NOTAM information including the complete closure of RWY 09R/27L and the use of RWY 08L/26R only as taxiway (not take-off/landing)

Alternative taxi clearance: -

Controller: Lufthansa Four Tango Uniform, taxi to Holding Position Runway



on the ePIB.

The crew is instructed to taxi to RWY 09L via NA1, N B, DB, D, K2, Z2 and to hold short of RWY 09R.

If the crew notices that RWY 09R is completely closed, ATC excuses itself and says: yes this runway is now used as a taxiway.

The crew is cleared for take-off on RWY 09L.

Zero-Niner Left via Bravo, Delta-Bravo, Kilo Two and Zulu Two, hold short of Runway Zero-Niner Right. Controller: Lufthansa Four Tango Uniform, crossing Runway Zero-Niner Right is approved, hold short of Runway Zero-Niner Left. Controller: Lufthansa Four Tango Uniform, line up and wait Runway Zero-Niner Left. Controller: Lufthansa Four Tango Uniform, cleared for takeoff Runway Zero-Niner Left, wind is calm, report airborne.


Post-run and debriefing questionnaires Post-run and debriefing pilot comments


4.1.1.3.7 SMA 05 In this scenario, the pilot had to taxi from Gate Z4 to Runway 09R. The visibility was set to IFR conditions. At Runway 09R the crew was instructed, to line up after another aircraft. This aircraft was simulated as being without an ADS-B transponder. Thus it was present in the outside view but not shown on the Airport Moving Map Display, while other traffic was shown.


The first aircraft, being number one for takeoff, started Takeoff, rejected it and remained with damaged tyres on the Runway. Meanwhile the pilot got his Takeoff clearance. EMMA-SMA-05 Runway Incursion on RWY 09R

(Initial takeoff phase) Remarks / ATC

Location Paris Charles-de-Gaulle (LFPG) Event scenario (Taxiway and Runway Incursion)

Objectives Primary: Evaluation of Runway Incursion Alerting vs. Traffic display only

Secondary: Assessment of airport moving map and traffic display.





Crew Task Taxi out to RWY 09R, perform takeoff and fly SID. Display Configuration: Airbus Standard PFD and Airport Moving Map on ND

SMAAS Route (if applicable): NA1 N B BD Y1 09R.27L



All known Traffic is displayed. One aircraft (AFR7780) is simulated as not having an ADS-B Transponder and is thus only visible in the outside view and not “known” by either the Displays or the Alerting System.

TCD Configuration Runway and Taxiway alerts enabled


Scenario Script The crew is instructed to taxi to RWY 09R via NA1, N, B, BD and Y1, and to hold short of RWY 09R.

The crew is instructed to line up, and No. 2 for takeoff, the other aircraft (AFR7780) is cleared for takeoff, but rejects takeoff at 130 kts due to an engine failure and remains on the runway with damaged tyres.

The alert will occur as soon as the ownship crew commences takeoff. If they notice that the RWY is not clear, they should be encouraged to commence takeoff all the same in order to demonstrate the alert.

Alternative taxi clearance: NA1, N, B, DB, D, DY, Y3.

Controller: Lufthansa Four Tango Uniform, taxi to Holding Position Runway Zero-Niner Right via November, Bravo, Bravo-Delta and Yankee One.

Controller: Lufthansa Four Tango Uniform, line up and wait Runway Zero-Niner Right, You are Number Two for takeoff.

Controller: Lufthansa Four Tango Uniform, cleared for takeoff Runway Zero-Niner Right, wind is calm, report airborne.


Post-run and debriefing questionnaires

Post-run and debriefing pilot comments


4.1.1.4 Participants

4.1.1.4.1 Evaluation Crew A total of six male pilots, all from Lufthansa, participated in the trials. The mean age of these pilots was 33,8 years (between 27 and 41 years old) an they had a mean total number of flight hours logged in their entire career in commercial aviation of 6333h (between 1000h and 11000h). Among the pilots were two Captains, one Senior Flight Officers and three First Officers. Three of them had a Flight Instructor education. Three of these pilots are flying an Airbus 320 family (A318, A319, A320, A321) airplane as their current type, one a Boeing 737 classic version (-300, -500) and two indicated a Boeing 747-400 as their current type.



One of the pilots had experience with airport moving map displays from previous campaigns at TUD. The pilots were asked to rate their familiarity with the airport at which the tests took place on a six-step scale with text statements, ranging from “It is my first time here” to “I know the layout of this airport almost better than that of my flat”. All of the pilots participating in the scenarios at Frankfurt (EDDF) named, that they are familiar (“I know this airport very well.”) with this airport. At Paris (LFPG) most of them indicated, that they had been several times there. Just two of them stated that they were familiar with the airport or knew it very well.

4.1.1.4.2 Assessment Team The core assessment team for all the trials consisted of two TUD research scientists. These two research scientist were systems engineers who participated in the functional and HMI design of the Surface Movement Awareness and Alerting System (SMAAS) and who were also responsible for the creation and extension of the questionnaires used in the experiment. They acted as the experiment leaders, did the briefing as well as the familiarisation, took down pilots’ comments and assisted pilots in filling the questionnaires. Furthermore, in the scenarios were it was necessary one of the scientists took the role of an ATC controller, giving the crew the instructions concerning taxi route clearances as well as take off and line up.

4.1.1.5 Schedule of Executed Test Runs The pilots came to TUD for the whole day and were first invited to fill out the Pilot-Intake-Questionnaire [6]. They were then briefed on the motivation and the functionalities of the Surface Movement Awareness and Alerting with a short power-point presentation. After a brief exchange with the attending pilot concerning the presentation, he was brought to the fixed-base Research Simulator. The two familiarization scenarios at Frankfurt Airport (EDDF) were run as described in chapter 2.1.3.2. Then the two SMAAS scenario taking place in Frankfurt were run before the lunch break. During the break, one of the TUD scientists prepared the simulator for the next scenarios taking place in Paris (LFPG). After the lunch, the pilot resumed the run of the scenarios and did scenarios SMA 03 through to SMA 05. After that, the pilot was asked to fill out the questionnaires described in chapter 2.1.5.2 and a debriefing session followed. The typical course of an evaluation day was the following, where T0 was an individual start time for each pilot. The time is given in minutes.

T0 - T0+30 Presentation - Briefing

T0+35 - T0+55 Training

T0+60 - T0+150 System evaluation – EDDF scenarios

T0+150 - T0+195 Lunch

T0+195 - T0+285 System evaluation – LFPG scenarios

T0+285 - T0+345 Debriefing

4.1.1.6 Metrics and Indicators In general, there are two different methods of gathering information for a validation process. The first method consists in the quantitative measurement of some given indicators for the system and the objective assessment of the gathered data. The second method relies on the qualitative assessment of



subjective measurements, such as user opinions, comments, feedbacks.

Since the objectives concerning the Surface Movement Awareness and Alerting System that was to be assessed during the Real Time Simulation trials are all linked to subjective notions such as the crew’s situation awareness, the perceived workload or the human machine interface, it was decided to use only the second method: qualitative assessment of questionnaires and debriefing interviews [6].

During the scenarios the pilots’ comments were documented by one of the TUD scientists. After the exercise, data was collected by means of several debriefing questionnaires and a debriefing interview. The gathered data was then methodically analysed and evaluated.

4.1.1.6.1 Debriefing Questionnaire A set of three dedicated debriefing questionnaires was used as well as two Cooper Harper Rating Scale sheets. The dedicated questionnaires can be found in the EMMA2 document 2-D616 Validation plan – airborne in chapter 9.6.

Dedicated questionnaires The detailed questionnaires enabled the pilot to rate individual elements using 7 or 10 step Likert scales. The first questionnaire details the pilot’s impressions on the airport moving map. The second questionnaire deals with the surface traffic display (CDTI) used in the system. The last questionnaire enables the pilot to state his views on the Surface Movement Awareness and Alerting System as a whole, including mostly the display of operational, clearance and traffic information, as well as the associated alerting mechanisms.

Cooper Harper System (CHS) The CHS is used by starting at the lowest maturity level (i.e. the question “Is the system usable?” at the bottom left-hand corner) and progressing up the table while the question can be answered with a “yes”. When a question has to be answered with a “no”, the actual pilot rating is determined by the detailed nature of the identified deficiency, as defined on the right hand side of the table. This process results in the evaluator selecting a rating in a ten-point scale ranging from 10 (the system is unusable) to 1 (the system is fully mature).



Figure 4-9: Cooper Harper System

Two Cooper Harper rating scales were used. One was used to assess the Ground to Air Database Upload Function including the electronic Pre-flight Information Bulletin (ePIB) and more particularly the MCDU interaction used in the trials. The other Cooper Harper rating scale was used to assess the Surface Movement Awareness and Alerting System as a whole.

4.1.1.6.2 Debriefing Interview The aim of the debriefing interview was to go deeply into the key points noticed in the observations during the trials or in the debriefing questionnaires. It helped to bring out the operational improvement possibilities of the system through different human factor aspects:

- System usability - Confidence in the system - Situation awareness

The debriefing interview also helped the team understand why some pilots may have a diverging answer from the majority of pilots, or from the expected answer. This was used intensively to understand the questionnaire results in the analysis phase.



4.1.2 Results to Operational Feasibility (RTS)

4.1.2.1 Operational Feasibility

4.1.2.1.1 Results The Operational Feasibility of the Surface Movement Awareness and Alerting Systems was assessed with detailed questioning of the pilots concerning the user acceptability of the system and also concerning their impressions on the Human Machine Interface used in the trials. The following figures show the given answers for six pilots from the RTS and two further pilots from the OST on a scale from 1 (disagreement) to 10 (agreement).

4.1.2.1.1.1 Human Machine Interface To assess the HMI, the satisfactory of the AMM display colours was evaluated. Figure 4-10 shows that most of the pilots are satisfied with the colours on the display. Two of the pilots mentioned that one point that could be improved is the contrast for a FMS chosen Runway (the FMS chosen Runway is white outlined while the Runway is light grey). One of the pilots remarked that the red alert which overlays a runway where the conflict is predicted, should stay until the danger is not a factor anymore. This answer was taken up at the Prague trials and resulted by an improper work of the software at this time.

Figure 4-10: AMM display colours are satisfactory

Also the interpretation of the different colour codes (Figure 4-11) or symbols (Figure 4-12) affects the HMI. In general the interpretation of both easily pleased the pilots. Just for the symbols, improvements could be made. One pilot desires that traffic on runway should be displayed directly in red. In his opinion that would increase situational awareness. One pilot felt that the different symbols for traffic with know and traffic with unknown traffic should not be different. There should be a difference by the traffic type.



Figure 4-11: The different colour codes used are easy to interpret

Figure 4-12: The used symbols are easy to interpret

Also the information arrangement influences the HMI feeling for the pilots. In Figure 4-13 the answers are shown. A trend to the agreement with the current arrangement is visible while two pilots are not fully satisfied with the arrangement. But there had been no additional comments by the pilots about the improvement of the current system.

Figure 4-13: Information is arranged conveniently



One point for the result shown in Figure 4-13 could be that some pilots do not fully agree with the statement that different types of information are easy to find which is shown in Figure 4-14.

Figure 4-14: Different types of information are easy to find

The interaction with the system is also an important point for HMI assessment. Most of the pilots thought that the interaction through the use of the MCDU is satisfactory for them. This shows that the idea of using the MCDU for this type of interaction with the electronic Pre-Flight Information Bulletin goes in the right direction.

Figure 4-15: AMM interaction concept is satisfactory

Figure 4-16 shows that the amount of the information presented on the AMM is not too large. Additionally essential information and information that could be removed if necessary should be noted. Just one pilot answered this question by noting that taxiway names are essential information. Two pilots kept it more generally and noted that this depends on the traffic amount and that the display is a very nice adaption of info with range selection.



Figure 4-16: AMM information amount is too large

A main problem using computer displays with textual information on it could be that this information could not be readable well. Therefore Figure 4-17 shows that most of the pilots thought that on the tested display the textual information are very well readable.

Figure 4-17: Textual information is very well readable

Pilots are familiar with ND which just displays the Runways on it. Therefore the use of an AMM on the ND could lead to an overload of information for some pilots. Error! Reference source not found. shows that just one pilot tend to see an overload of information with the use of AMM. His reason for that is noted below: “Display on ND is very good, but overload is possible when a route (flight) is inserted.”



Figure 4-18: With AMM the ND is overloaded with information

As for safety the closed runway symbology must be intuitive for the HMI as well. Figure 4-19 shows that the pilots thought that the symbols used here are intuitive.

Figure 4-19: Closed Runway symbology is intuitive and easy to understand

Also for the colour presentation in Figure 4-20 the thoughts are nearly the same as in Figure 4-19.

Figure 4-20: Closed Runway colour presentation is consistent with overall cockpit concept



To complete the view of symbology and colour presentation Figure 4-21 shows the opinions of the pilots according the closed taxiway presentation of information. We have exactly the same result here as we have for Runways.

Figure 4-21: Closed Taxiway representation is consistent with the closed Runway concept

As mentioned before one pilot thought about a different symbol for traffic with know and traffic with unknown heading. Figure 4-22 aims to this consideration and the pilots’ opinion is visible in this figure as well. Although he has this opinion he thought that the traffic symbol is acceptable. That was because he had a briefing before the trails started, presenting him these symbols. And in all day business he will get familiar with it. It was also mentioned before that the colour of the traffic symbols is acceptable but one pilot noted that it will be easier for him if traffic on a runway will turn red (Figure 4-23).

Figure 4-22: The shape of the traffic symbol is intuitive and acceptable



Figure 4-23: The colour of the traffic symbol is intuitive and acceptable

4.1.2.1.1.2 User Acceptability For user acceptability it is essential that the user will see an increase in safety, situational awareness and that the presented information is consistent with existing information. One essential point to gain user acceptability is that the system is easy to handle. A lot of pilots agree with the statement in Figure 4-24 that the AMM is easy to handle, although the interaction is not the same as it probably will be in future aircrafts.

Figure 4-24: Handling of the AMM

As mentioned in the Safety paragraph the number of selectable ranges is sufficient for most of the pilots (Figure 4-25). They can comfortably choose their preferred range setting or change it during taxiing to fulfil the requirements during different phases (taxi along taxiway, approaching runway, etc.)



1

2

3

4

5

6

7

frequency

21. The number of selectable ranges is sufficient.

Figure 4-25: The number of selectable ranges is sufficient

Also the display size is acceptable for them. Although there is a larger screen in the simulator (see chapter 2.1.1.2) some pilots thought that a larger screen could be better because the amount of information will increase.

1

2

3

4

5

6

frequency

22. The size of the display is acceptable.

Figure 4-26: Display size is acceptable

Also the AMM is accepted by the pilots as was already mentioned in HMI paragraph.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

23. The ND is overloaded with information if the Airport Moving Map is added.




0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

2) I like the way of presenting the FMS‐selected runway information on thdisplay.

Figure 4-28: Presentation of FMS-selected runway on the display

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

14. This traffic presentation is definitely an improvement over the systemwe have today.

Figure 4-29: Traffic presentation is an improvement over the systems we have today

1

2

3

4

5

6

frequency

22. It is essential that aircraft approaching the runways and aircraft taking oare displayed as well.

Figure 4-30: It is essential that aircraft approaching the runways and aircraft taking off are displayed as well?



4.1.3 Results to Operational Improvements (RTS)

4.1.3.1 Safety (HLO_2) Safety depends on situational awareness and workload. So a full picture about safety could only occur after the survey of these two points. At the beginning stands a general question about the support the AMM gives. The pilots feel heterogeneously supported by the system compared to the current system. Although a majority of the pilots totally agreed the AMM gives them support they miss with current systems, some pilots were not as enthusiastic. It must be mentioned that the pilots on the trials used different systems. Some of the Lufthansa pilots use the RAAS function (Runway Awareness Advisory System) and thus didn’t have the same baseline system to compare with as the others.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

2. The Airport Moving Map gives me support I miss with current systems

Figure 4-31: Does AMM give missed support?

As mentioned, a good situational awareness is necessary to operate safely. To allow an improvement in situational awareness with the Surface Movement Awareness and Alerting System, the number of selectable ranges must be sufficient so that the pilots can build a good mental picture of all the information depicted on the displays. This is all the more necessary when alerts come so that the pilot has a sufficient overall view of the situation.

1

2

3

4

5

6

7

frequency

21. The number of selectable ranges is sufficient.

Figure 4-32: The number of selectable ranges is sufficient



Figure 4-32 shows that the number of selectable ranges was deemed sufficient for the pilots to have all information they needed.

But not only should all necessary information be available. Also a quick recognition of that information is essential. That means that the pilots should get important information with a short head down time. Therefore an overloaded ND is not desirable – the pilot should not have to filter out unnecessary information. This is all the more important to ensure that an unnecessary overload of information doesn’t diminish the reactivity to an alert.

Most of the pilots thought that the tested ND implementation is not overloaded with information. But they mostly noted that it will be necessary to limit the objects that will be displayed on the ND – particularly the traffic at some big airports. It has to be noted here that the traffic simulated in the real time simulation was much less than it would have been in real life.

One pilot thought that the display is overloaded because in his opinion an overload of information could occur when a flight route is inserted in addition to the displayed information – but it is important to note here that the ranges at which the flight route can be observed, and the ranges used for the SMAAS are quite different. Indeed the AMM would be practically not readable when the pilot is studying his flight route and vice versa.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

23. The ND is overloaded with information if the Airport Moving Map is added.


4.1.3.1.1 Surface Movement Alerting When improvement of situation awareness isn’t enough to prevent a potential incident, alerting occurs in the Surface Movement Awareness and Alerting System. This alerting function’s goal is also to increase safety when the increase in awareness thanks to the Operational, Clearance, Traffic and Position Awareness Functions still isn’t enough to prevent wrong behaviour – or when another aircraft has an unexpected and potentially dangerous behaviour.

So for that reason, a question concerning an alert when attempting to take off from a runway other than the selected one in FMS was listed (Figure 4-34) Most of the pilots thought that it will be operationally relevant to have this alert because own mistakes will not lead to an endangerment of others.



1

2

3

4

5

6

7

frequency

20) An alert when attempting to take off from a runway other than the onselected in the FMS is operationally relevant.

Figure 4-34: Operational relevance of alerts when attempting to take off other than FMS selected

A similar question is whether the pilots want to be alerted when entering a runway that is completely closed or when trying to land on it (Figure 4-35).

The necessity of an alert was mostly agreed fully because in this case an error preceded and the pilots thought that they could roll or land on the runway.

1

2

3

4

5

6

7

frequency

24) I would like to have an alert when I am entering a runway that is completely closed, e.g. due to construction, or trying to land on.

Figure 4-35: Alert when entering closed runway or try to land on it

Also for closed Runway alerting in general was asked. That means not just for closed runways due to constructions that will cause damage in the a/c. Here also an alerting for operationally closed runways is demanded.

Figure 4-36 shows that the pilots thought similar to the preceding question but the relevance is not this high like it is for closed runway due to constructions.



1

2

3

4

5

6

frequency

25) This type of alert will make flying safer in the future.

Figure 4-36: This type of alert will make flying safer in the future

Also following correctly a clearance and route given by ATC is a safety feature. For that reason the taxi route cleared by ATC is taken into account in the system: the route is displayed in green on the AMM. The pilot can thus monitor his own position and compare it with the defined taxi route. If he leaves this taxi route the route starts to blink on the display to attract the attention of the pilot. Most of the pilots thought that this is a useful feature that increases safety Figure 4-37.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

28) The taxi route monitoring function is a useful feature; the blinking routattracts my attention to the display.

Figure 4-37: Taxi route monitor function is useful; blinking route attracts attention

Similar to a takeoff from a Runway other than the FMS selected, a takeoff from a taxiway bears a high risk. So an alert when a crew tries to take off from a taxiway will contribute a high value to flight safety. Most of the pilots thought the same and fully agreed with the statement (Figure 4-38).



1

2

3

4

5

6

frequency

31) Taxiway takeoff alerting is a valuable contribution to flight safety.

Figure 4-38: Taxiway takeoff alerting is a valuable contribution to flight safety

Preventing a takeoff from a taxiway just by displaying an AMM is not enough for some of the pilots as Figure 4-39 shows. That shows that an alert mentioned before is necessary to increase flight safety.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

32) If my moving map display shows that I am currently on a taxiway, I winever try to take off.

Figure 4-39: When AMM displays that I’m on taxiway I will never try to takeoff

The pilots’ comments about the improvement of the traffic presentation were mostly positive. Just one pilot was undecided. This pilot missed speed information for traffic items. This feature would probably lead to clutter if it was shown for all traffic, but could be implemented to show the speed of traffic going faster than the advised taxi speed, or for traffic in the immediate vicinity of ownship. The pilot particularly missed this information during an alert and felt it could, in that case, help him stay “in the loop”.

4.1.3.1.2 CDTI



0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

2. How would you rate the contribution of this traffic presentation to flighsafety, i.e. safety of ground operations?

Figure 4-40: Rate of the contribution of this traffic presentation to flight safety

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency


Figure 4-41: This traffic presentation is definitely an improvement over the systems we have today

In Figure 4-42 the pilots’ comments about the usage of the system especially at night prior to or during takeoff is listed. Most of them will use it as additional information. But mostly prior to takeoff or used by the second pilot. One pilot mentioned that he will never use it during takeoff because his eyes should be outside all the time and also the second pilot should watch the outside. If the runway is clear of traffic or not must be cleared before takeoff.

It was indeed shown in scenario SMA 05 that all the pilots looked outside during the take off and did not solely depend on what they saw in the CDTI. Indeed in this scenario one traffic item was simulated as not being equipped with ADS-B and was thus not assessed by the system: neither to be seen on the display nor to cause an alert. This traffic item was seen in the outside world and despite Low Visibility Conditions and the fact that no alert came from the system, the pilots still noticed the danger.

This is a key issue showing that CDTI can already be used to increase safety even if not all traffic items are properly equipped and confirms hypotheses H2-1.



1

2

3

4

5

6

frequency

20. Especially at night and when visibility is low, I would like to use this display prior to and during takeoff to check whether the runway is really cle

of traffic.

Figure 4-42: Use of CDTI in Low Visibility and Night to check Runway before Take Off

Although the danger of an overload of the display due to high information density is given, most of the pilots agree with the necessity to display approaching or aircraft taking off (Figure 4-43). That is because these aircraft are a real risk for them due to the high speeds involved in those flight phases.

1

2

3

4

5

6

frequency

22. It is essential that aircraft approaching the runways and aircraft taking oare displayed as well.

Figure 4-43: Aircraft approaching or taking off should essentially be displayed

4.1.3.1.3 Traffic Conflict Detection

1

2

3

4

5

6

7

frequency

18) Runway incursion alerting contributes to an increase in flight safety.

Figure 4-44: Runway Incursion alerting increases flight safety



In figure 4-44 we can see that the pilots all felt that Runway Incursion alerting will contribute to an increase in flight safety. This alerting function was deemed by the pilots as the most important part of the traffic conflict detection function.

0

1

2

3

4

5

6

frequency

19 ) W ith tax i route and runw ay‐re lated c leara nce (like lineup, taclearance) displayed on the movin g map, it is unlike ly tha t the re

b e a runw ay in cu rs ion .

Figure 4-45: Necessity of Runway Incursion Alerting despite augmented Situational Awareness

As shown in Figure 4-45, a majority of pilots thought that the Runway Incursion alerting function will stay necessary even if complete CPDLC upload to the displays is available, i.e. taxi route and runway-related clearances. This confirms the concept of the Surface Movement Awareness and Alerting System: the system must give all the information it has to the pilot so that the situational awareness increases. This leads in the best case to an absence of alerts. But in case the crew still isn’t fully aware of the situation and/or another aircraft makes a mistake, the alerting is still necessary.

0,5

1

1,5

2

2,5

3

3,5

frequency

23) Runway incursion alerting does not require a change of current procedures.

Figure 4-46: Runway incursion alerting doesn't require new procedures

Although all the pilots agreed that the runway incursion alerting function would increase flight safety, not all of them thought it could be introduced without changing the current procedures, as can be seen in figure 4-46. The alerts triggered by the system are informative (“runway incursion” callout) and thus do not give the pilots an instruction as to what they should do. Some pilots noted this should then be given through new procedures, to tell them how to act when the given alerts occur.



0,5

1

1,5

2

2,5

3

3,5

frequency

24. Traffic alerting on the ground should be limited to conflicting traffic onthe runway.

Figure 4-47: Traffic Alerting limited to Runway

Figure 4-47 shows that pilots largely differ on their opinions regarding traffic alerting outside the runway. While some fully disagreed that traffic alerting should be limited to conflicting traffic on the runway, some did agree with the sentence, while others had no particular opinion. All pilots agreed that this function is not as safety critical as the prevention of runway incursion, and found the system’s option to limit alerts on taxiways to a maximum of level 2 (master caution), while runway incursion alerting reaches level 3 (master warning). But some even found that it would cause nuisance alerts and would not increase safety, while others found the function useful.

4.1.3.2 Suitability of Behaviour and Working Performance (HLO_4)

4.1.3.2.1 Situation Awareness (LLO4.2) The pilots were asked about their feeling about the increasing of situational awareness provided by the Airport Moving Map. Figure 4-48 shows their answers. Most of the pilots appreciate the introduction of AMM concern in the increase of situational awareness. The question if it is possible to get lost on an airfield while using the AMM gives more adequate answers to situational awareness (Figure 4-49). Here the increase is not this distinct as shown in Figure 4-48. But this is because some of the experimental constrains could create such a meaning by the pilots. One pilot for example gave two answers. One he mentioned, and this is the answer that is given in this document, represents his opinion about the actuality of the AMM underlying database. This opinion shows a neutral attitude to the statement. But if the database will be up to date, as it must be to be established in all day airline operation, he will agree with the statement that it won’t be possible to get lost. To increase situational awareness it could be reasonable to highlight the FMS selected takeoff or landing runway. Nearly all pilots fully agree with that point because on complex airports this information will attract attention out of the user structures (Figure 4-50). Also the distinction of different closure levels is desired by the pilots to increase their situational awareness (Figure 4-51). Furthermore, the alerting mechanisms which occur when the pilot has lost the necessary situational awareness were deemed very useful by the pilots. Indeed all pilots agreed with the concept of alerting in case the pilot takes off on another runway as the one selected in the FMS (Figure 4-52). All the pilots also agreed that they would like to be alerted in case they were either landing on or attempting to enter a completely closed runway, e.g. because of constructions works (Figure 4-53). To know the own route to prevent search for them on maps, etc is essential for situational awareness (Figure 4-54). The pilots agree with it because it could be a problem for pilots which are new at an airport to find the correct taxiways. The pilots mostly do not fully agree with the statement, because at



the time the pilots are familiar with an airport the taxi route will just help them in critical situations and is not mainly responsible for increasing situational awareness because the pilots have the layout in their mind.

1

2

3

4

5

6

frequency

1. The Airport Moving Map helps to increase situational awareness.

Figure 4-48: AMM increase situational awareness?

0,5

1

1,5

2

2,5

frequency

8. With the Airport Moving Map, it is not possible to get lost on an airfieldany more.

Figure 4-49: With AMM it is not possible to get lost on airfield

1

2

3

4

5

6

7

frequency

1) The presentation of the FMS‐selected takeoff or landing runway on theairport moving map is relevant from an operational point of view.

Figure 4-50: FMS selected runway presentation is operationally relevant



0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

7) The distinction of different closure levels is operationally relevant.

Figure 4-51: The distinction of different closure levels is operationally relevant

1

2

3

4

5

6

7

frequency

20) An alert when attempting to take off from a runway other than the onselected in the FMS is operationally relevant.

Figure 4-52: An alert when attempting to take off from runway other than the FMS selected is operationally relevant

1

2

3

4

5

6

7

frequency

24) I would like to have an alert when I am entering a runway that is completely closed, e.g. due to construction, or trying to land on.

Figure 4-53: Alert when entering a completely closed runway



0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency


Figure 4-54: Usefulness of taxi route monitoring function

In Figure 4-55 the answers to the overall increase in situational awareness with ground traffic on the moving map is listed. All pilots agree or fully agree with the statement to display traffic on AMM to increase situational awareness.

1

2

3

4

5

6

7

frequency

1. How would you rate your overall traffic situational awareness with grountraffic on the moving map?

Figure 4-55: Overall situation awareness with ground traffic on AMM

In Figure 4-56 the answers according to the question if the presented traffic labels are operational useful is given. This information could help the pilots in identifying traffic and/ or to have additional information about them. Most of the pilots wish to have traffic labels or they think that they are intuitive to use. The pilot who disagrees with the statement wants no abbreviations for the callsigns, because in his opinion the pilots will not know all abbreviations but they can identify a company so that the company name on the display will be more useful.



0,5

1

1,5

2

2,5

3

3,5

frequency

9. The presented traffic labels are intuitive and operationally useful.

Figure 4-56: Usefulness of presented traffic labels

Figure 4-57 strengthens the answers that were given in Figure 4-55. Most of the pilots see an improvement over the systems used today.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency


Figure 4-57: Traffic presentation as improvement for the pilots

One important point to show that the tested system increases situational awareness is shown in Figure 4-58. Here the statement is given that the display allows the pilot to establish a correspondence between the traffic outside and the traffic on the map. Most of the pilots agree with this statement just the transformation from a two-dimensional to a three-dimensional system takes some time for them so that not a full agreement for some of them is given. But the most important point is that if display clutter occurs or many items are next to one another, the identifying could be hindered.



0,5

1

1,5

2

2,5

3

3,5

frequency

15. The display allows me to establish a correspondence between the traffisee outside and the traffic on the map.

Figure 4-58: Correspondence between traffic outside and traffic on map

4.1.3.2.2 Workload (LLO4.1) Information that are not needed and which are displayed could increase the pilots’ workload because their head down time increases while they are searching for the relevant information. Most of the pilots disagreed with the statement that the system displays too much information, although they do not fully disagree due to the low traffic rate in the simulator (Figure 4-59). One pilot mentioned that the flight phases which are displayed are not really necessary for him.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

13. Sometimes information I do not need is displayed.

Figure 4-59: Sometimes information not needed is displayed



0,5

1

1,5

2

2,5

3

3,5

frequency

14. Different types of information are easy to find.

Figure 4-60: Different types of information are easy to find

0,5

1

1,5

2

2,5

frequency

15. The concept of interaction with the Airport Moving Map is satisfactory

Figure 4-61: AMM interaction concept is satisfactory

Figure 4-62 shows a similar diagram than Figure 4-59. Although the pilot in Figure 4-59 thought that there are some information which are not necessary he also thought that the amount of information are not too much.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency

16. The amount of information on the Airport Moving Map is too large.

Figure 4-62: AMM information amount is too large



As for safety the taxi route feature will also be useful to assure the workload will not increase by the system (Figure 4-63). It will be easier to follow the taxiway due to the mental picture the AMM will allow and the taxi route must not bear in mind all the time. Also the correct taxiway could be found more easily.

0,5

1

1,5

2

2,5

3

3,5

4

4,5

frequency


Figure 4-63: Taxi route monitoring is useful; blinking route attracts attention

Figure 4-64 shows the results to the question if the presentation of ground traffic increases the workload. Two pilots mentioned that there will be a small increase in workload because they will see traffic on their display that is probably out of their visual sight. That means that they get information they normally don’t have and they have to decide if they have to be aware of them or not.

0,5

1

1,5

2

2,5

3

3,5

frequency

3. Does the presentation of ground traffic increase your perceived workloa

Figure 4-64: Does presentation of ground traffic increase workload?

EMMA2



4.2 Operational Flight Trials Results

4.2.1 Set Up

4.2.1.1 The validation platform TUD’s Navigation Test Vehicle, a Volkswagen LT 28 van, is operated by the Institute of Flight Systems and Automatic Control and was originally conceived as a mobile test platform for the development and validation of navigation sensors and equipment. It features an array of high-quality sensors suited for precise navigation and a measurement data acquisition and logging system. Post-processing software allows the generation of highly precise reference trajectories with a CEP95 of 0.44 m based on sensor logs.

Figure 4-65: TUD Van during tests on closed runway 04/22 on Prague airport (LKPR), February 6th, 2006 Furthermore, as a by-product, the high-precision navigation solution also provides the prerequisites for the verification and validation of equipment used on aircraft, particularly in the domain of cockpit displays for surface movement, either in the form of software prototypes or real hardware. Consequently, TUD’s Navigation Test Vehicle can be used to simulate a taxiing aircraft and enables low-cost live field tests of such on-board functions in a real airport environment, and thus an assessment of the potential impact of factors such as database and navigation accuracy (e.g. IRS drift and GPS outages or multi-path effects) or the quality of data link date (e.g. ADS-B).

4.2.1.1.1 Navigation System A Honeywell H764 inertial reference system forms the core of the van’s precision navigation system, which is supplemented by an array of various GPS receivers ranging from off-the-shelf to high accuracy measurement models. The sensor mounting positions were measured within centimetre accuracy to correct errors caused by different sensor locations. An overview to the sensor equipment is given in Table 4-1. A Wiener filter is used for the implementation of an optimum sensor data fusion combining inputs from the H764 and a Novatel RT20 GPS receiver with D-GPS capabilities. To provide differential correction data to the on-board receivers, a portable reference station can be installed at points of which the exact position is known. The range of the D-GPS broadcasts depends on the surrounding environment and can vary between several dozens and some hundred meters, provided there is a free line of sight between rover- and reference station. As a backup, D-GPS can be obtained from the ALF

EMMA2



service, which is available within 600 km of Frankfurt/Main, Germany. For the EMMA trials, however, no D-GPS correction data were used, because the shakedown trials had shown that a sufficient real-time navigation precision in the order of a few meters could be achieved with GPS only.

Figure 4-66: Antenna mounting plate on the roof of TUD's Navigation Test Vehicle

4.2.1.1.2 Equipment In the back of TUD’s Navigation Test Vehicle, one row of seats has been removed to make room for two custom-built, interconnected racks that hold the hardware and computer equipment. Two inverters in the back are used to convert the 12V DC supplied by the vehicle’s electric bus to 250V/50 Hz AC required to drive the PC equipment.

EMMA2



Figure 4-67: Hardware and computer racks in the back of TUD’s Navigation Test Vehicle

In its standard configuration, TUD’s Navigation Test Vehicle is equipped with at least two PCs interconnected by Ethernet. The Navigation PC is responsible for collecting sensor data and the real-time calculation of the GPS/IRS sensor data fusion. It is equipped with both a MIL-1553 and an ARINC 429 card to interface with common aircraft avionics. The other PC is used for multiple purposes, such as data logging, acquiring traffic data and calculating a traffic data fusion or hosting software prototypes of on-board functions such as an Airport Moving Map. Space permitting, further PCs can be added and connected to the vehicle’s Ethernet network. ADS-B live traffic as well as TIS-B data issued from Prag-Airport during the OST in November can be received using a Filser RT60 ADS-B receiver (purchased via Eurotelematik) that sends out traffic data via Ethernet. Furthermore, the vehicle is equipped with Wireless LAN (WLAN), which can be used to access TIS-B traffic acquired through Frankfurt airport’s Cooperative Area Precision Tracking System (CAPTS), a Mode-S multilateration surveillance system developed by Thales and Sensis. The TIS-B over WLAN data link was developed for the ETNA project, where this feature was designed to bring traffic awareness to airport vehicles, particularly the airport fire engines. The passenger seat served as a very simple cockpit mock-up for the pilot evaluations and was therefore fitted with a 15" off-the-shelf LCD display (conventional PC monitor) and a Logitech trackball as a basic Crew Control Device (CCD), both provisionally mounted by the help of Velcro® tape. Although the LCD used is far from the ruggedized, high-contrast AMLCD screens used in aircraft cockpits, the display readability was very good even in back light conditions. Another limitation of the screen used is that its diagonal is a factor of 1.5 larger than the 6" x 8" screens currently used in the Airbus A380.

EMMA2



Figure 4-68: LCD screen installation in front of passenger seat

For evaluations on the manoeuvring area, the van was equipped with a turning light, and licensed drivers with the required radio permissions which were kindly supplied by Prague airport authorities. In Prague, additionally, an autonomous ADS-B transponder requiring only power supply was installed on the van’s roof.

Sensor/ HW Manufacturer Technical Data Remarks H764 (INU) Honeywell Position

<1.0 [nmi/hr] CEP velocity <1.0 [m/s] RMS Heading 0.1 [deg] RMS Pitch, Roll 0.05 [deg] RMS

INU medium accuracy

RT20 L1 (GPS) Novatel 12 channel L1 20 cm CEP50 (RT2 DGPS)

High accuracy

RT2 L1/L2 (GPS) Novatel 12 CH L1/L2 2 cm CEP50 (RT2 DGPS)

High accuracy

SK8 (GPS) Trimble 8 CH L1 2 m CEP50 (DGPS)

Low cost receiver

GPS 35 Garmin 12 CH L1 5 m RMS (DGPS)

Low cost receiver

Ashtech G12 Thales 12 channel L1 40cm CEP50 (DGPS)

GA certified

reference station RT2, Hermes

Novatel, Phillips

L1/L2 RTCM Type 59N

reference point required.

EMMA2



modem (DGPS, VHF link)

RTCA Type 7 VHF Modem: 459,57Mhz

Odometer Bosch 56 pulse/round ABS controller ALF receiver (DGPS )

DeTex RTCM-104 v2.0 122,5 kHz

Available within 600 km radius of Frankfurt

MS5535 (Barometer/ Thermometer)

Intersema -40°C to +125°C 1-12 bar

Parport PC Interface

RT60 Filser 1090 Mhz, Enhanced Squitter

ADS-B and TIS-B receiver

Table 4-1: Sensor equipment of TUD’s Navigation Test Vehicle

4.2.1.2 Traffic Scenarios The scenarios were divided in two packages of scenarios. One package was developed to provide scenarios with the General Aviation Aircraft available and the other simulating traffic with two ground vehicles. For the scenarios with the ground vehicles, the south east located area was used, containing runway 04/22 and the surrounding taxiways M and P. TUD Van simulates the own aircraft, with the flight crew for the trials. A second vehicle from the Prague Airport simulates other traffic. For the scenarios with the General Aviation Airplane, trying to land on a Runway, Runway 31 was used, with the TUD van trying to “takeoff” from Runway 13.

EMMA2



Figure 4-69: Overview of Prague airport (LKPR)

4.2.1.2.1 EMMA-VAN-01: Take Off RWY 04 – Traffic On/Approaching RWY This scenario describes a takeoff from the active Runway 22. TUD Van stands “cleared for takeoff” on Runway 22. Meanwhile a vehicle simulating other traffic taxiing over Taxiway P and crosses the stop bar to Runway 22 and roll onto the Runway. EMMA-VAN-01 Take Off RWY 04 – Traffic On/Approaching RWY Remarks / ATC

Location Prague Airport (LKPR)

Crew Task Van: Start T/O (simulated through flight phase) from RWY 04 at the level of TWY R.

Display Configuration: Airport Moving Map on ND

Traffic LKPR Vehicle: Roll onto RWY 04 using Taxiway P.

Scenario Script The crew is cleared for takeoff on RWY 04.

Traffic rolls onto RWY 04 while ownship is already in take off mode.

Controller: Lufthansa Four Tango Uniform, cleared for takeoff Runway Zero Four, wind is calm, report airborne.




EMMA2




Logging of Traffic position and Alerts.

Figure 4-70: Take Off RWY 04 – Traffic On/Approaching RWY

4.2.1.2.2 EMMA-VAN-02: Take Off RWY 04 without clearance This scenario describes a takeoff from Runway 22. TUD Van received the line up clearance on Runway 22. The crew starts taking off from Runway 22. EMMA-VAN-02 Take Off RWY 04 without clearance Remarks / ATC


Crew Task Van: Start T/O (simulated through flight phase) from RWY 04 at the level of TWY R.


Traffic Not relevant.

Scenario Script The crew starts taking off from RWY 04 after receiving the line up clearance

Controller: Lufthansa Four Tango Uniform, line up and wait Runway Zero Four.





Logging of Alerts.

EMMA2



Figure 4-71: EMMA-VAN-02: Take Off RWY 04 without clearance

4.2.1.2.3 EMMA-VAN-03: Collision Hazard with other Traffic on Taxiway This scenarios describes a taxiing situation, where the own airplane rolls in front of a second airplane. EMMA-VAN-03 Collision Hazard with other Traffic on Taxiway Remarks / ATC


Crew Task Van: Roll along Taxiway M. Display Configuration: Airport Moving Map on ND

Traffic LKPR Vehicle: Roll along Taxiway M.

Scenario Script The crew rolls on Taxiway M (middle) towards RWY 22.

Traffic rolls behind the van and starts getting closer.

If necessary, the speed of the traffic will be artificially increased (simulation) to cause the alert.






TUD van rolls along Taxiway M to Taxiway P via Taxiway L. The vehicle simulating the following airplane rolls along the same Taxiways and starts getting closer during the scenario.

EMMA2



Figure 4-72: EMMA-VAN-03: Collision Hazard with other Traffic on Taxiway

4.2.1.2.4 EMMA-VAN-04: Collision with other Traffic on Taxiway In this scenario the own airplane taxi along Taxiway L from East to West and is cleared to cross Runway 04. The intruder rolls along Taxiway R from East to West. At the intersection of Taxiway R and Taxiway L the intruder causes a collision hazard. EMMA-VAN-04 Collision with other Traffic on Taxiway Remarks / ATC


Crew Task Van: Roll along Taxiway L from East to West, having been cleared to cross RWY 04.


Traffic LKPR Vehicle: Roll along Taxiway R from East to West.

Scenario Script The crew rolls on Taxiway L towards RWY 04.

Traffic rolls on Taxiway R, causing a collision hazard at the intersection.






EMMA2



Figure 4-73: EMMA-VAN-04 Collision with other traffic on taxiway

4.2.1.2.5 EMMA-VAN-05: Crossing RWY 04 without clearance In this scenario TUD Van rolls on Taxiway L towards Runway 04. The crew has not received any clearance. The crew crosses Runway 04. EMMA-VAN-05 Crossing RWY 04 without clearance Remarks / ATC


Crew Task Van: Roll along Taxiway L from East to West and cross RWY 04.


Traffic Not relevant.

Scenario Script The crew rolls on Taxiway L towards RWY 04 and crosses the runway without having received any clearance.





Logging of Alerts.

EMMA2



Figure 4-74: EMMA GA-01 Takeoff of RWY 13, Traffic on Approach RWY 31

Figure 4-75: EMMA-VAN-05: Crossing RWY 04 without clearance

4.2.1.2.6 EMMA GA-01 – EMMA GA02 EMMA-VAN-GA-01 Runway Incursion #1 / GA Final Approach Remarks / ATC


Van Crew Task Roll along Taxiway L from West to East and cross RWY 22.


EMMA2



GA Crew Task Final Approach on RWY 13.

Scenario Script The GA crew is in final approach on RWY 13.

The van crew rolls on RWY 22 towards RWY 13 and crosses the runway while GA is at 1000ft. CPDLC is not available.

As soon as the alert is triggered in the GA, the crew executes a Go Around. As soon as the alert is triggered in the Van, the crew exits the runway.

If no alert has come when the GA is at 700ft, the GA crew executes a go around.





Logging of Alerts from the Van and the GA.

EMMA-VAN-GA-02 Runway Incursion #3 / GA in Take Off mode Remarks / ATC


Van Crew Task Roll along Taxiway M from West to East and cross RWY 04.


GA Crew Task Simulate Take-Off on RWY04 by accelerating to a ground speed above 20kts.

Scenario Script The GA crew starts Take Off from RWY 04 after having received the clearance from ATC.

The van crew rolls on Taxiway M towards RWY 04 and crosses the runway.

As soon as the alert is triggered in the GA, the crew aborts take off. As soon as the alert is triggered in the Van, the crew exits the runway.

If no alert has come when the GA is at 40kts, the GA crew aborts take off.





Logging of Alerts from the Van and the GA.

During these trials it was foreseen to perform in principle two different scenarios.

1. An aircraft approaching a runway blocked by a vehicle.

2. An aircraft taking-off, but again with blocking vehicle on the present runway.

Initially the approach scenario was planned to happen on the closed Runway 22 with the intention to not disturb the normal traffic. The initial planning for the Take-Off scenario was to start the aircraft on an operating runway (RWY13) or alternatively also on the closed RWY22 and locate the ground vehicle in a blocking way either by crossing RWY13 on RWY22 or with the other option by crossing RWY22 on a taxiway.

After discussion with the local ATC and the airport operator the scenarios were adjusted in terms of impact on safety and traffic capacity. It is against controller rules to allow an approach or take-off on a closed runway, furthermore the blocking of an operational runway needs to be avoided. The finally proposed scenario outline for the approach was to simulate a landing on RWY 13, while the ground vehicle (TUD Van) crosses RWY 13 on RWY22. The aircraft would finish the approach with a go-

EMMA2



around and land afterwards. In the case of take-off, the scenario was limited to a simulation only. Since the airborne conflict detection algorithm identifies the take-off phase by a certain increase of the ground speed. More details on the algorithms applied for approach and take-off as well can be found in the SP2-Specification-GA-MovingMap_V1.0.doc.

No further planning had been made, it was foreseen to have a briefing with pilots, drivers, coordinators and tower controllers before the start of activities. This meeting was deemed to consider operational restriction etc. as well.

4.2.1.3 Participants

4.2.1.3.1 Evaluation Team Two male pilots, all from Czech Airlines, participated in the trials. The mean age of these pilots was 38 years (between 34 and 42 years old) an they had a mean total number of flight hours logged in their entire career in commercial aviation of 4950h (between 3400h and 6500h). Among the pilots were 1 Captain, who had a Flight Instructor Education and 1 Senior Flight Officer. The types the pilots are logged actual are the Airbus 320 family (A318,A319,A320,A321) and Boeing 737 classic version (-300,-500). Two of the pilots have experiences with airport moving map displays from previous campaigns at TUD respectively EMMA. Pilots were asked to rate their familiarity with the airport at which the tests took place on a six-step scale with text statements, ranging from “It is my first time here” to “I know the layout of this airport almost better than that of my flat”. The pilots at Prague named that they are very familiar (“I know this airport very well.”, “I know the layout of this airport almost better than that of my flat.”) with their airport. The GA Team consisted of one pilot and one research scientist. The pilot is a flight trainer employed by the flight club.

4.2.1.3.2 Assessment Team The core assessment team for all the trials consisted of two TUD research scientists. These two research scientist were systems engineers who participated in the functional and HMI design of the Surface Movement Awareness and Alerting System (SMAAS) and who were also responsible for the creation and extension of the questionnaires used in the experiment. They acted as the experiment leaders, did the briefing as well as the familiarisation, took down pilots’ comments and assisted pilots in filling the questionnaires. At the Prague Airport Trials two more research scientist from TUD joined the assessment crew. One of them was mainly responsible for the operation of the navigation equipment in the Navigation Test Vehicle. The other one drove the van at the Prague Airport. In addition to the four TUD engineers, one engineer from Diehl Aerospace and one engineer from Funkwerk Avionics attended the trials at Prague Airport. For the trials without the G/A Aircraft, an engineer from the Prague Airport Authority played the role of the traffic conflicting with the TUD van.


4.2.1.4.1 First TUD OST The first TUD on-site trials at the Prague Airport (LKPR) took place from the 25th to the 29th of August 2008. The time schedule of the trials can be seen in the figure below.

EMMA2



25.08.2008

Arrival26.08.2008

Trial Preparations

Pilot trial

27.08.2008

Pilot trial

G/A Trial Preparations

G/A trials

28.08.2008

ADS-B traffic monitoring

29.08.2008

Departure

CW 35 in 2008

Figure 4-76: Time schedule for first TUD OST During each trial without the G/A Aircraft the five scenarios described in chapter 3.1.3.5 were tested. Two pilots from Czech Airlines (CSA) took part in those trials. The pilots first invited to fill out the Pilot-Intake-Questionnaire (see annex). They were then briefed on the motivation and the functionalities of the Surface Movement Awareness and Alerting with a short power-point presentation. After a brief exchange with the attending pilot concerning the presentation and a brief training showing the interaction possibilities with the display, the scenarios EMMA2-VAN-01 through to EMMA2-VAN-05 were run. During these trials, the conflicting traffic was simulated by a vehicle from the Prague Airport Authority equipped with an ADS-B transponder and driven by one of their engineers. The typical course of an evaluation trial was the following, where T0 was an individual start time for each pilot.

T0 - T0+30 Presentation – Briefing

T0+30 - T0+40 Training

T0+40 - T0+130 System evaluation

T0+135 - T0+195 Debriefing The trials with the G/A Aircraft were conducted without the presence of a pilot in the TUD van. Only the assessment team, as described above, was present. Before those trials, a briefing took place in the tower of the Prague Airport to discuss the trials with the responsible controllers and the pilot of the G/A Aircraft. For a detailed report on the trials with the G/A aircraft please see the following chapter.

4.2.1.4.1.1 Safety Net for On-Site Testing For the approach scenario an abort point was defined during preparatory discussions with the Tower Controller. That abort point was chosen to be at the runway intersection with taxiway P (Papa). This intersection was defined and shall never be transgressed by the aircraft, meaning the cusp of the turn being well in front or in any case not beyond an imagined line through that point.

EMMA2



The Take-Off scenario only required an acceleration to a speed above 20kts, in order to simulate a take-off and trigger an alert. However, during the test while moving on the south-western part of the closed runway 22 no crossing of the operating RWY31/13 was allowed.

4.2.1.4.2 Second TUD OST The second TUD on-site trials at the Prague Airport (LKPR) took place from the 17th to the 21st of November 2008. The time schedule of the trials can be seen in the figure below. The goal of these trials was mainly to test the TIS-B traffic data at the Prague Airport which were not available in the first TUD OST and also to assist the DLR OST with the ATTAS (see chapter 3.1.6.1). On the 18th of November a briefing took place in the tower of the Prague Airport for all the participants of the DLR OST and TUD OST. After this briefing session, the TIS-B functionality was switched on in the tower for the whole duration of the TUD OST.

17.11.2008Arrival

18.11.2008Trial Preparations

ADS-B/TIS-B monitoring + trial preparations with ATTAS

19.11.2008ADS-B/TIS-B monitoring + trial preparations with ATTAS

20.11.2008ADS-B/TIS-B monitoring

21.11.2008Departure

CW 48 in 2008

Figure 4-77: Time schedule for second TUD OST

EMMA2



On the rest of the 18th of November and on the 19th of November the TUD Team assisted the ATTAS Team for the preparation of the Traffic Conflict Detection Trials and in particular in getting proper traffic information on-board. In parallel, the TUD Team made necessary modifications to its software to be able to correctly monitor and record ADS-B as well as TIS-B traffic information. The TUD Van was escorted all along the northern part of the airport by an engineer from the Prague Airport Authority. It was only possible to receive the TIS-B broadcast in the northern part of the airport. On the 20th of November, the TUD Van was posted outside the airport premises north of the tower so as to be able to have a good view of runway 24 and be able to receive ADS-B traffic information as well as TIS-B traffic information.

4.2.1.5 Metrics and Indicators In general, there are two different methods of gathering information for a validation process. The first method consists in the quantitative measurement of some given indicators for the system and the objective assessment of the gathered data. The second method relies on the qualitative assessment of subjective measurements, such as user opinions, comments, feedbacks. Since the objectives concerning the Surface Movement Awareness and Alerting System that was to be assessed during the Real Time Simulation trials are all linked to subjective notions such as the crew’s situation awareness, the perceived workload or the human machine interface, it was decided to use only the second method: qualitative assessment of questionnaires and debriefing interviews. During the first on-site trials at Prague, the TUD team noticed quite a lot of problems linked to the use of ADS-B traffic information. This is why it was decided to use quantitative measurements for this particular aspect. During the scenarios the pilots’ comments were documented by one of the TUD scientists. After the exercise, data was collected by means of several debriefing questionnaires and a debriefing interview. The gathered data was then methodically analyzed and evaluated.

4.2.1.5.1 Debriefing Questionnaire A set of three dedicated debriefing questionnaires was used as well as a Cooper Harper Rating Scale sheet. The dedicated questionnaires can be found in the EMMA2 document 2-D616 Validation plan – airborne [6] in the annex.

Dedicated questionnaires The detailed questionnaires enabled the pilot to rate individual elements using 7 or 10 step Likert scales. The first questionnaire details the pilot’s impressions on the airport moving map. The second questionnaire deals with the surface traffic display (CDTI) used in the system. The last questionnaire enables the pilot to state his views on the Surface Movement Awareness and Alerting System as a whole, including mostly the display of operational, clearance and traffic information, as well as the associated alerting mechanisms.

Cooper Harper System (CHS) The CHS is used by starting at the lowest maturity level (i.e. the question “Is the system usable?” at the bottom left-hand corner) and progressing up the table while the question can be answered with a “yes”. When a question has to be answered with a “no”, the actual pilot rating is determined by the detailed nature of the identified deficiency, as defined on the right hand side of the table. This process results in the evaluator selecting a rating in a ten-point scale ranging from 10 (the system is unusable) to 1 (the system is fully mature). Contrary to the Real Time Simulator Trials, only one Cooper Harper rating scale was used, since there was no MCDU mock-up available in the TUD Van, and thus no way of assessing the Ground to Air Database Upload Functionality in particular the MCDU interaction with the electronic Pre-flight Information Bulletin (ePIB). The Cooper Harper rating scale was used to assess the Surface Movement Awareness and Alerting System as a whole.

EMMA2



4.2.1.5.2 Debriefing Interview The aim of the debriefing interview was to go deeply into the key points noticed in the observations during the trials or in the debriefing questionnaires. It helped to bring out the operational improvement possibilities of the system through different human factor aspects:

• System usability

• Confidence in the system

• Situation awareness

The debriefing interview also helped the team understand why some pilots may have a diverging answer from the majority of pilots, or from the expected answer. This was used intensively to understand the questionnaire results in the analysis phase.

4.2.1.5.3 Traffic logs For the assessment of the quality of traffic information coming from ADS-B data and from TIS-B data, a logging function was implemented and used during the second on-site trials at the Prague airport. The function logged all available information about every traffic item, and whether the information was coming from the tower (TIS-B) or the aircraft itself (ADS-B).

4.2.2 Technical and Operational approval (FAV / TUD trials) The trials took place at the 27th of August. After a flight trip of 2 hours the Piper reached Prague airport at 10:45am local time. The next item on the agenda was a meeting with the tower controllers, in order to operationally decide upon order, timing and procedures of the tests. It was decided to have the Approach scenario first because the traffic density at Prague was low at that time. Hence, the Take-Off simulation followed.

Figure 4-78: Approach Scenario - Entry positions (FAV aircraft and TUD Van)

EMMA2



After that briefing the scenario outline was nearly similar to the planned one. The major change was that the ground vehicle (TUD VAN) was not moving but standing on a fix position on the runway for both scenarios. Another change was that the position of the TUD Van for the Approach scenario was now between intersections of RWY31 with taxiway D and F. The entry positions of the scenarios can be seen in the picture above.

Approach Scenario: At 1255 UTC the Piper started its engine in order to perform low radar approaches on RWY 31. As initiation point the compulsory report point TANGO (see picture below) was selected.

Figure 4-79: Approach Scenario (FAV aircraft)

At that point a permission for a low radar approach had been requested which shortly after confirmation by ATC began. For safety reasons ATC instructed, the approaching aircraft must never exceed the intersection of RWY31 with taxiway P (PAPA). By consideration of this abort point the Piper turned right heading back to TANGO for requesting the next approach loop. In this way 4 “missed” approaches and one landing had been performed always staying in contact with the TUD Van. The contact was established by LKPR owned handheld radios.

EMMA2



Figure 4-80: Recorded ADS-B track during approaches The picture above is a presentation of the recorded ADS-B track during the approaches. While closing the blocked runway the CDTI triggered alerts as soon as the distance to the as intruder identified target was less than 2 NM or the approach angle to the Runway direction diverted less than 15°. In the picture above the activation of the incursion alerts are shown by red rectangles, while the deactivation is indicated by green rectangles. The indication of a runway incursion on the CDTI can be seen below.

EMMA2



Figure 4-81: CDTI in FAV OST

All runway incursion are logged by the CDTI and attached with the actual timestamp synchronized by GPS, supporting the post-trial evaluation. So it is possible to visualize the position at which the alert was raised and discarded as well. The picture below shows the ADS-B broadcasted position at which the runway incursions were triggered.

Figure 4-82: ADS-B broadcasted positions at which the RWY incursions were triggered Take-Off Scenario:

EMMA2



After landing and a short briefing the take-off simulation followed. For this kind of scenario the closed runway 22 was used. While the aircraft was accelerating to above 20 kts on the south-western part of the runway in direction 04, the TUD-Van was standing between taxiways L and P. While performing the simulations the runway incursion was triggered normally as seen in the case of the approaches. At one time the indicated position of the target was overdue, because of signal losses. After a change of the Van’s position the signal was received again. Most likely the signal losses were due to the altitude profile of the runway and also the single antenna installation at the bottom of the aircraft. From six runs one was unsuccessful because of the overdue position, so in this case no runway incursion alert was triggered. The on-board system has been approved by deploying the ADS-B receiver (RTH60) as ground station in Egelsbach. The ADS-B receiver is the same as it is used within the TUD Van. The SMA functionality of the CDTI has been verified by using test equipment at the facilities of Funkwerk Avionics. In this way several scenarios could be simulated and the software was approved as ready for on-site testing. The final check if TUD Van and GA-aircraft work together happened at the Prague airport shortly before the trials. During the on-site trials the function of the SMA algorithm has been approved, the scenarios were set up to comply with the respective conditions of the following parameters: height above runway, speed and direction as well as distance to the target vehicle.

As a baseline for the post-trial evaluation the logging function of the CDTI-2000 has been applied. The log file contains records of ADS-B target positions as well as records of the own state. Furthermore the CDTI recorded the triggering time of runway alerts. All the records in the log are accompanied by a timestamp derived by the CDTI system clock which itself is synchronised with the GPS time derived by the Trimble 2000 GPS device.

4.2.3 Results to Technical Feasibility (FAV Trials) During the trials TUD realized some significant delays in position transmission via ADS-B, not between the TUD Van and the GA-aircraft but with other real life traffic. The aircraft installation comprises two GNSS data sources, one for the CDTI ownship depiction and one for the ADS-B out functionality. As the installations allowed to receive the ADS-B messages broadcasted by the own transponder, this enabled to compare the different positioning sources by GPS timestamp. It was also possible to attempt the comparison between the timestamps of the different records at the same position (the delay) but this would be more significant using one single source for both (ADS-B Out and ownstate recording). The following chart shows the result of this analysis:

Figure 4-83: Comparison between the positioning sources

EMMA2



It has to be stated that the comparison bases on two different GPS receivers which might output the same position at different timestamps, influencing the validity of this comparison. An in any case reliable comparison would than be between the determined positions at the same timestamp. In this way the differences between the two GPS receivers can be shown. Well, this does not show the deviation from the real life position, but one can see to what extent the position determination can divert using different equipment. During the approach scenario there were 247 datasets with identical timestamp (increment: 0,001 seconds) out of which the deviation measures values between 0 and 119 meters.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0 10 20 30 40 50 60 70 80 90 100 110 120

position deviations [m]

cum

ulat

ive

perc

enta

ge

Apparently the deviation does not transgress 20 meters in 50% of the cases.

It is furthermore necessary to conclude that ADS-B out is “depending” on the quality of the positioning data source and that for the time being no certification baseline (TSO or comparable) is existing prescribing installation criteria and requirements to be fulfilled. There are some standards defined and available but they can be seen as recommendation but not as prescription. Hence it is absolutely necessary to filter out ADS-B messages which are obviously not containing trustable respectively not reliable information. As criteria for DO260/DO260A compliant systems the NUC/NIC values need to be considered. An example may be to exclude all ADS-B messages with a NIC value below 5.

4.2.3.1 Debriefing Comments In general the integration in the cockpit was in a good viewable and reachable position. The HMI is perceived as easy understandable and somewhat intuitive. A very good feature is the automatic zoom in case of warnings to depict all included items and providing valuable information to the pilot. As the aircraft is operated only under VFR conditions the approach is performed with the out of window view, therefore it would be nice to have an aural alert in case of a runway incursion. The aural alert capability was not installed in the CDTI although it is capable of.

EMMA2



4.2.4 Results to Operational Feasibility (OST) The overall feasibility of the Surface Movement and Awareness and Alerting System was assessed in the on-site trials with the same questionnaires as for the Real Time Simulation and were analysed at the same time as the other trials. Please refer to chapter 4.1.6.6. Because the simulator trials use ideal data for the position of ownship and traffic a question for the estimation of the correct time an alert appears is not reasonable. But while using “real world” data at Prague airport this question immerged. One of the two pilots thought, that the alert was too late. That was a consequence of the latency of the position data ADS-B has. One of the pilots took the latency into account and thus noted that the system would have an appropriate timing of the alert, if the latency didn’t appear. The recordings made during both on-site trials at the Prague Airport showed a few interesting point about traffic data coming from ADS-B sources and from the TIS-B installation in the tower. One of the main problems noticed during the first on-site trials was positional discrepancies of some ADS-B data of aircraft landing and then rolling to their gates. This discrepancy could go up to several kilometres. This would be a very serious issue mostly in airports with parallel runways: in Prague it was possible to still have an alert when an aircraft was landing on the runway because of the heading, but if the trials had taken place in Frankfurt for example, there would have been no way of telling on which runway the aircraft would be landing. This positional discrepancy problem was totally solved with the TIS-B installation: all the TIS-B aircraft were shown at a correct position. Another problem seen in the first trials was a latency of traffic data of 4 to 10 seconds. The latency seemed dependant of the speed of the aircraft: the faster the aircraft, the less latency was noted. It was hard to analyse the latency with TIS-B data because a new problem surfaced there: the update rate was very poor. A further problem with the TIS-B installation as it was in Prague was that aircraft appeared only very late when they were landing on the airport. Some aircraft even only appeared once they had turned onto a taxiway. This would be a major problem for the runway incursion avoidance alerts: the aircraft would first be assessed by the system when it was too late. Furthermore, it was noticed that the installation of a TIS-B antenna on top of the tower caused problems receiving the necessary data in the vicinity of buildings. The introduction of TIS-B must then make sure that the TIS-B messages can be received on all taxiways.

EMMA2



5 Conclusions

Conclusions to the new A-SMGCS services During the on-board validation trials of the “green team” all EMMA2 on-board A-SMGCS services (with exception of the HUD functionality) were implemented and under evaluation. Following A-SMGCS services were operated by the pilots:

• Ground Traffic Display (GTD) • Traffic Conflict Detection (TCD) • Surface Movement Alerts (SMA) • Ground-To-Air-Database Upload • TAXI-CPDLC

Generally speaking, the overall system provided a medium maturity and could fulfil the operational expectations. The pilots approved its operational feasibility by accepting most of the operational requirements and new procedures (compare the checklist in section Error! Reference source not found.). The pilots also stated that the new and higher A-SMGCS services would increase safety and situation awareness. An increase of workload is not expected as long as pilots are familiar with input devices. In the following general results of each new service will be reported: Electronic Moving Map (EMM) The EMM was already validated in the previous EU projects BETA and EMMA. Therefore these services were not the subject of testing in EMMA2 but served as the current baseline in the trials. Ground Traffic Display (GTD) The GTD was successfully validated. In real time simulations the HMI was accepted and the operational feasibility was approved. In subsequent on-site trials the technical feasibility of TIS-B as supplier of the complete traffic situation for the GTD was shown as well, resulting in a complete positive validation. Real time simulations It was proven that its HMI design fits the pilots’ needs, and that it works reliable (with an update rate of 5Hz in the simulation trials), and in an intuitive and interactive way. The GTD in general is accepted by the pilots while they stress – as expected – that the outside view through the cockpit windows remains their main source of information. In terms of usability pilots agree that the GTD is very easy to use without inconsistency. They report that they didn’t need to learn a lot before they could use the GTD properly. The very intuitive approach of the GTD was appreciated and highlighted by comments in the debriefing sessions. - HMI With the addition of displayed traffic, the on-board display now provides pilots with a complete airport traffic situation, allowing a rapid situation assessment. They agree that it is easier to locate and identify aircraft while using the GTD. Furthermore the GTD allows pilots to utilise the display capabilities (e.g. range scale selection, brightness, map overlays). This was realised in the DLR setup by soft buttons on the display and the mode and range selector in the EFIS. In the TUD setup, this was realized by overhead buttons as well as the mode and range selector in the EFIS.

EMMA2



The different modes (PLAN, ARC and NAV) and the zoom functionality of both the DLR and the TUD display are highly appreciated and intuitive to use. According to pilots’ feedback the GTD maintains a balance between human and machine tasks. It permits them to retain the power to make decisions for those functions for which they are responsible. Pilots reported in the debriefing sessions that the head down times while using the GTD display (located on the navigation display) are acceptable. Furthermore, the pilots do not over-trust the system and stay aware that the GTD function relies on cooperative sensors (ADS-B) that are not present on all the traffic surrounding them. - Fulfilment of general requirements Pilots agree that the GTD service is capable of being used appropriately when operating within the movement area. In the opinion of the pilots the new display was well integrated in harmonisation with existing cockpits components. Furthermore the GTD enables pilots to interface and function efficiently while a continuous service is provided. - Fulfilment of surveillance requirements The pilots reported that the GTD provides accurate position information on all static and moving traffic within the movement area enabling them to work safely and efficiently. The identification and labelling of the surrounding traffic was guaranteed by the display as well. In the pilots’ opinion the GTD was sufficiently capable of updating surveillance data of the surrounding traffic. On-site trials: In the first On-site trials with the TUD Van, it was noted that the quality of ADS-B traffic data available was too poor - mainly because of important latency for slow vehicles – to display the data of rolling traffic in the direct vicinity of ownship. After tuning of the TIS-B function it was proven that TIS-B is capable to fulfil the needs to provide the complete traffic situation of the airport for the GTD. DLR test pilots were able to use the display during taxiing on PRG airport in its full functionality resulting in general acceptance of the GTD. The update rate was sufficient enough to provide the pilots with the complete current traffic situation. Traffic Conflict Detection (TCD) The Traffic Conflict Detection function was successfully validated. Pilots agree that the Traffic Conflict Detection function is capable of being used appropriately on the surface movement area. The visual warnings with different colour codes for level 1, level 2 (master caution) and level 3 (master warning) present on the EMM and on the PFD as well as audible warnings will attract the pilots’ attention in case of conflicts. The chosen HMI for the alerting was deemed intuitive, easy to interpret and acceptable by the pilots. In the on-site trials it was noted that the quality of ADS-B traffic data available was too poor - mainly because of important latency for slow vehicles - for the conflict detection function on the taxiways. In these conditions, only the runway related functions would be relevant. Surface Movement Alerting (SMA) The Surface Movement Alerting function was successfully validated through the real time simulation tests as well as the on-site trials. Pilots agreed that the alerts generated by the SMA are operationally relevant. The pilots added that even though the situation awareness is already improved by the EMM and the display of the electronic Pre-Flight Information Bulletin data on the EMM (see Ground-to-Air Database Upload), all the alerts covered by the SMA are still needed. The warning concept including visual warnings on the displayed runway on the EMM, textual warnings on the PFD as well as audible warnings was well received and accepted by the pilots.

EMMA2



Ground-To-Air-Database Upload The introduction of the Ground-to-Air Database Upload function into the cockpit to enable the display and the use of the electronic Preflight Information Bulletin data in the SMA does not lead to any undue increase of workload in critical flight phases. The pilot interacts with the Ground-to-Air-Database Upload function through the MCDU. The MCDU is a well known means and pilots are familiar using it. The pilots didn’t have any problems to interact with the dedicated MCDU Airport Menu to update or load restrictions. Further development could be the introduction of a trackball or computer mouse to interact with the system. Most of the pilots thought that is very useful to have information about runway or taxiway restrictions. And also the presentation of this information is very intuitive because the symbology and the colours used in the SMAAS system in the TUD display are well known and intuitive. One improvement could be to show different statuses for one runway. TAXI-CPDLC The pilots’ feedback for requesting and receiving clearances by TAXI-CPDLC for start-up, push-back, taxi-in, and taxi out was predominantly positive. It must be distinguished clearly between the positively rated graphical information on the EMM display and the rating of the usability of the CDTI, which served for selecting CPDLC messages to be sent and for the textual displaying of received clearances. The pilots stated that they were provided with a very effective human-machine interface in terms of the EMM display to operate a data link communication with ATC. Nevertheless, the input device in terms of the CDTI needs important upgrades if a further use of the CDTI is foreseen in simulation and flight trials. Using a CDTI as an additional input device for data link communication (instead of using the FMS) is appreciated by the majority of the pilots. In their opinion it might be too confusing to integrate the data transfer into the FMS resulting in a difficult handling of the TAXI-CPDLC. Pilots highly appreciated and trusted the displayed taxi routes and the clearances. The visualisation of taxi routes on the EMM was appealing for them. Especially the exocentric perspective view was used by the CM2 to gain advance information during taxiing. Pilots responded that they would like to use the TAXI-CPDLC in the future and that they agree with its concept given and realised in EMMA2. They agree that the used terminology and symbology to represent a taxi route were easy to interpret. Paper charts were only held ready as a back-up as expected. Pilots agree that a preliminary taxi route will not bring added value because the assigned route can change at short notice during initial approach (inbound) or preparing the a/c for departure (outbound), and thus is not a reliable information. Some of the proposed procedures are perceived as being clumsy. For example, pilots recommend that the CDD should issue departure clearance, start up and change to ground frequency in one step, when the situation allows it. To establish R/T as back-up Pilots appreciate the initial call at the beginning of each TAXI-CPDLC dialogue at each new ATC position. In the trials the crossing, as a safety critical clearance, was performed by voice communication. This led to the situation that the pilots in the cockpit had to switch off the red crossbar (received from the Ground controller as the clearance limit) on their EMM. This procedure is of course criticised by the pilots. They agree that this task must be performed by an ATCO. Further testing of adapted procedures would be needed here, e.g. crossing clearance by TAXI-CPDLC, or by a voice clearance combined with an automatic deletion of the displayed stop bar by the ATCO’s pressing of the “crossing clearance” button on their EFS.

EMMA2



Conclusions to Operational Improvements Having proven the services concerning their operational feasibility, questions to their contribution to operational improvements arise. In the present test setting, it has to be considered that the EMM was the baseline, which was compared against the new A-SMGCS services. With the introduction of higher A-SMGCS on-board services an increase in safety is expected especially for GTD with an E-OCVM V2-3 level of maturity. New services like TCD, SMA and TAXI-CPDLC are rather novel with a target level of maturity V1-V2 and thus were tested for their operational feasibility primarily. Operational improvements were measured subjectively. Pilots rated significantly that

- GTD is regarded as potential means for increased safety which eases pilots work considerably. With its indication of the surrounding traffic the GTD is an additional information source for the pilots to navigate and to manage the aircraft speed more safely and efficiently.

- GTD significantly increases the pilots’ situation awareness (by allowing an intuitive use, providing the appropriate amount of information, and as an extraordinary aid in locating traffic) especially under low visibility conditions.

- Pilots’ workload is not affected negatively when using the GTD. - Furthermore, pilots report that the use of a GTD will improve their performance. - TAXI-CPDLC is regarded as potential means for increased safety which will ease pilots

work considerably with appropriate input devices. The graphical presentation of the cleared taxi route on the EMM (compared to textual clearances only) will enable pilots to follow an assigned taxi route more safely and efficiently.

- An increase of workload while using the TAXI-CPDLC service could be observed and was explained by the pilots due to the fact that a novel and not-well designed input device was used.

- Situation Awareness was not affected by the use of TAXI-CPDLC. - The Ground to Air Database Upload function is regarded as potential means for increased

safety which will ease pilots work especially in large airports. - The Ground to Air Database Upload function significantly increases the pilots’ situation

awareness by enabling the loading of and the interaction with electronic Preflight Information Bulletin Data (eg. NOTAM information).

- Pilots’ workload is not affected negatively when using the Ground to Air Database Upload function.

- The SMA and the TCD are regarded as potential means for increased safety as they will diminish the risks of accidents and incidents on the surface movement area especially in low visibility conditions.

EMMA2



EMMA2



6 Annex I

6.1 References [1] EC/EUROCONTROL, European Operational Concept Validation Methodology – Version 2

March 2007 [2] EMMA2, Annex I – “Description of Work”, Sixth Framework Programme, FP6 – 513522,

version 0.35, 24.11.2005. [3] EMMA2, A-SMGCS Services, Procedures, and Operational Requirements (SPOR), 2-D1.1.1,

v0.13, July 2008, www.dlr.de/emma2. [4] EMMA2, Generic Experimental and Test Plan, 2-D6.1.1b, February 2008, www.dlr.de/emma2. [5] EMMA2, Validation Plan, 2-D6.1.1a, February 2008, www.dlr.de/emma2. [6] EMMA2, Validation Test Plan Airborne, 2-D6.1.6, v0.03, January 2009, www.dlr.de/emma2. [7] ICAO, Doc 9830 AN/452, Advanced Surface Movement Guidance and Control System (A-

SMGCS) Manual, First Edition, 2004 [8] Bortz, J. (1993). Statistik für Sozialwissenschaftler, Springer Verlag: Berlin Heidelberg. [9] Brooke, J.T. (1996). SUS: a "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, B. A.

Weerdmeester & A. L. McClelland (Eds.). Usability Evaluation in Industry. London: Taylor and Francis.

[10] Tattersall, A.J., Foord P.S. (1996). An experimental evaluation of instantaneous self-assessment as a measure of workload. Ergonomics, Vol. 39 (5), 740-749.

[11] Foyle, D. C., Andre, A., McCann, R. S., Wenzel, E. M., Begault, D. R. &. Battiste V. (1996). Taxiway Navigation and Situation Awareness (T-NASA) System: Problem, design philosophy and description of an integrated display suite for low-visibility airport surface operations. Journal of Aerospace, 105, 1411-1418

[12] Joseph, K.M., Domino, D., Battiste, V., Bone, R.S., Olmos, B.O. A Summary of Flightdeck Observer Data From SafeFlight 21 OpEval-2. DOT/FAA/AM-03/2.

[13] EMMA2, Prague A-SMGCS Test Report, 2-D6.3.1, v0.03, December 2008, www.dlr.de/emma2.

[14] EMMA2, Validation Comparative Analysis Report, 2-D6.7.1, v0.03, January 2009, www.dlr.de/emma2.

[15] EMMA2, Validation Test Plan – Prague, v0.04, 2-D6.1.3, December 2008, www.dlr.de/emma2. [16] Wipplinger, P. (2005). Untersuchung des Pilotenverhaltens bei HALS/DTOP-Anflügen,

Dissertation, Technische Universität Darmstadt. [17] EMMA2, Assessment Report for the Ground CPDLC Clearance function PRG/MXP”, 2-

D2.5.2. www.dlr.de/emma2.

EMMA2



6.2 Abbreviations Acronym Meaning A/C Aircraft ADS-B Automatic Dependent Surveillance - Broadcast AIF Airbus France S.A.S. AMM Airport Moving Map ANOVA Analysis of Variance ANS CR Air Navigation Services of the Czech Republic ARINC Aeronautical Radio Incorporated A-SMGCS Advanced Surface Movement Guidance and Control System ASSA Airport Surface Situation Awareness Questionnaire ATTAS Advanced Technology Testing Aircraft System ATC Air Traffic Control ATCO Air Traffic Controller ATM Air Traffic Management ATMOS Air Traffic Management and Operations Simulator ATN Aeronautical Telecommunication Network ATS Apron & Tower Simulator CAVOK Clouds and Visibilty OK CCD Crew Control Device CDD Clearance Delivery Dispatcher CDTI Cockpit Display of Traffic Information CHS Cooper Harper System CM1 Pilot in command 1

CM2 Pilot in command 2

CPDLC Controller-Pilot Datalink Communication CPT Captain CSA Czech Airlines CSTR Constraint CWP Controller Working Position DAS Diehl Aerospace Df Degrees of freedom DLH Deutsche Lufthansa DLR Deutsches Zentrum für Luft- und Raumfahrt DM Downlink Message DMAN Departure Manager EC European Commission EDDF Frankfurt Airport EEC Eurocontrol Experimental Center EFIS Electronic Flight Instrument System EFS Electronic Flight Strips EHAM Airport Amsterdam EMM Electronic Moving Map EMMA2 European Airport Movement Management by A-SMGCS, Part 2 ENG Engine Display E-OCVM European - Operational Concept Validation Methodology ePIB electronic Pre-flight Information Bulletin EU European Union

EMMA2



Acronym Meaning EUROCONTROL European Organisation for the Safety of Air Navigation F F value in the F statistics FAV Funkwerk Avionics FCU Flight Control Unit FMS Flight Management System F/O First Officer GEC Ground Executive Controller GECO Generic Experimental Cockpit GPS Global Positioning System GPS/IRS Global Positioning System / Inertial Reference System, GPS/WAAS Global Positioning System / Wide Area Augmentation System GTD Ground Traffic Display HLO High Level Objectives HMI Human-Machine Interface HSI Horizontal Situation Indicator HUD Head-Up Display ICAO International Civil Aviation Organisation ID Identifier IP Internet Protocol ISA Instantaneous Self Assessment ISDN Integrated Services Digital Network IV Independent variable LAN Local Area Network LCD Liquid Crystal Display LFPG Charles-de-Gaulle Airport LKPR Prague Airport LLO Low Level Objectives LVP Low Visibility Procedures M Mean MAEVA Master European Validation Plan MCDU Multipurpose Control and Display Unit MLAT Multi-Lateration N Number of subjects NAV Navigation Display ND Navigation Display NM Nautical Miles NOTAM Notice To Airmen OST On-site Trials P p-value (error probability that the measured mean belongs to the H0 hypothesis) PAS Park Air Systems AS PF Pilot Flying PFD Primary Flight Display PNF Pilot Non Flying PRG Prague QE-OF EMMA2 Operational Feasibility Questionnaire (for Pilots) QE-QI EMMA2 Operational Improvement Questionnaire (for Pilots) RAAS Runway Awareness Advisory System RAD/NAV Radio Navigation RTS Real Time Simulation RWY Runway

EMMA2



Acronym Meaning SD Standard Deviation SF/O Senior Flight Officers SID Standard Instrument Departure SMA Surface Movement Alert SMAAS Surface Movement Awareness and Alerting System SMR Surface Movement Radar SP Sub-Project STAR Standard Terminal Arrival Route SUS System Usability Scale (Questionnaire) TCD Traffic Conflict Detection TCP Transport Control Protocol TEC Tower Executive Controller THAV Thales Avionics TIS-B Traffic Information System Broadcast TSD Traffic Situation Display TUD Technische Universität Darmstadt TWY Taxiway UM Uplink Message UTC Coordinated Universal Time VDLm2 VDL Mode 2 VFR Visual Flight Rules VFW Vereinigte Flugtechnische Werke V&V Verification and Validation WILCO Will Command WP Work-Package

6.3 List of Figures Figure 3-1: DLR’s Generic Experimental Cockpit (GECO).............................................................. 9 Figure 3-2: PFD, NAV, ENG and FCU in GECO ........................................................................... 10 Figure 3-3: Architecture of Cockpit & Tower Simulation, Pseudo Pilots and DMAN.................... 11 Figure 3-4: Taxi Route in EXE02, Pt. 1 outbound (Gate 34 RWY24)........................................ 24 Figure 3-5: Taxi Route in EXE02, Pt. 2 inbound (RWY24 Gate 34).......................................... 24 Figure 3-6: Taxi Route in EXE04, Pt. 1 outbound (Gate 34 RWY06)........................................ 25 Figure 3-7: Taxi Route in EXE04, Pt. 2 inbound (RWY06 Gate 34).......................................... 25 Figure 3-8: Taxi Route in EXE06, Pt. 1 outbound (Gate 34 RWY24)........................................ 26 Figure 3-9: Taxi Route in EXE06, Pt. 2 inbound (RWY24 Gate S6) ......................................... 26 Figure 3-10: Taxi Route in EXE06, Pt. 3 outbound (Gate S6 RWY24) ................................... 27 Figure 3-11: DLR ATTAS Test Aircraft (D-ADAM) during OST in PRG................................... 60 Figure 3-12: DLR ATTAS Test Aircraft (D-ADAM) parking position during OST in PRG........ 61 Figure 4-1: View of the TUD fixed-base Research Flight Simulator .............................................. 65 Figure 4-2: Scenario FAM 01 .......................................................................................................... 67 Figure 4-3: Scenario FAM 02 .......................................................................................................... 68 Figure 4-4: Scenario SMA 01 .......................................................................................................... 70 Figure 4-5: Scenario SMA 02 .......................................................................................................... 71 Figure 4-6: Scenario SMA 03 .......................................................................................................... 72 Figure 4-7: Scenario SMA 04 .......................................................................................................... 74 Figure 4-8: Scenario SMA 05 .......................................................................................................... 75 Figure 4-9: Cooper Harper System .................................................................................................. 79 Figure 4-10: AMM display colours are satisfactory ...................................................................... 80

EMMA2



Figure 4-11: The different colour codes used are easy to interpret ................................................ 81 Figure 4-12: The used symbols are easy to interpret...................................................................... 81 Figure 4-13: Information is arranged conveniently........................................................................ 81 Figure 4-14: Different types of information are easy to find ......................................................... 82 Figure 4-15: AMM interaction concept is satisfactory................................................................... 82 Figure 4-16: AMM information amount is too large ..................................................................... 83 Figure 4-17: Textual information is very well readable................................................................. 83 Figure 4-18: With AMM the ND is overloaded with information ................................................. 84 Figure 4-19: Closed Runway symbology is intuitive and easy to understand................................ 84 Figure 4-20: Closed Runway colour presentation is consistent with overall cockpit concept ....... 84 Figure 4-21: Closed Taxiway representation is consistent with the closed Runway concept ........ 85 Figure 4-22: The shape of the traffic symbol is intuitive and acceptable....................................... 85 Figure 4-23: The colour of the traffic symbol is intuitive and acceptable ..................................... 86 Figure 4-24: Handling of the AMM............................................................................................... 86 Figure 4-25: The number of selectable ranges is sufficient ........................................................... 87 Figure 4-26: Display size is acceptable.......................................................................................... 87 Figure 4-27: With AMM the ND is overloaded with information ................................................. 88 Figure 4-28: Presentation of FMS-selected runway on the display ............................................... 88 Figure 4-29: Traffic presentation is an improvement over the systems we have today ................. 88 Figure 4-30: It is essential that aircraft approaching the runways and aircraft taking off are

displayed as well?.......................................................................................................................... 88 Figure 4-31: Does AMM give missed support? ............................................................................. 89 Figure 4-32: The number of selectable ranges is sufficient ........................................................... 89 Figure 4-33: With AMM the ND is overloaded with information ................................................. 90 Figure 4-34: Operational relevance of alerts when attempting to take off other than FMS selected

91 Figure 4-35: Alert when entering closed runway or try to land on it ............................................. 91 Figure 4-36: This type of alert will make flying safer in the future ............................................... 92 Figure 4-37: Taxi route monitor function is useful; blinking route attracts attention .................... 92 Figure 4-38: Taxiway takeoff alerting is a valuable contribution to flight safety.......................... 93 Figure 4-39: When AMM displays that I’m on taxiway I will never try to takeoff ....................... 93 Figure 4-40: Rate of the contribution of this traffic presentation to flight safety .......................... 94 Figure 4-41: This traffic presentation is definitely an improvement over the systems we have

today 94 Figure 4-42: Use of CDTI in Low Visibility and Night to check Runway before Take Off.......... 95 Figure 4-43: Aircraft approaching or taking off should essentially be displayed .......................... 95 Figure 4-44: Runway Incursion alerting increases flight safety.................................................... 95 Figure 4-45: Necessity of Runway Incursion Alerting despite augmented Situational Awareness

96 Figure 4-46: Runway incursion alerting doesn't require new procedures ..................................... 96 Figure 4-47: Traffic Alerting limited to Runway.......................................................................... 97 Figure 4-48: AMM increase situational awareness? ...................................................................... 98 Figure 4-49: With AMM it is not possible to get lost on airfield................................................... 98 Figure 4-50: FMS selected runway presentation is operationally relevant .................................... 98 Figure 4-51: The distinction of different closure levels is operationally relevant.......................... 99 Figure 4-52: An alert when attempting to take off from runway other than the FMS selected is

operationally relevant .................................................................................................................... 99 Figure 4-53: Alert when entering a completely closed runway ..................................................... 99 Figure 4-54: Usefulness of taxi route monitoring function .......................................................... 100 Figure 4-55: Overall situation awareness with ground traffic on AMM...................................... 100 Figure 4-56: Usefulness of presented traffic labels...................................................................... 101 Figure 4-57: Traffic presentation as improvement for the pilots ................................................. 101 Figure 4-58: Correspondence between traffic outside and traffic on map ................................... 102 Figure 4-59: Sometimes information not needed is displayed ..................................................... 102

EMMA2



Figure 4-60: Different types of information are easy to find ....................................................... 103 Figure 4-61: AMM interaction concept is satisfactory................................................................. 103 Figure 4-62: AMM information amount is too large ................................................................... 103 Figure 4-63: Taxi route monitoring is useful; blinking route attracts attention ........................... 104 Figure 4-64: Does presentation of ground traffic increase workload? ......................................... 104 Figure 4-65: TUD Van during tests on closed runway 04/22 on Prague airport (LKPR), February

6th, 2006 105 Figure 4-66: Antenna mounting plate on the roof of TUD's Navigation Test Vehicle ................ 106 Figure 4-67: Hardware and computer racks in the back of TUD’s Navigation Test Vehicle ...... 107 Figure 4-68: LCD screen installation in front of passenger seat .................................................. 108 Figure 4-69: Overview of Prague airport (LKPR) ....................................................................... 110 Figure 4-70: Take Off RWY 04 – Traffic On/Approaching RWY .............................................. 111 Figure 4-71: EMMA-VAN-02: Take Off RWY 04 without clearance ....................................... 112 Figure 4-72: EMMA-VAN-03: Collision Hazard with other Traffic on Taxiway....................... 113 Figure 4-73: EMMA-VAN-04 Collision with other traffic on taxiway ....................................... 114 Figure 4-74: EMMA GA-01 Takeoff of RWY 13, Traffic on Approach RWY 31 .................... 115 Figure 4-75: EMMA-VAN-05: Crossing RWY 04 without clearance ....................................... 115 Figure 4-76: Time schedule for first TUD OST........................................................................... 118 Figure 4-77: Time schedule for second TUD OST ...................................................................... 119 Figure 4-78: Approach Scenario - Entry positions (FAV aircraft and TUD Van) ....................... 121 Figure 4-79: Approach Scenario (FAV aircraft) .......................................................................... 122 Figure 4-80: Recorded ADS-B track during approaches ............................................................. 123 Figure 4-81: CDTI in FAV OST .................................................................................................. 124 Figure 4-82: ADS-B broadcasted positions at which the RWY incursions were triggered ......... 124 Figure 4-83: Comparison between the positioning sources ......................................................... 125

6.4 List of Tables Table 3-1: Independent variable and levels in RTS........................................................................ 12 Table 3-2: TAXI-CPDLC standard procedures for an outbound flight .......................................... 15 Table 3-3: TAXI-CPDLC standard procedures for an inbound flight ............................................ 17 Table 3-4: TAXI-CPDLC advanced procedures for an outbound flight......................................... 19 Table 3-5: TAXI-CPDLC advanced procedures for an inbound flight........................................... 21 Table 3-6: Scenario Description Overview RTS (Part 1, June 2008) ............................................. 22 Table 3-7: Scenario Description Overview RTS (Part 2, October 2008) ....................................... 23 Table 3-8: List of GECO call signs for every test run .................................................................... 28 Table 3-9: Demographical Description of the pilots participating at the DLR RTS....................... 28 Table 3-10: Test Schedule including treatment, allocation of Pilots to the Cockpit and ATCOs to the

CWP positions, Validation Area and the used traffic scenario ..................................................... 30 Table 3-11: Metrics and Indicators (RTS) ........................................................................................ 32 Table 3-12: QE-OF - Means, N, SD, and P-Value................................................................................ 45 Table 3-13: SUS- Means, N, SD, and P-Value ..................................................................................... 47 Table 3-14: QE-OI - Means, N, SD, and P-Value for safety-related items ...................................... 53 Table 3-15: QE-OI - Means, N, SD, and P-Value for capacity/efficiency related items.................. 54 Table 3-16: Two-way repeated measures ANOVA “ISA Workload” .............................................. 55 Table 3-17: ASSA - Means, N, SD, and P-Value ................................................................................. 56 Table 3-18: Two-way repeated measures ANOVA “ISA Situation Awareness”.................................. 57 Table 3-19: Pilots’ raw data, M, and SD of the ISA (Workload).......................................................... 58 Table 3-20: Pilots’ raw data, M, and SD of the ISA (Situation Awareness)......................................... 58 Table 3-21: Pilots’ executed Test Runs in the ATTAS Test Aircraft ................................................... 62 Table 4-1: Sensor equipment of TUD’s Navigation Test Vehicle ................................................ 109

EMMA2



7 Annex II

EMMA2



7.1 Raw Data (DLR RTS)

7.1.1 QE-OF

ID P1 P2 P3 P4 P5 P6 P7 P8 2-G

GTD 6 6 3 6 6 6 5 5 TXC 5 5 3 4 3 2 5 5

3-G GTD 6 6 4 6 5 6 5 5 TXC 4 4 4 4 4 2 4 5

5-G GTD 6 6 4 6 4 6 6 4 TXC 5 5 4 3 4 2 5 6

7-G GTD 4 6 6 5 4 6 6 6 TXC 3 3 6 5 2 6 4 6

9-G GTD 6 6 5 6 6 4 6 4 TXC 2 3 5 4 3 1 4 4

11-G GTD 5 6 4 5 2 3 4 4 TXC 5 3 4 4 2 2 4 4

12-G GTD 6 4 5 6 2 1 4 4 TXC 3 4 5 2 2 1 4 4

14-G GTD 6 5 6 6 6 5 5 6 TXC 3 4 6 3 5 4 4 6

15-G GTD 6 5 5 6 6 5 5 5 TXC 2 4 5 3 3 5 5 6

16-G GTD 5 5 4 n.a. 6 5 6 5 TXC 2 4 4 5 5 5 5 4

17-G GTD 6 4 5 6 6 6 6 6 TXC 2 4 5 6 6 6 5 4

19-G GTD 6 5 5 6 6 5 6 6 TXC 5 5 5 2 6 2 3 6

1-S 6 6 5 6 5 6 6 6 2-S 6 6 5 6 5 6 6 6 3-S 6 6 5 4 6 6 6 5 4-S 6 6 5 5 6 6 5 6 5-S 6 6 5 6 6 n.a. 6 6

10-R 6 5 5 5 6 5 6 5 2-H 6 5 5 6 5 5 5 6 3-H 5 5 5 6 5 5 6 5 4-H 4 5 5 5 2 5 6 5

EMMA2



ID P1 P2 P3 P4 P5 P6 P7 P8 5-H 1 5 5 5 3 1 4 5 7-H 6 6 6 5 3 5 5 6 8-H 5 6 5 5 5 5 5 5 9-H 5 6 5 4 3 5 4 4

10-H 5 6 5 6 5 5 4 6 11-H 6 6 5 6 6 5 6 5 14-H 6 6 6 6 6 6 5 5 15-H 3 5 4 4 1 6 2 4 18-H 6 5 5 5 n.a. 2 5 4 23-H 6 3 5 5 5 5 6 4 24-H 5 5 5 6 2 4 4 4 30-H 4 3 4 5 2 1 4 5 35-H 3 4 4 4 1 2 4 5 36-H 4 3 4 3 2 3 3 4 38-H 3 3 3 3 2 4 3 4 39-H 3 4 3 3 n.a. 1 1 2 40-H 5 5 4 2 5 2 5 6 42-H 3 4 5 1 2 1 5 5 47-H 5 4 5 5 5 1 6 4 48-H 6 5 5 2 2 5 5 4 2-T 6 6 5 6 6 6 6 3 3-T 4 5 3 3 1 1 3 5 6-T 4 5 4 n.a. 5 n.a. 5 2 8-T 2 4 4 3 2 3 4 2 10-T 1 2 2 2 2 2 1 3 11-T 2 4 5 6 6 5 3 5 13-T 2 1 2 4 n.a. 1 4 5 14-T 1 2 3 2 1 3 4 4 15-T 1 2 3 2 2 2 5 2 16-T 2 2 3 2 2 3 4 5 17-T 2 3 5 4 2 4 4 5 18-T 2 n.a. 3 n.a. 3 2 4 5 19-T 1 3 4 n.a. 1 2 4 4 22-T 5 4 3 5 3 2 5 6 23-T 5 4 3 1 5 5 6 6 25-T 6 4 3 1 6 3 5 6 26-T 5 4 4 5 5 5 4 6 27-T 2 4 5 4 2 3 5 6 28-T 1 6 5 6 2 5 6 4

7.1.2 QE-OI (Safety) ID P1 P2 P3 P4 P5 P6 P7 P8

1P-01 3 5 5 5 4 4 6 5 1P-02 5 6 6 6 6 5 6 5 1P-04 5 4 6 5 2 4 6 5 1P-06 6 6 5 6 6 5 6 6 1P-08 5 5 6 5 6 5 6 n.a.

EMMA2



7.1.3 QE-OI (Capacity/Efficiency) ID P1 P2 P3 P4 P5 P6 P7 P8

1P-03 5 5 5 4 6 5 6 6 1P-05 4 4 3 5 1 2 4 5 1P-07 6 6 5 6 6 4 6 6

7.1.4 SUS ID P1 P2 P3 P4 P5 P6 P7 P8 1 5 5 5 4 3 3 5 5 2 4 2 1 4 3 3 2 2 3 3 4 3 2 2 3 4 4 4 1 2 1 1 2 4 1 1 5 4 4 2 2 3 3 3 4 6 3 3 2 3 2 3 2 2 7 5 3 5 3 3 4 4 5 8 3 3 3 3 2 3 1 1 9 4 3 4 4 2 4 4 4

10 1 3 1 2 2 3 1 2

7.1.5 ASSA ID P1 P2 P3 P4 P5 P6 P7 P8

1 5 4 2 3 1 1 2 4 2 4 4 3 4 3 4 3 4 3 4 4 4 5 4 4 5 4 4 5 4 4 4 4 4 5 4 5 3 3 3 3 2 2 3 4 6 4 3 3 3 3 2 3 5 7 5 4 4 4 5 4 4 5 8 5 4 4 4 5 5 4 5 9 5 5 5 4 5 5 5 5

10 2 2 3 2 1 2 2 2 11 5 4 4 5 5 4 5 5 12 5 5 4 4 4 4 5 4 13 5 5 4 4 5 4 4 4 14 4 4 4 4 4 3 4 5 15 5 4 5 4 5 4 5 5

airborne validation results part a · emma2 airborne validation results part a save date:...

Documents