the challenge of routine flight operations loukia d. loukopoulos r. key dismukes immanuel barshi...

THE CHALLENGE OF ROUTINE FLIGHT OPERATIONS

Loukia D. LoukopoulosR. Key DismukesImmanuel Barshi

NASA Ames Research CenterMoffett Field, CA

U.S.A.

22nd European Annual Conference on Human Decision Making and Control2-4 June, 2003

Linköping University, Sweden

2

MOTIVATION

Complex operating environment (cockpit)• pilots• number of tasks• timing of tasks

FlowsChecklists

ProceduresPhase of flight structure

? • INTERACTIONS among tasks and operators

• Characterize nature of interaction• Characterize demand such interaction places on cognition• Errors?• Solutions?

3

METHODSComprehensive approach

• Cockpit jumpseat observation of routine flights (~100 hours) • Boeing 737; 2+ air carriers; 1- 4 hours per leg; U.S.-wide• notes; informal interviews

• Training (initial, transition/upgrade)• Written materials (manufacturer and carrier FOMs)• Incident reports (ASRS, NTSB)

Qualitative analysis applies to all phases of flight * use Preflight for illustration (Taxi Out in paper) *

4

real

Idealmanufacturer perspective (manuals)

ideal

assumed

PERSPECTIVES - layering

Assumedcarrier perspective (FOMs, training)

Errors pilot perspective (ASRS reports)

Real observer perspective (jumpseating)

5

Manufacturer’s representation? (ideal)

6

PHASE OF FLIGHT - ideal (manufacturer)

Goals:• Prepare aircraft for next phase of flight

Considerations:• Safety• Design specifications

Perspective:• Detached from operational reality

Preflight - ideal (manufacturer)

Begin checklist

Checklist complete

Begin checklist

Checklist complete

ENGINE START & PUSHBACK

Ask for checklist

Ask for checklist

CAPTAINCAPTAIN FIRST OFFICER FIRST OFFICER

CLEARANCE

checklist

checklist

procedure

procedure

8

Carrier’s representation? (assumed)

9

PHASE OF FLIGHT - assumed (carrier)

Goals:• Prepare aircraft for next phase of flight

Considerations:• Safety• Design specifications

+ Additional constraints + Priorities (fuel, time, passenger comfort)+ Type of operations (short/long-haul, turn-arounds,

regions/airports, fleet)+ Philosophy (safety, efficiency)+ New technologies (ACARS, on-board performance computer)+ Coordination with agents (dispatch, maintenance, company)

Perspective:• Suggestive of operational reality

Preflight - assumed (carrier)

Obtain ATIS

Obtain clearance

Review paperwork

Prepare/review charts

(Passenger count)

(Load Sheet)

Program FMC

Begin checklist

Checklist complete

Begin checklistChecklist complete


Review paperwork

Sign flight release


Review Load Schedule

Review FMC

Takeoff brief

Ask for checklist

Ask for checklist


CLEARANCE

procedure

procedure

checklist

checklist

Cabin Attendant

Gate Agent

ACARs / OPC

Interphone

Ground/Company/Dispatch

Frequencies

11

1

2

3

4

A

B

C

D

PHASE OF FLIGHT - assumed (carrier)

BOEINGFinal Flight Deck Preparation (down to the line)

Challenge ResponseFlight deck preparation CompletedLight test CheckedOxygen and interphone CheckedYaw damper OnNavigation transfer and display switches AUTO & NORMALFuel _ lbs. & pumps OnGalley power OnEmergency exit lights ArmedPassenger signs SetWindow heat OnHydraulics NormalAir conditioning and pressurization _ pack(s), bleeds On, setAutopilots DisengagedInstruments Cross-checkedAutobrake RTOSpeed brake Down detentParking brake SetStabilizer trim cutout switches NormalWheel well fire warning CheckedRadios, radar, transponder, and HUD SetRudder and aileron trim Free & zeroPapers AboardFMC/CDU SetN1 and IAS bugs set

BOEINGFinal Flight Deck Preparation (below the line)

Challenge ResponseDoors ClosedFlight deck windows LockedAir conditioning pack switches OffAnti collision light switch On

+ x

- y

if z then C’


INFORMATION

EVENT

PILOT-driven operation

Activities are:• Linear: task B always follows task A, in this sequence • Controllable: tasks initiated by pilot, independently

• Predictable: information available when needed communication possible when necessary

monitor

12

COGNITIVE ASPECTS - (carrier perspective)

Activities are:

Linear, Controllable, Predictable

Are these good outcomes?

√ Standardization (between pilots/crews)

√ Accurate, efficient, “safe” performance

Training and repeated Practice on the line build:

• Automaticity (reduced cognitive requirements)

• Habit (speed, accuracy)

• Reliance on triggers (memory aid)

• (task 2 triggers task 3)

• (Captain’s request for the checklist cues FO to begin checklist)

13

… and so it should all work as expected…

… or maybe not?!

PREFLIGHT Captain’s ASRS Report (edited)

“At this point the scheduled flight is about 1.5 hours behind schedule. Prior to our arrival we experienced some problems with the autopilot system and had to call maintenance for approval to continue operations in accordance with the MEL. I discussed this with FO who contacted company, and called for fuel. I went inside the terminal to call Maintenance.

I headed back to the aircraft… the passengers were boarding and the FO was at the back of the aircraft. As we both got into the cockpit, the FO started the Load Manifest and I completed the write-up in the logbook. I then started the Before Start checks but was distracted by the events of a final baggage and passenger count. There were other flights on the ground and I had seen the fuel truck moving about.

With the passengers and bags onboard and the doors closed, the engines were started and a call for taxi made. The main runway has been closed for construction and we were given a shorter runway. After a look at the takeoff data, winds, and weights, we advised Tower we needed a different runway. When cleared for takeoff we had not completed the checklist so we advised Tower we needed some more time. Once completed, we departed with a climb in VFR conditions. After climbing through 12000 ft I was scanning the instruments and noticed the fuel quantity only showed 980 lbs.

The series of events from the first landing until takeoff was about 15 minutes. I feel there were many distractions that interrupted the flows on the preflight check and items were skipped.”

Obtain ATIS

Obtain clearance

Review paperwork


(Passenger count)

(Load Sheet)

Program FMC

Begin checklist

Checklist complete



Review paperwork

Sign flight release



Review FMC

Takeoff brief

Ask for checklist

Ask for checklist


CLEARANCE

PREFLIGHT errors attributed to “distractions” (incident reports)

Omitted review of (SID) charts - distractions - speed violation on departure

Failed to program new departure - interruptions during preflight - discover after takeoff

Forgot to request updated PDC - distractions; misled because preflight already complete - take off with expired PDC

Skipped checklist item - interruption - push back with Seat Belt signs off

Omit requesting Load sheet - defer during preflight; ACARS inoperative - takeoff without load sheet

Omitted check of circuit breakers - busy with preflight; rushing - unable to start engine after pushback

Never finished review of maintenance log - interruptions - aircraft flown with “open” item in logbook

Checklist item read but not verified - interruption - pushback with emergency door slides armed

Entered wrong weight into FMS - tail strike at takeoff

Missed checking main tank fuel quantity - distractions - departed with insufficient fuel

Neglected to add MEL to flight release - multiple distractions, busy with preflight - discovered en route

Skipped over checklist item - interruption - departed without lobgook

Skipped over checklist item - interruptions - discover insufficient fuel after pushback

Omitted flow and checklist items - interruptions; delay; change in departure runway - discover insufficient fuel at 12000 ft

Forgot to call maintenance - distractions - depart with hatch installed backwards

Forgot to request PDC - deferred when instructed to request by voice, not ACARS - discover after takeoff

Forgot to complete preflight flow - interruption - took off with APU running

Missed aircraft not refueled - checklist interrupted - return to airport after taking off

Skipped over checklist item - fuel pumps deferred during preflight because refueling - engine starvation in flight

Improper setting of pressurization during preflight flow - interruptions - cabin altitude warning light in cruise

Read but not verify checklist item - distractions - pushback with throttles open, damage to aircraft

Forgot logbook at ramp - kept deferring to check it; distractions; busy with preflight - discovered en route

16

•Wide range of outcomes

•Large number of omissions not caught by later checklist

•Consequences spill into all phases

•Potential for accidents

•$$$ time, fuel, ramp mishaps, taxiway/runway incursions, rejected takeoffs

ERRORS

17

Line representation (real)

Obtain ATIS

Obtain clearance

Review paperwork


(Passenger count)

(Load Sheet)

Program FMC

Begin checklist

Checklist complete



Review paperwork

Sign flight release



Review FMC

Takeoff brief

Ask for checklist

Ask for checklist


CLEARANCE

procedure

procedure

checklist

checklist

Cabin Attendant

Gate Agent

ACARs / OPC

Interphone

Ground/Company/Dispatch

Frequencies

Preflight - real (compiled observations)

procedure

Interruption

FO busy

Interruption

busy frequencyKeep trying

Ask for checklist

Resume flow

Inoperative item

Time pressure

Inoperative item

Flight release still not picked up

no time, familiarity

Interruption

Delay at gate

Ramp and/or Ground?

Flight plan/Departure runway change

Conduct exterior walk-around

no time, familiarity

New PDC

Still refueling

Data unavailable

Passenger count unavailable

Call maintenance

Look for ops/gate agent

Double-check charts

Resume checklist

Confirm Mx responded

Confirm Mx departed

Confirm resolution

Confirm logbook on board

Check charts

Defer programming FMC

Communicate with companyCompute new performance #s

Re-program FMS

Re-program FMC

Check fuel quantity and pumps

Re-brief

Re-flow trim & other settings

New flight release/PDC?Re-set MCP

Takeoff brief

Request passenger count

19

1

2

3

4

A

B

C

D

PHASE OF FLIGHT - real

*monitor z

z1z2z3z4z5

Activities are:

Linear:

x y

z

*Wait


BOEINGFinal Flight Deck Preparation (down to the line)

Challenge ResponseFlight deck preparation CompletedLight test CheckedOxygen and interphone CheckedYaw damper OnNavigation transfer and display switches AUTO & NORMALFuel _ lbs. & pumps OnGalley power OnEmergency exit lights ArmedPassenger signs SetWindow heat OnHydraulics NormalAir conditioning and pressurization _ pack(s), bleeds On, setAutopilots DisengagedInstruments Cross-checkedAutobrake RTOSpeed brake Down detentParking brake SetStabilizer trim cutout switches NormalWheel well fire warning CheckedRadios, radar, transponder, and HUD SetRudder and aileron trim Free & zeroPapers AboardFMC/CDU SetN1 and IAS bugs set

BOEINGFinal Flight Deck Preparation (below the line)

Challenge ResponseDoors ClosedFlight deck windows LockedAir conditioning pack switches OffAnti collision light switch On

CLEARANCE

PILOT-driven

Dynamic

Inter-dependency of tasks and operators

Unpredictable interactions (nature and timing)

ENVIRONMENT-driven operation

Predictable:

Controllable:

20

• Multiple interactions among operators and tasks

(within, outside cockpit)• Pervasive and unnoticed: most often dealt with successfully• Training

• does not adequately reflect reality • does not adequately prepare pilots for line flying (learn on the job)

• Interactions are not well characterized• Effects (challenges) are underestimated• Vulnerability to errors are not appreciated

• Neither is the potential for such errors to lead to accidents

Results

21

• Characterize nature of interaction• Characterize demand such interaction places on cognition

So what are these Interactions?

22

Interactions 1

Pilots are constantly being interrupted

Interruptions demand attention (to note and assess)

and action (to resolve)

Observations from the jumpseat

• The first officer monitored the Ground frequency throughout the preflight preparations. A call with departure information interrupted her as she was setting the pressurization panel (a prescribed step in the procedure). She continued “dialing” the destination altitude before interrupting the procedure to acknowledge the call from Ground. She then copied the memorized information on the load sheet before resuming the interrupted procedure.

23

Interactions 2

Pilots must often perform >1 tasks concurrently•Some combinations are often practiced together•Others combinations are new & must be carried out on the spot

From the jumpseat…

• The captain monitored the cockpit interphone while performing his preflight duties and expecting to be contacted by the push back crew. A flight attendant notified him of a potential issue with a passenger. He relied on the cabin crew to resolve it with the help of the Gate Agent and continued with the preflight procedure. At the same time, he continued to monitor the conversation taking place right outside the cockpit door, intending to intervene if necessary.

24

Interactions 3

Pilots do not/cannot always initiate actions• Circumstances may not be right for an action• Information/tool/agent may not be available

From the jumpseat…

• CA reached the point on his preflight procedure calling for verification of sufficient fuel on board. He looked around for a fuel slip but quickly realized that it had not yet been brought to the cockpit - that implied that refueling was still underway. He formed the intention to return to the fuel quantity action item later and proceeded on with the remaining procedural steps.

25

Challenge 1

In order to respond to the multiple, complex, and unpredictable interactions, a pilot must …

Suspend/Defer activities

i.e., must resume the suspended task … or

start the deferred task at a later time

The first officer monitored the Ground frequency throughout the preflight preparations. A call with departure information interrupted her as she was setting the pressurization panel (a prescribed step in the procedure). She continued “dialing” the destination altitude before interrupting the procedure to acknowledge the call from Ground. She then copied the memorized information on the load sheet before resuming the interrupted procedure.

requires• forming an intention (acknowledge call, copy information)

• maintaining the intention (while continuing primary activity)

• remembering what/when suspended/deferred (call from Ground)

• noticing when it is time to resume/start intended task (once finished with primary activity)

26

Challenge 2In order to respond to the multiple, complex, and

unpredictable interactions, a pilot must …

Interleave activities

i.e., combine habitual or novel activities …

in previously-encountered or new ways

requires• shift attention between activities• remembering the activities • remembering to check on the progress of each activity• deciding when/how often to check progress

27

Research Implications

• Events in the environment continue to flow in habitual sequence and with their usual timing

• “Triggers” for sequence of activities are disrupted or even lost altogether

Memory

esp. Prospective Memory (for intentions)

Attention

continuous sampling of environmental stimuli monitoring (essentially a continuous PM demand)

Pilots are vulnerable to errors under such circumstances because…

28

Practical Implications (carriers)

• Acknowledge pilot fallibility• Examine procedures and checklists, incident (ASAP)

reports• Blank-slate approach

• source• nature• timing of challenges in everyday operations

• Design robust procedures• warning signs (red flags)• triggers for items at risk (tie critical functions to fixed

reference points)

• traps (strict decision criteria not to cross unless all expected activities fulfilled)

• Develop (better) guidance for Monitoring

Cognitive Performance in Aviation Training and Operationshttp://human-factors.arc.nasa.gov/flightcognition/

30

Backup slides from here on...

PANEL OFF light --- Extinguished Set panel --- As desired WARNING: Do not key HF radio while airplane is being fueled. Injury to personnel or fire may result

FLIGHT ALTITUDE indicator --- cruise altitudeLANDING ALTITUDE indicator --- destination field elevationPressurization mode selector --- AUTO AUTOMATIC FAIL light --- Extinguished

PreflightB737 Ops Manual

32

TAXI - ideal (manufacturer)

Goals:• bring aircraft to departure runway• configure it for takeoff (next phase of flight)

Considerations:• safe operation• within performance limitations and design specifications

TaxiB737 Ops Manual

Receive taxi clearance

Start checklist

Checklist complete

Receive takeoff clearance

Start checklist

Checklist complete


Start taxiing

Ask for checklist


Ask for checklist

Line up with runway

Taxi - ideal (manufacturer)

TAKEOFF


35

PHASE OF FLIGHT - ideal (manufacturer)

A

B

C

D

1

2

3

4


Activities are:

• Linear: task B always follows task A, in this sequence

• Controllable: tasks initiated by pilot, independently

CLEARANCE

• Predictable: information available when needed communication possible when necessary

36

TAXI - assumed (carrier)

Goals:• bring aircraft to departure runway• configure it for takeoff (next phase of flight)

Considerations:• Safe operation• Within performance limitations and design specifications

+ Carrier (additional) considerations + Priorities (fuel, time, passenger comfort)+ Type of operations (short/long-haul, turn-arounds,

regions/airports, fleet)+ Philosophy (safety, efficiency)+ Technologies on-board (ACARS, laptop)+ Coordination with agents (Ground, Tower, dispatch, other

aircraft)


Start checklist

Checklist complete


Start checklist

Checklist complete

Start taxiing

Ask for checklist


Ask for checklist

Line up with runway

Taxi - assumed (carrier)

Taxi Clearance

MONITORGround

Company

Takeoff Clearance

MONITOR Ground

Company/Dispatch

MONITOR Captain taxiing

TAKEOFF


TAXI-OUT -real Captain’s ASRS Report # 425357 (edited)

“The aircraft we had for this flight had an inoperative fuel gauge, and an inoperative APU… DFW was in the midst of a departure rush. Since we had to do a crossbleed start, we did not do the After Start checklist immediately...

There were many distractions leading up to this incident which should have been warning signals.

There was a time pressure element, we were running late and knew we had an airplane change in a short ground time in Denver before our next flight.

The APU and the necessity for the crossbleed start precluded the normal flow of calling for flaps and the before takeoff checklist as we taxi from the gate.

I was distracted by calling for taxi, so ground control knew we were ready to move.

Immediately after that call we were given a complicated taxi route with hold short and follow instructions and we were concentrating on finding the taxiways in the dark…”

We got clearance to taxi and requested a spot to do the start. We stopped ... as instructed and completed the start. As we were finishing, ground control was giving instructions to aircraft taxiing mentioning us in reference. We heard this, completed the after start checklist, and told ground we were ready to taxi.

We were given instructions to hold short of taxiway WJ, give way to opposite direction traffic, then proceed N to taxiway Z to taxiway HY to taxiway Y across the bridge to taxiway J to taxiway EF for takeoff on runway 17R, follow an ACR aircraft. With all the above, taxiing to a new and unfamiliar runway (for this crew) in the dark, we didn't complete the before takeoff checklist. As we were cleared for takeoff and applied power, the takeoff warning horn sounded and we immediately realized that the flaps had not been extended for takeoff.


Start checklist

Checklist complete


Start checklist

Checklist complete

TAKEOFF

Start taxiing

Ask for checklist


Ask for checklist

Line up with runway

TAXI-OUT errors attributed to “distractions” (ASRS incidents)

Failed to start engine #-2 - distracted while discussing special operations for destination; omitted checklists - delay takeoff

Omitted call for flaps -rushed to clear ramp/gate area for arriving aircraft -aborted takeoff

Omitted flaps - crew discussing problem with APU, delayed flaps due to snow - aborted takeoff

Flaps incorrectly set, missed noticing during checklist - crew busy with fuel problem, runway changes, programming FMC - aborted takeoff

Omit above-line Checklist - running late, checklist interrupted by Tower, unexpected clearance for takeoff - abort takeoff

FO failed to monitor CA - busy checking and correcting calculations of load data - taxi past hold short line

FO failed to monitor CA – busy with flow; night taxi – taxi in wrong direction

CA taxied without having fully understood instructions - busy looking at other aircraft on taxiway and ramp - ground controller issued warning

FO failed to monitor CA - busy with pre-takeoff preparations - aircraft crossed hold short line

Flaps incorrectly set - late paperwork ; runway change; programming FMC; short taxi; rushed to accept takeoff clearance- aborted takeoff

Started taxi without clearance - trouble-shooting problem with engine start - nearly hit ground handler

Started taxi without clearance - crew discussing taxi instructions - struck pushback tug

Omitted flaps - checklist interrupted by Tower; crew rushed to accept takeoff clearance - aborted takeoff

Omitted flaps - checklist interrupted by thrust reverser light; crew busy troubleshooting - aborted takeoff

Misunderstood Tower instruction - new FO on IOE, CA coaching FO - taxi onto runway w/o clearance

Confuse own position on taxiway diagram - new terminal; studying NOTAMs; runway change – taxied into ditch

Started taxi without clearance – rushed by other aircraft waiting to pull into gate; radio congestion; marshaller’s headset inoperative – query by ground controller

Omitted checking reason for bleed air indicator light-busy with delayed engine start and checklists - takeoff without troubleshooting

Fail to confirm flap position - evaluating heavy rain showers; rushed to accept takeoff clearance - aborted takeoff

Neglected to set flaps -preoccupied with new departure clearance and packs-off operation -aborted takeoff

FO failed to monitor CA -runway change; busy reprogramming FMC -taxied past intended taxiway

Omitted checklist - busy with delayed engine start and checklists; rushed to accept takeoff clearance - flaps not set, aborted takeoff

Incorrect trim setting - checklist interrupted after item had been read but not verified - aborted takeoff


TAKEOFF


Start checklist

Checklist complete


Start checklist

Checklist complete

Start taxiing

Ask for checklist


Ask for checklist

Line up with runway

Taxi Clearance

MONITORGround/Company

Takeoff Clearance

MONITOR Captain taxiing

Taxi-out - real (compiled observations)MONITOR

Ground//Dispatch

Acknowledge clearance

Form mental picture of taxi routeConfirm CA’s understanding

Remember taxi instructionsId taxiways and turns

Remember to follow aircraftIdentify aircraft to follow

Remember to hold shortId correct place to hold short

New/ Additional taxi instructions

Unfamiliarity with airport


Strobes

LandinglightsRadar?

“Clear” ramp areaConsult charts

Shoulder harnesses

FMC update

“Clear” runway

APU off for 2 min before off-loading

Keep head up/ outside

Calculate & reset Performance data

Inform Company (new #s, delays)

Cross check with CA

FMC: program/verify

Keep trying

MONITORaircraft position on chart

MONITORTower frequency

MONITOR airport traffic

Acknowledge clearanceConfirm CA’s understanding

Defer takeoff flaps

Defer checklist

Restart engineRepeat checklists

Brief new runway

Check charts

De-icing Checklist

APU? Packs?

Repeat Checklist Take control of aircraft while finishing checklist

Accept new runway?

Consult charts

Keep trying

Consult charts

Before/After StartChecklist

Form mental picture of route

Resume checklist

Just-in or new load data

busy frequency

Interruption

Ice/Snow

Ice/Snow

traffic, FO busy

Change in takeoff runway

Change in takeoff sequence FO’s leg

Ramp and/or Ground?

Extended taxi delay

Busy airport/ departure rush

Unfamiliarity with airport

Delayed engine start

busy frequency

41

components

MOTIVATION: study complex operating environment as an entity

nature (constraints)

operator(s)

external agents

tasks

hardware/software

• Characterize nature of interactions• Characterize their effects on the (routine flight) operation• Develop or help guide development of tools/solutions to mitigate errors

cognitive abilities

tools (solutions)

flows checklists

procedures (SOP)

phase of flight structure

(mfgr/engineers) (design - $$$ -

technology)

interactions ?

?

number timing

the challenge of routine flight operations loukia d. loukopoulos r. key dismukes immanuel barshi...

Documents

phases of flight

idealphase of flight

assumedphase of flight

assumed carriergoals

tasks interactions

phase of flightconsiderations

assumed carrier x yif

hours boeing