Air Safety: End of the Golden Age?

Arnold Barnett

Blackett Memorial Lecture

How should we measure aviation safety?

“NTSB studies show that, from 1993 through 1996, scheduled US carriers averaged only 0.2 fatal accidents per 100,000 flight hours, less than half the fatal accidents rate for the four-year period a decade earlier.

--Wall Street Journal, 8/11/97

Two problems with the statistic fatal accidents per 100,000 flight hours:

The numerator and

the denominator!

• The generic term “fatal accident” blurs the distinction between a crash that kills one passenger out of 300 and another that kills 300 out of 300.

• Measuring activity by “flying hours” misses the point that most accidents occur on landing or takeoff.

Airline Report Cards(Air Travelers Association)

Score = 100 - 10,000 (Z/N)

where Z = number of fatal crashes over 1987-96

N = number of flights over 1987-96 (1000’s)

Letter Grades: A: 90 -100 B: 80-89.9

C: 70-79.9 D: 60-69.9 F: below 60

This grading system:

• Again uses the generic term “fatal accident”

• Is quite arbitrary, and sometimes assigns vastly different grades because of statistically meaningless differences

Example:• Airline #1

200,000 flights over 1987-96No fatal accidents

Letter Grade: A

• Airline #2200,000 flights over 1987-96

One fatal accident, which kills one passenger out of 100

Letter Grade: F

What about hull losses per 100,000 departures?

(This is a popular one.)

Consider two hull losses this year:• Southwest Airlines, Boeing 737, Burbank, CA

Passengers on board: 137 Passengers killed: 0

• Alaska Airlines, MD-80, off Los Angeles

Passengers on Board: 83 Passengers Killed: 83

No difference?

Measure of Safety Performance Over a Past Period:

Death Risk Per Randomly Chosen Flight

Question: If a person chooses a flight at

random from among those of interest (e.g. UK domestic jet flights over the period 1990-95), what is the probability that he will not survive it?

This death risk per flight statistic has conceptual advantages compared to the other statistics just discussed.

What Conceptual Advantages?

• Ignores length and duration of flight, which are virtually unrelated to mortality risk

• Weights each crash by the percentage of passengers killed

• Easy to calculate and understand

First-World Domestic Jet Services

Death Risk per Flight, 1990-99:

1 in 13 million

At a mortality risk of 1 in 13 million per flight, a passenger who took one flight per day would on average travel for 36,000 years before dying in a plane crash.

Passenger Mortality Risk for Various World-wide Jet Services, 1990-99

Type of Service Death Risk per Flight

First-World Domestic 1 in 13 million

International withinFirst World 1 in 6 million

International Between FirstAnd Developing Worlds 1 in 1 million

Within Developing World 1 in 500,000

Passenger Mortality Risk Arising from Criminal/Terrorist Acts, Scheduled First-World Jet Services Over 1990-99

Type of Service Death Risk per Flight

US:Domestic 0International 0

First World Outside US:Domestic 0International 1 in 2 billion

This record is all the more remarkable because of several successful acts of sabotage in the late 1980’s.

Two Possible Reasons for the Quiet Decade:

• The desire to do harm to First-World air travelers genuinely diminished.

• Improved security measures deterred some potential attacks and foiled others.

Unfortunately, neither of these explanations is especially convincing.

October 2000:

“Fear of terrorist attacks after the explosion of violence in the Middle East hammered global airline shares Friday.”

--Reuters

Passenger Mortality Risk Arising from Runway Collisions, Scheduled First-World Jet Services Over 1990-99

Type of Service Death Risk per Flight

US:Domestic 1 in 100 millionInternational 0

First World Outside US:Domestic 0International 0

We were asked to investigate the following question:

How great a threat do US runway accidents pose to domestic airport operations in the next two decades?

If there are N aircraft operations at a given airport in a given year, then:

To a first approximation, one might expect that the risk of a runway accident would vary with N2.

Why?

1) The number of flights that could (theoretically) collide is (N2-N)/2.

2) The Quadratic Model is conceptually attractive.

3)N2 is widely used in airspace collision-risk models.

But, to the extent possible, it is desirable to go beyond merely stating conjectures, and to test hypotheses and “approximations” against empirical evidence.

A most interesting data set

The 40 US runway incursions in 1997 that:

(1) were judged by experts to have “extremely high” accident potential

and(2) took place under known conditions of

reduced visibility (night, sunrise/sunset).

The N2-hypothesis passed a Chi-squared statistical test with flying colors.

(The test was based on the spread of the 40 dangerous events across US airports.)

Intriguingly, the hypotheses that dangerous events varied across airports with either N or N3 did not pass Chi-squared tests.

Overall, runway collisions over the next two decades could take 600 lives among US jet passengers, and cause 200 serious injuries.

(Mid-range projection)

Estimated Runway Collision Death Risk per Flight, US Domestic Jets 2003-22:

1 in 25 million

(Four times the actual risk in the 1990’s.)

Passenger Mortality Risk Arising from Mid-Air Collisions, Scheduled First-World Jet Services Over 1990-99

Type of Service Death Risk per Flight

US:

Domestic 0International 0

First World Outside US:

Domestic 0International 0

(Based on 100 million flights)

Is it safe to adopt free-flight?

(Operations Research, Nov 2000)

A B

C

DE

F

Present Routings: A-E-F-B and C-E-F-DFree Flight Routings: A-B and C-D

Under certain assumptions, free-flight would:

• Reduce the likelihood of path intersections

• Tend to reduce the crossing angles of paths that intersect

Why is the latter point important?

Because:

At present, emergency warnings go off in air-traffic control towers when two planes come within five miles of one another, regardless of the angle at which they are converging.

Consider two planes on a collision course that have just come within five miles of one

another.

For example:5

C

3.5

3.5

“Resolution Time” As a Function of Angle of Convergence

Convergence Angle Time to Resolve Emergency

180º 18 seconds 90° 25 10 30º 70 5 30°

(Assume both planes going 500 mph)

Because of fewer path crossings and longer times to resolve emergencies, the geometric consequences of free-flight might act to reduce mid-air collision risk.

However:

All these apparent benefits of free-flight could be more than outweighed by a decline in “situational awareness” on the part of air traffic controllers.

How Does It All Add Up?

Aviation Safety: Time To Stop Worrying?

Arnold Barnett