Lead Time Aviation Verification

Onset and Cessation of Ceiling and Visibility Flight Category Conditions (IFR, MVFR, VFR)

at FAA Core Airports

NWS Aviation Services Branch

Aviation Services Virtual Workshop

April 16, 2013

2

Lead Time Accuracy Requirements

• Identified by Joint FAA/NWS Traffic Flow Management (TFM) Weather Requirements Working Group (TRWG)

• Performance-Based Weather Information Requirements– Linked to Decisions for Traffic Management Initiatives– Accuracy needed at decision points prior to onset and cessation

• New Verification Method, Different than Traditional Method– Probability of Detection (POD) and False Alarm Ratio (FAR)

only at time of onset and cessation, not throughout event duration

• Verification Metrics to Baseline Current Support against New Performance-Based Requirements– NWSH Performance Branch developed modified version of Stats

on Demand

3

Lead Time Accuracy Requirements

Accuracy Criteria Needed for Time of Onset and Cessation of IFR, MVFR, VFR Conditions

IFR, MVFR, VFR Conditions for Core Airports

Lead Time

Projection

Accuracy

(minutes)

Probability of

Detection

False Alarm Ratio

Timing Error

(minutes)

Time of Onset

0 - ≤ 2 h ± 15 min ≥ 80% ≤ 20% ≤ 15 min

> 2 - ≤ 4 h ± 30 min ≥ 75% ≤ 25% ≤ 30 min

> 4 - ≤ 6 h ± 45 min ≥ 70% ≤ 35% ≤ 45 min

> 6 - ≤ 8 h ± 60 min ≥ 65% ≤ 40% ≤ 60 min

Time of Cessation

0 - ≤ 2 h ± 15 min ≥ 80% ≤ 20% ≤ 15 min

> 2 - ≤ 4 h ± 30 min ≥ 75% ≤ 25% ≤ 30 min

> 4 - ≤ 6 h ± 45 min ≥ 70% ≤ 35% ≤ 45 min

> 6 - ≤ 8 h ± 60 min ≥ 65% ≤ 40% ≤ 60 min

4

Lead Time to Cig/Vis Event Example

Obs

00Z TAF

0:00 3:00 6:00 9:00 12:00 15:00 18:00 21:00 0:00

VFR MVFR IFRMVFR Onset 0536Z

VFR Cessation 0536Z

MVFR Forecast Onset 0500Z

VFR Forecast Cessation 0500Z

5

TAF Lead Time - IFR Onset POD

2-4 Hour Lead Time to Onset of IFR Conditions at Core Airports

Pro

babi

lity

of D

etec

tion

Num

ber of Events/F

orecasts

75%

2012 2013

6

TAF Lead Time – IFR Onset FAR

2-4 Hour Projection to Onset of IFR Conditions at the Core 30 Airports

25%

Num

ber of Events/F

orecastsF

alse

Ala

rm R

atio

2012 2013

7

TAF Lead Time - IFR Onset Timing Error

Mean Absolute Timing Error: 2-4 Hour Lead Time to Onset of IFR Conditions

30 min

Num

ber of Events/F

orecastsT

imin

g E

rror

(m

inut

es)

2012 2013

8

TAF Lead Time - IFR Cessation POD

2-4 Hour Lead Time to Cessation of IFR Conditions at Core Airports

75%

Num

ber of Events/F

orecastsP

roba

bilit

y of

Det

ectio

n

2012 2013

9

TAF Lead Time - IFR Cessation FAR

2-4 Hour Projection to Cessation of IFR Conditions at the Core 30 Airports

25%

Num

ber of Events/F

orecastsF

alse

Ala

rm R

atio

2012 2013

10

TAF Lead Time - IFR Cessation Timing Error

Mean Absolute Timing Error: 2-4 Hour Lead Time to Cessation of IFR Conditions

30 min

Num

ber of Events/F

orecastsT

imin

g E

rror

(m

inut

es)

2012 2013

11

TAF Lead Time - MVFR Onset POD

2-4 Hour Lead Time to Onset of MVFR Conditions at Core Airports

75%

Num

ber of Events/F

orecastsP

roba

bilit

y of

Det

ectio

n

2012 2013

12

TAF Lead Time - MVFR Onset FAR

2-4 Hour Projection to Onset of MVFR Conditions at the Core 30 Airports

25%

Num

ber of Events/F

orecastsF

alse

Ala

rm R

atio

2012 2013

13

TAF Lead Time - MVFR Onset Timing Error

2-4 Hour Lead Time to Onset of MVFR Conditions at Core Airports

30 min Num

ber of Events/F

orecastsT

imin

g E

rror

(m

inut

es)

2012 2013

14

TAF Lead Time - MVFR Cessation POD

2-4 Hour Lead Time to Cessation of MVFR Conditions at Core Airports

75%

Num

ber of Events/F

orecastsP

roba

bilit

y of

Det

ectio

n

2012 2013

15

TAF Lead Time - MVFR Cessation FAR

2-4 Hour Projection to Cessation of MVFR Conditions at the Core 30 Airports

25%

Num

ber of Events/F

orecastsF

alse

Ala

rm R

atio

2012 2013

16

TAF Lead Time - MVFR Cessation Timing Error

Mean Absolute Timing Error: 2-4 Hour Lead Time to Cessation of MVFR Conditions

30 min Num

ber of Events/F

orecastsT

imin

g E

rror

(m

inut

es)

2012 2013

17

TAF Lead Time - VFR Onset POD

2-4 Hour Lead Time to Onset of VFR Conditions at Core Airports

75%

Num

ber of Events/F

orecastsP

roba

bilit

y of

Det

ectio

n

2012 2013

18

TAF Lead Time - VFR Onset FAR

2-4 Hour Projection to Onset of VFR Conditions at the Core 30 Airports

25%

Num

ber of Events/F

orecastsF

alse

Ala

rm R

atio

2012 2013

19

TAF Lead Time - VFR Onset Timing Error

2-4 Hour Lead Time to Onset of VFR Conditions at Core Airports

30 min Num

ber of Events/F

orecastsT

imin

g E

rror

(m

inut

es)

2012 2013

20

TAF Lead Time - VFR Cessation POD

2-4 Hour Lead Time to Cessation of VFR Conditions at Core Airports

75%

Num

ber of Events/F

orecastsP

roba

bilit

y of

Det

ectio

n

2012 2013

21

TAF Lead Time - VFR Cessation FAR

2-4 Hour Projection to Cessation of VFR Conditions at the Core 30 Airports

25%

Num

ber of Events/F

orecastsF

alse

Ala

rm R

atio

2012 2013

22

TAF Lead Time - VFR Cessation Timing Error

Mean Absolute Timing Error: 2-4 Hour Lead Time to Cessation of VFR Conditions

30 min

Num

ber of Events/F

orecastsT

imin

g E

rror

(m

inut

es)

2012 2013

23

Summary

• Lead Time Accuracy Requirements– Decision process needs timing of Onset/Cessation of events

impacting TFM on scale of tens of minutes

• New Verification Method, Different than Traditional Method– Preliminary Flight Category verification results show significant gap

between accuracy baseline and decision process needs– TAF hourly time scale too coarse to meet need of TFM decisions– Model guidance time scale likely to lead to similar results

• Next Steps for Lead Time Assessment Tool– Wind Direction Changes, Thunderstorms, Model Guidance– Accessible on Performance Management Website– Training for Lead Time Verification Method and Results Interpretation