casa’simpact on radar-based wind assessments

1
CASA’s Impact on Radar-Based Wind Assessments Don J. Rude * , Ellen J. Bass * and Brenda Philips ** * University of Virginia, ** University of Massachusetts This work is supported primarily by the Engineering Research Centers Program of the National Science Foundation under NSF award number 0313747. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the National Science Foundation. DISCUSSION Since wind speed plays a critical role in the definition of severe thunderstorms, the purpose of this study was to measure the impact of CASA radar data on wind assessment performance. The results reported herein clearly indicate that for wind speed assessments CASA radar data can simultaneously • increase wind speed estimates • reduce assessment error • increase confidence This outcome is particularly promising given that • data are from experimental radars • participants were given minimal training • the radar display and computer-interface were unfamiliar • many participants said CASA data confirms their mental model The increase in wind speed assessments supports research by Brown and Wood (Brown et al., 2005) which predicts that increased spatial sampling results in higher radial velocity readings. The shift in warning decisions when given CASA data is especially notable because only one experimental task was covered by an actual warning according to NWS archives. As systems such as CASA, NEXRAD SuperRes, and MPAR (Heinselman, Priegnitz, Manross, Smith, & Adams, 2008) come on- line, the forecasting community may need to revisit policies and thresholds for issuing warnings. Future Work The current study could be enhanced by the systematic control of radar beam attributes. Update rate, beam height, wind-to-beam intersection angle, and sampling fidelity each influence performance. Future work could quantify the impact of these attributes individually. Additional measures of performance could be used including the size of warnings, their duration, and lead time. Finally, a standardized and rigorous training tool for CASA data would help remove any novelty effects from future studies. METHODOLOGY Introduction The Center for Collaborative Adaptive Sensing of the Atmosphere (CASA) is creating a new paradigm for radar systems based on low-cost networks of X- Band radars that collect high spatial and temporal resolution data from the lower troposphere. CASA operates a 4 node radar test bed in south-west Oklahoma to research the capabilities of this new technology and to understand how CASA data will impact decision making by users such as National Weather Service forecasters. We hypothesize that CASA technology will better inform forecasters, improve their performance, and ultimately improve severe weather warnings. This study was designed to measure the impact of CASA data on wind speed assessments, a critical component of the severe thunderstorm warning process. Experimental Task Forecasters predicted winds speed at the ground, 2 to 5 minutes into the future at specific locations using only NEXRAD data or CASA and NEXRAD data combined. Case studies were created from data collected in the Oklahoma test bed. Ground sensor readings from the Oklahoma Mesonet (McPherson et al., 2007) provided ground truth. This ground truth was used to calculate absolute wind speed assessment error. Six scenarios, each approximately 12 minute long, were chosen from May 7, 2008, where a cold-core system produced small bowing cells and supercells. Participants made predictions based on these data. The scenarios had similar but discrete storm cells producing straight line winds in the 20-50 kt range at locations with adequate CASA and NEXRAD coverage. Participants Sixteen National Weather Service forecasters (12 male, 4 female) with operational experience ranging from 5 to 25 years (M = 14.4 years, SD = 5.9) participated in the experiment at the 33rd Annual meeting of the National Weather Association in Louisville, KY, October 11-16, 2008. Independent Variables Weather Scenario. Weather scenario refers to the 6 selected atmospheric situations in time order, 1st to 6th. Data Source. Participants evaluated weather scenarios with only NEXRAD data or both CASA and NEXRAD data. Data source indicates if radar data were supplied from “NEXRAD Only” or “CASA & NEXRAD”. Example Weather Scenario (#5) The two images in the top row are from NEXRAD’s KFDR radar at 21:58 and 22:02 UTC. Beam height 6.4 kft @ 65 nmi range. The images in the bottom row are from CASA’s KSAO radar at 21:58, 21:59, 22:00, 22:01, and 22:02 UTC. Beam height 0.4 kft @ 2 nmi range. At 22:05 UTC sensors at the CHIC Mesonet station reported a 47 kt max wind speed. Wind Speed Assessment The mean wind speed assessments were statistically higher when both CASA and NEXRAD data were available as opposed to when only NEXRAD data were available (F (1,28) = 15.264, p = 0.001). • Mean with CASA & NEXRAD: 40.1 kt • Mean with NEXRAD Only: 33.9 kt Absolute Wind Speed Assessment Error The mean of the absolute values of the wind speed assessment errors were statistically smaller when both CASA and NEXRAD data were available as opposed to when only NEXRAD data were available (F (1,28) = 7.309, p = 0.012). • Mean with CASA & NEXRAD: 7.9 kt • Mean with NEXRAD Only: 11.5 kt Confidence Ratings 0 5 10 15 20 25 1-Not confident 2 3 4 5 6 7-Very Confident CASA & NEXRAD NEXRAD Only Assessment Confidence Forecaster confidence assessments were ranked statistically higher when both CASA and NEXRAD data were available as opposed to when only NEXRAD data were available (χ 2 = 16.030, df=1, p < 0.0005). • Mode with CASA & NEXRAD: 5 • Mode with NEXRAD Only: 4 RESULTS Dependent Variables Wind Speed Assessment. Wind speed assessment is the ground level wind speed forecasted for the target location by the participant to the nearest 1 knot. Absolute Wind Speed Assessment Error. The absolute value of the difference between the wind speed assessment and the automated ground sensor reading. Assessment Confidence. After providing their wind speed assessment participants were asked “how confident are you in this estimate” on a scale from 1-“Not Confident” to 7-“Very Confident”. Warning Decision. After providing their confidence rating participants were asked “do these radar based winds indicate a warning is needed” and a “Yes” or “No” response was recorded. REFERENCES Brown, R. A., Flickinger, B. A., Forren, E., Schultz, D. M., Sirmans, D., Spencer, P. L., et al. (2005). Improved Detection of Severe Storms Using Experimental Fine-Resolution WSR-88D Measurements. Weather and Forecasting, 20(1), 3-14. Heinselman, P. L., Priegnitz, D. L., Manross, K. L., Smith, T. M., & Adams, R. W. (2008). Rapid Sampling of Severe Storms by the National Weather Radar Testbed Phased Array Radar. Weather and Forecasting, 23(5), 808-824. McPherson, R. A., Fiebrich, C. A., Crawford, K. C., Elliott, R. L., Kilby, J. R., Grimsley, D. L., et al. (2007). Statewide Monitoring of the Mesoscale Environment: A Technical Update on the Oklahoma Mesonet. Journal of Atmospheric and Oceanic Technology, 24(3), 301-321. Warning Decision The proportion of decisions “Yes” to warn was statistically higher when both CASA and NEXRAD data were available as opposed to when only NEXRAD data were available (χ 2 = 7.251, df=1, p = 0.007). • With CASA & NEXRAD: 30 of 47 • With NEXRAD Only: 6 of 48 Scenario 5 with NEXRAD Only had all “No” responses (8) whereas CASA & NEXRAD had all “Yes” responses (7) and one missing response. ACKNOWLEDGMENTS We thank the 16 forecasters who volunteered to participate in the study described herein. Also, we thank Patrick Marsh and Ron Przybylinski for their forecasting insights, and the NWA for hosting us at their annual meeting.

Upload: others

Post on 05-Apr-2022

2 views

Category:

Documents


0 download

TRANSCRIPT

CASA’sIm

pact on Radar-Based W

ind Assessm

ents

Don J. Rude*, Ellen J. Bass

*and Brenda Philips**

*University of Virginia, **University of Massachusetts

This work is supported primarily by the Engineering Research Centers Program of the National Science Foundation under NSF award number

0313747. Any opinions, findings, conclusions, or recommendationsexpressed in this m

aterial are those of the authors and do not necessarily

reflect those of the National Science Foundation.

DISCUSSIO

N

Sin

ce w

ind

sp

eed

pla

ys

a cr

itic

al

role

in t

he d

efin

itio

n o

f se

ver

e

thu

nd

erst

orm

s, t

he

pu

rpo

se o

f th

is s

tud

y w

as

to m

easu

re t

he

imp

act

of

CA

SA

rad

ar

dat

a o

n w

ind

ass

essm

ent

per

form

ance

.

The

resu

lts

rep

ort

ed h

ere

in c

lear

ly i

nd

icat

e th

at f

or

win

d s

pee

d

asse

ssm

ents

CA

SA

rad

ar d

ata

can s

imu

ltan

eousl

y

•in

crea

se w

ind

sp

eed

est

imat

es

•re

duce

ass

essm

ent

erro

r

•in

crea

se c

on

fid

ence

Th

is o

utc

om

e is

par

ticu

larl

y p

rom

isin

g g

iven t

hat

•d

ata

are

fro

m e

xp

erim

en

tal

rad

ars

•p

arti

cip

an

ts w

ere

giv

en m

inim

al

train

ing

•th

e ra

dar

dis

pla

y a

nd

co

mp

ute

r-in

terf

ace

wer

e u

nfa

mil

iar

•m

an

y p

arti

cip

an

ts s

aid

CA

SA

dat

a co

nfi

rms

their

menta

l m

od

el

The

incre

ase

in w

ind

sp

eed

ass

essm

ents

su

pp

ort

s re

sear

ch b

y B

row

n

and

Wo

od

(B

row

n e

t al.

, 2

00

5)

wh

ich p

red

icts

that

incre

ased

sp

atia

l

sam

pli

ng r

esu

lts

in h

igher

rad

ial

velo

cit

y r

ead

ings.

The

shif

t in

war

nin

g

dec

isio

ns

when g

iven C

AS

A d

ata

is e

spec

iall

y n

ota

ble

bec

ause

on

ly

one

exp

erim

en

tal

task

was

co

ver

ed b

y a

n a

ctual

war

nin

g a

cco

rdin

gto

NW

S a

rch

ives

.

As

syst

em

s su

ch a

s C

AS

A,

NE

XR

AD

Sup

erR

es,

and

MP

AR

(Hein

selm

an,

Pri

eg

nit

z, M

anro

ss,

Sm

ith,

& A

dam

s, 2

00

8)

com

e o

n-

line,

the

fore

cast

ing c

om

mu

nit

y m

ay n

eed

to

rev

isit

po

licie

s and

thre

sho

lds

for

issu

ing w

arn

ing

s.

Future W

ork

The

curr

ent

stud

y c

ou

ld b

e en

hance

d b

y t

he s

yst

em

ati

c co

ntr

ol

of

rad

ar

bea

m a

ttri

bute

s. U

pd

ate

rate

, b

eam

heig

ht,

win

d-t

o-b

eam

in

ters

ecti

on

ang

le,

and

sam

pli

ng f

ideli

ty e

ach i

nfl

uence

per

form

ance

. F

utu

re w

ork

cou

ld q

uanti

fy t

he

imp

act

of

thes

e at

trib

ute

s in

div

iduall

y.

Ad

dit

ional

mea

sure

s o

f p

erfo

rmance

co

uld

be

use

d i

nclu

din

g t

he

size

of

war

nin

gs,

their

du

rati

on,

and

lea

d t

ime.

Fin

all

y,

a st

and

ard

ized

and

rig

oro

us

train

ing t

oo

l fo

r C

AS

A d

ata

wo

uld

help

rem

ove

an

y n

ovelt

y e

ffec

ts

fro

m f

utu

re s

tud

ies.

METHODOLOGY

Introduction

Th

e C

ente

r fo

r C

oll

abo

rati

ve

Ad

apti

ve

Sen

sin

g o

f th

e A

tmosp

her

e (C

AS

A)

is

crea

tin

g a

new

par

adig

m f

or

radar

sys

tem

s b

ased

on

lo

w-c

ost

net

wo

rks

of

X-

Ban

d r

adar

s th

at c

oll

ect

hig

h s

pat

ial

and

tem

po

ral

reso

luti

on d

ata

fro

m t

he

low

er

tro

posp

her

e. C

AS

A o

per

ates

a 4

no

de

radar

tes

t bed

in

so

uth

-wes

t O

kla

ho

ma

to

rese

arch

the

cap

abil

itie

s o

f th

is n

ew t

echn

olo

gy

and

to

und

erst

and

ho

w C

AS

A

dat

a w

ill

imp

act

dec

isio

n m

akin

g b

y u

sers

su

ch a

s N

atio

nal

Wea

ther

Ser

vic

e

fore

cast

ers.

We

hyp

oth

esiz

e th

at C

AS

A t

echn

olo

gy w

ill

bet

ter

info

rm

fore

cast

ers,

im

pro

ve

thei

r per

form

ance

, an

d u

ltim

atel

y im

pro

ve

sever

e w

eath

er

war

nin

gs.

Th

is s

tud

y w

as d

esig

ned

to

mea

sure

the

impac

t o

f C

AS

A d

ata

on

win

d

spee

d a

sses

smen

ts, a

crit

ical

com

po

nen

t o

f th

e se

ver

e th

und

erst

orm

war

nin

g

pro

cess

.

Experim

ental Task

Fo

reca

ster

s p

red

icte

d w

ind

s sp

eed a

t th

e gro

un

d, 2

to

5 m

inu

tes

into

the

futu

re a

t

spec

ific

lo

cati

ons

usi

ng o

nly

NE

XR

AD

dat

a o

r C

AS

A a

nd

NE

XR

AD

dat

a

com

bin

ed.

Cas

e st

ud

ies

wer

e cr

eate

d f

rom

dat

a co

llec

ted

in

th

e O

kla

hom

a te

st

bed

. G

roun

d s

enso

r re

adin

gs

fro

m t

he

Okla

ho

ma

Mes

onet

(McP

her

son

et

al.,

200

7)

pro

vid

ed g

roun

d t

ruth

. T

his

gro

un

d t

ruth

was

use

d t

o c

alcu

late

ab

solu

te

win

d s

pee

d a

sses

smen

t er

ror.

Six

sce

nar

ios,

eac

h a

pp

roxim

atel

y 1

2 m

inu

te l

on

g,

wer

e ch

ose

n f

rom

May

7,

200

8, w

her

e a

cold

-co

re s

yste

m p

rod

uce

d s

mal

l bo

win

g c

ells

and

su

per

cell

s.

Par

tici

pan

ts m

ade

pre

dic

tion

s b

ased

on

th

ese

dat

a. T

he

scen

ario

sh

ad s

imil

ar b

ut

dis

cret

e st

orm

cel

ls p

rodu

cin

g s

trai

gh

t li

ne

win

ds

in t

he

20

-50

kt

ran

ge

at

loca

tio

ns

wit

h a

deq

uat

e C

AS

A a

nd

NE

XR

AD

cover

age.

Participants

Six

teen

Nat

ional

Wea

ther

Ser

vic

e fo

reca

ster

s (1

2 m

ale,

4 f

emal

e)w

ith

oper

atio

nal

exp

erie

nce

ran

gin

g f

rom

5 t

o 2

5 y

ears

(M

= 1

4.4

yea

rs, SD

= 5

.9)

par

tici

pat

ed i

n t

he

exp

erim

ent

at t

he

33

rd A

nnu

al m

eeti

ng o

f th

eN

atio

nal

Wea

ther

Ass

oci

atio

n i

n L

ou

isvil

le,

KY

, O

ctob

er 1

1-1

6, 2

00

8.

Independent Variables

•Weather Scenario.

Wea

ther

sce

nar

io r

efer

s to

the

6 s

elec

ted

atm

osp

her

ic

situ

atio

ns

in t

ime

ord

er, 1

st t

o 6

th.

•Data Source.

Par

tici

pan

ts e

val

uat

ed w

eath

er s

cenar

ios

wit

h o

nly

NE

XR

AD

dat

a o

r b

oth

CA

SA

an

d N

EX

RA

D d

ata.

Dat

a so

urc

e in

dic

ates

if

rad

ar d

ata

wer

e

supp

lied

fro

m “

NE

XR

AD

On

ly”

or

“CA

SA

& N

EX

RA

D”.

Example W

eather Scenario (#5)

The

two

im

ages

in t

he

top

ro

w a

re f

rom

NE

XR

AD

’sK

FD

R r

adar

at

21

:58

and

22

:02

UT

C.

Bea

m h

eig

ht

6.4

kft

@ 6

5 n

mi

range.

The

images

in t

he

bo

tto

m r

ow

are

fro

m C

AS

A’s

KS

AO

rad

ar a

t

21

:58

, 2

1:5

9,

22

:00

, 2

2:0

1,

and

22

:02

UT

C.

Bea

m h

eig

ht

0.4

kft

@

2 n

mi

ran

ge.

At

22

:05

UT

C s

enso

rs a

t th

e C

HIC

Mes

onet

stat

ion

rep

ort

ed a

47

kt

max w

ind

sp

eed

.

Wind Speed Assessment

The

mea

n w

ind

sp

eed

ass

essm

ents

wer

e st

atis

tica

lly h

igh

er w

hen

bo

th C

AS

A a

nd

NE

XR

AD

dat

a w

ere

avail

ab

le a

s o

pp

ose

d t

o

when o

nly

NE

XR

AD

dat

a w

ere

avail

able

(F

(1,2

8)=

15

.26

4,

p =

0.0

01

).

•M

ean w

ith C

AS

A &

NE

XR

AD

: 4

0.1

kt

•M

ean w

ith N

EX

RA

D O

nly

: 3

3.9

kt

Absolute W

ind Speed Assessment Error

The

mea

n o

f th

e ab

solu

te v

alu

es o

f th

e w

ind

sp

eed

ass

essm

ent

erro

rs w

ere

stat

isti

call

y s

mall

er

when b

oth

CA

SA

and

NE

XR

AD

dat

a w

ere

avail

able

as

op

po

sed

to

when

on

ly N

EX

RA

D d

ata

wer

e av

ail

able

(F

(1,2

8)=

7.3

09

, p

= 0

.01

2).

•M

ean w

ith C

AS

A &

NE

XR

AD

: 7

.9 k

t

•M

ean w

ith

NE

XR

AD

On

ly:

11

.5 k

t

Confidence Ratings

05

10

15

20

25

1-Not

confident

23

45

67-Very

Confident

CASA & NEXRAD

NEXRAD Only

Assessment Confidence

Fo

reca

ster

co

nfi

dence

ass

essm

en

ts w

ere

rank

ed

stat

isti

call

y h

igher

when b

oth

CA

SA

and

NE

XR

AD

dat

a w

ere

avail

able

as o

pp

ose

d t

o w

hen

on

ly N

EX

RA

D d

ata

wer

e av

ail

able

(χ2

= 1

6.0

30

,

df=

1,

p <

0.0

00

5).

•M

od

e w

ith C

AS

A &

NE

XR

AD

: 5

•M

od

e w

ith N

EX

RA

D O

nly

: 4

RESULTS

Dependent Variables

•Wind Speed Assessment.

Win

d s

pee

d a

sses

smen

t is

th

e gro

un

d l

evel

win

d s

pee

d f

ore

cast

ed f

or

the

targ

et l

oca

tion

by

the

par

tici

pan

tto

th

e

nea

rest

1 k

no

t.

•Absolute Wind Speed Assessment Error.

The

abso

lute

val

ue

of

the

dif

fere

nce

bet

wee

n t

he

win

d s

pee

d a

sses

smen

t an

d t

he

auto

mat

ed g

roun

d

sen

sor

read

ing.

•Assessment Confidence

. A

fter

pro

vid

ing t

hei

r w

ind

spee

d a

sses

smen

t

par

tici

pan

ts w

ere

asked

“h

ow

co

nfi

den

t ar

e yo

u i

n t

his

est

imat

e”on

a

scal

e fr

om

1-“

No

t C

on

fid

ent”

to 7

-“V

ery

Co

nfi

den

t”.

•Warning Decision.

Aft

er p

rovid

ing t

hei

r co

nfi

den

ce r

atin

g p

arti

cip

ants

wer

e as

ked

“d

o t

hes

e ra

dar

bas

ed w

ind

s in

dic

ate

a w

arn

ing i

s nee

ded

and

a “

Yes

” o

r “N

o”

resp

on

se w

as r

eco

rded

.

REFERENCES

Bro

wn,

R.

A.,

Fli

ckin

ger

, B

. A

., F

orr

en,

E.,

Schult

z,

D.

M.,

Sir

mans,

D.,

Spen

cer,

P.

L.,

et

al.

(2

00

5).

Im

pro

ved D

etec

tio

n o

f S

ever

e S

torm

s U

sin

g E

xper

imenta

l

Fin

e-R

esolu

tio

n W

SR

-88

D M

easu

rem

ents

. Weather and Forecasting

, 20

(1),

3-1

4.

Hei

nse

lman,

P.

L.,

Pri

eg

nit

z,

D.

L.,

Manro

ss,

K.

L., S

mit

h,

T.

M.,

& A

dam

s, R

. W

.

(20

08).

Rap

id S

am

pli

ng o

f S

ev

ere

Sto

rms

by t

he

Nati

onal

Wea

ther

Radar

Tes

tbed

Phase

d A

rray R

adar.

Weather and Forecasting

, 23

(5),

80

8-8

24

.

McP

her

son,

R.

A.,

Fie

bri

ch,

C.

A.,

Cra

wfo

rd,

K.

C.,

Ell

iott

, R

. L

., K

ilb

y,

J. R

.,

Gri

msl

ey,

D.

L.,

et

al.

(2

00

7).

Sta

tew

ide

Mo

nit

ori

ng o

f th

e M

esosc

ale

Envir

on

ment:

A T

ech

nic

al

Up

date

on t

he

Okla

ho

ma M

eso

net

. Journal of

Atmospheric and Oceanic Technology,

24

(3),

30

1-3

21.

Warning Decision

The

pro

po

rtio

n o

f d

ecis

ions

“Y

es”

to w

arn w

as s

tati

stic

all

y

hig

her

when b

oth

CA

SA

and

NE

XR

AD

dat

a w

ere

avai

lab

le a

s

op

po

sed

to

when o

nly

NE

XR

AD

dat

a w

ere a

vail

able

(χ2

=

7.2

51

, d

f=1

, p

= 0

.00

7).

•W

ith C

AS

A &

NE

XR

AD

: 3

0 o

f 4

7

•W

ith N

EX

RA

D O

nly

: 6

of

48

Sce

nari

o 5

wit

h N

EX

RA

D O

nly

had

all

“N

o”

resp

onse

s (8

)

wher

eas

CA

SA

& N

EX

RA

D h

ad a

ll “

Yes

” re

spo

nse

s (7

) and

one

mis

sin

g r

esp

onse

.

ACKNOWLEDGMENTS

We

thank

the

16

fo

reca

ster

s w

ho

vo

lun

teer

ed t

o p

arti

cip

ate

in

the

stud

y d

escr

ibed

her

ein

. A

lso

, w

e th

ank

Pat

rick

Mar

sh a

nd

Ro

n P

rzyb

yli

nsk

ifo

r th

eir

fo

reca

stin

g i

nsi

ghts

, an

d t

he

NW

A

for

ho

stin

g u

s at

their

an

nual

mee

tin

g.