Bill Kuo

DTC Management Board Meeting

24th January 2013

1

Topics for DTC MB meeting:AOP 2013 plan and prioritiesExecution of AOP 2013:

NOAA-UCAR Cooperative AgreementTransition to next phase of DTC

DTC Science Advisory Board membershipDTC sponsors’ funding and prioritiesDTC Executive Committee (EC) meeting

preparationChallenges

2

Planning for AOP 2013Preliminary guidance for DTC is ‘flat budget,’ however,

potential cuts are possible. DTC has taken into account the initial guidance provided

by DTC MB during the October 2012 MB meetingFinal budget for DTC will most likely not be known until

Spring 2013DTC Task Leads will present:

Accomplishments in 2012Proposal for 2013

DTC MB will need to decide on:What tasks fall within the 85% budget level?The ranking of tasks between 85% to 100% budget

level

3

DTC funding sources (in $K)Funding source FY2011 FY201

2Differen

ce

NOAA/OAR 2,994 2,925 -69

NOAA/HFIP 708 460 -248

USWRP 281 190 -91

GSD 250 250

AFWA 718 855 137

NCAR 250 250

NSF 100 100

Carry-over 129 268* 139

Total 5,430 5,298 -132* Due to period of performance for some projects, some funding is ‘committed carry-over’ for AOP 2012.

4

DTC budget allocations (in $K)Funding allocation

FY2011 FY2012

Difference

Director’s Office 755 746 -9

Visitor Program 200 200

Mesoscale Modeling

1,037 1,029 -8

Hurricane 1,197 1,012 -185

Data Assimilation 772 569 -203

Ensemble 846 827 -19

Verification 623 915 *292

Total 5,430 5,298 -132*Note verification support for T&E activities is included in the Verification task in AOP 2012. In AOP 2011, they resided with task areas conducting the test. 5

Execution of AOP 2013DTC AOP has adopted the period of performance of

March 1 – February 28.Current NWS-UCAR Cooperative Agreement (CA)

will end by August 2013. NWS indicated that:NCAR budget through August 2013 can be

transferred via NWS-UCAR CARemainder of funds needs to be transferred using

OAR-NSF CA (subject to NSF cost-recovery fee)NOAA will administer a competitive RFP in 2013

for the next DTC CAWhen will the process be completed?How would this impact AOP 2014 planning?

6

DTC Science Advisory BoardDTC Science Advisory Board consists of 14 members.Six SAB members whose terms will expire by June

2013:Brian Colle, SUNY Stony BrookJames Doyle, NRL Bob Dumais, ARLCliff Mass, U. of Washington (chair)Tom Henderson, ESRLDavid Bright, AWC

DTC MB needs to discuss:Who should be retained?Nomination of new membersIs the operation of SAB effective? Suggestion for change?

DTC Science Advisory Board

STC Sponsors’ Funding and PrioritiesDTC funding sponsors include NOAA/OAR, NOAA/HFIP,

USWRP, GSD, AFWA, NCAR, NSFHFIP, USWRP, AFWA and NSF have provided guidance on

the allocation of their funding to support specific tasks that fall within the core areas of DTC

NOAA/OAR, GSD, and NCAR have allowed flexibility to allocate funds according to priorities set by DTC MB

Recently, questions have been raised with regards to the allocation of NOAA/OAR funds:Should NOAA (or OAR) provide guidance on the priority for

NOAA/OAR funds (instead of the entire DTC MB)?This change will have a significant impact on DTC operation,

as NOAA/OAR is the largest funding source for DTC

9

DTC EC Meeting PreparationDTC EC meeting will be held 12 Feb 2013 in Silver

SpringExecutive Committee will:

Review and approve DTC AOP 2013 and prioritiesReview and approve DTC SAB membership

DTC MB needs to identify important issues/topics for DTC EC discussionNext phase of DTC, RFP process, use of NOAA/OAR

fundsFuture direction of DTCDTC partnership between NOAA, AFWA, NCAR, and NSFOthers?

10

ChallengesSome DTC task areas are becoming subcritical due to budget

reduction:Should we consider consolidation of task areas?Should DTC focus on certain aspects (e.g., physics)?

SAB recommended DTC put greater emphasis on T&E at the expense of community support. Are we ready to take a large step back from conducting tutorials for

our publically-released software packages?Should we reduce the number of packages that we support to the

community?How can DTC be more effective in R2O?

Development and testing of next-generation NWP systemsMigration toward a unified modeling system (a recommendation from

the UCACN)Development of 10-year strategic plan for EMCEstablishment and operation of an “ECMWF-like” facility

11

SummaryDTC has come a long way toward establishing

a community facility with a robust management structure and planning process, strong community connection and partnership among sponsors.

Joint decision making process by the DTC MB is a very important mechanism for:Setting priorities for DTCMaintaining the partnership among sponsorsSetting future direction of DTC

12

Louisa Bogar Nance

Community Interactions

13

Outline

14

Software SystemsCommunity Outreach Events

Software SystemsFramework for bringing together operational capabilities and research innovations to accelerate the transition of new technology into operations by facilitating carefully-controlled extensive T&E

Close collaboration between DTC & developers is critical to the success

of this work!

15

Software System PhilosophyShared resource w/ distributed development that

includes capabilities of current operational systemsOn-going development maintained under mutually

agreed upon software management planCode repository maintained under version control

softwareProtocols for proposing & approving modifications to

the softwareTesting standardsCode review committee

Additional testing standards to more thoroughly check integrity of evolving code base

16

Accelerating R2O Transitions for GSIBefore and After: An Example

GSD initiates merging the cloud analysis code to the existing GSI

GSD/DTC commit code changes to the GSI trunk as a trial case to set up the GSI R2O transition procedure

17

2008

2010

2009

2012

2011

Issues related to:• Version control• Code portability• Development

coordination• Formal code commit

procedure for external groups

• Coding standards • Standard pre-commit

tests

Can we wait this long for any other R2O transition?

Accelerating R2O Transitions for GSIBefore and After: An ExampleGSD initiates merging the code to the existing GSI

GSD starts to use the GSI repository

Establish DTC community GSI repository with the multi-platform feature

GSD/DTC commit code changes to the GSI trunk(s) as a trial case to set up the GSI R2O transition procedure

Now, it takes about one week for GSI Review Committee to review a code change proposal and about one day to commit.

18

2008

2010

Establish EMC operational GSI repository

2009

DTC leads effort to form the GSI Review Committee (GRC) and set up the GSI R2O procedure (including repository syncing)

2012

GRC finalizes the R2O procedure

2011DTC commits portability related changes to the GSI trunk(s)All GRC members start to follow the

R2O transition procedure

Lessons learned through the DTC’s experience

with GSI are being applied to all other

software systems we work with!

Current Software SystemsWRF – NWP model + pre- and post-processors

UPP – New package in 2011 – community code mgmt plan in process of being implemented

Model Evaluation Tools (MET) – verification packageGridpoint Statistical Interpolation (GSI) data assimilation system,

including GSI-hybrid capabilityWRF for Hurricanes - set of tools for tropical storm forecasting,

including coupled atmosphere and ocean system & stand alone GFDL vortex tracker

Modular end-to-end ensemble system (repository & code mgmt plan established during AOP 2011)

NOAA Environmental Modeling System (NEMS) (repository established during AOP 2011 – working towards code mgmt plan)

19

Software system management for GSI, NEMS and ensemble systems has greatly benefitted from having DTC staff members, Hui Shao and Eugene Mirvis, co-located with EMC staff!

Making New Capabilities Available to Operations (2009-present)WRF (including atmospheric component of HWRF)Enhanced interoperability for NMM-E, including moving nest

Radiation – RRTMGCumulus – Tiedtke, NSAS and Grell (uncoupled only)

New capabilities (3-nest) and physics updates from AOML/HRDHWRFExtension of POM coupling to Eastern Pacific (URI)GSINCAR/MMM’s aerosol optical depth data assimilation functionNumerous contributions from GSD (e.g., cloud analysis) and

GMAO (both new features and enhancements to existing features)METBaldwin-Elmore spatial significance tool (DTC Visitor Program)Capability to convert TRMM satellite data into MET readable

format20

Contributions to Operational Software SystemsEnsemble systemBias correction and downscaling for SREFEnsemble Kalman FilterWorking w/ EMC & ESRL to set-up a code

management plan – DTC will likely be a major contributor to this effort

NEMSEnhanced portability (DTC staff and DTC

Visitor Program)Co-leading movement towards common

repository for external NCEP libraries21

Publically-Released Packages

Philosophy Current PackagesPeriodic releases made

available to the community that include latest developments of new capabilities & techniquesAdditional testing, including

multiple computing platforms and compiler options

Centralized support (in collaboration with developers)Software downloadsDocumentationEmail helpdeskTutorials (online and onsite)

WRFUPPHWRFGFDL vortex trackerGSIMET

22

Registered Users

SoftwareInitial

Release

Registered Users

2009

Registered Users

2013

WRF Dec ‘00 ~1,400* 17,225*

MET Jan ’08 ~300 ~2,000

GSI Sept ’09 0 679

HWRF Aug ’11 0 500

GFDL vortex tracker

Aug ‘12 0 217

23

*All WRF users required to re-register starting in 2008 – number corresponds to those who have registered since 2008

AOP 2012 Code Releases and Tutorials

Software Code releases

Onsite Tutorials

WRF

v3.4 – 6 Apr 12

v3.4.1 – Aug 12

28 Jan – 1 Feb ’13

UPPv2.0 – 1 Nov

1228 Jan – 1 Feb

’13

HWRFv3.4a – 29

Aug 12none

GFDL vortex tracker

v3.4a – 29 Aug 12

none

GSIv3.1 – 20 Jul

1221-23 Aug 12

METv4.0 - 25 Jun

124-5 Feb 12

24

AOP 2013 Code Releases and Tutorials85-100%

Software Code releases

Onsite Tutorials

WRFv3.5 – Apr 13v3.5.1 – Aug

13none

UPP v2.1 – Apr 13 none

HWRF v3.5a – Jun 13 none

GFDL vortex tracker

v3.5a – Jun 13 none

GSI v3.2 – Jul 13 none

METv4.1 - Apr 13v4.2 - TBD

Annual25

AOP 2013 Code Releases and TutorialsJust beyond 100%

Software Code releases

Onsite Tutorials

WRFv3.5 – Apr 13v3.5.1 – Aug

13none

UPP v2.1 – Apr 13 none

HWRF v3.5a – Jun 13 Annual

GFDL vortex tracker

v3.5a – Jun 13 Annual

GSI v3.2 – Jul 13 Annual

METv4.1 - Apr 13v4.2 - TBD

Annual26

Software System for AOP 2013 – 85%

27

ID Activity Description

MM1 WRF/UPP repository maintenance, public release & user support (includes only ARW portion)

MM2 NMME user support

MM5+

EN2

NEMS - implement code management plan, repository maintenance, enhance portability and physics options

HU1 HWRF repository maintenance, public release and user support

HU2 HWRF physics interoperability

HU3 HWRF scripting maintenance

DA1 GSI code management & repository maintenance, public release & user support

DA4 GSI-hybrid coordination, code management and repository maintenance

DA7 Community-base GSI observations pre-processing capability

VX1 Community support - MET release (including MET-TC), user support & annual tutorial

Discussion Items for Software SystemsMaintenance of SREF code repository does

not currently fall within the 100% scenario (only NEMS portion of work currently part of plan) - what does this mean for the future of this repository?

Community support – are we ready to take a large step back from conducting tutorials for our publically-released software packages?

What type of planning should DTC be doing wrt potential upcoming transitions of HWRF to NEMS?

28

Community Outreach Events

Important mechanism for bringing together research and operations to discuss how to work together to advance NWP

29

Outreach efforts – AOP 2012DTC-sponsored events

Mesoscale ModelingAnnual WRF Users Workshop (25-29 Jun 2012)

EnsemblesMini-workshop w/ GIFS-TIGGE working group (June

2012)DTC & NUOPC Ensemble Design Workshop (10-12 Sept

2012)

Verification – invited presentationsUnidata Triennial Users Workshop (3; July 2012)Earthcube Workshop (Dec 2012)Full-day tutorial for Turkish Met Service

30

DTC & NUOPC Ensemble Design Workshop

Focus: Quantification & characterization of uncertaintyMain conclusion: more scientific approach is needed to answer ensemble design questionsPlan for the future:1.Establish standard set of

metrics that will allow for useful inter-comparison of ensemble formulations & dealing w/ uncertainty

2.Establish a small set of target parameters

3.Establish a global ensemble data archive for research (e.g. Ensemble ICs and perturbations, forecasts from major centers etc.)

4.Establish clean experimental program

BAMS paper by Scott Sandgathe, Brian Etherton, Barb Brown & Ed Tollerud

31

DTC-Sponsored Events – AOP 201385% Scenario:

Annual WRF Users Workshop

Proposed but did not make 100%Verification workshopGSI Workshop

32

Given the DTC’s mission to serve as a bridge between the research and operational NWP communities, are we stepping back too much from sponsoring workshops?

Jamie Wolff

Mesoscale Modeling

Collaborators:NOAA’s Environmental Modeling CenterNOAA’s Earth System Research LaboratoryNCAR’s Mesoscale and Microscale Meteorology DivisionNorth Carolina State UniversityDivisión de Energías Renovables, CIEMAT, Madrid, SpainUniversity of Washington

33

Activity Description Status

WRF-based community code maintenance and support:

Repository maintenance, email support, code releases, tutorial

Ongoing

Physics interoperability for WRF-based system In progress

Enhancement of NEMS-based code management:Technical discussions, friendly user release, FSOE for internal T&E

In progress

Establish a Mesoscale Model Evaluation Testbed (MMET)*:

Define process for R2O transition, provide datasets and baseline results for cases of interest

Complete

Continue to conduct extensive T&E through comprehensive research innovation inter-comparisons and Reference Configuration designation:

AFWA: WRF version difference and LIS input data set impact*NOAA: Surface drag parameterization schemes impact on a High Resolution Window WRF-ARW baseline configuration

AFWA – Complete

NOAA – In

progress

34

Mesoscale Modeling AOP 2012 Activities

Key Accomplishments

Inter-comparison Testing and EvaluationMMET

35

WRF Testing and Evaluation (T&E)End-to-end system: WPS, WRFDA, WRF, UPP, and METTest Period: 1 July 2011 – 29 June 2012Retrospective forecasts: 48-h warm start forecasts

initialized every 36 h w/ DADomain: 15-km CONUS gridEvaluation:

Surface and Upper Air ((BC)RMSE, bias) Temperature, Dew Point Temperature, Winds

Precipitation (GSS, frequency bias) 3-h and 24-h accumulations

GO IndexStatistical Significance Assessment

Compute confidence intervals (CI) at the 99% level Apply pair-wise difference methodology Compute statistical significance (SS) and practical significance (PS)

36

Functionally similar operational environment testingWRF Data Assimilation and 6-hr warm start

WRFDAv3.3.1 + WRFv3.3.1 w/ LoBCs from LIS w/ Noahv2.7.1

WRFDAv3.4 + WRFv3.4 w/ LoBCs from LIS w/ Noahv2.7.1WRFDAv3.4 + WRFv3.4 w/ LoBCs from LIS w/ Noahv3.3

Evaluation included:Impact assessment of WRF system versionPerformance assessment of the LIS input data set

Current AFWA Op Configuration

Microphysics WRF Single-Moment 5 scheme

Radiation SW and LW Dudhia/RRTM schemes

Surface Layer Monin-Obukhov similarity theory

Land-Surface Model Noah

Planetary Boundary Layer Yonsei University scheme

Convection Kain-Fritsch scheme

37

WRF Inter-comparison T&E

Background Error FilesUsed gen_be to produce seasonal background error

covariance filesCold start cases initialized at 00 and 12 UTC daily for ~15

days during each season (GFS only - no SST or LIS)Pseudo single observation test

Resulting analysis increment from a single observation of the v-component of the wind with 1 m/s innovation

Seaso

n

Dates of cold start

runs

Sum

mer

20110723 –

20110804

Fall 20111016 –

20111030

Winte

r

20120123 –

20120207

Sprin

g

20120401 –

20120415

Sum

mer:

201

207

21

18

Win

ter:

20

120

11

906

WRF v3.3.1 – v3.4 Results

39

SS (light shading) and PS (dark shading) pair-wise differences for the annual aggregation of surface temp, dew point and wind BCRMSE and bias aggregated over the full set of cases and the entire integration domain

Regional Temperature Bias Verification

WRF v3.3.1 w/ Noah v2.7.100 UTC 12h

forecast00 UTC 24h

forecast

WRF v3.4 w/ Noah v2.7.1

40

GO IndexVersion Difference

41

Key Accomplishments

Inter-comparison Testing and EvaluationMMET

42

Testing Protocol MotivationWide range of NWP science innovations

under development in the research community

Testing protocol imperative to advance new innovations through the research to operations (R2O) process efficiently and effectively.Three stage process:

1)Proving ground for research community

2)Comprehensive T&E performed by the DTC

3)Pre-implementation testingat Operational Centers 43

Mesoscale Model Evaluation Testbed (MMET)

What: Mechanism to assist research community with initial stage of testing to efficiently demonstrate the merits of a new developmentProvide model input and

observational datasets to utilize for testing

Establish and publicize baseline results for select operational models

Provide a common framework for testing; allow for direct comparisons

Where: Hosted by the DTC; served through Repository for Archiving, Managing and Accessing Diverse DAta (RAMADDA)

www.dtcenter.org/eval/mmet

44

MMET Cases Initial solicitation of cases from DTC Science Advisory Board

Members and Physics Workshop Participants – great response and enthusiasm towards endeavor

Cases current available within MMET 20090228 – Mid-Atlantic snow storm where North American Mesoscale

(NAM) model produced high QPF shifted too far north 20090311 – High dew point predictions by NAM over the upper Midwest

and in areas of snow 20091007 –High-Resolution Window (HIRESW) runs underperformed

compared to coarser NAM model20091217 – “Snowapocalypse ‘09”: NAM produced high QPF over

mid-Atlantic, lack of cessation of precipitation associated with decreasing cloud top over eastern North Carolina

20100428-0504 – Historic Tennessee flooding associated with an atmospheric river event

20110404 – Record breaking severe report day 20110518-26 – Extended period of severe weather outbreak covering

much of the mid-west and into the eastern states later in the period 20111128 – Cutoff low over SW US; NAM had difficulties throughout the

winter of breaking down cutoff lows and progressing them eastward 20120203-05 – Snow storm over Colorado, Nebraska, etc.; NAM

predicted too little precipitation in the warm sector and too much snow north of front (persistent bias)

User C

ase

#1

45

User Case #1 (Jimenez and Dudhia)

20100428-20100504 – Extended case focused on historic Tennessee flooding event

Forecasts: WRF v3.4 ARW baseline configuration namelist from DTC

WRF v3.4 ARW namelist with topo_wind=1 activated CONUS domain at 15km resolution

Utilized IC and BC files provided by DTC for model initializationUtilized observation files provided by DTC for verification

46

User Case #1 (Jimenez and Dudhia)Wind Speed Time Series

47

User Case #1 (Jimenez and Dudhia)Wind Speed Error (topo_wind=1)

Average wind speed across the domain• topo_wind=1• Observed

00 UTC 20100428 through

00 UTC 20100504 (every 3 hours)

Underforecast

Overforecast

48

Defaulttopo_wind

=1

User Case #1 (Jimenez and Dudhia)Wind Speed 6-day Average Error

Status of testing:• Overall 6-day domain average with topo_wind=1 smaller than default• Reduces diurnal mean bias but does not capture full diurnal amplitude• Looking into reduction of convective mixing and vertical transport of momentum causing overall lower speeds

49

Proposed Activities for 2013

50

Mesoscale Modeling AOP 2013 Proposed ActivitiesID Activity Description

MM1

WRF/UPP community code maintenance and support

MM2

WRF-NMM support

MM5

Enhancement of NEMS-based code management

MM6

MM7

Continue to conduct extensive T&E through comprehensive research innovation inter-comparisons and Reference Configuration designation

MM8

Continue implementation and maintenance of MMET

HMT

Add 2 Atmospheric River cases from HMT to MMET

51

Extensive inter-comparison T&E and RC designationsMM6: AFWA - Two WRF-ARW configurationsMM7: NOAA - Two NEMS-NMMB configurations

NAM physics suite vs. Thompson mpAdditional Western US verification focus for HMT

MM8: Expansion of Mesoscale Model Evaluation Testbed (MMET)Establish NMMB baselines for all cases (existing and new)Maintain infrastructure

Update baselines with new versionsWork with community to ensure utilization

Add additional cases 4 more cases from EMC priority list2 HMT Atmospheric River cases

AOP 2013 Proposed ActivitiesTesting and Evaluation

52

Hurricane

Ligia R. Bernardet

External collaborators:NOAA Environmental Modeling Center

NOAA Geophysical Fluid Dynamics LaboratoryNOAA Atlantic Oceanographic and Meteorological Laboratory

NCAR Mesoscale and Microscale Meteorology DivisionUniversity of Rhode Island

University of California – Los AngelesFlorida State University

53

Hurricane AOP 2012 ActivitiesActivity Description Status

HWRF repository maintenance, public release and user support

Ongoing

HWRF interoperability – Thompson microphysics

In progress

HWRF FSOE to match 2012 operational

Competed

HWRF 2012 operational Reference Configuration

Completed

T&E FSOE: HWRF cumulus sensitivity Completed

T&E FSOE: HWRF atmos-ocean fluxes Completed

Sensitivity experiments: Thompson microphysics in HWRF

Current– will complete in Feb

Diagnostics of large scale environment in HWRF

Completed54

POM Flux Test

55

BackgroundHRD (Uhlhorn and Cione) compared HWRF retro forecasts for 2011 against buoys and showed that HWRF ocean does not respond (=does not cool as much as obs) when storm goes by

Example: Katia 09/01/11 init12 UTC and buoy passage 9/4 12 UTCZ

• Fluxes from HWRF atmosphere to ocean are truncated in POM (75%)

• DTC ran 2012 season: control HD12 (75% fluxes) and modified HDFL (100%) 56

Atlantic track and intensity

Track ME: HD12 and HDFL very similarInt MAE: HDFL SS better at 3 lead timesInt bias: HD12 lowers intensity and helps overintensification at long lead timesHurricane Leslie (12L) is the storm with largest impact (large and slow)Pacific impact is much smaller (POM 1D)

57

Leslie bias and 09/04 00Z case

• HD12 and HDFL tracks are similar

• HDFL reduces intensity (as expected).

• Is it because of low SST under storm?

58

Leslie bias and 09/04 00Z case48-h SST control – flux exp At 48 h, control has

cooler SST than flux exp (contrary to linear interpretation)

X = storm center

X

59

Cumulus sensitivity test

60

Test of HWRF sensitivity to cumulus schemes

12 24 36 48 60 72 84 96 108 120HNSA                  HKF1      HTDK            

12 24 36 48 60 72 84 96 108 120HNSA  HKF1    HTDK  

Track

Intens

Tested HWRF SAS, new SAS, Tiedtke, Kain-FritshHWRF SAS performs best for track; differences in intensity have little statistical significance

Statistical Significance 95%Green= HWRF SAS betterRed = HPHY SAS worse

61

Case study: Katia init 09/02/11 18 Z

78-h forecast isotachs (E-W x-section)

HPHYHNSA

HTDKHKF1

Tracks: similarIntensity: different (HPHY, HTDK intensify)

SHIPS diagnostics of shear: initially similar, later different. Intensifiers have lower shear.

Highlights cumulus effects on and control on intensification

62

Large scale diagnostics

63

BackgroundMotivation

EMC is preparing to implement basinscale HWRF in ‘14/15

Extensive collective work in data assimilation, moving nests, trans-Atlantic POM

Need to identify large scale errors – Vx of HWRF 3D fields never done beforeDTC diagnostic study

Evaluated cold-started basinscale HWRF large scale fields

Identified issues that deserve further investigation (hypotheses)

Created benchmark

Example of basinscale domain

64

MethodologyBHWRFforecas

tfields

GFS analy

sisfields

Compute paired

differences

Accumulate differences by

forecast lead time

570 forecast cases 615 forecast cases

~730 possible forecast cases from 2011060318 to 2011112506

Cold-started from GFS analysisRun by EMC

PRE13HI

surface pressureskin temperature3D temp3D u and v3D rel. hum.3D sp. hum.3D geopotential

65

Highlight: 600-hPa zonal wind speed

Basinscale biasSeptember 2011 – 72-h forecastAfrican jet too weak in HWRF

GFS BiasSeptember 2011 – 72-h forecastIn GFS jet displaced to south

66

Highlight: surface temperatureBasinscale biasJune 2011 – 24-h forecastHWRF cold over dry continental areasSuggests issue with inland ice

GFS BiasJune 2011 – 24-h forecastNo significant biases

67

Thompson microphysics

68

InteroperabilityEMC (S. Trahan) has created the basic interoperability

Ability to advect various microphysics mixing ratios and number concentrations (Ferrier only advects one species)

New nest-parent interpolation routines which communicate all microphysics variables (for Ferrier or other microphysics)

DTC improving MP-radiation interfaceTesting by DTC

Irene and Earl, with stationary and moving nestsWinter storm with single domain and stationary nest

DebuggingTests, diagnostics, code analyses uncovered bugs in nest-parent

interpolationEMC corrected; work in progress

DTC-EMC collaboration in MP

69

HWRF w/Thompson MP (winter storm)

Most recent problem solved: snow coming from grid1 into grid2 has a sharp discontinuity (also cloud ice number concentration). Caused by an array dimensioned incorrectly

70

The sum of ice and snow mass is passed from MP to radiation

Their radius is assumed to be small at cold temperatures

Effectively, snow is counted as small particles, with massive (and incorrect) impact on shortwave radiation reflection

Solution: compute effective radii of cloud ice, snow, cloud droplets in manner consistent with microphysics scheme – for Thompson, Ferrier etc.

Implemented in WRF-ARW in RRTMG (RRTMG being tested by EMC for 2013 HWRF)

Will transfer to HWRF *and* NMM-B

Radiation code issues: DTC work

71

Hurricane AOP 2013 ActivitiesID Activity Description

HU1 HWRF repository maintenance, public release and support

HU2 HWRF interoperability

HU3 HWRF scripts maintenance

HU4 HWRF Tutorial (currently beyond 100%)

HU5 T&E: HWRF FSOE – innovation testing

HU7 T&E: Diagnostics and sensitivity experiments for HWRF

HU6 T&E: HWRF FSOE – innovation testing (currently beyond 100%)

72

T&E for AOP 2013

Is it cost effective to maintain a HWRF FSOE to do 1 T&E this year?? Recommend expanding to at least 2 T&E activities

73

Work with the research and operational communities to determine priorities

Follow up on 2012 large-scale diagnostic work (hypothesis were formulated and require testing)Noah LSM and initialization of ice-covered land

Could improve surface temperature inconsistencies and African jet placement

Prepare for storm surge and flooding coupling Repeat basinscale evaluation with newer datasets (GSI-Hybrid)

Expand diagnostics to precipitation (CMORPH and Stage IV near/over land)Feature-based analysis of subtropical high and upper level high/lowsSwitch tool to MET, which now provides spatial verificationStatistics conditional on SHIPS: how does model perform when shear is

high/weak; when moisture is high/low etc SAB suggestions

Intercomparison with other HFIP modelsAcquire/organize datasets for case studies

Diag & sensitivity exper - possibilities

74

Work with the research and operational communities to determine priorities

Noah LSM (beyond diagnostic)Could improve surface temperature inconsistencies and

African jet placementLaureano thesis indicates improvement for landfalling storms

Thompson microphysicsImprovement noted in parallel COAMPS could also benefit

HWRFTest innovations devised by HFIP grants program, if

availableDa-Lin’s vertical level distributionFovell’s modified physics

T&E- possibilities

75

Hui ShaoAFWA, NCEP/EMC, NOAA/ESRL,

NASA/GMAO, NCAR/MMM, AOML/HRD, University of Oklahoma

Data Assimilation Task

76

Data Assimilation: AOP 2012 Activities

Activity Description Status

Software Systems: GSI code management & repository maintenance, public release & user support

Ongoing

Software Systems: GSI tutorial Completed

T&E: GSI baseline tests for AFWA Ongoing

T&E: GSI-hybrid for HWRF Ongoing

T&E: NCAR DART EnKF System (2011 leftover)

Completed

T&E: Impact of Radio Occultation Data on HWRF Forecasts

Ongoing

77

GSI Baseline Tests for AFWAMotivation: Assist AFWA with determining appropriate initial configuration of GSI for operational implementation (proper set-up and definition of background error covariance.

0 6 12 18 24 30 36 42 48

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Black-box Run of GSI in SLP forecastsBetween 5/14/2012 And 7/25/2012

T51A (GSI) RMSE T91A (WRFDA) RMSE

T51A (GSI) BIAS T91A (WRFDA) BIAS

FORECAST HOURS

Se

a L

eve

l Pre

s. (

mb

)

78

Mechanism for AFWA-DTC Communications

AFWF real-time parallel GSI runs:

Updates/changes are periodically brought into parallel runs. Focus on evaluating the overall performance of GSI.DTC real-time & retrospective GSI runs using functionally-similar operational environment: Focus on testing incremental changes.• Real-time: “sync” testbed,

uncover the issues• Short-term retrospective: test

individual changes, tackle the issues

• Extensive retrospective: impact study w/ SS, test research/developmental components

AFWF real-time operational WRFDA runs.

• Benchmark

• Oper config

• Benchmark

• Parallel run config

• Archived data /background for retro runs

• DA system switch

• Oper config (updated)

• Benchmark

• Developmental config (suggested from the DTC)

Pa

th

wa

y t

o “

O”

79

0 1 2 3 4 5 6 7 8 9 10

-5

0

5

10

15

20

Background Comparison

AFWA(UKMET)-BiasDTC(GFS)-BiasAFWA(UKMET)-RMSEDTC(GFS)-RMSE

Variables

Bia

s/R

MSE (

Vari

able

Unit

)

Functionally Similarity Check

Only differences are input fields (background and observations) and individual changes to be tested.

0 1 2 3 4 5 6 7 8 9 10-2

0

2

4

6

8

10

12

14

Analysis Comparison

AFWA BIAS

DTC BIAS

AFWA RMSE

DTC RMSE

Variables

Bia

s/R

MSE (

Vari

able

Unit

)

Variable No. 1 2 3 4 5 6 7 8 9

Variable Name T T-120 T-131 UV UV-220

UV-231 q q-120 Ps-

180,181,187

AFWA Northern Hemisphere Domain

80

Real-Time Runs

N<1: GFS+ARW better

N>1: GSI+ARW better

GSI: NAM BE (no GPSRO)

GSI: GFS BE+GPSRO

GSI+ARW runs switched to AFWA parallel run configuration

AFWA GO index:where Sw is the sum of the skill scores, weighted by lead time, for wind speed, dew point temperature, temperature, height at various levels and surface, and mean sea level pressure.

81

N< 1: GFS BE+GPSRO better

N>1: NAM BE/GFS BE (No GPSRO) better

Retrospective Runs: What caused the drop?

NAM BE: Northern Hemisphere BE computed based on NAM forecasts.

GFS BE: Global BE computed based on GFS forecasts.

RAP BE: Global BE tuned for the RAP. combination of global/regional (balance = GFS, Lengthscales/variance = NAM)

N< 1: GFS BE+GPSRO better

N>1: NAM BE betterGFS BE (No GPSRO) betterRAP BE better

NAM BEGFS BERAP BE NAM

BEGFS BERAP BE

Wind Analysis RMSETemperature Analysis RMSE 82

Retrospective Runs: Background Errors (BE)

Vertical LengthscaleHorizontal LengthscaleStandard Deviation

GFS BE

NAM BE

Analysis Inc. from single T

obs testBE factors for Stream Function

Messages Passed to the “Operations”For Northern Hemisphere, the NAM BE or tuned global

BE w/ regional scaling is recommended at current stage.

The DTC is testing the impact of application specific BE, by comparing with the NAM BE experiments. Based on the pending results, application specific BE may be recommended (resolution may play a role here!).

For Southern Hemisphere, BE should be examined separately since the model errors are expected to be larger than those in Northern Hemisphere.

Action taken: AFWA is going to test the NAM BE in their real-time parallel runs.

84

GSI-Hybrid Test for HWRFGlobal GSI-hybrid implemented May 2012. Regional

system is still under development.DTC member of HFIP tiger team targeting at the 2014

implementation of GSI-hybrid for HWRF (EMC (team lead), ESRL, AOML, U of O)

DTC T&E activities focusing on: Building up baseline tests for cross comparison with

other teams’ work:Cross covariance contributed by the ensemble BE

under current NCEP global GSI-hybrid setupGSI-hybrid versus GSI benchmark runs

Testing alternative/reference configurations:Partial/full cycling runs versus cold start/warm

start runsVarying weights of the static BE and ensemble BE

85

GSI-Hybrid Benchmark Tests

NoDAGSI-3DVARGSI-HybridBest Track

•GFS ensemble input: DTC tests show mixed results on both track & intensity forecasts.

•GSI-hybrid using regional ensembles is under investigation and will be added to these benchmark tests.

•Varying weights of static BE and ensemble BE and their impacts are under investigation.

86

Partial Cycling of GSI-hybrid Case study of the 1-d cycling of the GSI-

hybrid prior to the TCvital time shows positive impact on track and intensity forecasts.

NCEP/EMC and NOAA/ESRL are running warm-start (6hr HWRF forecast as background) and full cycling through TC life time. Cross examination will be done once their initial test results are available.

1-d Cycling No cycling

GFS

1-d Cycling

1-d Cycling

Best track

87

Data Assimilation AOP 2013 ActivitiesID Activity Description

DA1 GSI code management & repository maintenance, public release & user support

DA2 GSI Tutorial (currently beyond 100%)

DA3 GSI Workshop (currently beyond 100%)

DA4 GSI-hybrid coordination, code management and repository maintenance

DA5 GSI baseline for AFWA

DA6 GSI-hybrid for HWRF

DA7 Community-base GSI observations pre-processing capability

88

Community-base GSI Observations Pre-processing Capability • GSI observations are required in BUFR format.

Conventional data go through a sophisticated QC procedure prior to being assimilated by GSI. Satellite data are dumped into a data trunk and their QC and bias correction procedure are performed inside GSI.

• The data format conversion and the data QC procedure have become interest of active GSI community users, especially those who would like to ingest additional data types.

• AFWA little_r data files are in ASCII format and, along its pre-processing tool, are used already by AFWA operations and some community users.

• NCEP/EMC has worked with AFWA to adopt the NCEP/NCO pre-processing package. However, the code has not reached the community code standard.

89

Community-base GSI Observations Pre-processing Capability: Wish list

NCEP/AFWA Pre-processing System

• Flexibility to ingest new observation types

• Independent format converter (multiple input format)

• Configurable platform-specific QC

• Different model background for QC

• Code portability• User friendly interface• Configurable setup• Modularized code• Documentation• User support

90

Plans for Setting Up an Initial Community Capability• Review and test the current AFWA/NCEP data

collection and processing procedures in an operational environment.

• Establish a DTC code repository to facilitate version control of the pre-processing capability development.

• Design an acceptable workflow system and user interface that will meet AFWA’s operational requirements.

• Develop essential scripting and coding to accommodate the desired initial capability of the pre-processing procedures, such as adding initial configuration management to the scripts and codes requested by AFWA.

• Develop an automatic and portable configuration and compilation utility so that the system can be easily ported to different computing environments.

• Begin to document the established community capability.

Resources allocated to this activity reduced from original proposal – longer time frame required to achieve ultimate goals 91

DTC will be testing developmental capabilities – focus of tests will be determined by consultation with AFWA. Potential ideas include: ARW BE (domain specific)Increased model topNew NCEP regional bias correction (BC) scheme for

radiance data (by blending global coefficients and ozone information with regional BC)

Moisture channel radiance QC and impactsRadiance channel selectionGPSRO assimilation

Only limited configurations can be tested. Running short-term tests (<monthly) will allow more capabilities to be tested but with less confidence in the results.

GSI Baseline Test for AFWA

92

Aspect of GSI-hybrid to be tested will be determined and coordinated through consultation with EMC and appropriate developers (the “tiger” team). Potential focus:Input from regional ensemblesOptimal weighting for static and ensemble BEFull cycling of GSI-hybridMoving domain for GSI-hybrid

Only limited configurations can be tested (case studies but testing more configurations, longer-period of tests but limited testing components).

GSI-hybrid Tests for HWRF

93