1

DevelopmentalTestbedCenterReportAOP2016Activities

1April2016–31March2017

1 Introduction

TheDevelopmentalTestbedCenter(DTC)isadistributedfacilitywithcomponentsattheNationalCenterforAtmosphericResearch(NCAR)andtheNationalOceanicandAtmosphericAdministration(NOAA)EarthSystemResearchLaboratory(ESRL)GlobalSystemsDivision(GSD).ThepurposeoftheDTCistoprovidealinkbetweentheresearchandoperationalcommunitiessoNumericalWeatherPrediction(NWP)researchcanbeefficientlytransferredtooperations.Inaddition,theDTCprovidestheresearchcommunityaccesstothelatestoperationalNWPcodepackagesforresearchapplications.TheDTCmeetsitsgoalsby:maintainingandsupportingcommunitycodepackagesthatrepresentthelatestNWPtechnology,performingextensivetestingandevaluation(T&E)ofnewNWPtechnology,developingandmaintainingastate-of-the-artverificationpackage,andconnectingtheNWPresearchandoperationalcommunitiesthroughworkshopsanditsvisitorprogram.Overthepastyear,DTCactivitieswereorganizedintofivefocusareas:Verification,DataAssimilation(DA),Hurricanes,RegionalEnsemblesandGlobalModelTestBed(GMTB).

FundingfortheDTCisprovidedbyNOAA’sNationalWeatherService(NWS)andOfficeofOceanicandAtmosphericResearch(OAR),theAirForce(AF),NCAR,andtheNationalScienceFoundation(NSF).ThisreportprovidesadescriptionoftheactivitiesundertakenbytheDTCbetween1April2016and31March2017.TheseactivitiesincludethosedescribedintheDTC2016AnnualOperatingPlan(AOP),aswellasafewcarry-overactivitiesfromtheDTCAOP2015.TheperformanceperiodfortheGMTBis1Julythrough30June.ThisreportalsoprovidesastatusupdateontheGMTBactivitiesthrough31March2017.

1.1 DTCManagement

TheexternalmanagementstructureoftheDTCincludesanExecutiveCommittee(EC),aManagementBoard(MB),andaScienceAdvisoryBoard(SAB).Currentmembershipsarelistedbelow.TheMBandECareresponsibleforapprovingtheDTCAnnualOperatingPlan(AOP),whichdefinestheworktobeundertakenbytheDTCinagivenyear,whereastheSABischargedwithprovidingtheDTCDirectorwithadviceonfuturedirectionsoftheDTCandreviewingproposalssubmittedtotheDTCVisitorProgram.

TheDTChosteditsannualSABmeetingatNCAR’sFoothillsCampusinBoulder,CO,on14-15September2016.ThepurposeofthismeetingwastodiscussstrategicfuturedirectionsfortheDTC.Participationinthisannualmeetingwasstrongerthanusual,with16SABmembersparticipatingin-personandthe17thmemberparticipatingremotely.Day1ofthemeetingconsistedofbriefingsfromtheDTC’soperationalpartners,anoverviewpresentation,highlightsbytaskareaandabreakoutgroupdiscussiononbuildingcommunity.Day2startedoffwithapresentationonDTC’scommunityinteractions,followedbybreakoutgroupdiscussionsbytaskarea.ThemeetingwrappedupwithabriefinganddiscussionofSABrecommendations.GeneralandtaskspecificrecommendationsstemmingfromthismeetingarepostedontheDTCwebsite(http://www.dtcenter.org/SAB/SAB-recommendations-Sept2016.pdf).

InMarch2016,theNationalCentersforEnvironmentalPrediction(NCEP)director,BillLapenta,chargedtheUCARCommunityAdvisoryCommitteeforNCEP(UCACN)withconductingareviewoffour

2

representativetestbedsmanagedorco-managedbyNCEPinassociationwithoneoftheNCEPCenters.TheDTCwasoneoftheselectedtestbeds.ThereviewcommitteefortheDTCconsistedof:ShuyiChen(chair–UniversityofMiami),PeterNeilley(TheWeatherCompany),LanceBosart(StateUniversityofNewYork[SUNY]-Albany)andAndyBrown(UKMetOffice).Aspartofitsreview,theDTCmanagementhostedaDTCOverviewwebinaron30Septembertoprovidebackgroundinformationtotheexternalreviewpanel,followedbyanonsitevisitbythereviewcommitteeon6-7October2016.TheonsitevisitconsistedofpresentationsbyDTCmanagementandtaskleadsandone-on-onediscussionsbetweenthereviewcommitteeandacross-sectionofDTCstaff.

DTCExternalManagementCommittees:

ExecutiveCommittee ManagementBoard JimHurrell NCAR JoshHacker NCAR MikeFarrar NOAA/NWSBillLapenta NOAA/NWS JoeKlemp NCAR FredToepfer NOAA/NWSRalphStoffler AirForce MichaelGremillionAirForce StanBenjamin NOAA/OAR/ESRLKevinKelleher NOAA/OAR JeffCetola AirForce Jian-WenBao NOAA/OAR/ESRL

ScienceAdvisoryBoardAdamClark NationalSevereStormsLaboratory(NSSL)RobertFovell SUNY–AlbanyKristenCorbosiero SUNY–AlbanySharanyaMajumdar UniversityofMiamiKathyGilbert NationalCentersforEnvironmentalPrediction(NCEP)/WeatherPredictionCenterGeoffDiMego NCEP/EnvironmentalModelingCenter(EMC)JenniEvans PennsylvaniaStateUniversityDavidGochis NCARS.R.Gopalakrishnan NOAA/AtlanticOceanographicandMeteorologicalLaboratory(AOML)DavidVollmer UnitedStatesAirForce(USAF)AcademyTomAuligne JointCenterforSatelliteDataAssimilation(JCSDA)TimWhitcomb NavalResearchLaboratory(NRL)BradColman ClimateCorporationZhuoWang UniversityofIllinoisKellyMahoney CooperativeInstituteforResearchinEnvironmentalSciencesRussSchumacher ColoradoStateUniversityKayoIde UniversityofMaryland

Overthepastyear,theDTChostedtwoMBmeetings:atwo-hourconferencecallon31October2016anditsannualin-personMBmeetingon18-19January2017atNCAR’sFoothillsCampusinBoulder,CO.ThefocusoftheOctoberconferencecallwastoreportonrecommendationsfromtheSABanddiscussinitialguidanceonprioritiesforAOP2017.ThepurposeoftheJanuarymeetingwastodiscussandrefinetheDTC’sproposalforAOP2017anddiscussnominationsforSABmemberstoreplacecurrentmemberswhosetermexpiresinJune2017.

DTCmanagementparticipatedintwoDTCECconferencecalls(17May2016,29September2016)andtheannualin-personECmeetingatNWSHeadquartersinSilverSpring,MD,on3March2017.RecentDTCaccomplishments,recommendationsfromtheSAB,proposedactivitiesforAOP2017,andthefuturedirectionoftheDTCwerediscussedatthein-personmeeting.TheECalsoapprovedtheDTCDirector’sproposaltorotateoffsixSABmemberswhosetermsexpireinJune2017(RobertFovell,KristenCorbosiero,SharanyaMajumdar,GeoffDiMego,JenniEvans,andKellyMahoney)andaddsixnewSABmembers(three-yeartermbegins1July2017).ThesixnewSABmembersare:VincentLarson

3

(UniversityofWisconsin-Milwaukee),XuguangWang(UniversityofOklahoma),TomGalarneau(UniversityofArizona),PhilPegion(ESRL/PhysicalSciencesDivision),RustyBenson(GeophysicalFluidDynamicsLaboratory[GFDL])andVijayTallapragada(NCEP/EMC).QuarterlyreportsontheprogresstodatewerealsopreparedforeachactivityanddistributedtotheECandMBmembers.

1.2 CommunityInteractions

MaintainingstrongtiestoboththeresearchandoperationalNWPcommunitiesiscriticaltotheDTC’sabilitytosuccessfullymeetitsmission.Overthepastyear,strongtieswiththeoperationalcommunityweremaintainedthroughtheDTC’sinteractionswithourpartnersattheoperationalcenters(i.e.,EMCandAirForce)bothatthemanagementlevelandthroughourteamleadinteractionswiththeappropriateteamleadsand/orfocalpointsattheoperationalcenters.TheDTCalsoworkedtowardstrengtheningitstiestothebroaderresearchcommunitythroughworkshops,tutorialsandtheDTCVisitorProgram.InformationonDTC-sponsoredtutorialsisprovidedinSection2.3.TheDTCalsoengagesthecommunitythroughthedistributionofitsnewsletter“Transitions”thatservesasaforumfortheresearchandoperationalcommunitiestoshareinformation.Overthepastyear,theDTCdistributedthreeissuesofTransitions.AllissuesofTransitionscanbeaccessedat:http://www.dtcenter.org/newsletter/.

1.2.1 CommunityOutreachEvents

InJune2016,theDTCco-hostedwithNCAR’sMesoscaleandMicroscaleMeteorology(MMM)Laboratorythe17thWeatherResearchandForecasting(WRF)Users’WorkshopatNCAR’sCenterGreenCampusinBoulder,CO.ThefirstdayconsistedoflecturesonWRFsoftwareandbestcomputingpractices,followedbya3-dayworkshopconsistingof67talksandnearly80posters.Thelastdayconsistedoffourmini-tutorialsontheMesoscaleModelEvaluationTestbed(MMET),VisualizationandAnalysisPlatformforOcean,AtmosphereandSolarResearchers(VAPOR),ensemblepredictionandNCARCommandLanguage(NCL).TheMMETinstructionalsessionwasorganizedandconductedbyDTCstaff.About190peoplefrom19countriesattendedtheworkshop(http://www2.mmm.ucar.edu/wrf/users/workshops/WS2016/WorkshopPapers.php).

AlsoinJune2016,theDTCco-hostedwithNCEP/EMCthe7thEnsembleUsers’WorkshopattheNOAACenterforWeatherandClimatePrediction(NCWCP)inCollegePark,MD.Thisworkshopattractedmorethan150participantsrepresentingabroadcross-sectionofexpertiserangingfromensembledeveloperstotheendusersofensembleproducts.Theover-archinggoaloftheworkshopwasdetermininghowtosupporttheNWSasitmovestowardaseamlessoperationalensembleforecastsystematstorm-toglobal-scales,fromshort-termtoseasonaltimescales,usingatmosphere-onlytoocean-waveandcoupledensemblepredictionsystems.Theworkshopconsistedoforalandposterpresentations,aswellasopendiscussionovera3-dayperiod.Workshoppresentationsandalistofattendeesarepostedathttp://www.dtcenter.org/events/workshops16/ensembles/.AreportontheworkshopisalsoavailableontheDTCwebsite(http://www.dtcenter.org/eval/ensembles/).

1.2.2 DTCVisitorProgram

TheDTCVisitorProgramsupportsvisitorstoworkwiththeDTCtotestnewforecastingandverificationtechniques,modelsandmodelcomponentsforNWP.Thegoalistoprovidetheoperationalweatherpredictioncenters(e.g.,NCEPandAirForce)withoptionsfornear-termadvancesinoperationalweatherforecastingandtoprovideresearcherswithNWPcodesthatrepresentthelatestadvancesintechnology.Italsooffersanopportunityforvisitorstointroducenewtechniquesthatwouldbeof

4

particularinteresttotheresearchcommunityintothepublicly-releasedsoftwaresystemssupportedbytheDTC.

Overthepastyear,theDTCprovidedsupportforonevisitorprojectselectedin2014(seeTable1.2.2-1),fourprojectsselectedin2015(seeTable1.2.2-2)andfourprojectsselectedforfundingin2016(seeTable1.2.2.-3).Duringthistimeperiod,Dr.Roebbercompletedhisprojectandsubmittedhisprojectreport,soallprojectsawardedin2014arenowcomplete.Mr.Otkinandcolleaguesalsocompletedtheirprojectandsubmittedtheirprojectreport.Theremaining2015projectsarenearingcompletion,withonlytheprojectreportsremaining.Forthe2016projects,Mr.IaconoandMr.HendersonhavecompletedtheirprojectandsubmittedtheirprojectreportandonlythereportremainsforDr.Niyogi’sgraduatestudentprojectbySubashiniSubramanian.Theotherprojectsawardedin2016arewellunderwayandtheprojectsawardedin2017areeitherunderwayoranticipatedtogetunderwayinthecomingmonths.Allvisitorprojectreportsreceivedoverthepastyearareavailableonthe“VisitorProgram”portionoftheDTCwebsite(http://www.dtcenter.org/visitors/).Inadditiontoprojectreportsandrelevantcodedeliverables,theDTCstartedschedulingvisitorseminarsduringtheirfinalDTCvisitthatareopentolocalareascientistsaswellasremoteparticipants.FeedbackfromboththevisitorsandlocalareascientistsaboutthisincreasedexposureoftheDTCvisitorprojectshasbeenoverwhelminglypositive.Oneadditionalprojectisintheprocessofbeingawarded.

Table1.2.2-1.2014VisitorProjectsPI Institution ProjectTitle

PaulRoebberUniversityofWisconsin-Milwaukee

Demonstrationproject:Developmentofalargememberensembleforecastsystemforheavyrainfallusingevolutionaryprogramming

Table1.2.2-2.2015VisitorProjectsPI Institution ProjectTitle

JasonOtkinUniversityofWisconsin-Madison

ObjectbasedverificationfortheHRRRmodelusingsimulatedandobservedGOESinfraredbrightnesstemperatures

GretchenMullendore(MariuszStarzec)

UniversityofNorthDakota

Mesoscalemodelintercomparisonatconvection-allowingresolutionusingMODE

DevNiyogi(XingLiu) PurdueUniversity

ImprovingWRFweatherforecastthroughenhancedrepresentationofcropland-atmosphereinteractions

JoelBedardUniversityofQuebec-Montreal

Implementationandvalidationofageo-statisticalobservationoperatorfortheassimilationofnear-surfacewindsinGSI

Table1.2.2-3.2016VisitorProjectsPI Institution ProjectTitle

MichaelIacono/JohnHenderson

AtmosphericandEnvironmental

Research

TestingrevisionstoRRTMGcloudradiativetransferandperformanceinHWRF

RobertFovell SUNY-Albany ImpactofplanetaryboundarylayerassumptionsonHWRFDevNiyogi(Subashini

Subramanian)PurdueUniversity DevelopingcapabilityinidealizedHWRFforassessingtheimpactoflandsurfaceontropicalcycloneevolution

ShaowuBao CoastalCarolinaUniversityEvaluationofthemicrophysicsschemeinHWRF2016versionwithremote-sensingdata

5

Table1.2.2-4.2017VisitorProjectsPI Institution ProjectTitle

PatrickSkinner UniversityofOklahoma

QuantifyingthevalueofradardataassimilationintheCommunityLeveragedUnifiedEnsembleusingobject-basedverificationmethods

KarinaApodaca ColoradoStateUniversityR2OtransitionoftheGOES-RGLMlightningassimilationcapabilityinGSIforuseintheNCEPGDAS

BillGallus IowaStateUniversityUseoftheCLUEtoexamineimportanceofmixedphysicsinensembles

PaulRoebberUniversityofWisconsin-Milwaukee

AnAdaptiveBayesianModelCombination(BMC)PostProcessorfortheHRRR-TLEForecastSyste

JiangZhu UniversityofAlaska-FairbanksAdvancedDataAssimilationTechniquesAppliedtoaRegionalHighResolutionRapidRefreshModelinAlaska(HRRR-Alaska)

Ting-ChiWu ColoradoStateUniversity

EvaluationoftheNewlyDevelopedObservationOperatorsforAssimilatingSatelliteCloudandPrecipitationObservationsinGSIwithintheHWRFsystem

2 SoftwareSystems

Toserveasabridgebetweenoperationsandresearch,theDTCprovidesaframeworkforthetwocommunitiestocollaborateinordertoacceleratethetransitionofnewscientifictechniquesintooperationalweatherforecasting.Thisframeworkisbasedonsoftwaresystemsthatareasharedresourcewithdistributeddevelopment.Thecurrentoperationalsystemsareasubsetofthecapabilitiescontainedinthesesoftwaresystems.Ongoingdevelopmentofthesesystemsismaintainedunderversioncontrolwithmutuallyagreeduponsoftwaremanagementplans.TheDTCcurrentlyworkswiththefollowingsoftwaresystems:

• WeatherResearchandForecasting(WRF)–NWPmodel+pre-andpost-processors• HurricaneWRF(HWRF)-setoftoolsfortropicalstormforecasting,includingacoupled

atmosphereandoceansystem• UnifiedPost-Processor(UPP)• GridpointStatisticalInterpolation(GSI)dataassimilation(DA)system• EnsembleKalmanFilter(EnKF)DASystem• Modularend-to-endensemblesystem• ModelEvaluationTools(MET)–Verificationpackage

TheDTCdoesnotgenerallycontributetothedevelopmentofnewscientifictechniquesforthesesoftwarepackages.ThetwoexceptionsareMETdevelopmentandsomelimitedphysicspackagedevelopmentforWRFtoaddressshort-comingsbroughttolightbyDTCT&E.TheDTCcontributestothesoftwaremanagementofallofthesesystemsandusersupportforthepublicly-releasedsystems(WRF,HWRF,UPP,GFDLvortextracker,GSI,EnKFandMET).Allsoftwaremanagementandusersupportactivitiesarecollaborativeeffortswiththedevelopers,wheretheexactroleoftheDTCdependsonthesoftwarepackage.ThemaindevelopersofthesepackagesareaffiliatedwithEMC,ESRL,NCAR,GlobalModelingandAssimilationOffice(GMAO)oftheNationalAeronauticsandSpaceAdministration(NASA),NationalEnvironmentalSatellite,DataandInformationService(NESDIS),JCSDA,GFDL,UniversityofRhodeIsland(URI)andtheHurricaneResearchDivision(HRD)ofNOAA’sAOML.

TheDTCisworkingwithEMCtounifytheverificationsystemsbetweenthetwoorganizationsthroughMETandMETViewer,MET’saccompanyingdatabaseanddisplaysystem.DTCstaffvisitedEMCforaweekduringearlyMay.Thisvisitconsistedof18meetingswithapproximately50EMCstafftodiscuss

6

theircurrentverificationpracticesandimmediateneeds.InformationgatheredduringthesemeetingsissummarizedinarequirementsdocumentthatwasreleasedtoallparticipantaswellastheEMCDirectoron15September2016.TheDTCisusingthisinformationtodevelopaunifiedverificationsystem,calledMET+,throughNextGenerationGlobalPredictionSystem(NGGPS)fundingoutsideoftheDTC.Briefly,MET+isasetofpythonwrapperstosimplifysetting-upandrunningMETtoallowresearcherstoleveragetheirownuniquealgorithms,andsystematicallyplotthefieldsandresults.

FortheGMTB,theDTChasbeenworkingwithEMCandthephysics-developmentcommunity(ESRL,NCAR,NRL,anduniversities)toestablishaCommonCommunityPhysicsPackage(CCPP)thatwillserveasaframeworkforefficientlydevelopingandtransitioningcurrentandnext-generationphysicsparameterizationsintooperationstomeettheneedsofNGGPS.AnotherimportantcomponentofthisworkisestablishinganInteroperablePhysicsDriver(IPD)thatprovidesaframeworkforphysicalparameterizationsuiteswithintheCCPPtointerfacewithdifferentdynamiccores.Overthepastyear,therequirementsfortheIPDandCCPPhaveundergoneextensivereviewandrefinement,andinformedasoftwaredesign.TheinitialdesignwaspresentedtoEMCandtheNationalUnifiedOperationalPredictionCapability(NUOPC)PhysicsInteroperabilitygroup,aswellasseveralsubsequentrevisionsthatincorporatedfeedbackfromthesegroups.Thisdesigndocument,aswellasthephysicsaliasinglayer(alsoknownasIPDv4)developmentperformedbyGFDLandmadeavailableforFV3inMarch2017,servedasthefoundationfortheinitialCCPPandIPD.GMTB’scontributiontotheIPDeffortwillenableconstructingphysicssuitesatruntimebyparsingauser-friendlyconfigurationfile,allowingforrunningtheparameterizationswithintheCCPPinaveryflexiblemanner.

TheGMTBintendstofacilitateanenvironmentfortheCCPP,definedasasetofpracticesinwhichdesign,development,anddeploymenthappensimultaneouslyandrapidlyinthesameecosystem.Tosupportthisbasicprincipleofmodernsoftwaredesign,theGMTBalsocompleteddocumentsthatdescribetheconceptandmanagementoftheCCPPdevelopmentandsoftware,aswellasaproposedgovernancestructuretomanagetheevolutionoftheCCPP.TheconceptanddesignoftheCCPPdescribesanecosystemfordevelopmentandtransitionofphysics,whereanEMCdevelopercaneasilydeveloponhisownorengageexternaldevelopers.Codeiselevatedtosupportedoroperationalstatusfollowingasuiteofscientifictests.Detailsofthesetestsaretobedeterminedbyagovernancestructureyettobefinalized.AproposalforthisgovernancestructureisincludedintheGMTB’sCCPPRoadmap.TheGMTBtestharnesswillplayakeyroleinmakingthesetestsaccessibletoalldevelopers.Acodemanagementplanwascreatedtosupporttheecosystemandmeetthegovernanceneeds.

Duringthefinaltwomonthsofthereportingperiod(Feb-March2017),theCCPPwascreatedwithplaceholderphysics–askeletontoguidetheconnectiontorealphysics.Fortestingpurposes,theGMTBSingleColumnModel(SCM)wasmodifiedtocallasuitecomposedoftheCCPPplaceholderphysicsthroughtheIPD.Thisimplementationcanbeconsideredthesimplest“dycorecap”fortheIPD–theSCMreplacesactualdynamicswithadvectiveforcing,butitenablesthetranslationofSCMstateanddiagnosticvariablestothosewithinthephysicssuite.ThefirstCCPPreleaseisplannedforearly2018.NCARandGSDphysicsdevelopershavebeenengaged,andwillbeconnectingnewphysicsunderseparatefunding

Buildingondocumentationeffortsfromyear1,comprehensivetechnicalwebdocumentsarenowservedfromtheDTCwebsite,describingboththebackgroundandcurrentfunctionoftheIPDv2usedduringtheNGGPSdynamicalcoretestandtheinitialmemberoftheCCPP,the2016operationalGlobalForecastSystem(GFS)physicssuite.ThecontentofbothdocumentswasgeneratedroughlyequallybytheNCARandGSDmembers,drawingonexpertisefromallcontributorsasappropriate.AreviewbyphysicsexpertsatNOAAEMCwassolicitedandobtained,withfeedbackintegratedintothefinaldocuments.ThedocumentationfortheIPDandCCPPcanbeaccessedatthefollowingURLs,

7

respectively:http://www.dtcenter.org/GMTB/gmtb_ipd_doc/andhttp://www.dtcenter.org/GMTB/gfs_phys_doc/.

Theprocessforupdatingthedocumentationgoingforwardisinfluxatthistime.AttherequestofEMC,sourcefilescontainingtheDoxygen-formattedcommentsusedtogeneratethedocumentationwebpagesweremergedintothetrunkofEMC’scoderepository.Inaddition,allexternalfiguresandfilesnecessarytogeneratethedocumentationhavebeenplacedinsidea‘docs’directorynexttothephysicssourcefiles.Asaresult,documentationmaybeversioncontrolledanditispossibleforbothDTCstaffandthoseatNOAAEMCtogeneratethedocumentationoutput.AlthoughaplanforfullyintegrateddocumentationforallcomponentsofNGGPSisnotcomplete,theDTChasprovidedinputtodecision-makerswithinNOAAregardingtheuseofDoxygenastheappropriatetoolandfordefiningtheprocessfordeveloperstoupdatedocumentationasdevelopmentprogresses.

2.1 SoftwareManagement

Whilespecificsoftwaremanagementplansdifferbetweenthevarioussoftwarepackages,theyallcontainthefollowingelements:

• Coderepositoriesmaintainedunderversioncontrolsoftware.• Protocolsforproposingmodificationstothesoftware,whetherthemodificationsaresimply

updatestocurrentfeatures,bugfixesortheadditionofnewfeatures.• Testingstandardsproposedsoftwaremodificationsmustpasspriortobeingcommittedtothe

coderepository.• Additionaltestingstandardsusedtomorethoroughlychecktheintegrityoftheevolvingcode

base.

Givenallthesesoftwarepackagescontinuetoevolveovertime,alltestingstandardsmustbeupdatedperiodicallyinordertomeetthemaintenancerequirementsofthecodebase.Overthepastyear,theDTCcontinuedtocollaboratewiththevariousdevelopergroupsontheseongoingsoftwaremanagementactivities.TheDTCalsocontinuedtoprovideapathwayfortheresearchcommunitytocontributetothedevelopmentofthesesoftwaresystems.Noteworthyeventsfromthisworkoverthepastyearare:

• WRF–Overthepastyear,worktowardsaddinganoptiontoruntheAdvancedResearchWRF(ARW)dynamiccorewithanewsmoothedterrain-followinghybrid-verticalcoordinatewascompleted.ThiscodewillbeincludedinthenextreleaseofWRFandwillbebackwardscompatibleifthenewoptionisnotselected.All2016operationalHWRFforecastsystemcapabilitieswerecommittedtotheWRFtrunk,andweremadeavailableforthenextcommunityreleaseofWRF.ThesecapabilitiesincludetheGFShybridEddy-DiffusivityMass-Flux(EDMF)PlanetaryBoundaryLayer(PBL)scheme(availableforbothARWandNonhydrostaticMesoscaleModelontheEgrid-NMME),updatestothesurfacefluxexchangeswiththecoupledocean(NMMEonly),updatedscale-awareSimplifiedArakawa-Schubert(SAS)cumulusparameterization(ARWandNMME),alandfalloptionfortheidealizedtropicalcyclone(TC)capability(NMMEonly),aswellasmiscellaneousbugfixesandtuningparameters.

• UPP–TheDTCcontinuedtoworkcloselywithEMCtomanagetheUPPcodebasethroughregularbi-monthlymeetings.TostreamlinetheeffortstokeepthecommunityUPPinsyncwithEMC’soperationalUPP,thecommunityUPPsourcecodewasmigratedtoabranchoftheEMCrepositoryforeasiercodesharingandsyncing.ThroughcollaborationswithoperationaldevelopersandsupportedDTCvisitors,themostrecentcommunityreleaseofUPPincludedfull

8

GRIB2outputcapability,alongwithnewmicrophysics-specificreflectivityoutputandsyntheticsatellitefields.

• HWRF–TheDTCcontinuedtosupportHWRFdevelopersinusingandaddinginnovationstothecoderepository.TheDTCcompleteddevelopmenttoenablethemultistormconfigurationtorunusingthesamesetupastheoperationalconfiguration,withtheexceptionofoceancoupling,andaddedthiscapabilitytotheHWRFrepository.FixeswerecommittedtotherepositorytoenablebackwardscompatibilityforthepreviousoperationalHWRFconfiguration.Additionally,theDTCdevelopedanewcapabilitytostartaHWRFrunfromtheWRFcomponentwhenusingwrapperstoaidHWRFcommunitydeveloperswhowanttomodifyandruntheatmosphericforecastcomponentwithoutgainingexpertiseonthefullend-to-endsystem.Theabilitytosimulatelandfallwithintheidealizedtropicalcyclonecapability,aninnovationdevelopedbyaDTCvisitor,wastransitionedtotheHWRFcoderepository.TheHWRFrepositorywasadaptedtoaccommodatethetransitionoftheWRFandWRFPre-processingSystem(WPS)repositoriesfromSubversion(SVN)toGitandthecommunityGSIrepositorytransitiontoVLab.UpdatedproceduresandinstructionswerepublishedtotheHWRFdevelopers’webpageandassistancewasprovidedviatheHWRFhelpdesk.TheDTCprovidedcoordinationofdevelopmentactivitiesbychairingtheHWRFdevelopers’committeebi-weeklymeetings.Additionally,DTCprovidedenhancedsupportfordeveloperscontributingtotheHWRFsystem,includingHurricaneForecastImprovementProject(HFIP)-fundedprincipalinvestigators.Tofacilitateinter-developercollaboration,theDTCcontinuedtohostanhwrf-contribrepositoryforpeer-to-peersharingofcode.TheNCARandGSDstaffconductedthisworkjointly.TheHWRFv3.7aandv3.8aUsers’GuideswerepublishedasGSDtechnicalnotes:

Biswas,M.K.,L.Carson,C.Holt,L.Bernardet,2016:CommunityHWRFUsersGuideV3.7a.NOAATechnicalMemorandumOARGSD-46,doi:10.7289/V5SJ1HMD,144pp.

Biswas,M.K.,L.Carson,K.Newman,L.Bernardet,C.Holt,2017:CommunityHWRFUsers’GuideV3.8a,NOAATechnicalMemorandumOARGSD-47,doi:10.7289/V5/TM-OAR-GSD-47,149pp.

• GSIandEnKF–Overthepastyear,theDTCmadeafewcriticalupgradestocurrentcodemanagementandsupporteffortsfortheDAsystems.TheDTCtransitionedthehelpdesktoRequestTracker(RT)forbettertrackingofusers’requestsandquestions.TheDTCbuiltanewcommunityrepository(svn)onNOAA’sVlabserverandsuccessfullytransitionedallcommunityrepositorydeveloperstothisnewrepository.TheDTCalsoinitiatedeffortstoworkwithEMCtoimprovetheefficiencyofcodemanagementfrombothsidesasfollows:startedeffortstounifythecodebuildtool(usingcmake)forGSI,EnKF,andNCEPI/Olibraries;convertedtheuser’sguidestoLaTeXtosharewithalldevelopersthroughthecoderepository;andtransitionedcommunityutilities(e.g.,formatconversion,diagnosticplottingscripts)totheEMCrepository.TheseeffortswillcontributesignificantlytotheunificationoftheDTC-EMCcoderepositoriesoverthecomingyear.On-goingeffortsincluded:supportGSI/EnKFusersthroughthenewcoderepositoryandhelpdesk,performcodereviewsforeachproposedcodeupdateandsynchronizetheDTCcommunitycoderepositorywiththetrunkofEMC’soperationalrepository,coordinatetheGSI/EnKFdevelopmentamongdistributeddevelopersbychairingtheDAReviewCommitteeandhostingthreereviewcommitteemeetings.AnotableoutcomeofthereviewcommitteemeetingswastheadditionofJCSDAasthe10thmemberofthecommittee.

ForAOP2016,theDTCscaledbackitsworkwiththeNOAAEnvironmentalModelingSystem(NEMS)/NonhydrostaticMultiscaleModelontheBgrid(NMMB)duetothelackofanyT&Eactivitiesutilizingthis

9

softwarepackage.WhiletheDTCisnolongeractivelyengagedincodemanagementorusersupportforthiscodebase,itdidcontinuetotesttheportabilityoftheNEMSsoftwarepackageandassociatedlibrariesthroughregressiontestingforregionalapplicationsontheNCARsupercomputer,Yellowstone.Overthepast6months,significantinfrastructureandcodearchitecturechangesoverthepastsixmonthsledtofailedportabilitytestsandretrofittingtoanewplatformisconsideredbeyondthescopeoftheDTCactivitiesatthistime.RelevantfeedbackwasprovidedtoEMCNEMSdevelopers.

2.2 VerificationToolDevelopment

Overthepastyear,theDTCverificationteamcompletedtwoMETreleases.METv5.2wasreleasedtothecommunityon15August2016andMETv6.0on3April2017.Approximately20majorenhancementsand50smallerbugfixesweresupportedtothecommunityinthereleases,includingtheupgradetoreading/writingNetworkCommonDataForm(NETCDF)4,a25%speed-upoftheGridpointStatisticalInterpolation(GSI)diagnostictool,andconfigurationfilechangestomakeMETeasiertoset-up.

Tosupportglobalverification,thegrid-to-gridverificationtoolswereupdatedtoincludeaconfigurationoptiontoapplygridboxareaorcosine-latitudeweightingtothecomputationofcontinuousstatistics,suchasAnomalyCorrelation,RootMeanSquareErrorandBias.AlltoolsareprovidedwithGRIB1/2tablesupportfornon-NCEPtablessuchasUKMetOfficeandEuropeanCenterforMediumRangeForecasting(ECMWF).TheASCII2NCtool,whichcreatesNetCDFfilesfrompointobservationsinASCIIformat,wasenhancedtoincludeconfigurationoptionstospecifytheexpectedfrequencyofobservationsandomitoutputwhennotenoughvaliddataarepresent.ThisfeaturewasrequestedbytheNEMSGlobalAerosolComponent(NGAC)group.A“DESC“column(shortfordescription)wasaddedtoMETtoallowuserstoaddadescriptorformoreeffectivestratificationofstatisticsinbothMETandMETViewer.TosupportNOAA’stropicalcyclone(TC)verification,theTC-pairstool,whichmatchesTCforecastsandobservations,wasupdatedtohandleinterpolatedmodelswhosemodelidendsin'3',readprobabilisticforecastsfromthe“E-deck”fileformat,andincludemoreflexiblewaysofpassingthetooldifferentfilenamesforthebestandoperationaltracks.

SeveralmeasuresandmethodswereaddedtoMETtosupportverificationoftotalcloudfractionontheglobalscale.NewinterpolationmethodswereaddedtoallowinterpolationofmodeloutputtobehandledinthemannersimilartotheWorldWideMergedCloudAnalysis(WWMCA)cloudfractionmapping.ThetoolforregriddingWWMCAwasenhancedtodrawadditionalfields,suchasSatelliteID(SatID)andpixelage,fromthebinarycloudanalysisfiles.Also,theGen-VX-masktool,whichcreatesabitmappedmaskedarea,wasupdatedtocomputethesolarazimuthandanglevaluesbasedonlocationandtimeofdaytoallowforthederivationofday/nightmask.

TheMETViewerdatabaseanddisplaysystemisanotherverificationtoolunderdevelopmentduringtheAOP.Itwasmodifiedtosupportthenewfileformats,statisticsand“line”typesintroducedviatheMETv5.2andMETv6.0releases.TheDTCmodifiedtheloadinglogictohandlespecialcasesintroducedbydifferentversionsoftheNOAAmesoscaleandensembleverificationstatisticdatabasefileformat(VSDB).TheinterfacewasalsoenhancedtocomputethemeanandaratioofseveralcurvestosupporttheRegionalEnsemblesteam.Basicdatabasepurgingscriptswerealsogenerated.METViewerv1.9-1.12werereleased,includingtheadditionofaTaylorDiagramtemplate,enhancementstotheeventequalizationlogicandtheMethodforObject-basedDiagnosticEvaluation(MODE)attributecomputationforEMC’sMesoscaleandGlobalbranches,respectively.METViewerdatabasedesignwasinterrogated,severalchangesweremadetospeedupdataloadingandquerying.Thesechangesincludedpartitioningthedatabase,notattemptingtoloademptyfiles,andprovidingthecapabilitytoremoveasingleormultiplerecords,ifneeded.Newdatabasetechnologies(e.g.Couchbase)werealso

10

investigatedandfoundpromising.TheDTCalsoinvestigatedoptionstomodifytheMETViewerinterfacetoallowuserloginsandsaveuserpreferences.

TeleconswithNOAAMETandMETVieweruserswereheldeverymonth,startinginOctober2016.Inviteesinclude25engagedNOAAstafffromEMC,WeatherPredictionCenter(WPC)andGSD.ThediscussionsweredrivenbyquestionsarisingfromincreaseduseofMETandMETViewer.Inresponsetothesemeetings,theDTCstaffansweredmanyquestions,fixedseveralbugsintroducedwithloadingVSDBdataintoMETViewer,improvedthespeedofloadingMETViewerandaddednewfeaturestobothMETandMETViewer.METViewercodewasmovedintoGitHubtoallowbothDTCnodesandexternalcollaboratorsaccess.Additionally,planninganddevelopmentoftheinitialpythonwrappersaroundthecomponents,calledMET+,wasdistributedacrossbothDTCnodes.TheGSDnodecontributionestablishedagreaterunderstandingoftheMET+components(METandMETViewer)andallowedmoremeaningfulcollaborationbetweenthenodes.

2.3 Publicly-ReleasedSystems

TheDTCcurrentlycollaborateswithdevelopersonsevensoftwaresystemsthatundergoapublicreleaseprocess:WRF,UPP,HWRF,GFDLvortextracker,GSI,EnKFandMET.Assistancecontinuedtobeofferedthroughemailhelpdesksforallpackages.Informationregardingthetimingandversionofthemostrecentrelease,alongwiththecurrentnumberofregisteredusersandaveragehelpdeskticketspermonthforeachpackagearelistedinTable2.3-1.Table2.3-2containsalistofthewebaddressesforeachsoftwarepackage’susers’page.

Table2.3-1:Codereleases,numberofregisteredusersandnumberofhelpdeskticketspermonthforthepublicly-releasedsoftwarepackagessupportedbytheDTCoverthepastyear.

SoftwarePackagePublicRelease

Version Timing RegisteredUsers Helpdeskticketspermonth

WRFV3.8 April2016

~32,700 ~400V3.8.1 August2016

UPP V3.1 September2016 ~740 ~10HWRF V3.8a November2016 1399

~30GFDLVortexTracker V3.5b September2013 617

GSI V3.5 August20161,687

~20EnKF V1.1 August2016 ~2-5MET V5.2 August2016 3180 ~20-25

Table2.3-2:Userspagewebsitesforpublicly-releasedsoftwarepackages.SoftwarePackage UsersWebsites

WRF http://www.mmm.ucar.edu/wrf/users/

UPP http://www.dtcenter.org/upp/users/

HWRF http://www.dtcenter.org/HurrWRF/users/

GSI http://www.dtcenter.org/com-GSI/users/

EnKF http://www.dtcenter.org/EnKF/users/

MET http://www.dtcenter.org/met/users/

11

InadditiontogeneralMETusersupport,theDTCverificationteamactivelyrespondedtorequestsfromNOAAusersregardingtheuseofMETandMETViewer.Accomplishmentsoverthepastyearinclude:1)increasednumberofMETandMetViewerusersfromNOAA(from74to100);2)workedwithNCEPEnvironmentalModelingCenter(EMC),WeatherPredictionCenter(WPC),andClimatePredictionCenter(CPC)toidentifyrequirementsnecessarytounifyverificationbetweenDTCandEMC;3)whitepaperelucidatingtherequirementssubmittedtopointsofcontactatEMCandEMCdirector;4)startedexploringdatabasedesignstohandlelargedatasets;and5)participatedintheNGGPSVerificationandValidationTeamdiscussions.

TheDTCrevampedtheonlinetutorialsforbothGSIandEnKF.Theonlinetutorialswereredesignedtoprovidemoreuser-friendlyinstructionsalongwiththelatestGSIandEnKFcapabilities.Testcaseswerecarefullyselectedwithnewtestingperiodsanddomains,coveringupdatedandadditionalconfigurationswithalternativedataassimilationtechniques[3DVar,3Dhybrid,4DhybridEnsembleVariational(EnVar)],datatypes(conventionalandsatelliteradiancedata),andforecastmodels(ARW,HWRF,NMMB,WRF-chem,GFS).

2.4 DTC-supportedsoftwarecontainers

Manytimesthebiggesthurdlewhenrunninganewsoftwaresystemisgettingitsetupandcompiledontheintendedcomputerplatform.Buildingcomplexsystemsthatrequireanumberofexternallibrariescanbealargeissueforuserstoovercome.Inordertorelievesomeofthisdifficulty,anewtechnologyreferredtoasa“container”hasbeendevelopedthatallowsforcompletesoftwaresystemstobebundledandshippedtousers.Thecontainersincludeeverythingthatisneededtorunthesoftwarecomponent,includingtheoperatingsystem(toolsandlibraries)andcode-thusallowingfortheusertoquicklyproduceoutputwithoutbeingdelayedbytechnicalissues.

ContainershavebeenestablishedoutsideoftheDTCforportionsofanend-to-endNWPsystem,includingWPS,WRF,andNCL.DuringAOP2016,DTCstaffestablishedcontainersfortheUPPandMETsoftwaresystems.Inaddition,datasetsthatmakeuptwoMesoscaleModelEvaluationTestbed(MMET)caseswerebundledinacontainer.ContainersareavailableviaGitHubtorunMET(https://github.com/NCAR/container-dtc-met)andtoruntheend-to-endsystem(includingMETandMMETdataset(https://github.com/NCAR/container-dtc-nwp).Byestablishingtheseadditionalcontainers,theDTCisassistingtheusercommunity(especiallystudents)withefficientlyrunningNWPcomponentsinanefforttofosterconnectionswithfuturecollaborators.

3 TestingandEvaluationT&EactivitiesundertakenbythedevelopersofnewNWPtechniquesfromtheresearchcommunityaregenerallyfocusedoncasestudies.However,inordertoadequatelyassessthesenewtechnologies,extensiveT&Emustbeperformedtoensuretheyareindeedreadyforoperationalconsideration.DTCT&Egenerallyfocusesonextendedretrospectivetimeperiods.Thecasesselectedincorporateabroadrangeofweatherregimesrangingfromnull,toweakandstrongevents.Theexactperiodschosenvarybasedonthephenomenonoffocusforthetest.ThetechniquetobetestedmustbepartofthecoderepositoriessupportedbytheDTCtoensurethatthecodehasreachedacertainlevelofmaturity.TheDTC’sevaluationoftheseretrospectiveforecastsincludesstandardverificationtechniques,aswellasnewverificationtechniqueswhenappropriate.Allverificationstatisticsundergoastatisticalsignificance(SS)assessmentwhenappropriate.Byconductingcarefullycontrolled,rigoroustesting,includingthegenerationofobjectiveverificationstatistics,theDTCisabletoprovidetheoperationalcommunitywithguidanceforselectingnewNWPtechnologieswithpotentialvalueforoperationalimplementation.DTCtestingalsoprovidestheresearchcommunitywithbaselinesagainstwhichtheimpactsofnew

12

techniquescanbeevaluated.Thestatisticalresultsmayalsoaidresearchersinselectingmodelconfigurationstousefortheirprojects.

3.1 RegionalEnsembles

MesoscaleNWPsystemsareutilizedinbothresearchandoperationalforecastingapplicationsandcanbeconfiguredtosuitabroadspectrumofweatherregimes.DuetothenumberofapproachesdevelopedandofferedbyNWPsystems,itisnecessarytorigorouslytestselectconfigurationsandevaluatetheirperformanceforspecificapplications.

OnepaperassociatedwithapastRegionalEnsemblesactivitywaspublishedinMonthlyWeatherReview:

Jankov, I., J. Berner, J. Beck, H. Jiang, J.B. Olson, G. Grell, T. G. Smirnova, S. G. Benjamin, J. M. Brown, 2017: A performance comparison between multiphysics and stochastic approaches within a North American RAP ensemble. Mon. Wea. Rev., 145, 1161-79.

3.1.1 MesoscaleModelEvaluationTestbed(MMET)

TheMesoscaleModelEvaluationTestbed(MMET;http://www.dtcenter.org/eval/meso_mod/mmet)providestheopportunityfortheresearchcommunitytoconducttheirownT&Eofanewtechnique.DatasetsforanumberofcasesdeemedtobeofhighinterestbyEMCaredistributedviaRAMADDA,aRepositoryforArchiving,ManagingandAccessingDiverseDAta(http://ramadda.org/).MMETdatasetsincludeavarietyofinitializationandobservationdatasets,aswellasbaselinesforselectoperationalconfigurations.Casesofinterestand/orpersistentoperationalmodelissueswereidentifiedthroughouttheyearbyleveragingadirectlinktoEMC’sModelEvaluationGroup(MEG)throughDTCstaffparticipationinMEGweeklytelecons.Alistofoperationalcasesofinterestand/orpersistentmodelweaknesseswascompiledbasedontheseweeklydiscussionsandpublicizedontheMMETwebpage(http://www.dtcenter.org/eval/meso_mod/mmet/additional_cases.php).

Inthepast,asnewversionsoftheWRFandNEMScodewerereleased,MMETcaseswerereruntoprovidecurrent,baselineresultsfortheusercommunity.Whilethefullend-to-endsystemisnolongerupdatedonanannualbasis,previousversionswillremainavailablethroughtheRAMADDAdataserver.Rather,thisyear,operationalmodeloutputforseveralNWPsystems(bothdeterministicandprobabilistic)wereevaluatedandtheobjectiveverificationscoresprovidedtotheresearchcommunitythroughMMET.TheoperationalforecastsystemsincludedtheNorthAmericanMesoscale(NAM),RapidRefresh(RAP),High-ResolutionRapidRefresh(HRRR),andHurricaneWRF(HWRF).ThesebaselinesareprovidedforallnewandexistingMMETcasesforeachavailableoperationalmodel.Twonewcaseswereestablishedthisyear:1)acaseoverAlaska(20150826:TyphoonAtsaniremnantsaffectingAlaska)and2)ahurricanecase(20160928-29:HurricaneMatthew).AnothernewadditionthispastyearistheevaluationoftheStormScaleEnsembleofOpportunity(SSEO)datacollectedfromthe2016HazardousWeatherTestbed(HWT)SpringExperiment.ThisevaluationincludeddeterministicperformanceresultsforeachindividualSSEOmemberalongwithprobabilisticresultsfromtheensembleasawholeforselectvariables;theplotsforselectvariables,thresholds,andmetrics,aswellastheMETViewerXMLsusedtogeneratetheplots,areavailableviaRAMADDA.

Inaddition,communityoutreacheventscontinuetobeprovidedbytheDTCtopromoteenhancedconnectionswithfuturecommunitycollaboratorsandpromotetheuseofMMETdatasets.A1.5hourinstructionalsessionwasofferedduringthe17thWRFUsers’WorkshopinJune2016toraiseawarenessaboutandhighlightthetoolsavailabletothecommunity-at-largethroughMMET.AposterwaspresentedonMMETattheAmericanMeteorologicalSociety(AMS)AnnualMeetinginJanuary2017;

13

Harrold,M.,J.K.Wolff,andT.Hertneky,2017:MesoscaleModelEvaluationTestbed(MMET):HelpingConnecttheResearchandOperationalCommunities.28thConferenceonWeatherandForecasting/24thConferenceonNumericalWeatherPrediction,Seattle,WA,January24-27,2017.

andaproposaltoholdashortcourseonusingDTC-supportedcontainers(includingMMETcases)atthe2018AnnualAMSmeetingisbeingwrittenforsubmissionattheendofApril.Finally,amanuscriptwaspreparedandacceptedforpublicationintheNovembereditionoftheBulletinoftheAmericanMeteorologicalSociety(BAMS).

Wolff,J.K.,M.Harrold,T.Hertneky,E.Aligo,J.Carley,B.Ferrier,G.DiMego,L.Nance,Y.-H.Kuo,2016:MesoscaleModelEvaluationTestbed(MMET):AresourcefortransitioningNWPinnovationsfromresearchtooperations(R2O).Bull.Am.Met.Soc.DOI:http://dx.doi.org/10.1175/BAMS-D-15-00001.1

3.1.2 HRRREnhancements

TheHigh-ResolutionRapidRefresh(HRRR)modelbecameoperationalatNCEPon30September2014,andhasfoundwideacceptancebyforecastersinsideandoutsideoftheNationalWeatherServiceasguidanceforavarietyofweatherphenomenaacrossallseasons.TheHRRRusestheARWdynamicalcore,aphysicspackagethathasprovedeffectiveatcapitalizingonthecloud-permittingresolutionofthemodel,anduniqueinitializationproceduresusingradarandsatellitedata,aswellasconventionalin-situobservations,togetherwithaRapidRefresh(RAP)forecast.Inviewoftheimportanceofaccurateshort-termforecastsforvulnerablecoastalareas,particularlyalongtheGulfandAtlanticcoasts,theDTCinvestigatedthevalueofexpandingtheHRRRdomaintowardtheeastandsouth.Atthetimethisworkwasoriginallyproposed,theoperationalNCEPHRRRforecastsonlyextendedto15h,butESRLisnowrunningtheHRRRexperimentallyto36h.ThelateralboundariesoftheHRRRdomainareoftenwithinthecirculationofnortheastcoastalsnowstormsandland-fallingtropicalcycloneswhenthesesystemsarewithina24-36-hstrikingdistanceoftheUSmainland.ExpandingtheboundariesofthecurrentHRRRdomainthusbecameincreasinglyimportantfortheselongerforecastlengths.

ThreetropicalcyclonecasestudieswereconductedwithHRRRversion2(HRRRv2),whichistheversionimplementedoperationallyatNCEPinAugust2016.Themodelinitializationtimesforthethreestormswerechosensuchthatthecenterofeachstormwasnearorslightlyoutsidethestandarddomainbutwellinsidetheextendeddomain.Thestormsandinitializationtimesofinterestwere:Bonnie(12UTC27May2016),Colin(00UTC6Jun2016)andHermine(12UTC31Aug2016).ExtendeddomainsimulationswerealsoconductedforHurricaneHermine,withandwithoutlightningdataassimilation,toaforecastlengthof36h.Herminewaschosenbasedonthelargenumberoflightningstrikesoccurringinanareaofrelativelylowsimulatedreflectivityvaluesatmodelinitializationtime.

Projectdeliverablesweretwo-fold:(1)upgradedproceduresforinitializationofconvection-permittingmodelsoveroceanareaspronetodeepconvection,mesoscaleconvectivesystemsandtropicalcyclones,and(2)arecommendationfordomainconfigurationandphysicssuiteforanexpandedHRRR.Theresultsofourexperiments,particularlythoseforHurricaneHermine,suggestedforecastvalueassociatedwithexpandingtheHRRRdomaintothesouthandeast.ResultsofsensitivityteststolightningdataassimilationpromptedinclusionofalightningreflectivityproxyalgorithmwithintheexperimentalHRRRv3,foreventualimplementationatNCEPin2018.Amorecomprehensivesummaryofthisworkisavailableonlineat:http://www.dtcenter.org/eval/ensembles/dtc_expanded_domain_report.docx

14

3.1.3 Testingandevaluationofsmoothedterrain-followingcoordinateinWRF

Oncethenewsmoothed,terrain-followingcoordinatewascommittedtotheRAP/HRRRrepositoryinlate2016,theGSDnodeofDTCbeganimmediatetestingtoanalyzetheimpactsofthisnewverticalcoordinate.ThisnewfeaturewasemployedintheexperimentalRAPandHRRRmodelsrunatGSD,andanumberofcoldstartRAPrunswereconductedascontrolledtestsofthenewverticalcoordinate.Itquicklybecameclearthataproblemexisted,asnumerousmodelcrashesfollowedandspurious/excessivejet-levelwindcharacteristicswerefounduponanalysisofmodelresults.ThesefindingsweresharedwiththeWRFdevelopers.Abugrelatedtothemapscalefactorwasfoundandcorrectedinearly2017.

FollowinganewversionofthecodebeingcommittedtotheGSDrepository,testingbeganagaininMarch2017toassessthehybridverticalcoordinate.Initialqualitativetestsusingcold-startRAPforecastsproducedstableresults.Differenceswerefoundtobenegligibleatinitializationtime,butbysixhoursintotheforecasts,250-hPawindspeedvalueswithinthecoreoftheupper-leveljetforthehybridcoordinatewerefoundtoexceedvaluesinforecastsusingtheoriginalverticalcoordinate.Outsideofthemajorupper-leveljet,anumberofareaswerefoundwherewindswereslowerforthehybridcoordinate.Mostofthedifferenceswerefoundtooccurover,ordownwindofmajormountainranges,specificallyinthewesternareasofNorthAmerica.

Furthertestingwasundertakenwithaweek-long,fully-cycledRAPexperiment,conductedfortheperiodbetween7-13March2017andanotherfive-dayexperimentfor3-7September2016.Resultsfromthesetworetrospectiverunsindicatedthehybridcoordinateresultsweresensitivetothestrengthoftheupper-leveljet.FortheSeptember2016experiment,ContiguousUnitedStates(CONUS)-wide,upper-levelrelativehumidityandwindspeedRMSEandbiasimprovedbyastatisticallysignificantamountwhenthehybridcoordinatewasused.FortheMarch2017timeperiod(Fig.3.1.3-1),similarstatistically-significantreductionsinRMSEandbiaswerefoundfortheCONUSasawhole;however,windspeedbiasfortheWesternCONUSshowedastatistically-significantdegradation.Theupper-leveljetwasstrongerintheMarch2017retro,andthereforeitishypothesizedthatthesmoothed,terrain-followingcoordinateisresultinginwindsthatareslightlytoostrongovermountainousterrainduringsynopticallyactiveperiods.Itshouldbenotedthatverificationbelowabout500hPashowednostatisticallysignificantdifferencesbetweenthetworetrospectivesimulations,highlightingtheimpactofthehybridverticalcoordinateonjet-levelwinds,particularlyinmountainousregions.

Afinalweek-longsimulationwasconductedwiththeHRRRfrom3-10September2016toassesstheimpactofthesmoothed,terrain-followingcoordinateatconvective-resolvingresolution.Resultsfromthisretrospectivesimulationshowedavastlyreducedimpactofthehybridcoordinate.Nostatisticallysignificantdifferenceswerefoundbetweenthetwoverticalcoordinatesouttothefull12hoursofeachforecast,andonlyminor,non-statisticallysignificantdifferenceswerefoundat12hours,limitedtoregionsabove200hPa.Onepossibleexplanationforthesefindingsisthatthehybridcoordinateissensitivetoresolution,withtheconvective-allowingmodelbeingabletobetterresolvemountainousterrain,therebyminimizinganydifferencesfoundbetweentheretrospectiveruns.Givenanypotentialdecreasedsensitivityathigherresolutions,itisalsopossiblethattheretroperiodofSeptember2016didnotcontainupper-levelwindspeedvaluesnecessarytoillustrateadifferenceinverticalcoordinatesat3-kmresolution.

Overall,thehybridverticalcoordinateappearstoproducethelargestimpactatupperlevels,particularlyforwindspeed.Inaddition,thesedifferencesareamplifiedovermountainousterrain,wherethelargestdisplacementsofthehybridverticalcoordinatefromthetraditionalverticalcoordinatearefound.

15

Resultsalsoshowedthatasresolutionincreases,theexpecteddifferencesbetweenthetwoverticalcoordinatesdecrease,likelyrelatedtoimprovedterraininhigher-resolutionmodels.

Giventhesefindingsandapparentsensitivitytoseasonandresolution,theNCARnodeoftheDTCwillinvestigatethehybridverticalcoordinatethroughanumberofMMETcasestudiesoveravarietyofdifferentsynopticconditions.TheseresultsareforthcomingandafinalreportforthistestingactivitywillbeavailableontheDTCwebsitebytheendofJune2017.Amanuscripthighlightingfindingsforthisactivitywillbesubmittedbeforetheendof2017.

Figure3.1.3-1.Hybridcoordinate(red)andterrain-followingcoordinate(blue)resultsfromtheRAPretrospectiveforecastsfor7-13March2017.CONUSRMSEisshownfortemperature(upperleft),relativehumidity(uppercenter),andwindspeed(upperright),andbiasforwindspeed(lowerleft).WindspeedbiasforthewesternCONUSisshowninthelowercenterplot,andatimeseriesofaveragedwindspeedbiasforthewesternCONUSfrom300-150hPaisshowninthelowerright.

3.1.4 AddressinguncertaintythroughstochasticparameterperturbationswithintheHRRRensemble

Inmostexistingregionalensemblesystems,model-relateduncertaintyisaddressedbyusingmultipledynamiccores,multiplephysicssuites,oracombinationofthesetwoapproaches.Whiletheseapproacheshavedemonstratedpotential,itistime-consumingandcostlytomaintainsuchsystems,especiallyinoperations.Inordertomovetowardamoresustainableandunifiedsystem,stochasticparameterperturbationswithintheHRRRphysicssuitewereinvestigatedwithafocusonplanetaryboundarylayer(PBL)andLandSurfaceModel(LSM)processes.

ForAOP2016,theRegionalEnsembleteamestablishedatestharnessusingtheRocotoWorkflowManagementSystemtoconductfunctionallysimilarend-to-endtestingoftheHRRRmodelinbothadeterministicandensemblemode.ThistestharnessincludesMETverificationtaskstoevaluatethedeterministicandprobabilisticforecastoutput.TheinclusionofMETintheworkflowprovidesthe

16

opportunitytonotonlyverifythefinalproducts,buttoalsoiterativelyadjusttheensembledesignwhileexamininghowprobabilisticstatisticschangewhendifferentapproachesareutilized.

Duetothehighlevelofcomplexityofrunningafrequentlyupdating(hourly),highspatialresolution(3km),largedomain(ContiguousUnitedStates-CONUS)ensemblesystem,extensivehighperformancecomputing(HPC)resourceswereneededtomeetthisobjective.AproposalwaswrittenandsupercomputingresourceswereprovidedthroughtheNCARStrategicCapability(NSC)projectsupport.ThisHPCallocationallowedforamoreextensivesetoftestsleadingtomorerobustresultsthanwouldhaveotherwisebeenpossible.

Asafirststeptowarddesigningthetest,numeroussensitivitytestsofstochasticparameterperturbations(SPP)appliedtovariousparameterswithinthePBLschemeandincombinationwithmorecommonlyusedstochasticapproachesStochasticKineticEnergyBackscatter(SKEB)andStochasticPerturbationofPhysicsTendencies(SPPT),werecarriedout.AvarietyoftestswerealsoperformedinvolvingtheLSMscheme.Preliminaryresultshighlightinginitialtestingofselectspatialandtemporalde-correlationlengthscalesofsoilmoistureperturbationswerepresentedaspostersatthe2016FallAGUmeeting

Wolff,J.K.,I.Jankov,J.Beck,L.Carson,J.Frimel,M.Harrold,H.Jiang:M.Xu,2016:AddressingmodeluncertaintythroughstochasticparameterperturbationswithintheHighResolutionRapidRefresh(HRRR)ensemble.Presentedat2016FallMeeting,AGU,SanFrancisco,CA,December12-16,2016.andthe2017AnnualAMSmeeting.

Andthe2017AnnualAMSmeeting

Beck,J.,I.Jankov,H.Jiang,J.K.Wolff,M.Harrold,J.Frimel,andL.Carson,2017:AnevaluationofstochasticphysicswithintheHighResolutionRapidRefreshEnsemble(HRRRE)andtheimpactsofHighPerformanceComputing(HPC).3rdSymposiumonHighPerformanceComputingforWeather,Water,andClimate,Seattle,WA,January24-27,2017.

Basedontheoutcomeofthesensitivitytests,aretrospectiveexperimentwasdesigned.Inadditiontothestochasticensembleconfiguration,theplanincludedacontrolensembledesignedtoincludeavarietyofPBLandLSMschemestorepresentthecurrentstateofregionalensembleconfigurations.TheSSEOobtainedfromNSSLandSPCcolleaguesforalimitedvariabledatasetservedasasecondbaselineforthistest.AdeterministicHRRRrunwithoutperturbationswasalsoperformedtoprovideabaselinetomakesuretheSPPperturbationsdidnotintroduceanunrealisticbias.Severalextendedretrospectiverunshavebeencompletedandtheteamisintheprocessofanalyzingtheresults.AreportonthefindingswillbeavailableontheDTCwebsitebytheendofJune2017.TheRegionalEnsembleteamwillalsobepreparingamanuscriptforsubmissiontoMonthlyWeatherReview,asafollowuptorecentlypublishedresultsfortestingofSPPintheRAPframework.

3.1.5 WRFtestingandevaluationactivity

InresponsetoWRFconfigurationrecommendationsfromNCAR’sMMMdivision,theAFrequestedtheDTCconductaWRFconfigurationtestwithWRFv3.8.1toprovidecriticalinformationforapossibleoperationalimplementation.InadditiontotransitioningtoanewversionofWRF,theseconfigurationupdatesincludedmigrationtotheThompsonaerosol-awaremicrophysicsschemeandtheupdatedRapidRadiativeTransferModelforGlobalClimateModels(RRTMG)radiationscheme.Toaddressthisrequest,DTCstaffconductedanend-to-endtestandevaluationactivitytoassessthesensitivityofreplacingtheAirForce’scurrentWRFv3.5.1operationalconfiguration,whichwaspreviouslytestedbytheDTC,withaproposedconfigurationforv3.8.1.Forthistest,theDTCgeneratedretrospectiveWRFv3.8.1forecastsforthesamecasesusedforitspriorWRFv3.5.1testingactivity(1July-30September

17

2011and1January-31March2012)andcomparedthesenewretrospectiveforecaststothearchivedv3.5.1forecasts.DetailsoftheWRFv3.5.1andv3.8.1configurationsaresummarizedinTable3.1.5-1.Aprojectwebpageisbeingfinalizedandwillincludepertinentinformationregardingthetestsetupandthefullsuiteofresults,alongwithacomprehensivefinalreport(http://www.dtcenter.org/eval/meso_mod/afwa_test/wrf_v3.8.1/index.php).

Table3.1.5-1:Physicssuiteconfigurationsettingsforversion3.5.1and3.8.1,alongwithothernamelistdifferencesbetweenversions.

PhysicsSuite v3.5.1 v3.8.1 Othernamelistchanges(v3.5.1→v3.8.1)

Microphysics WSM5 Thompsonaerosol-aware timestep:90s→60s eta_levels rh2qv_method:1→2 icloud:1→3 aer_opt:1→3 swint_opt:turnedon ysu_topdown_pblmix:turnedon use_aero_icbc:true diff_opt:1→2 dampcoef:0.05→0.2 epssm:0.1→0.5 scalar_adv_opt:0→1

Radiation(LW/SW) RRTM/Dudhia RRTMG/RRTMG

SurfaceLayer Monin-ObukhovsimilaritytheoryRevisedMM5Monin-Obukhov

LandSurface Noah NoahPBL YSU YSUConvection Kain-Fritsch Kain-Fritsch

Thetestingmethodologyallowedforpair-wisedifferencestobecomputedbetweenv3.5.1andv3.8.1,includinganassessmentofbothstatisticallysignificant(SS)andpracticallysignificant(PS)pair-wisedifferences.Consistentwiththesignificantchangesassociatedwiththev3.8.1configuration,alargenumberofSSandPSpair-wisedifferenceswereobservedforboththesurfaceandupperairmetrics.Briefly,intermsofBCRMSE,anumberofPSdifferenceswereseenfor2-mtemperatureanddewpointtemperature,generallyfavoringAFv3.8.1;veryfewofthedifferencesfor10-mwindspeedwerePS(notshown).Intermsofbias(seeFig.3.1.5-1),acoldtemperaturebiasat2mwasgenerallyobservedforbothconfigurations;however,AFv3.8.1wasgenerallythepreferredconfiguration,withafewexceptions.For2-mdewpointtemperature,PSdifferencesforbiasgenerallyfavoredAFv3.8.1.WhileveryfewPSdifferenceswerenotedfor10-mwindspeed,theSSdifferencesforbiasfavoredAFv3.5.1.Intermsofupperairverificationresults,upper-airtemperaturebiasshowedAFv3.8.1asthepreferredconfigurationovertheWestduringthesummer;resultsweremoremixedovertheWestduringthewinterandintheEastforbothseasons.Upper-airdewpointtemperaturebiasshowedAFv3.5.1wasfavoredintheWest,whileintheEast,AFv3.8.1wasfavoredduringthesummerwithmoremixedresultsinthewinter.Upper-airwindbiasSSdifferencesgenerallyfavoredAFv3.5.1inthesummerwithsomePSdifferences,whilewinterwasmixedwithonlyoneinstanceofPSdifferences.

WhenexaminingtheGOIndex(Fig.3.1.5-2),askillscoredevelopedbytheAF,AFv3.5.1wasshownasthebetterperformerforboththe00and12UTCinitializationsduringthesummerandwinterseasons.Basedontheoverallresultsfortheindividualmetrics,aninvestigationintothecauseoftheAFv3.8.1degradationintermsoftheGOIndexwasconductedandshowedthatremovalofRMSEfor400hPaheightfromthecalculationresultedinareversalofperformance,whereAFv3.8.1waspreferredforallbutthe00UTCwinteraggregation.

18

Figure3.1.5-1.Timeseriesplotof(a)2-mAGLtemperature(°C)(b)2-mAGLdewpointtemperature(c)10-mAGLwindspeedmedianmeanerror(bias)forthe00UTCinitializationsaggregatedacrossthesummercasesfortheEast(solid)andWest(dashed)verificationdomains.AFv3.5.1isinblueandAFv3.8.1inred.Theverticalbarsattachedtothemedianrepresentthe99%CIs.

Figure3.1.5-2.BoxplotsofGOIndexvaluesaggregatedacrossthesummerandwinterseason,stratifiedbyinitializationtime,where00UTCisinredand12UTCisinbluefor(a)thestandardGOIndexcalculationand(b)theGOIndexcalculationwithout400hPaheight.Themedianvalueisthethickblacklinelocatedatthevertexofthenotches,thenotchesaroundthemedianareanapproximationofthe95%confidenceaboutthemedian,thewhiskers,denotedbytheblack,dashedlines,denotethelargestvaluesthatarenotoutliers,andthecirclesrepresenttheoutliers.

b)

c)

a)

19

3.2 Hurricanes

3.2.1 ImpactofThompsonmicrophysicsinHWRF

2013T&EactivitiesrevealedthatThompsonmicrophysicsinthe2013versionofHWRFproducedimprovementsintrackfortheAtlantic(AL)basin,butdegradedthetrackforecastsfortheEasternNorthPacific(EP)basin.GiventhesignificantupgradestotheoperationalHWRFsystemafterthe2013version,performancewhenusingtheThompsonmicrophysicsschemewithinthe2015HWRFwasre-evaluated.ThisT&EactivitywasdesignedinclosecollaborationwiththeEMCHWRFteamtoinform2016pre-implementationtesting,wheretheThompsonandadvectedFerrier-AligomicrophysicsschemeswerebothcandidatesforreplacementoftheoperationalFerrier-Aligomicrophysicsscheme.ThefocusoftheDTC’sevaluationwastodeterminetheimpactofreplacingtheoperationalFerrier-AligomicrophysicsschemewiththeThompsonmicrophysicsscheme.TheoperationalFerrier-Aligoschemeadvectstotalcondensateonly,whereastheThompsonschemeadvectsindividualspecies.TheexperimentsincludedfivestormsfromtheALbasinandelevenstormsintheEPbasinthatoccurredduringthe2014and2015seasons.ParticularemphasiswasplacedonEPbasinstormsinresponsetothe2013T&Eresults.Priortoconductingtheretrospectivetest,boththeThompsonschemeandthepartialcloudiness(PC)schemewithintheRapidRadiativeTransferModelforGlobalClimateModels(RRTMG)parameterizationweremodifiedinanefforttounderstandandaddressthecauseoftheincreasedtrackerrorintheEPbasin.ThesemodificationsincludedfallspeedchangeswithintheThompsonmicrophysicsschemeandalterationstotheRRTMGpartialcloudinessschemetoimplementabugfixandchangethelowerlimitofthesnowandiceparticlesize.ThemajorityoftheworkonthisT&EactivitywascompletedpriortothisreportingperiodandwasdescribedfullyintheAOP2015report.ResultsrevealedthattheexperimentalconfigurationproducedimprovedtrackandintensityforecastsintheALbasin.However,intheEPbasin,theexperimentalconfigurationimprovedthespatialdistributionofclouds,buttheseimprovementsdidnottranslateintoimprovementsintrackandintensityforecasts.

ThefullreportforthisThompsonmicrophysicsevaluationisnowavailableontheDTCwebpage:http://www.dtcenter.org/eval/hwrf_thomp2016/.

TheDTCpresentedresultsfromthisworkduringthe32ndAMSHurricaneandTropicalMeteorologyConference(April2016)andthe17thAnnualWRFUsers’Workshop(June2016):

Holt,C.,M.Biswas,Z.Zhang,S.Trahan,L.Bernardet,G.Thompson,K.Newman:Anevaluationofalternativespecies-advectingmicrophysicsschemesinHurricaneWRF,32ndConferenceonHurricanesandTropicalMeteorology,18-22April2016,SanJuan,PR.

Thompson,G.,L.Bernardet,K.Newman,M.Biswas,andC.Holt:Towardsimprovingexplicitlyresolvedandsub-grid-scalecloudsinHurricaneWRF.32ndConferenceonHurricanesandTropicalMeteorology,18-22April2016,SanJuan,PR.

Holt,C.M.K.Biswas,Z.Zhang,S.Trahan,L.R.Bernardet,G.Thompson,K.M.Newman:Anevaluationofalternativespecies-advectingmicrophysicsschemesinHurricaneWRF.WRFUsers’Workshop,27-30June2016,Boulder,CO.

3.2.2 HWRFphysicsadvancementForAOP2016,theDTCpartneredwithDTCVisitorProgramPrincipleInvestigatorsandsubjectareaexpertstohelpcoordinateandtesttheperformanceofalternatephysicsschemesandinnovationsrelativetothecurrentparameterizationswithintheHWRFphysicssuite.Physicsadvancementsconsideredfortestingcoveredradiation,planetaryboundarylayer(PBL),microphysicsandcumulus

20

parameterizations,summarizedinTable3.2.2-1.InadditiontocoordinationandsupportforHWRFphysicsdevelopers,theDTCevaluatedthecodereadinessofcandidatephysicsadvancementsandconsultedwiththeEMChurricaneteamontopprioritiesforHWRF2017pre-implementationtesting.

Table3.2.2-1.Candidatephysicsadvancementsfortestingandevaluation.Innovationsselectedfortestingareindicatedinbold.

PhysicsDeveloper Institution Scheme DescriptionM.Iacono,J.Henderson AER–DTCVisitor

ProgramPIRRTMGradiation Cloudoverlapmethodology

G.Thompson NCAR/RALandDTC RRTMGradiation Modifiedpartialcloudinessscheme

G.Thompson NCAR/RALandDTC Thompsonmicrophysics

EnhancedschemebasedonAOP2015T&Eresults

S.Bao CCU–DTCVisitorProgramPI

advectedFerrier-Aligomicrophysics

InvestigationofadvectedmicrophysicswithinHWRF

R.Fovell U.Albany–DTCVisitorProgramPI

YonseiUniversityPBL HWRFsensitivitytoalternativePBL/surfacelayerschemes

G.Grell,E.Grell NOAA/ESRL–NGGPSPI

Grell-Freitascumulus Replacementschemeforscale-awareSimplifiedArakawa-Schubertscheme

RetrospectivecaseswererunforfourstormsintheALbasin(Edouard,Gonzalo,Matthew,Fiona)andtwostormsintheEPbasin(Patricia,Dolores)thatoccurredduringthe2014-2016hurricaneseasons.Alimitednumberofstormswererunduetocomputationalconstraints.Inanefforttoincreasestormdiversitywithlimitedresources,126hourforecastswererunevery18hours,with12hourforecastsforallinitializationsforcyclingpurposes.Thisrestrictionlimitsthesamplesizeforforecastleadtimesbeyond12hours,decreasingthelikelihoodofobtainingstatisticallysignificantdifferences.Fourparallelexperimentswereruntotestthesensitivityofthethreeexperimentalphysicsconfigurations.Thecontrol(CL)wasrunusingthe2016operationalHWRFdefaultsettings.TwoRRTMGcloud-radiationexperimentswereconductedtotestthesensitivityofanalternatecloudoverlap(CO)methodologyandtheimpactofamodifiedpartialcloudinessscheme.Additionally,acumulusparameterizationreplacementtest(GF)wasruntoinvestigatetheimpactoftheGrell-Freitas(GF)schemecomparedtotheoperationalscale-awaresimplifiedArakawa-Schubert(SAS).

AnewcloudoverlaptechniquefortheRRTMGradiationparameterization,exponential-random(ER),wastestedasareplacementforthedefaultmaximum-randomassumption.TheERtechniquealterstheoverlapofcontinuouscloudlayerstoallowforanexponentialtransitionfrommaximumtorandom.Otherapplicationshaveshownthismethodtobemorerealisticrelativetoradarmeasurementswithinverticallydeepclouds,addingmotivationtotestwithinHWRF.ThetrackandintensityresultsintheALbasinsuggestedmodestreductionsintrackerror,particularlybeyond2days(Figure3.2.2-1).However,thesedifferencesarenotstatisticallysignificant(SS).Absoluteintensityerrorsindicatesmallernon-SSmeanerrorsoutto30hours,withmixedimpactthroughouttheintermediateandlongerleadtimes(Figure3.2.2-1).Boththeexperimentalandcontrolconfigurationsexhibitedanegativeintensitybias(notshown),withreducednon-SSmeanbiasesforCObeyond3days.DuetolimitedcasesintheEPbasin,resultsarependingtheadditionofstormstothesample.

ApartialcloudinessschemewasimplementedwithintheRRTMGradiationschemeforthe2015operationalHWRFsystemtoaddressexcessiveshort-waveradiationreachingthesurfaceduetotransparencyofSAScloudstoRRTMGandalackofstratusrepresentation.Adjustmentsweremadetotherelativehumiditythresholdmethodologytofurtheraddresssolarradiationbiases.PriortestsofthesemodificationsusingWRF-ARWoverCONUSresultedinreducedsolarradiationbiases.TheseupdatesweretestedtoassesswhetherschemeimprovementswouldtranslatetotheHWRFsystem.SimilartotheCOconfiguration,trackerrorsintheALbasinhadatendencyforslightlyreducedmeanerrorsforthePCconfigurationrelativetoCLatthelongestleadtimes.Again,thesedifferenceswere

21

notSS(Figure3.2.2-1).AbsoluteintensityerrorsweresmallerforthePCconfigurationbeyond48hours,withSSdifferencesfavoringPCatthe84-,90-,and108-hourleadtimes.ThePCconfigurationexhibitedanegativeintensitybiasof-5to-10kts(notshown),similartotheCLconfiguration.Meanintensitydifferenceswithrespecttoleadtimeweremixedandnon-SS.AswiththeCOconfiguration,resultsfromtheEPbasinarependingincreasedsamplesize.

Figure3.2.2-1.Meantrackerrors(left)andabsoluteintensityerrors(right)intheALbasinwithrespecttoleadtime.TheCL(operational)isinblack,PCingreen,andCOinblue.Pair-wisedifferences(experimentminuscontrol)areshowninlightshadeswith95%confidenceintervals.

TheGFschemeemploysanensembleapproachtorepresentconvection,usingacollectionofparametersandalgorithmstorepresentconvectivetriggers,verticalmassflux,andclosures.Additionally,theschemeisscale-aware,makingitsuitableforHWRF’snestedgridconfiguration.Trackerrorsindicatestatisticallysignificant(SS)differencesbetweentheGFandCLforsomeoftheearlyleadtimes(18-himprovementand48-h,54-hdegradation),whereasmeandifferencesshownon-SSsmallertrackerrorsforGFbeyond84-hours(Figure3.2.2-2).ThemeanintensitybiasforGFappearstobesmallerthanthatfortheCL,butthedifferencesarenotSS(Figure3.2.2-2).TheoperationalHWRFisknowntounder-predictintensityforstrongALstorms,whichisheavilyrepresentedwiththissample.TheGFconfigurationtendstoreducethisdominatetendency;however,errorbarssuggesttheGFconfigurationmaytendtowardover-prediction.IntheEPbasin,intensitytracessuggestatendencyfortheGFconfigurationtobetterrepresentrapidintensification(RI)forspecificinitializationtimes.AdditionalRIcasesintheEPbasinareunderinvestigation.

AdditionalevaluationisunderwaytoverifyHWRF-simulatedbrightnesstemperatures(BT)againstGeostationaryOperationalEnvironmentalSatellite(GOES-13,channel4)BTs.Currently,HurricaneMatthewhasbeenverifiedontheparentdomain(d01)andinnermostnest(d03).Inadditiontothefouraforementionedconfigurations,asupplementaltest(PC+CO)wasruntoincorporatethecombinedimpactofthePCandCOinnovations.Figure3.2.2-3demonstratesthedifferentattributesofeachconfigurationanddomain,shownbyprobabilitydensityfunctions(PDFs)ofthebrightnesstemperature.Fortheparentdomain,theobservedBTfrequencyincreasessteadilyupto280K,followedbyasharpincreasepeakingat290K.TheGFandCOconfigurationspeakatcoolertemperatures,whereasthePCandPC+COexperimentsdemonstrateanimprovementastheyshiftthePDFtowardswarmerBTs.Fortheinnernest,theobservationaldistributionisapproximatelyuniform.Conversely,themodel-simulatedBTPDFsareclearlybi-modal.Notably,onbothdomainsthemodel-simulatedBTsallunderestimatetheobservedfrequencyofBTfrom235-275K.

22

Figure3.2.2-2.Meantrackerrors(left)andmeanintensityerrors(right)intheALbasinwithrespecttoleadtime.CL(operational)isinblack,GFinred,pair-wisedifferences(GF-CL)areshowningreywith95%confidenceintervals.

Fig.3.2.2-3.Probabilitydensityfunctions(PDFs)oftheobserved(yellowline)andmodel-simulatedbrightnesstemperaturesfortheGF(green),PC+CO(purple),CO(blue),PC(red),andCL(black)simulationsatforecasthour24ond01(left)andd03(right).

Fractionsskillscore(FSS)wasalsocomputedtoshowtheskillofeachconfiguration(notshown).ResultsindicatetheGFconfigurationperformsthebestforallBTthresholdsexceptthewarmest(greaterthan290K)ontheparentdomain,whereasthePCconfigurationdemonstratedthebestskillontheinner-nestforBTthresholdsgreaterthan250K.Notably,noneoftheconfigurationsworsenedHWRF’sabilitytoreproducetheobservedBTPDForsubstantiallydegradedtheFSSrelativetotheCL.

Theresultsofthepre-implementationtestingweretoadoptthePCinnovationsforthe2017operationalHWRFconfiguration.TheGFconfigurationcontinuedtodemonstratepromisewithlargerpre-implementationtestsperformedbyEMC,howeverreproducibilityissueswhenrunningwithadifferentnumberofprocessorscauseddelays,resultinginadeferreddecisionontheGFconfigurationforthe2018HWRFimplementation.Finally,inclusionoftheCOinnovationintoHWRFwastableduntil

23

developersimplementanamelistoptionforthenewoverlapmethod.Furtheranalysisofallthreeconfigurationsareunderwayforinclusionintheprojectfinalreport,whichwillbepostedontheDTCwebsite(http://www.dtcenter.org/eval/hwrf_GF_PC_CO/).Additionalcasesarebeingaddedtoincreasethesample,particularlyintheEPbasin.Furthermore,casestudiesfocusingonparticularstormsareongoingtobetterunderstandthebehaviorofeachphysicsinnovation.

3.3 DataAssimilation

OnepaperassociatedwithpastDAactivitieswaspublishedinBAMS:

H.Shao,J.Derber,X.-Y.Huang,M.Hu,K.Newman,D.Stark,M.Lueken,C.Zhou,L.Nance,Y.-H.Kuo,andB.Brown,2016:BridgingResearchtoOperationsTransitions:StatusandPlansofCommunityGSI.Bull.Amer.Meteor.Soc.,97,1427–1440,doi:10.1175/BAMS-D-13-00245.1.

3.3.1 RegionalEnsembleBasedDAT&E

UnderAOP2015funding,theDTCbuiltanexperimental4DhybridEnVardataassimilationsysteminthecontextofRAP,evaluatedthecodereadinessforregionalapplications,andprovidedfeedbacktodevelopersonitsinitialassessmentofforecastimpactsof4DhybridEnVardataassimilation.Throughouttheprocessofsettingupandtestingthiscapability,theDTCreportedbugfixesandmissingcapabilitiestotheGSIdevelopers.Replacingthe3DhybridEnVardataassimilationstepintheRAPworkflow,4DhybridEnVardataassimilationgeneratedminimalimpactsontheanalysesandforecasts.Inadditiontoreportingontheoutcomeofitstest,theDTCidentifiedareasthatneedmoreworktoimprovethecurrent4DEnVarcapabilities.Thefinalreport,whichisavailableontheDTCwebpageathttp://www.dtcenter.org/eval/data_assim/4denvar/rap_15km/,providesadetaileddescriptionoftheexperimentsanddiscussionoftheresults.

TheDTCalsopresentedtheresultsattwoworkshops:

H.Shao,M.Hu,C.Zhou,K.Newman,X.Zhang,andC.Holt,2016:Testingandevaluationoffour-dimensionalensemblevariationaldataassimilationforregionalweatherforecasts.The7thNOAATestbedandProvidingGroundsWorkshop,CollegePark,Maryland.

C.Zhou,M.Hu,K.Newman,H.Shao,andX.Zhang,2016:InitialassessmentoftheGSI-based4Dhybridensemble-variationaldataassimilationanditsapplicationforregionalforecasts.The17thAnnualWRFUsers'Workshop,Boulder,Colorado.

K.Newman,M.Hu,C.Zhou,andH.Shao,2016:InvestigatingthecapabilityofGSIfour-dimensionalensemblevariationaldataassimilationforWRF-ARWapplications.The17thAnnualWRFUsers'Workshop,Boulder,Colorado.

3.3.2 HighResolution(3km)EnVarTestingandEvaluation

TheDAT&EactivityforAOP2016focusedon4DhybridEnVarcapabilitiesforhigh-resolutionregionaldataassimilationincontextoftheHRRRsystem.HRRRcurrentlyusesaGSI-based3DhybridEnVarDAsystem,whichusestheglobalensemble(~30km)fortheensemblebackgrounderrorcalculationandone-hourARW(3-km)forecastsinitializedwithRAP(13km)analysesfromthepreviouscycle(socalled“pre-forecast”)astheDAbackground.ThisT&Eactivitywasdividedintotwofocusareas:1)demonstrationof4DhybridEnVarsystemforHRRR,and2)feasibilityandimpactassessmentforfastcyclingof4DhybridEnVar.

24

3.3.2.1 Demonstrationof4DhybridEnVarsystemforHRRR

TheDTCsetuptwoworkflows:1)aworkflowbasedonoperationalHRRRsystemand2)aworkflowthatreplacedthe3DhybridEnVarDAsystemwithanexperimental4DhybridEnVarsystem.Similartothe3DhybridEnVar,the4DhybridEnVarwasconfiguredtouseensembleinputfromtheglobalensembletocomputetheflow-dependenterrorcovariance.Notethateach4Danalysisrequiresmultiple-timelevels(3timelevelsforthistest)ofensembleandbackgroundinputfiles,incontrasttotheoneensembleandonebackgroundfilerequiredforeach3Danalysis).TheDTCusedtheseworkflowstoconducthourlyupdateexperiments.Duetocomputingconstraints,theDTCselectedareducedHRRRdomainforthetest.Thistestfocusedonthetimeperiod3-10September2016,whichincludedafast-evolvingconvectivescaleevent.Figure3.3.2.1-1showsthetestdomainwiththeanalyzedreflectivityfromthe4DhybridEnVarexperimentat1200UTCon8September2016(leftpanel)andtheobservedradarreflectivityat1155UTCon8September2016(rightpanel).

Figure3.3.2.1-1.Thetestdomainand4DhybridEnVaranalyzedreflectivityat1200UTCon8September2016(leftpanel)andtheobservedradarreflectivityat1155UTCon8September2016(rightpanel).

Figure3.3.2.1-2showsthedomain-averagedRMSEforthewindbackgroundandanalysesfrom3Dand4Druns.Theresultsindicatethe4DhybridEnVartechniqueimprovesthefitofbothbackgroundandanalysestoobservations.Similarresultswerealsofoundforhumidityandtemperatureatmostofverticallevels.Fortheforecasts,theimpactsofthe4Dtechniquebecamelargerforlongerforecastrangeandresultedinremarkabledifferencesfromthe3Dresults.Figure3.3.2.1-3showsthedifferencebetweenthesimulatedreflectivityforecastsfromthe4Dand3Drunsatdifferentleadtimes.Thepatterninthedifferencefieldspointstotheimpactsofthe4Dtechniqueontherainbandlocationsandmagnitudeatconvectivescales.Atforecasthour6,themagnitudeofthedifferenceislargersuchthatthereflectivitydifferencesareofthesameorderofmagnitudeastheobservedreflectivity.TheDTCiscurrentlyperformingmorein-depthdiagnosticsandanalysisofverificationresultsfromthistestingactivity.AdetailedreportisexpectedtoavailablebetheendofMayandwillbepostedontheDTCwebpage.

Inaddition,theDTCexaminedtheimpactsofreplacingtheGFSensemblewithahigh-resolutionARWensembleinthehybridruns.Acasestudyshowsthata3DhybridrunusingtheARWensemble(notshownhere)canresultinlargedifferencesin6-hforecastscomparedwith3DhybridrunsusingtheGFSensemble.Thisoutcomeindicatestheensemblerepresentationatconvectivescalescanleadtoimpactssimilartoreplacing3Dwith4Ddataassimilation.

25

Figure3.3.2.1-2.Thetimeseries(leftpanel)andverticalprofilesofdomainaveraged(rightpanel)RMSEsforthewindbackgroundandanalysesgeneratedfromthe3D(redlinesforbackgroundandgreenlineforanalyses)and4D(blacklinesforbackgroundandbluelinesforanalyses)hybridEnVarexperiments.

Figure3.3.2.1-3.Differencebetween1-hforecasts(leftpanel)and6-hforecasts(rightpanel)ofmodelsimulatedreflectivityproducedbythe4Dand3DhybridEnVarexperiments.

3.3.2.2Feasibilityandimpactassessmentforfastcyclingof4DhybridEnVar

Basedontheoutcomefromthefirstfocusarea,theDTCmodifiedtheworkflowtoincreasetheanalysisupdatefrequencyandcycledthehybriddataassimilationtoevery15minutesthroughoutthepre-forecasthour.TheDTCalsoaddedthecapabilitytoGSItoperformsub-hourly(inminutes)analysisupdates.Thenewworkflowsforthe3Dand4Dsub-hourlycyclingdataassimilationsystemforHRRRareshowninFigure3.3.2.2-1.Duetothesubstantialincreaseincomputationalresourcesassociatewithgoingtosub-hourly,theDTConlyperformedtestsforasubsetoftestingperiod(8-10September2016)usedforthefirstfocusarea.Theresultsshowedthatboththe3Dand4Dsub-hourlycyclinganalysisproducedabetterfittoobservationsthanthatofthe3Dhourlycycling,followedbyneutraltoslightlynegativeimpactsatthefollowinghours(figuresnotshown).Thesub-hourly3Dand4DEnVarconfigurationsmayneedtobetunedintermsofobservationtimewindowsforassimilation,orobservationerrormayneedtobeadjustedforhigh-frequencycycling.Itisalsoimportanttokeepinmindthatthesesub-hourlyexperimentsusedaGFSensemble,whichmayleadtorepresentationissuesforbackgrounderrorsatconvectivescales.TheDTCisperformingfurtherdiagnosticstocomparethe3Dand4Dresults.AdetailedreportisexpectedtoavailablebytheendofMayandwillbepostedtotheDTCwebpage.

3Dbackground4Dbackground3Danalysis4Danalysis

26

Figure3.3.2.2-1.Sub-hourly3D(leftpanel)and4D(rightpanel)hybridEnVarworkflowdiagramsshowingtheevolutionofthe15-minuteassimilationcyclespriortothe24-hourforecastissuedonthehour.NotethatmultipleGFSensemblesand30-minHRRRforecastsateachcycleareneededfor4DhybridEnVarassimilation.

3.4 GlobalModelTestBed(GMTB)

3.4.1 PhysicsTestbed

TofacilitatethedevelopmentofanadvancedphysicssuiteforNWS’sNGGPS,theDTCisdevelopingauniform‘testharness’toenablein-depthinvestigationofvariousphysicalparameterizations.ThistestharnessiscurrentlybeingusedbytheGMTBforitsT&Eactivities,andhasbeenmadeavailabletocommunityscientistsworkingwiththeGMTB.Asanexample,developersoftheGrell-FreitascumulusparameterizationranpreliminarytestsusingtheGMTBtestharnesstopreparecodeforamorecomprehensivetest,whichwillbeconductedbytheGMTBstaff.Thetestharnessmimicsthelogicalprogressionfortestingnewlydevelopedparameterizationsthattypicallytakesplacewithinthescientificcommunity.Componentsaregraduallyaddedasonemovesthroughthehierarchyuntilthefullforecastmodelcomplexityisreached.ItisdesignedtocomplementboththeexistingtestingprotocolatEMCandindependenttestingtypicallyperformedbyparameterizationdevelopers.Figure3.4.1-1illustratesthehierarchicaltiersofthetestharness,representshowtheDTCenvisionsthedivisionofeffort(GMTB’slikelyroledenotedbyblue)andshowshowtheharnessfitswithinEMC’sexistingtestingframework.

3.4.1.1 SingleColumnModel

AspartoftheGMTBphysicstestharness,aSingleColumnModel(SCM)thatmakesuseoftheIPDhasbeendevelopedandlightlytested.TheSCMisdrivenbyspecifyinganinitialprofilerepresentingthethermodynamicstateandaccompanyinghorizontalwinds.Changestotheprofilecausedbylarge-scaleadvectionareappliedthrough“forcing”terms,whicheffectivelyreplacethedynamicsofathree-dimensionalmodel.Aphysicssuitecalculatessub-gridscaleprocessesandchangestheprofileinconcertwiththeappliedforcing.

DesignoftheSCMfocusedoncommunity-friendlinessbyminimizingexternaldependenciesandusingcommunity-sanctionedcodingpractices.UsingtheSCMonlyrequiresthecmakeutilityforbuildingandtheFortrannetCDFlibrary(I/O)tobeinstalledandaccessible.Python-basedscriptsforplottingandanalysis,easilysetupwithaneditableconfigurationfile,areincludedinthedistribution.Asofnow,theGFSphysicssourcecodeisbundledwiththeSCMsourcecodeasaseparaterepository.Fortestingpurposes,thiscodeisupdatedoccasionallytoworkwiththetop-of-trunkGFSphysicscode,althoughit

27

willlikelyonlysupportspecific,taggedversionsofthiscodeinthefuture.ThecodehasbeentestedonalatemodelMacintosh,aswellasonNOAA’sResearchandDevelopment(R&D)machine(Theia)andNCAR’sYellowstone.Inaddition,aGMTBSCMUser’sGuideandtechnicaldocumentationwasdevelopedwithDoxygenandisavailableontheDTC’swebsite:http://www.dtcenter.org/GMTB/gmtb_scm_doc/.

Figure3.4.1-1.DiagramillustratingthetestinghierarchyplantosupportphysicsdevelopmentforNGGPS.LRindicateslowresolution,MRmediumresolution,andHRhighresolution.Colorshadingindicateswherethedifferentgroupsareanticipatedtofocustheirefforts(red–physicsdevelopers,blue–GMTBtaskwithintheDTC,andgreen–EMC).PPstandsforphysicsparameterization.

Sofar,theSCMissetuptorunindividualcaseslikethosesuppliedbytheGlobalEnergyandWatercycleEXchanges(GEWEX)GlobalAtmosphericSystemStudies(GASS)program.Thesecasesoftenderiveinitialconditionsandadvectiveforcingfromfieldcampaigns,andareintendedtostudyspecificphysicalphenomenaandhowtheyarerepresentedbyphysicssuites.The“catalog”ofcasestousewiththeSCMisawork-in-progress,withoneshallowconvectivecasebasedonthetransitionfromstratocumulus-to-cumulusasobservedduringtheAtlanticStratocumulustocumulusTransitionEXperiment(ASTEX)fieldcampaignandonedeepconvectivecaseasobservedduringtheTropicalWarmPool–InternationalCloudExperiment(TWP-ICE)fieldcampaign.Bothcasesareinitializedandforcedbasedonobservationsmadeduringtheirrespectivefieldcampaigns.Althoughbothcasesusehorizontaladvectivetendencieswithprescribedverticalmotion,itispossibletoconfiguretheSCMtousetotaladvectivetendenciesandrelaxationforcingasdescribedinRandallandCripe(1999).Goingforward,theGMTBwilladdmorecasestothecatalog,includingcasesthatwillrequirechangestotheunderlyingGFSphysicscode(e.g.,abilitytoturnoffspecifiedphysicsschemeswithinthesuite).ThecasesaddedbytheGMTBcanalso

28

serveasanexampleforcommunitymemberstoaddcasesofinterest.Inaddition,theSCMissetuptoeasilyrunusingforcingensemblesthatcanbeusedtounderstandaphysicssuite’sresponsetouncertaintyintheforcing.

3.4.1.2 WorkflowforLow/MediumResolutionGlobalForecastTests

Buildingonpreviousprogress,theGMTBsuccessfullyestablishedanend-to-endworkflowsystemforrunningNEMS/GlobalSpectralModel(GSM)andUPP(stronglyleveragingEMCcapabilities),aswellasrunningDTC-contributedcomponents.Theend-to-end-workflowsystemreachedamaturestatethatallowedforrunningatestoftheGrell-Freitas(GF)convectiveparameterization(seedescriptioninsection3.4.2).TheDTC-contributedworkflowcomponentsforcreatingPython-basedforecastplots(e.g.temperature,moisture,convectivevs.non-convectiveprecipitation)andverificationresults(e.g.,near-surface,upper-air,andprecipitationverification)continuedtobeupgradedtoincludeadditionalfeaturesandflexibility.AscripttoplottropicalcyclonetracksforeachmodelinitializationwascreatedandaddedtotheautomatedworkflowanumberofconfigurationfilesforMETViewer(auserinterfaceforplottingMEToutput)weremodifiedandupgradedtogenerateimprovedverificationplots.Thisworkincludedaddingthe‘scorecard’capabilitytotheverificationarsenal;the‘scorecard’isawaytosummarizepatternsintheperformancedifferencesbetweentwoconfigurations,includinglevelofsignificance,forspecifiedmetrics,variables,levels,regions,andtimes.The‘scorecard’wasdevelopedbytheNCARVerificationteamwithNGGPSfundingandwasmadeavailabletoGMTBforbeta-testingaheadofitsreleasetoEMCandothers.

Workisalsounderwaytoexpandthetestbedcapabilitiestoequipphysicsdeveloperswithawiderangeoftoolstoassessstrengthsanddeficienciesofphysics.ThecapabilitytoproducebiasinformationfromGSIdiagnosticfiles,whichprovideO-B(observation–background)informationwillsoonbeavailable.Inaddition,theGMTBiscollaboratingwithNGGPSPIJasonOtkintoincludesyntheticsatelliteoutputfromUPPtohelpwithevaluatingthemodel’sabilitytoaccuratelysimulatecloudsandmoisture.TheGMTBhasbeeniteratingwithJ.OtkinandhisteamtouserawmodeloutputfromtheGFtesttorunthroughUPPinordertotestupdatestotheradiativetransfermodelemployedbyUPP.Tropicalcyclogenesisverificationisintheprocessofbeingimplemented.Inaddition,thecapabilitytoperform6-hglobalprecipitationverificationisunderdevelopment. TheGMTBrevitalizeditscollaborativedialoguewithEMC’sglobalteamwithrespecttomigratingfromtheircurrentscriptingarchitecturetoaunifiedglobalworkflow,whichincludesattendingbi-weeklymeetingshostedbyEMC.TheGMTBisactivelytestingandrunningthenewRocoto-basedworkflow(v3.0.0)inpreparationforthenexttestingeffort,whichwillincludecycledDA.

Inaddition,theteamhascontinuedtomanagethescripts,andconfigurationfilesusedintheGMTBworkflowthroughaGitrepositoryonVLab.AsubstantialefforthasalsobeenputforthtodocumenttheGSM/UPPworkflow,aswellasthediagnosticandverificationworkflow.AstheGMTBtransitionstoEMC’sunifiedglobalworkflow,thedocumentationwillcontinuetobeupdatedaccordingly.SimilartootherGMTBdocumentation,theworkflowdocumentationusesDoxygen.

3.4.2 Grell-Freitasconvectiveparameterizationtest

TheGMTBconductedatestoftheGrell-Freitas(GrellandFreitas2014)convectiveparameterizationtoprovideinputontheestablishmentanddevelopmentofanadvancedphysicssuiteforNOAA’sGFS.Thisparameterizationwasselectedfortesting,throughconsultationwithEMCandtheNGGPSProgramOfficeandPhysicsTeam,becauseofitspotentialforimprovingforecasts.Itisastate-of-the-artschemethatincludesascale-awarefeature,whichmakestheschemesuitableforuseacrossawiderangeof

29

modelresolutions.Itincorporatesanensembleapproachinitsrepresentationofconvection,whichallowsperturbationbystochasticfieldsfordeterministicforecasting,aswellasensembledataassimilation.Flux-formverticaltracertransport,wetscavenging,andaerosolawarenessarealsooptionsinthisscheme.Thescheme’smaturity,itshistoryofoperationaluseatNCEPintheRAP,andthefactthatitsdevelopmentisfundedbyNGGPSalsoledtochoosingtheGFschemefortesting.

ThistestwasconductedusingGMTB’shierarchicaltestbed,whichcurrentlyconsistsofaSCMandaworkflowforrunningtheGFS.Inbothcases,acontrolusingtheGFSoperationalSimplifiedArakawaSchubert(SAS)convectiveschemewascreated(GFS-SAS)andcomparedagainsttheexperimentalconfiguration(GFS-GF)whosephysicssuitewasthesameasGFS-SASexcepttheGFdeepandshallowconvectionschemeswereusedinplaceofSAS(Table3.4.2-1).Theentiretestresults,aswellasthefinalreport,arepostedontheDTCwebsite.Selectresultsarepresentedbelowtohighlightthekeyfindings.

Table3.4.2-1.TablesummarizingthebasicelementsofGrell-Freitasconvectiveparameterizationtest.

FortheSCM,asinglecasebasedonadeepconvection-focusedfieldcampaignwasusedtoprovideinsightintohowGFS-SASperformscomparedtoGFS-GF.ThetestingparadigmfollowstheonedescribedinRandalletal.(2003)andZhangetal.(2016),namelyinitialconditionsandcolumnforcingarederivedfromobservationsobtainedduringIntenseObservationPeriods.TheatmosphericphysicssuitethatmakesuptheSCMisallowedtorespondtotheforcingbygeneratingparameterizedcloudsandprecipitation,radiativeheating,verticalmixing,etc.Givenidenticalforcing,theGFS-GFsuiteproducedsmallerconvectivetendenciesandamuchlowerconvectiveprecipitationratiothantheGFS-SASsuite(Fig.3.4.2-1).GFS-GFreducedthedrybiasintheboundarylayerandgenerallyproducedahighercloudfractionduringthedeepconvectiveperiodcomparedtoGFS-SAS.

Theglobalforecastswererunatarelativelylowresolution(~34km),infree-forecastmode(nodataassimilationorcycling)andwithouttuningofthephysicssuite.TheoperationalGFSanalyseswereusedtoinitializetheretrospectivecoldstartforecastscoveringthetimeperiodJune–August2016(JJA).TheglobalmodelforecastsdisplayedbehaviorsimilartotheSCM,inthesensethatGFS-GFhadlowerconvectiveprecipitation(Fig.3.4.2-2).DifferencesbetweenprecipitationcharacteristicsofGFS-SASandGFS-GFwerenoticeableovertheCONUSdomain.Whilethe6-hprecipitationfrequencybiasesshowedaprominentdiurnalsignalforbothconfigurationsatthe0.01”threshold(Fig.3.4.2-3),aswellas0.1”and0.25”thresholds(notshown),GFS-GFproducedalargerdiurnalsignalinthefrequencybias,withsimilarmagnitudestoGFS-SASat18-UTC,butlowerbiasat00UTC.

30

Figure3.4.2-1:Meanprofilesoftemperaturetendencies(Kday-1)fortheactivephaseoftheTWP-ICEcase.Colorsdenoteforcing(red),PBLscheme(green),convectiveschemes(deep+shallow,blue),andmicrophysicsscheme(purple).Linetypesdenotethephysicssuite:GFS-SAS(solid)andGFS-GF(dashed).Tendenciesduetolongwaveandshortwaveradiationareinorangeandbrown,respectively.

Figure3.4.2-2:Average6-haccumulatedconvectiveprecipitation(mm)overthethree-monthtestperiod(JJA2016)atthe120-hforecastleadtimeforGFS-GF-GFS-SAS.

Theglobalforecastswererunatarelativelylowresolution(~34km),infree-forecastmode(nodataassimilationorcycling)andwithouttuningofthephysicssuite.TheoperationalGFSanalyseswereusedtoinitializetheretrospectivecoldstartforecastscoveringthetimeperiodJune–August2016(JJA).TheglobalmodelforecastsdisplayedbehaviorsimilartotheSCM,inthesensethatGFS-GFhadlowerconvectiveprecipitation(Fig.3.4.2-2).DifferencesbetweenprecipitationcharacteristicsofGFS-SASandGFS-GFwerenoticeableovertheCONUSdomain.Whilethe6-hprecipitationfrequencybiasesshowedaprominentdiurnalsignalforbothconfigurationsatthe0.01”threshold(Fig.3.4.2-3),aswellas0.1”and

31

0.25”thresholds(notshown),GFS-GFproducedalargerdiurnalsignalinthefrequencybias,withsimilarmagnitudestoGFS-SASat18-UTC,butlowerbiasat00UTC.

Figure3.4.2-3:Frequencybiasof6-haccumulatedprecipitation(in)forGFS-SAS(red)andGFS-GF(green)aggregatedovertheCONUSdomainforthe0.01”thresholdasafunctionofforecastleadtime(h)forJJA2016.Theverticalbarssurroundingtheaggregatevaluerepresentthe95%CIs.

Withthecopiousamountofverificationresultsproducedforthistest,a“scorecard”wasastraightforwardwaytoidentifypatternsintheperformancedifferencesbetweenthetwoconfigurations,includinglevelofsignificance,forspecifiedmetrics,variables,levels,regions,andtimes.Thescorecardforglobalsub-regionshelpedidentifythatupper-airwindspeedhadthefeweststatisticallysignificant(SS)differencescomparedtoothervariables,asshowninFig.3.4.2-4forNorthernHemisphere(NH).TheNHclearlysignaledGFS-SASasperformingbetterfortheearlierpartoftheforecastperiod.However,GFS-GFwasthebetterperformerfortemperaturebiaslaterintheforecastperiod.Follow-updiagnosticsindicatedtheimprovedperformanceofGFS-GFforthismetricwasrelatedtotheGFS-SASwarmingupprogressivelyovertheNHthroughouttheforecastperiod.

Tropicalcyclonetrackerrorsaveragedoverallthebasinsweresimilarforbothmodelconfigurations(Fig.3.4.2-5a).WhileaccuracyinTCintensityforecasts(Fig.3.4.2-5b)isnotexpectedofamodelrunatthiscoarseresolution,itisinterestingtonotethatstormsproducedbyGFS-SASaremoreintense(notshown)andhavelessabsoluteintensityerrorthanthoseproducedbyGFS-GF.

TheGMTB’stestingandevaluationoftheGFcumulusparameterizationillustratedthecomplexity--yetscientificusefulness--ofconnectinganewschemetotheGSM.ThesuccessofthistestwasheavilydependentoninteractionsamongandinvestmentbytheGMTB,thephysicsdeveloper,andEMC’sGlobalTeam.TheclosecollaborationanditerationwiththedeveloperhelpedensuretheGMTBproperlyconnectedtheGFparameterizationwithintheGSMcode.Inaddition,thecollaborationwiththeEMCGlobalTeamwasessentialtotheGFtest.TestingwiththeSCMresultedinseveralkeyfindings,withonefindingaligningwithresultsfromthe3-Dglobalforecasts.WhilenotallresultsfromtheSCMcouldbetranslatedtothefullglobalforecasts,thistesthighlightedtheutilityandprocessofthehierarchicaltesting.Movingforward,itwillbenecessarytofurtherengagewithEMCandcontinuetogetfeedbackregardingdesiredverificationmethodsanddisplays,whichwouldthenbeprioritizedforfutureimplementationinthetestbed.

32

Figure3.4.2-4:ScorecarddocumentingtherelativeperformanceofGFS-SASandGFS-GFovertheNHformeanbiasandRMSEfortemperature,relativehumidity,andwindspeedbyforecastleadtimeandverticallevelforJJA2016.Green(red)shadingindicatesGFS-GF(GFS-SAS)wasbetterthanGFS-SAS(GFS-GF)atthe95%significancelevel.Smallgreen(red)arrowsindicateGFS-GF(GFS-SAS)wasbetterthanGFS-SAS(GFS-GF)atthe99%significancelevel.Largegreen(red)arrowsindicateGFS-GF(GFS-SAS)wasbetterthanGFS-SAS(GFS-GF)atthe99.9%significancelevel.GreyshadingindicatesnostatisticallysignificantdifferencesbetweenGFS-SASandGFS-GF.

Figure3.4-6:(a)Meantrackerrors(nm),and(b)meanabsoluteintensityerrors(kt)with95%confidenceintervalswithrespecttoleadtime(h)forGFS-SAS(red),GFS-GF(green)andtheirpairwisedifferences(black)inallbasinsforJJA2016.

33

3.4.3 CICEtest

Overthepastyear,theGMTBcompletedatestofTheLosAlamosseaicemodel(CICE),whichstemmedfromtheNGGPSSeaIceModelingWorkshoporganizedbyGMTBinFebruary2016.Asdescribedintheworkshop’sfinalreport(link),participantsrecommendedthetentativeadoptionofCICE,pendingfollow-uptesting,andaddressingconcernsraisedregardingmodelgovernanceanddifferencesinstaggeringbetweenthegridsusedinNCEP’sUnifiedGlobalCoupledSystem(UGCS)oceanmodelsandCICE.

Thetestplan,devisedjointlybyGMTB,EMC,theworkshopcommitteeandinterestedworkshopparticipants,includedexperimentsoveraone-yearperiod,withCICEbeingruninastandaloneframework,forcedbyatmosphericandoceanicfieldsfromtheNCEPoperationalClimateForecastSystemversion2(CFSv2).GMTBconductedthe30-dayforecastruns,andevaluationwasperformedjointlybyGMTB,EMC,andESRL.

Theexperimenthadthreephases,asdescribedinTable3.4.3-1,withvaryingmodelresolutionandapproachestoinitializingtheatmosphereandoceanfields,aswellasconstrainingtheSeaSurfaceTemperature(SST).

Table3.4.3-1.CICEmodelresolutionatthepole(km),datasetforatmosphericinitializationandforcing,datasetforoceaninitialization,andmethod/datasetforoceanforcing.

CICE Atmos Init and Forcing Ocean Init Ocean Forcing

Phas

e 1 30 km CFSv2 1.00 CFSv2 1.00 CFSv2 10 6-hourly forcing

2 15 km CFSv2 0.20 CFSv2 0.50 CFSv2 0.50 6-hourly forcing

3 15 km CFSv2 0.20 CFSv2 0.50 Freely evolving

Generallyspeaking,thePhase-1andPhase-2forecastsattheendofthemonth-longintegrationsareingoodagreementwiththeCFSv2initialconditionsatthebeginningofthenextmonth,indicatingaverygoodforecastfortheendofthemonth.Themajorexceptionsareinthesummerseasonsofbothhemispheres,whereexcessivemeltingoccurs(Fig.3.4.3-1).Follow-updiagnosticsindicatedthatbasalmelting,causedbywarmSSTs,wasthecauseoftheexcessivemelting.

Figure3.4.3-1.Iceextent(1012m2)duringmonth-longintegrationsforPhase1(dashed)and2(solid)intheNorthern(top)andSouthernhemisphere(bottom).ThecirclesindicatetheinitialconditionsfromCFSv2atthebeginningofeachmonth.

34

WhentheSSTfromCFSv2forcingwasreplacedbyfreely-evolvingSSTfromtheCICEinternalmixed-layeroceanmodel,basalmeltingwasreducedandpredictionoficeextentattheendofsummerwassubstantiallyimproved(Fig.3.4.3-2).Comprehensiveresultsfromthistestcanbefoundinthereport(link)andonthetestwebsite(link).

Figure3.4.3-2.SameasFig.3.4.3-1,exceptforPhase2(solid)andPhase3(dashed).

3.5 CloudVerification

AttherequestoftheAF,theDTCinvestigatedapproachesforevaluatingcloudpredictionsfromNWPmodelsandstatisticalpredictionsofcloudproperties.ThisevaluationincludednumerousNWPmodelforecastsutilizedbytheAF.RawmodeloutputfromtheARWlimitedareamodel,overtheNorthernHemisphere,andtheAF