research to operations (r2o, r2x and o2r) sid-ahmed boukabara the jcsda summer colloquium, august 7...
TRANSCRIPT
Research To Operations(R2O, R2X and O2R)
Sid-Ahmed Boukabara
The JCSDA Summer Colloquium, August 7th, 2015, Fort Collins, CO
NOAA/NESDIS/STAR & JCSDANOAA/NESDIS/STAR & JCSDA
2
Contents
What Is R2O or R2X in General?1
What is an O2R and why is it important?2
What is needed for a successful R2O/O2R?3
Specific Example of the JIBB and S45
How do we measure Success in R2O?4
Conclusions and Lessons Learned6
What Is R2O or R2X in General?
R2O is the process by which a scientific project led by a researcher, proven to provide added value, is transitioned to the Operational environment
R2X refers to the transition of a research project outcome to operations, applications, and services (for example to decision makers). R2O is an example of R2X
3
Why is R2O Important for NOAA?(and other operational partners)
R2O is an efficient way to leverage research done in NOAA and in the outside research community, to improve NOAA operational systems
R2O allows NOAA to maximize the return on investment (ROI) of its outsourced research
R2O allows NOAA to remain competitive, and at the edge of science in its products.
4
R2O ?
R2O means often different things to different people
5
Researcher in Academia
NESDIS
NWS
STAR
OSPO
R2O
EMC
NCO
R2O
R2O
R2O
JCSDA R2O Concept
Goal: Accelerates the use of satellite data in NWP centers
Ensures resources are in place to achieve successful transition: supercomputer, a software integration team, etc.
Ensures consistency and demonstrates benefits with operational systems like: Hybrid GDAS system (global), HWRF (Hurricane forecast).
Scientific efforts in
satellite DA in academia,
CIs
Scientific efforts in
satellite DA in NOAA/OAR
Scientific efforts in
satellite DA in research community
Scientific efforts in satellite DA in NOAA/NESDIS
(funded by GOES-R PGs, JPSS PGs,
etc)
Products, techniques, improvements, with direct and immediate relevance
to NWP Operational Models
(both global and regional)
Made consistent with Operational Systems
JCSDA’s own DA
Activities
NWP Operational CentersOngoing Baseline
Improvement in NWP centers
Objective: Improvements in Forecast skills
Diverse R&D activities
Select projects for R2O
Transition
7
Contents
What Is R2O or R2X in General?1
What is an O2R and why is it important?2
What is needed for a successful R2O/O2R?3
Specific Example of the JIBB and S45
How do we measure Success in R2O?4
Conclusions and Lessons Learned6
What is an O2R and why is it important?
O2R infrastructure brings the Operational environment to researchers
It allows scientists to perform their research in a system that is compatible with operational needs
When a successful research project is performed in an O2R environment, it increases its chances to transition to Operations (R2O)
Without an O2R environment, R2O is at best difficult, (often does not happen). 8
O2R Is the Bridge Between R and O
9
Research Operations
Valley of Death
O2R
How does O2R/R2O work?
10
11
Contents
What Is R2O or R2X in General?1
What is an O2R and why is it important?2
What is needed for a successful R2O/O2R?3
Specific Example of the JIBB and S45
How do we measure Success in R2O?4
Conclusions and Lessons Learned6
Components of an O2R/R2OInfrastructure(1)
The infrastructure is not just the supercomputer:What Money can Buy….
Supercomputer (Hardware, basic software, and IT) (almost the easiest part!) Scientific Software Integration (this is the hardest part: keep in synch with
operational models) User Support & Documentation Management of resources to allow R2O and O2R to be sustained Documented Utilities to help scientists (CRTM, assessment, formatting tools) Constant engagement between Research and Operations Rigorous software configuration system to track changes Rigorous (and independent) testing mechanism to demonstrate added value
12
Components of an O2R/R2OInfrastructure (2)
What Money can NOT Buy…. And perhaps more importantly: the right culture.
• Transparency of R2O protocols and willingness to accept research performed by researchers & work collaboratively (on the operational partner)
• Willingness to work/start with operational system and follow protocols and accept constructive feedback (on the research partner)
• Sense of Mutual respect between Researchers and Operational partners: respect for innovation in research and pragmatism in the operational implementation
13
Accept that R2O is not for everyone: careful choice of research and operations partners is important
14
Contents
What Is R2O or R2X in General?1
What is an O2R and why is it important?2
What is needed for a successful R2O/O2R?3
Specific Example of the JIBB and S45
How do we measure Success in R2O?4
Conclusions and Lessons Learned6
Metrics for Assessing O2R Success
When R2O transition has occurred: assess specific scientific metric(s) that was(ere) improved upon transition to operational environment. Example: Anomaly correlation, hurricane track
improvement, etc.
When R2O transition has not occurred yet, assess progress in the maturity of the project, as a result of the O2R infrastructure
15
Example of a project that led to an R2O transition (Improved QC)
16
Significant
Southern hemisphere 500-hPa height ACC for 10 September – 24 October 2012 for the control (black) and LNVD (red).
Bottom: Variation between LNVD (red) and the control (operational QC of MODIS winds: black) based on forecast hour.
All ACC differences outside of the bars are significant at the 95% confidence level.
The circle represents a statistically significant improvement for the Day-4 and Day-5 forecasts.
Plot courtesy of D. Santek, UW (S4 project)
How do we assess R2O-maturity?(Case of JCSDA/NESDIS)
17
NOAA Technical Readiness Level (TRL)TRL 1 Basic principles observed and reported
TRL 2 Technology concept and/or application formulated
TRL 3 Analytical and experimental critical function and/or characteristic proof-of-concept
TRL 4 Component/subsystem validation in laboratory environment
TRL 5 System/subsystem/component validation in relevant environment
TRL 6 System/subsystem model or prototyping demonstration in a relevant end-to-end environment
TRL 7 System prototyping demonstration in an operational environment
TRL 8 Actual system completed and “mission qualified” through test and demonstration in an operational environment
TRL 9 Actual system “mission proven” through successful mission operations
TRLs 1-2 are nominally considered Research, TRLs 3-5 are Development, TRLs 6-8 are Demonstration, and TRL 9 is Deployment, Implementation, or Operational Transition.
FF
O
Dir
ecte
d R
esea
rch
Vis
itin
g S
cie
nti
st
Pro
gra
m
JCSDA Modes of Operation
R2O
Pat
h
18
Contents
What Is R2O or R2X in General?1
What is an O2R and why is it important?2
What is needed for a successful R2O/O2R?3
Specific Example of the JIBB and S45
How do we measure Success in R2O?4
Conclusions and Lessons Learned6
19
Supercomputer for Satellite Simulations and data assimilation Studies (S4)
• Overview:– NESDIS, in collaboration with UW
is making S4 available to scientists involved in satellite data assimilation and other activities related to it.
– One of the purposes is to consolidate data assimilation activities performed in the research community
– and to encourage the usage of NOAA operational models/tools/codes installed on it.
– Open policy for users (since hosted in academic institution).
– Only open source/non-restricted data allowed on S4.
– Status: Upgraded in 2014.
The S4 system, hosted by University of Wisconsin.
S4 technical Description
Brief Technical Description The S4 system is a Linux cluster (Dell/Intel hardware), 4700 CPU cores (60 Tflops) with 18TB of total RAM 1700TB in storage nodes Hosted in the UW/SSEC Data Center (with UPS) Able to run 4 GSI/GFS experiments simultaneously at T1534
System Administration and Resource Allocation SSEC provides system administration services and security
protocols for the system, NESDIS/STAR provides overall guidance on system
resource allocation and selection of users as well as scientific support (code management, software integration, etc)
20
S4 Intented ActivitiesThis system is intended to allow primarily (but not
exclusively) the following major activities: (1) Undertake satellite data assimilation experiments at global and/or regional
scales and the assessment of their impacts on forecast models skills, using currently flying satellite sensors and allowing scientists to test new science/methodology and
(2) In support of the activity above, undertake all necessary satellite data simulations, calibration, algorithms development/improvement, radiative transfer modeling and validation, quality control (QC) procedures, etc
(3) Perform Observing System Simulation Experiments (OSSEs) for new sensors (such as GOES-R and JPSS).
• This component is expected to address key questions that NESDIS constantly faces: (a) what is the impact and the added value of flying new sensors, (b) what is the optimal orbital configuration, (c) what are the best mitigation strategies possible, from an NWP perspective, given the budget and other constraints, (d) how to best coordinate with other national and international partners to achieve the most optimal Earth satellite observing system
21
S4 Governance & Management
22
• S4 is governed by a NESDIS/STAR policy memo identifying roles and responsibilities• IT support is handled by the host institution UW.
• Software Integration is handled by NESDIS (the same team handles the software integration on the JCSDA sister machine JIBB)
• To request an account, a simple email to STAR POC will suffice. Accounts usually set up the same day.
• The S4 committee includes NESDIS/STAR director, deputy, and three senior program scientists (JCSDA, JPSS and GOES-R), makes recommendations concerning 90% of the machine's resources. The remaining 10% is managed by UW.
Policies and Code management on S4
Code management is the hardest part• The software integration team is dedicated to making sure the NOAA codes installed on S4
are in synch with the operational models (including scripts, codes, datasets, libraries, etc)• SVN is extensively used when appropriate.
The following is a list of S4 Policies:– User account request– Managing resources– Account expiration– Tools and softwares guidance– User guide
Compiler & Software Availabilityo Intel Fortran / C++o Gfortran /gcc/g++o Netcdf 3 & 4o Matlabo IDLo NCO 4o GRADSo HDF 4 & 5o NCL
23
How much did Projects benefit from S4?
24
Project Title Project Leads Operational Partner Point of Contact at Oper. Partner
Expected Date (best guess) for
R2O
R2O Current Stage(TRL)
TRL before using S4
1. A New Quality Control Method for Satellite-derived Polar Winds in the NCEP GDAS/GFS
David Santek, and Brett Hoover
NCEP Andrew Collard, John Derber
Dec. 2015 8 6
2. Proactive Quality Control (PQC) based on Ensemble Forecast Sensitivity to Observations (EFSO)
Daisuke Hotta, Tse-Chun Chen and Eugenia Kalnay
NCEP Daryl Kleist, Mark Iredell
January 2016 7 3
3. Development of Community Satellite Data Thinning and Representation Optimization Tool (CSTROT)
Tong Zhu and Sid Boukabara
NCEP Andrew Collard June 2016 6 2
4. A near real time regional satellite data assimilation system for high impact weather research and forecasts
Jun Li, Jinlong Li, and Pei Wang
NHC and EMC NHC: Mark DeMaria, John L. BevenEMC: Vijay Tallapragada
October 2014 to NHCJune 2016 to EMC
3 1
5. Tropical cyclone data assimilation of MSG SEVIRI all-sky infrared satellite radiances
Milija Zupanski and Man Zhang
NCEP/EMC was Steve Lord begin in 2016 6 2
6. Studying tropical cyclone genesis and rapid intensification with the NCEP GSI data assimilation system
Zhaoxia Pu NCEP/EMC Vijay Tallapragada Dec. 2015 6 3
7. Assessment of Temperature and Humidity Prediction with Microwave and Hyperinfrared Satellite Data Assimilation over United States
Jianjun Xu, and Alfred Powell,Jr
STAR Mike Kalb &Sid Boukabara
June 2016 3 1
Maturity Assessment of S4 projects
25
Added value of S4 O2R Environment measured by the maturity-index (before and after using the S4)
Examples of S-NPP Projects Benefiting from O2R(Not directly funded by JCSDA, but supported through the O2R: HPC, user support, Porting of codes, etc)
# PI/Scientist Project Title Main Sensor used
1 Jinlong Li Near real-time assimilation system development for improving tropical cyclone forecast with NPP/JPSS sounder data
-CrIS, ATMS
2 Banglin Zhang -Improve Hurricane Structure Monitoring and Intensity Forecast Using NPP ATMS and GCOM-W AMSR2
-ATMS
3 Jianjun Xu Application of hyper-spectral satellite data in severe weather forecasts with GSI/WRF regional data assimilation system
-CrIS
4 Lin Lin -Evaluation of BG-Remapped ATMS on Hurricane Sandy Forecast & Test of Mie and DDA Scattering in the Simulation of ATMS
-ATMS
5 Marouane Temimi Assessment of Assimilating NPP/JPSS ATMS Ground Surface Observations -ATMS
6 Li Fang, et al. Dual Assimilation of Microwave and thermal-infrared observations of soil moisture into NLDAS for improved drought monitoring
- ATMS, VIIRS
7 Marouane Temimi Assessment of Assimilating NPP/JPSS ATMS Ground Surface Observations -ATMS
8 Hu Yang Scientific Support for JPSS Instrument Calibration -S-NPP Sensors (ATMS)
9 Daisuke Hotta Proactive-QC based on Ensemble Forecast Sensitivity to Observations (EFSO) -All S-NPP assimilated sensors (ATMS)
10 Xinjia Zhou BTs validation of CRTM simulations against AVHRR, MODIS and VIIRS BTs in SST bands
-VIIRS
11 Johnathan W. Smith
WRF-Chem data assimilation of NOAA-Unique CO retrievals: Impact study on tropospheric ozone enhancement observed during the 2010 AEROSE Campaign
-IASI (potential application to CrIS)
12 Agnes Lim Huei Ni Impact analysis of LEO Hyperspectral Sensor IFOV size on the next generation NWP model forecast performance
-CrIS
Resources installed for Global and Regional DA and OSSE Activities
The following models & tools are installed on S4: Global Data Assimilation System (GDAS) (including the new Hybrid
Ensemble system) Grid Statistical Interpolation (GSI) Weather Research & Forecast (WRF) (WRF-ARW and WRF-NMM) Hurricane Weather Research & Forecast (HWRF) Microwave Integrated Retrieval System (MIRS) Community Radiative Transfer Model (CRTM) Verification Statistics Data Base (VSDB) Ocean Model HYCOM Etc…
The following models & tools will be installed in the future: NOAA Environmental Modeling System (NEMS) Unified Post Processor (UPP) Land Information System (LIS) Etc..
Major projects on jibb
General data impact and OSSE experiments for Several Sensors
Next generation NOAA hybrid data assimilation system port
Hyperspectral IR methodology Hyperspectral IR cloudy radiances Satellite wind impact experiments Lightning assimilation methodology FFO projects of the JCSDA NASA Roses Projects of the JCSDA
28
JIBB and S4 Benchamarking
Efforts have been devoted to compare satellite data experiments results on S4 and JIBB, to those performed on the NOAA R&D
Benchmarking performed for currently operational GDAS (GSI+GFS)
The results show that the operational version of GDAS is running well on S4 and JIBB (both T382 and T574)
Efforts included the benchmark of the hybrid data assimilation system.
29
Anomaly correlation performed using GDAS system on NOAA R&D machine (Vapor), JCSDA JIBB machine and the S4 system. Benchmarking performed by J. Jung and the STAR/JCSDA SI team. Courtesy of J. Jung.
30
Contents
What Is R2O or R2X in General?1
What is an O2R and why is it important?2
What is needed for a successful R2O/O2R?3
Specific Example of the JIBB and S45
How do we measure Success in R2O?4
Conclusions and Lessons Learned6
Lessons learned for an effective R2O
Importance of the O2R environmentImportance of availability of an accessible HPCImportance of a transparent R2O protocolImportance of code portability & synchonizationImportance of enforcing the exclusive usage (by researchers)
of the operational systems Importance of a rigorous traceability of changesImportance of the S4 and JIBB approaches
Importance of nurturing a culture of mutual respect between research and operations partners
31
Coordination & Synchronization: JCSDA Accelerated R2O coordination for SNPP/ATMS
32
• SNPP Was launched Nov 2011
• Dec 2011, the JCSDA coordinated the signature of the R2O Action Plan for ATMS R2O between NESDIS and NWS
• The main points were: Who, does what and when and where (between NESDIS and NWS)
• Strong backing from STAR and NCEP Directors (present at kick off meeting)
• 7 months after launch, ATMS was assimilated operationally in NOAA
• Important of ‘coordinate & synch’
Conclusions JCSDA and NESDIS are involved in R2O by making significant
resources available to scientists and collaborators (HPC, support, ..)
The two major activities targeted are: (1) Advanced Satellite data assimilation and (2) OSSE activities.
The O2R infrastructure is more than the supercomputer: components include software management, integration and user support, etc
The JIBB and S4 offer an O2R that is both operationally relevant as well as open and inclusive
S4 and JIBB access offers an O2R environment conducive of innovation, but with a clear R2O objective.
33
34
Questions?