joint center for satellite data assimilation an overview jim yoe, jcsda chief administrative officer...

Joint Center for Satellite Data AssimilationAn Overview

Jim Yoe, JCSDA Chief Administrative Officer

JCSDA Summer Colloquium Santa Fe, NM 1

Outline What (or who) is the JCSDA? Why is there a JCSDA? What does the JCSDA do? How Does the JCSDA function? What Successes has the JCSDA had? Challenges and Opportunities Summary Colloquium Forward


What is the JCSDA?


4

NASA/Earth Science Division

US Navy/Oceanographer andNavigator of the Navy and NRL

NOAA/NESDIS NOAA/NWS

NOAA/OAR

US Air Force/Director of Weather

JCSDA Partner Agencies

JCSDA Summer Colloquium Santa Fe, NM

Why is there a JCSDA?


6

Mission:

…to accelerate and improve the quantitative use of research and operational satellite data in weather, ocean, climate and environmental analysis and prediction models.

Vision:

An interagency partnership working to become a world leader in applying satellite data and research to operational goals in environmental analysis and prediction

JCSDA Vision and Mission


What does the JCSDA do?



From idea to implementation (R2O)

• Basic research

• Applied research

• Research to Operations transition (JCSDA)

• Operational implementation (at NCEP, AFWA, FNMOC)

8


JCSDA Science Priorities

Radiative Transfer Modeling (CRTM) Preparation for assimilation of data from new instruments Observations affected by clouds and precipitation Assimilation of land surface observations Assimilation of ocean surface observations Atmospheric composition; chemistry and aerosol

Driving the activities of the Joint Center since 2001, approved by the Science Steering Committee

Overarching goal: Help the operational services improve the quality of their prediction products via improved and accelerated use of satellite data and related research

How does the JCSDA Function?


JCSDA Modes of Operation

Directed research Carried out by the individual partners Mixture of new and leveraged funding from other programs JCSDA plays a coordinating role (ideally)

External research Grants or Contracts awarded following proposals submitted

to Federal Funding Opportunities administered by the partners

Open to the broader (external) research community Funding awarded competitively, peer review process

Visiting Scientist Program



JCSDA Working Groups Virtual teams consisting of working level scientists

from multiple JCSDA partners and stakeholders Coordination of effort and direct collaboration between

partners on specific topics Working Groups formed to date

CRTM – team working since early 2008 Hyperspectral IR sounding Microwave sensors Oceans Land Atmospheric composition

Hurricane Forecast Improvement Program (HFIP)

Joint JCSDA/HFIP Workshop on satellite data assimilation focusing on improved hurricane track and intensity prediction in 2011

Support for JCSDA scientist to work on cloudy radiance data near the vortex

Potential future involvement in JCSDA external research (FFO)


GOES-R Program

Support for 4D-VAR development SEVIRI radiance assimilation (ABI

proxy) External Research (FFO)

Project related to GLM data exploitation Support for JCSDA computing needs

(via NASA/GSFC and NESDIS/STAR)


Joint Polar Satellite System (JPSS) Program

Support for preparations for assimilation of CrIS and ATMS radiances CRTM upgrades Data reformatting Data ingest

Latency Study Support?


International collaboration

ECWMF ECMWF-JCSDA Workshop on Assimilating Satellite

Observations of Clouds and Precipitation into NWP models June 15-17, 2010

OSSE (Nature Run, validation, experimental design, …) Ensemble Prediction, data assimilation development, ODB

WMO/CGMS JCSDA organized/hosted Fourth WMO WS on Data Impacts

in Sedona, AZ in May ,2012 ESA

NESDIS ground support for ADM => improved data latency for North American users


JCSDA Outreach Activities


JCSDA Seminars

Typically one or two seminars per month in WWB

Mix of national and international speakers on wide variety of satellite and data assimilation related topics

Attended both in person and virtually (Webex and phone-in)


Annual JCSDA Science Workshop

Usually Conducted in the Spring Growing attendance

~90-100 participants in 2010 ~ 150 in 2011

Features Reports by JCSDA management JCSDA Working Groups Funded JSDI and FFO investigators In-kind contributors (i.e. Partner Agencies)



JCSDA Summer Colloquium

Two-week summer school Important because of lack of US academic programs

in DA Target is to conduct every other year

Program of lectures given by world-renowned experts

19 participants (almost all Ph.D. students or post-docs) from 3 countries, including the US

Sponsored by all JCSDA partners

20

JCSDA Computing

• JCSDA management working with NASA and GOES-R program to secure JCSDA operated supercomputing resource that can be made available to external investigators to test algorithms in the context of operational partner systems

• Management and location of the resource still being discussed; will likely be collocated with existing NASA or NOAA resources

21JCSDA Summer Colloquium Santa Fe, NM

JIBB Projects and Users• NPP transition to operations impact experiments • General data impact experiments and OSEs• Next generation NOAA hybrid data assimilation system port• Hyperspectral IR methodology • Hyperspectral IR cloudy radiances • Satellite wind impact experiments • Lightning assimilation methodology

– (Fuelberg)

22

Jibb Initial System Capabilities(after NOAA-funded expansion)

• Compute – IBM iDataPlex– 576 total cores; 6.3 TF Peak Computing– 48 Compute Nodes

• Dual-socket, hex-core 2.8 GHz Intel Westmere with 24 GB of RAM

• Quad Data Rate Infiniband Network (32 Gbps) in a 2-to-1 blocking fabric

• Storage– 8 IBM x3650 Storage Servers– 2 IBM DS3512 Storage Subsystems– 200 TB Total– IBM GPFS File System

• Ancillary Nodes– 2 Login Nodes– 2 Management Nodes

JCSDA Update, HPCRAC, January 11, 2012

23

Jibb Total System Capabilities

• Compute – IBM iDataPlex– 3,456 total cores; 37.8 TF Peak Computing– 288 Compute Nodes

• Dual-socket, hex-core 2.8 GHz Intel Westmere with 24 GB of RAM

• Quad Data Rate Infiniband Network (32 Gbps) in a 2-to-1 blocking fabric

• Storage– 8 IBM x3650 Storage Servers– 2 IBM DS3512 Storage Subsystems– 400 TB Total– IBM GPFS File System

• Ancillary Nodes– 2 Login Nodes– 2 Management Nodes


24

Management approach• Jibb Configuration Board

– Routine decisions on system admin issues– Membership: Riishojgaard, Boukabara, Yoe, Devaliere, Pratt

(JCSDA), Duffy, Romney, Sinno (NASA/GSFC)• Jibb Resource Allocation Board

– Policy-level decision on allocation of run-time and storage space– Membership: JCSDA Executive Team

• System Administration– NCCS

• User account sponsor group– Signs off on account request– Membership: L. P. Riishojgaard, S.-A. Boukabara, J. Yoe


25

What Success has the JCSDA Had?


26


Key Accomplishments Common assimilation infrastructure (EMC, GMAO, AFWA) Community radiative transfer model (CRTM - all partners) Common NOAA/NASA land data assimilation system (EMC, GMAO, AFWA) Numerous new satellite data assimilated operationally, e.g. MODIS (winds

and AOD), AIRS and IASI hyperspectral IR radiances, GPSRO sensors (COSMIC, GRAS, GRACE), SSMI/S, Windsat, Jason-2, NPP/ATMS

Advanced sensors tested for operational readiness, e.g. ASCAT, MLS, SEVIRI (radiances)

Ongoing methodology improvement for sensors already assimilated, e.g. AIRS/IASI (CrIS), GPSRO, SSMI/S

Improved physically based SST analysis Adjoint sensitivity diagnostics Emerging OSSE capability in support of ADM, COSMIC-2, JPSS, GOES-R,

Decadal Survey and other missions Establishment of IT platform to emulate operational environments (JIBB) Systematic Data Denial Experiments

27

Challenges and Opportunities

Budget Pressures Impact partners singly and collectively Impact national satellites program Increasing budget scrutiny

New satellites and sensors Data volumes challenging

Scope Moving beyond global NWP – climate, space wx, etc.

New Facility See next slides


NOAA Center for Weather and Climate Prediction

29

• Four-story, 268,762 square foot building in Riverdale, MD will house 800+ Federal employees, and contractors• 5 NCEP Centers (NCO, EMC, HPC, OPC, CPC)

• NESDIS Center for Satellite Applications and Research (STAR)

• NESDIS Satellite Analysis Branch (SAB)

• OAR Air Resources Laboratory

• Includes 40 spaces for visiting scientists

• Includes 500 seat auditorium/ conference center, library, deli, fitness center and health unit

Move Schedule• February 2, 2012 – data center setup began!• April 7, 2012 – Building operations

transferred to new owner– Furniture being installed and wired

• July 30, 2012 – begin phased move-in– Front offices – Non-operational groups

• August 2012 – dual operations• September 2012 – complete move• Post-election 2012 – ribbon-cutting ceremony

30

February 2, 2012

Summary• Progress in many JCSDA activity areas

– Impacts achieved for all operational partners• Computational resources for external partners• Major new developments in data assimilation

underway within all JCSDA partners• Extensive collaboration• Major challenges remain


Forward What Should You Take Away?

Understanding of Data Assimilation and Applications

Familiarity with the JCSDA Institutions and Individuals

Motivation To use exploit satellite data more effectively in

your research and applications


Back up slides


Satellite Data used in NWP

• HIRS sounder radiances• AMSU-A sounder radiances• AMSU-B sounder radiances• GOES sounder radiances• GOES, Meteosat, GMS

winds• GOES precipitation rate• SSM/I precipitation rates• TRMM precipitation rates• SSM/I ocean surface wind

speeds• ERS-2 ocean surface wind

vectors

• Quikscat ocean surface wind vectors

• AVHRR SST• AVHRR vegetation

fraction• AVHRR surface type• Multi-satellite snow cover• Multi-satellite sea ice• SBUV/2 ozone profile and

total ozone• Altimeter sea level

observations (ocean data assimilation)

• AIRS• MODIS Winds• COSMIC

~33 instrumentsJCSDA Summer Colloquium Santa Fe, NM

34

• EMC WRF Developmental Test Center, NASA/ NOAA/DoD Joint Center for Satellite Data Assimilation

• CPC Climate Test Bed

• NHC Joint Hurricane Test Bed

• HPC Hydrometeorological Test Bed

• SPC Hazardous Weather Test Bed with NSSL

• SWPC Space Weather Prediction Test Bed with AFWA

• AWC Aviation Weather Test Bed

• OPC linked with EMC’s Marine Modeling and Analysis Branch

Test BedsService – Science Linkage with the Outside Community

35

joint center for satellite data assimilation an overview jim yoe, jcsda chief administrative officer...

Documents

jcsda function

multiple jcsda partners

operational satellite

assimilation of data

environmental analysis

operational goals

operational services

quantitative use of