www.esmf.ucar.edu enhancements for hydrological modeling in esmf cecelia deluca/ncar (esmf) december...
DESCRIPTION
Earth System Modeling Framework (ESMF) Component-based framework that targets tightly coupled, computationally demanding weather and climate applications written in Fortran ESMF provides standard interfaces for model components ESMF provides common utilities and tools for routine modeling functions such as regridding between components Status Initiated by NASA in 2002 and developed and managed by a multi-agency consortium More than 76 ESMF components in the community, used by the NASA Modeling, Analysis and Prediction Program, the Department of Defense Battlespace Environments Institute, the National Weather Service, and many othersTRANSCRIPT
www.esmf.ucar.edu
Enhancements for Hydrological Modeling in ESMF
Cecelia DeLuca/NCAR (ESMF)
December 19, 2008AGU Fall Meeting
www.esmf.ucar.edu
Outline• ESMF overview• Results and projects relevant to the hydrological community• ESMF + data services and semantic technologies• A peek at the future
www.esmf.ucar.edu
Earth System Modeling Framework (ESMF)• Component-based framework that targets
tightly coupled, computationally demanding weather and climate applications written in Fortran
• ESMF provides standard interfaces for model components
• ESMF provides common utilities and tools for routine modeling functions such as regridding between components
Status• Initiated by NASA in 2002 and developed and managed by a multi-agency consortium• More than 76 ESMF components in the community, used by the NASA Modeling,
Analysis and Prediction Program, the Department of Defense Battlespace Environments Institute, the National Weather Service, and many others
www.esmf.ucar.edu
GEOS-5
surface fvcore gravity_wave_drag
history agcm
dynamics physics
chemistry moist_processes radiation turbulence
infrared solar lake land_ice data_ocean land
vegetation catchment
coupler
coupler coupler
coupler
coupler
coupler
coupler
• Each box is an ESMF gridded component or coupler component• ESMF State objects carry data between components• Every component (including couplers) has a standard interface to facilitate exchanges• The ESMF architecture enables the assembly of many different systems
ESMF Application Example
Part of theGEOS-5 AtmosphericGeneral CirculationModeling system
www.esmf.ucar.edu
Interoperability and standardization of interfaces
• There are only three ESMF component methods: Initialize, Run, and Finalize (I/R/F)
• Users associate their native code I/R/F methods with an ESMF Component through a registration process
• I/R/F methods cascade down the tree
• Small set of standard arguments:
Child GridComp “Atmosphere”
Parent GridComp “Hurricane Model”
Finalize
Child GridComp “Ocean”
Finalize
Child CplComp “Atm-Ocean Coupler”
Finalize
Call Initialize Call FinalizeCall Run
Initialize Run Finalize
Initialize
Initialize
Initialize
Run
Run
Run
AppDriver (“Main”)
Call Initialize Call FinalizeCall Run
call ESMF_CompRun (myComp, importState, exportState,clock, phase, blockingFlag, rc)
www.esmf.ucar.edu
Outline• ESMF overview• Results and projects relevant to the hydrological community• ESMF + data services and semantic technologies• A peek at the future
www.esmf.ucar.edu
Data representation options1. Representation in index space (Arrays)• One or more logically rectangular tiles
store indices and topology• Sparse matrix multiply for regridding with
user supplied interpolation weights• Very general data representation, but
limited interoperability since not muchsemantic info is encoded by the framework
2. Representation in physical space (Fields)• Built on Arrays + some type of Grid• Regridding using parallel on-line interpolation weight generation, bilinear or
higher order• Standard high-level data structures increase interoperability
Supported Array distributions
www.esmf.ucar.edu
Coupling options in ESMF
• Lots of flexibility incoupling approaches
• Single executable• Multiple executable• Coupling
communicationscan be set up and calledin a coupler
• Coupling communications can be setup in a coupler but calleddirectly from within components – useful when it is inconvenient to return from a component in order to perform a coupling operation
• Recursive components for nesting higher resolution regions• Ensemble management, including sequential execution of ensemble members
Comp A Comp B
Comp A
Comp B
Contributed by U Maryland
Coupler
Array send/recv
Multiple executable options
www.esmf.ucar.edu
New Features and Applications
The Flow-Following Finite Volume Icosahedral Model (FIM) from NOAA GSD is converting to ESMF to couple to National Weather Service models
• ESMF v3.1.1 beta release (September 2008) includes◦ unstructured mesh class◦ observational data streams◦ on-line parallel regridding◦ standard metadata packets
• Consortium of U.S. operational and research centers (NOAA, NASA, DoD) has entered pilot phase of development of multi-model global ensemble using ESMF components
ESMF parallel regrid:• Grid-Grid, Grid-Mesh,
Mesh-Mesh• Bilinear and higher order
National Unified Operational Prediction Capability
www.esmf.ucar.edu
Targeting greater interoperability• ESMF is a toolkit with many options for coupling, and ESMF compliance alone
has not meant plug-and-play• Several groups have been creating additional guidelines, conventions, and
constraints for increased interoperability• Example: a NUOPC operational consortium committee has reached detailed
agreements on the following:◦ Componentization level - atm, ocean, land, ice, atm physics and dynamics◦ Clock - how time is passed between models and modules◦ Ownership – what models and modules are allowed to modify data held in
common (e.g. aerosol species and concentrations)◦ Level of data representation – index space or physical space◦ Metadata – the description of the data, grid, clock, etc.◦ Coupling approach - NUOPC will provide a standard template.◦ Discussions are underway on IO, post-processing, and component invocation
points
www.esmf.ucar.edu
Parallel weight calculation and regrid:Results for ESMF in CCSM
Interp.noise
Interpolation noise in the derivative of the zonal wind stress
grid index in latitudinal direction
• dTAUx/dy - closely related to the curl of the wind stress, which drives the upper ocean circulation and is therefore important to represent accurately in climate simulations
• Mapped from the T62 (2-degree) Community Atmospheric Model (CAM) grid to the gx1v5 (384x320 irregularly spaced)
• Noise is calculated as deviation of a point from the sum of itself plus four neighbors
• 33% reduction in noise globally compared to previous CCSM bilinear approach
• CCSM is now using ESMF for their standard remappingBlack = bilinear
Red = higher-orderESMF v3.1.1Green = higher order ESMF v4.0.0
www.esmf.ucar.edu
Performance Portablity30+ platform/compiler combinations regression tested nightly, new ports include gfortran (4.3), Solaris, and (almost) Windows
Performance at the petascale…The chart at right shows scaling of the ESMF sparse matrix multiply, used in regridding transformations, out to 16K processors.(ESMF v3.1.0rp2)
Plot from Peggy Li, NASA/JPLTested on ORNL XT4 in a variety of configurations .-N1 means 1 core per node.
ESMF Sparse Matrix Multiply Run-Time
www.esmf.ucar.edu
Outline• ESMF overview• Results and projects relevant to the hydrological community• ESMF + data services and semantic technologies• A peek at the future
www.esmf.ucar.edu
Metadata export from ESMF (v4.0)• ESMF holds information about grids, component inter-relationships• Users can associate additional metadata with ESMF classes (Grids, Arrays, Fields, States,
Components) as name/value pairs in Attribute packages• Attribute packages can be standard (e.g CF conventions for Fields) or custom• Packages can be nested – for example, metadata for an import State contains metadata for
all the Fields in the State.• Metadata can be exported in plain text or XML format …• then ingested into portal, where users can search and browse components and datasets via
the portal, “trackback” to the model configuration that created a dataset, create dynamic comparison tables for specific properties, create private or shared workspaces, etc.
• Excellent support for component and model inter-comparison projects.
Image courtesy of Rocky Dunlap, Georgia Institute of technology
The path from model metadata to web interface involves many format translations along the way
www.esmf.ucar.edu
Numerical Techniques for Global Atmospheric Models Summer Colloquium
• Held at NCAR during summer 2008, about 80 participants
• 9 atmospheric dynamical core components archived
• Running 22 test cases at up to 5 resolutions
• Search, browse, trackback, comparison tables
• Automated upload of simulation data, publishing, and comparison
• Data search and browse via ESG Gateway
• Demonstrates how knowledge base can combines with ESMF
http://dycore.ucar.edu
Curator projectsponsored by
www.esmf.ucar.edu
Work in progress• Translation of ESMF interfaces into web service interfaces to support loose
coupling, multi-language coupling, and integration of components into service oriented architectures (Curator project)
• Integration of CCSM ESMF version with Kepler workflow tool (PhD student at Istanbul Technical University)
• Automated coupler generation (PhD student at Georgia Tech)• New TeraGrid award: invoke CCSM ESMF version from an ESG-based science
gateway on the TeraGrid - data and metadata from the run are automatically stored back to the portal to support intercomparison projects
• Working with Cray, Microsoft and NASA to finish port to Windows HPC 2008
Image courtesy of Rocky Dunlap, Georgia Institute of technology
www.esmf.ucar.edu
A peek at the future …2009 2010 2011 2012 2013 2014 2015
InteroperabilityCompliance definitionsand verification tools
Tutorial development andclassroom deployment
Integration with workflowsGeneration of service interfaces for loose couplingMetadata links to data and other services
Automated model assembly andrun environmentBased on metadata files
Numerical/computational maintenance and development