www.esmf.ucar.edu

Strategic PlanImplementation

Cecelia DeLuca/NCAR (ESMF)

December 17, 2008ESMF Board/Interagency Meeting

www.esmf.ucar.edu

Strategic objectives through 20101. Complete functionality and increase ease of use2. Integrate ESMF into science gateways3. Support integration into applications

www.esmf.ucar.edu

Release Plan2002 2003 2004 2005 2006 2007 2008 2009 2010

ESMF v1Prototype

ESMF v3Index Space OperationsESMF_ArraySparseMatMul()

ESMF v4Grid OperationsESMF_GridCreate()ESMF_FieldRegrid()

ESMFv5StandardizationBuild, init, data types, error handling, …

Public release May 2008ESMF v3.1.0r

ESMF v2Components, VM and UtilsESMF_GridCompRun()

Complete re-write ofdata and grid structuresfor generality and robustness

www.esmf.ucar.edu

Outstanding functionality

From ESMF Strategic Plan: 1. complete an ArrayBundle class that enables optimized data transfers of multiple Arrays2. complete the Field and FieldBundle communications methods3. integrate contributed Location Stream code for the representation of observational data4. complete arbitrary distributions for structured Grids, introduce multi-tile Grid methods,

and implement exchange Grids5. complete the interface to a general regridding package (still need conservative regrid)6. complete the interface to unstructured Meshes7. complete a C interface8. complete a hierarchical attribute package9. complete a rudimentary IO package10. complete a compliance tester for components

www.esmf.ucar.edu

Data representation options

ESMF data structures were reworked and released in ESMF v3, starting May 2006.Two options:

1. Representation in index space (Arrays)• One or more logically rectangular tiles

store indices and topology• Sparse matrix multiply for regridding with

user supplied interpolation weights• Highly scalable - no global information held

locally

2. Representation in physical space (Fields)• Built on Arrays + some form of Grid• Grids may be logically rectangular, unstructured mesh, or based on location of

observations• Regridding using parallel on-line interpolation weight generation, bilinear or higher order

Supported Array distributions

www.esmf.ucar.edu

Coupling options

• Single executable• Multiple executable• Coupling

communicationscan be set up and calledin a coupler

• Coupling communications can be setup in a coupler but calleddirectly from within components – useful when it is inconvenient to return from a component in order to perform a coupling operation

• Recursive components for nesting higher resolution regions• Ensemble management, including sequential execution of ensemble members

Comp A Comp B

Comp A

Comp B

Contributed by U Maryland

Coupler

Array send/recv

Multiple executable options

www.esmf.ucar.edu

Results for ESMF higher orderregrid in CCSM

Interp.noise

Interpolation noise in the derivative of the zonal wind stress

grid index in latitudinal direction

• dTAUx/dy - closely related to the curl of the wind stress, which drives the upper ocean circulation and is therefore important to represent accurately in climate simulations

• Mapped from the T62 (2-degree) Community Atmospheric Model (CAM) grid to the gx1v5 (384x320 irregularly spaced)

• Noise is calculated as deviation of a point from the sum of itself plus four neighbors

• 33% reduction in noise globallycompared to current bilinear approach

Black = bilinearRed = higher-orderESMF v3.1.1Green = higher order ESMF v4.0.0

www.esmf.ucar.edu

Usability Metrics: Bugs

40% reduction from peak (230 to 140)

www.esmf.ucar.edu

Usability Metrics: Feature Requests30% reduction from peak (110 to 76)

www.esmf.ucar.edu

Usability Metrics: Support RequestsHolding steady

www.esmf.ucar.edu

Port and test status

• 30+ platform/compiler combinations regression tested nightly, new ports include gfortran (4.3) and Solaris, Windows in progresshttp://www.esmf.ucar.edu/download/platforms

• Received award to employ the TeraGrid Build & Test service to access additional platforms for testing

• New test harness for automating the test coverage of a huge grid and distribution parameter space

www.esmf.ucar.edu

Strategic objectives through 20101. Complete functionality and increase ease of use2. Integrate ESMF into science gateways3. Support integration into applications

www.esmf.ucar.edu

Metadata export from ESMF (v4.0)

• ESMF holds information about grids, component inter-relationships• Users can associate additional metadata with ESMF classes (Grids, Arrays, Fields, States,

Components)• Metadata is stored as name/value pairs in Attribute packages• Attribute packages can be standard (e.g CF conventions for Fields) or custom• Packages can be nested – for example, metadata for an import State contains metadata for

all the Fields in the State.• Metadata can be exported in plain text or XML format …• then ingested into portal, where users can search and browse components, more (Curator

talk)

Image courtesy of Rocky Dunlap, Georgia Institute of technology

Preview of the portal for dynamical core comparison developed jointly with ESG

www.esmf.ucar.edu

More on science gateways …

• Work in progress: translation of ESMF interfaces into web service interfaces to support loose coupling and integration of components into service oriented architectures

• Will enable invocation of ESMF components and applications from a portal in a Grid-based (e.g. Teragrid) gateway

• Data and metadata from the run stored back to the portal

Image courtesy of Rocky Dunlap, Georgia Institute of technology

www.esmf.ucar.edu

Strategic objectives through 20101. Complete functionality and increase ease of use2. Integrate ESMF into science gateways3. Support integration into applications

www.esmf.ucar.edu

Support and broaden the customer base

HAF

GAIM

Jan 2008

www.esmf.ucar.edu

ESMF adoption

• 76 components in the community – most recent additions are CCSM4 dead and data components. (Live coming next…)

• 4 large production or operational codes running ESMF:◦ NCEP GFS◦ GEOS-5◦ HYCOM-CICE◦ COAMPS-NCOM

• Goals from strategic plan for end 2010:◦ 100 components◦ 20 large production or operational codes running ESMF

Image courtesy of Rocky Dunlap, Georgia Institute of technology

www.esmf.ucar.edu

A peek at the future …2009 2010 2011 2012 2013 2014 2015

InteroperabilityCompliance definitionsand verification tools

Tutorial development andclassroom deployment

Integration with workflowsGeneration of service interfaces for loose couplingMetadata links to data and other services

Automated model assembly andrun environmentBased on metadata files

Numerical/computational maintenance and development