cf metadata conventions: governance, support, and future karl e. taylor program for climate model...
Post on 13-Dec-2015
Embed Size (px)
- Slide 1
CF Metadata Conventions: Governance, Support, and Future Karl E. Taylor Program for Climate Model Diagnosis and Intercomparison Lawrence Livermore National Laboratory Presented to the GO-ESSP Community Workshop Paris, France 11 June 2007 Slide 2 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Intro to CF metadata conventions Climate and Weather Forecast Metadata that makes data files self-describing Examples of CF metadata Coordinate information needed to locate data in space and time Further grid information (e.g., grid cell bounds, area) Standard names in conjunction with other attributes helps users determine whether data from different sources are the same physical quantity and helps distinguish variables within archives Some processing info. (e.g., zonal mean, climatological processing) Slide 3 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Brief history of CF Started with COARDS standard, and developed through efforts of a handful of volunteers 2003: CF 1.0 released 2005: CF white paper discussing future governance circulated 2006: White paper revised and presented to WCRP WGCM 2006: Current governance structure established 2003-present: increasing acceptance of CF and adoption by several community coordinated projects: IPCC AR4 archive conforms PMIP, HTAP, regoinal groups, seasonal groups, ENSEMBLES Slide 4 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Governance structure Original CF authors turned over control to two working committees: Conventions Committee Standard Name Committee Committee work done via email and web discussion (Trac) Support provided by: BADC: Alison Pamment (50%) standard names PCMDI: Kyle Halliday (20%) web site support WCRP/WGCM has been approached to appoint a CF Governance Panel charged with responsibility for stewardship of CF. Slide 5 PCMDI BASC / CRC 17 May 2007 K. E. Taylor What happened at last GO-ESSP meeting? Discussed CF governance & procedure for making changes Some pressing CF issues were identified and in some cases individuals volunteered to try to make progress Augmented grid information, especially for less structured grids. Relationship between CF and GIS grids & metadata Standard name issues (including dialects, profiles, hierarchy structures) Relationship between CF and netCDF4 CF handling of in situ observations How to handle discovery information Sample datasets and reference implementations Slide 6 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Agenda for today Governance issues what needs fixing? Underlying principles any adjustments needed? Technical issues whats been done? Can we make progress on any of the discussion topics? Supportive software status reports Any additions? Other concerns? Slide 7 Conventions Committee: Support and Governance Agenda Item GO-ESSP Community Workshop Paris, France 11 June 2007 Slide 8 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Current committee members Karl Taylor (chair) Kyle Halliday (secretary) Balaji John Caron Jonathan Gregory Tom Gross Steve Hankin Jamie Kettleborough Russ Rew Rich Signell Slide 9 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Terms of reference Further develop (revise) CF conventions Consider implementation of CF metadata conventions in other file formats (besides netCDF) Determine what is meant by CF conformance Slide 10 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Current procedure for modifying/extending CF Anyone can propose a change Discussion opens & a member of the committee volunteers to moderate A provisional resolution is reached by consensus and summarized by moderator Reference files are produced illustrating the new feature If not a trivial change, trial implementations are carried out. If problems emerge iterate When consensus reached, the proposed change is provisionally accepted. (advanced implementers may begin adoption) Collection of changes evaluated as a package leading to new CF versions released by consensus Test reference files become part of the test data Slide 11 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Problems encountered. Progress on some proposals, but few have been carried through to implementation. Contrast with standard names committee Few actions have taken place With a couple of exceptions, the volunteer moderation of discussion topics has floundered Procedure for introducing changes outlined in white paper: Email Web-based Trac system Jonathans thoughts Slide 12 Issues Concerning Underlying Principles of CF Agenda Item GO-ESSP Community Workshop Paris, France 11 June 2007 Slide 13 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Underlying principles Special needs of observations Should the scope of metadata accommodated by CF include: Description of experiment Description of model Description of higher-order grid-characteristics (e.g., relationship between u & v grids, which grid cells are connected to which) Competing needs of data producers & application developers CF extensions can be implemented faster by producers who sometimes are impatient for changes to be formally adopted Should requirement be relaxed that all CF metadata reside in the same file as the data itself? Bundling of common definitions of a quantity into I.D.s (defined by a combination of attributes including the standard name + others) Aggregation layers CF profiles Slide 14 Miscellaneous Technical Issues Agenda Item GO-ESSP Community Workshop Paris, France 11 June 2007 Slide 15 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Miscellaneous technical issues Ensembles (including realization_weight and other issues): Alison Time axis issues for forecasts: Jonathan Definition of the geoid: Alison Use of anomaly: Alison or Jonathan Subgrid variation description issues (use of where): Jonathan Time and calendar issues for paleoclimate simulations: Jean-Yves? Balaji? Slide 16 The Future of CMOR Karl E. Taylor Program for Climate Model Diagnosis and Intercomparison Lawrence Livermore National Laboratory Presented to the GO-ESSP Community Workshop Paris, France 11 June 2007 Slide 17 PCMDI BASC / CRC 17 May 2007 K. E. Taylor The Climate Model Output Rewriter (CMOR) Provides a uniform interface for contributing CF-compliant data (adhering to good practices) to the growing number of MIPs (e.g., CMIP, AMIP, CFMIP, PMIP, APE, HTAP) For IPCC AR4, nearly all model output in the archive was written through CMOR. Assures CF-compliance and all MIP-established output requirements are met. Traps many metadata errors when files are produced. See http://www-pcmdi.llnl.gov/software/about_software.php Slide 18 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Project-specific CMOR tables facilitate and ensure consistency of model output from contributing groups Proper specification of several coordinate attributes, including: Correct standard name axis, positive, and formula_terms attributes, as appropriate Proper specification of several variable attributes, including: Correct standard name Required dimensions cell_methods attribute A capability to Reorder axis order Reverse axis direction (or translate longitude dimension) Convert units (through udunits) Slide 19 PCMDI BASC / CRC 17 May 2007 K. E. Taylor CMOR flagged common errors, including Pointing out when required metadata are omitted. Rejecting incorrect metadata (wrong units, inadmissible attribute values, etc.) Rejecting inconsistent coordinate dimensions passed by user to CMOR. Rejecting non-monotonic coordinate values or inconsistent boundary values, as passed by user. Rejecting values that are clearly unrealistic (likely indicating improper units conversion or incorrect sign). Slide 20 PCMDI BASC / CRC 17 May 2007 K. E. Taylor What changes will be implemented in the next 6 months? Station data (needed, for example, by the HTAP project), including Station names Station locations Output from regional models With logically rectangular grids But with non-cartesian longitudes and latitudes i.e., data(i,j), lon(i.j), and lat(i,j) Slide 21 PCMDI BASC / CRC 17 May 2007 K. E. Taylor Longer term prospects for CMOR Difficult to see extending CMOR to write the grid- description files mandated by the GridSpec proposal. Probably possible, however, to write the actual data files through CMOR (and leave it to other software to produce the special GridSpec files).