Important ESDIS 2009 tasks review

Kent Yang, Mike Folk

The HDF Group

April 1st, 2010

04/19/23 1Annual briefing to ESDIS

Outline

• HDF-EOS Website• HDF OPeNDAP support• HDF-EOS5 Augmentation tool

04/19/23 2Annual briefing to ESDIS

HDF-EOS Web Sitehttp://hdfeos.org

04/19/23 3Annual briefing to ESDIS

Main page: http://hdfeos.org

• Reorganize the overall structure

• Update the main page

04/19/23 404/19/23 4Annual briefing to ESDIS

Help: http://www.hdfeos.org/help

04/19/23 504/19/23 5Annual briefing to ESDIS

Examples

• Added a new “Examples” section

- C, Fortran, IDL and MatLab examples to access HDF-EOS2 and HDF-EOS5 Grid and Swath files

- Examples on how to use tools such as HEG, GrADS, NCL, Using IDV via OPeNDAP to access HDF-EOS 2 and HDF-EOS5 data

604/19/23 6Annual briefing to ESDIS

Software

• Evaluated 35 tools and libraries

• Added 11 tools and libraries

• Added detailed descriptions

November 3-5, 2009 7HDF/HDF-EOS Workshop XIII04/19/23 7Annual briefing to ESDIS

November 3-5, 2009 HDF/HDF-EOS Workshop XIII 8

Forum

04/19/23 8Annual briefing to ESDIS

HDF-EOS website statistics

Current Period: Nov. 1st 2009 – Feb. 28th 2010

Previous Period: July 4th 2009 – Oct. 31st 2009

04/19/23 9Annual briefing to ESDIS

Definitions

Visits – The total number of visits to the website.Pageviews – The total number of pages viewed on the website. Repeated views of a single page are counted.

Pages/Visit – The average number of page views per visitor.Bounce Rate – The percentage of single-page visits. (Total number of visits viewing one page) / (Total number of visits)

Avg. Time on Site – The average visit duration.

% New visits – The percentage of visits by people who had never visited the website before.

Organic – Web traffic which comes from unpaid listing at search engines or directories.

04/19/23 10Annual briefing to ESDIS

HDF-EOS Website Statistics Comparison

Jul Aug SepOct

04/19/23 11Annual briefing to ESDIS

Month by Month Visits Comparison

04/19/23 12Annual briefing to ESDIS

Traffic Sources

- Visitors who typed in the URL directly

-Visitors who clicked a link from another website

- Visitors who used a search engine

04/19/23 13Annual briefing to ESDIS

04/19/23 14Annual briefing to ESDIS

Pageviews

04/19/23 15Annual briefing to ESDIS

Top Pages Viewed

04/19/23 16Annual briefing to ESDIS

04/19/23 17Annual briefing to ESDIS

04/19/23 18Annual briefing to ESDIS

04/19/23 19Annual briefing to ESDIS

04/19/23 20Annual briefing to ESDIS

Current work

04/19/23 21Annual briefing to ESDIS

Current Work

• Collect sample files for NASA HDF4/HDF-EOS2 products

• Provide special HDF-EOS description for each sample file

• Examine the access of each sample file with widely used tools

• Provide the codes and the plots generated from the tools for each sample file

04/19/23 22Annual briefing to ESDIS

Tools to access the HDF data

• NCL

• MatLab

• IDL

04/19/23 23Annual briefing to ESDIS

04/19/23 24

Datapool Example

04/19/23 24Annual briefing to ESDIS

HDF OPeNDAP Work Update

04/19/23 25Annual briefing to ESDIS

HDF5 –OPeNDAP Handler

• Four official releases from OPeNDAP website• The latest release date is February 2010• Add the support for HDF-EOS5 Aura swath files• Aura MLS, OMI, HIRDLS files are served via

OPeNDAP• URL:

http://opendap.org/download/hdf5_handler.html

04/19/23 26Annual briefing to ESDIS

http://hdfeos.org/software/ncl_opendap_mls.php

Access Aura MLS data via OPeNDAP

04/19/23 27Annual briefing to ESDIS

HDF4 OPeNDAP Handler

04/19/23 28Annual briefing to ESDIS

Phase 1 work review

• Time: April 2009 – October 2009• Enhanced the original HDF4-OPeNDAP handler• Some HDF-EOS2 data can be served via

OPeNDAP• Provided a version to GES DISC

04/19/23 29Annual briefing to ESDIS

Phase 2 work

• Time: November 2009 – July 2010• Goal: Enhance the HDF4-OPeNDAP handler so

that more NASA HDF-EOS2 and HDF4 data can be served via OPeNDAP

04/19/23 30Annual briefing to ESDIS

Phase 2 work

• Comprehensive examinations of all NASA HDF-EOS2 data products

- NASA Data pool in the previous slides• Come up with the re-design based on the

examinations• Re-implement the HDF-EOS2 module of HDF4-

OPeNDAP handler• Add the support for handling some NASA pure

HDF4 data

04/19/23 31Annual briefing to ESDIS

Phase 2 - Current Status

• We finished the examination of NASA HDF-EOS2 products

• We finished the design note of HDF-EOS2 module of HDF4-OPeNDAP handler

• We finished implementing 90% features of the HDF-EOS2 module

• We provided a customized version for GES DISC to test

04/19/23 32Annual briefing to ESDIS

Phase 2 – Current Work Highlights

• Focus on making the data accessible by Java-based OPeNDAP clients

• Strictly Follow CF/COARD conventions• Handle different HDF-EOS2 cases

04/19/23 33Annual briefing to ESDIS

Phase 2 work plan

• Time: Now to July 14th, 2010• Finish all features for the HDF-EOS2 module• Handle Hybrid HDF-EOS2 files• Handle some NASA HDF4 files

04/19/23 34Annual briefing to ESDIS

HDF-EOS5 augmentation tool

04/19/23 35Annual briefing to ESDIS

HDF-EOS5 and NetCDF-4

• Enabling NetCDF4 to access HDF-EOS5 data Conversion Augmentation

• One file can be used for both EOS5 and NetCDF-4.

• Note that EOS5 users are not affected at all.

HDF5

HDF-EOS5HDF-EOS5

file HDF5

NetCDF4NetCDF-4

file

AugmentedHDF-EOS5

file

Augmentation

04/19/23 36Annual briefing to ESDIS

Goals

• Augmented file must be readable through NetCDF-4 API as well as HDF-EOS5 API

• How? Read info in the HDF-EOS5 part.

Transform it to conform to NetCDF-4 model. .

Append the new info.

+ =

04/19/23 37Annual briefing to ESDIS

Past Work Review

Grid Swath

OMI L2GOMI L3GTESMLSHIRDLS

OMIMLSHIRDLSTES

• Not all Aura HDF-EOS5 files use projection for storing dimension values.

• For example, OMI L2G grid files store lat/lon in data fields.

• TES grid files use both ways.

Collected and Examined different HDF-EOS5 data

04/19/23 38Annual briefing to ESDIS

Past Work Review

• To follow the netCDF4 data model, we need users to provide the information about the dimension fields.

• MLS and HIRDLS data are vertical profilers. The dimension fields are Pressure and Time.

• We need similar information for other instruments.

Provided a customized version for HIRDLS team

04/19/23 39Annual briefing to ESDIS

Future Work

• Handle other cases:

OMI L2G,future HIRDLS/MLS grid• Design a general input file for insufficient

information• Continue testing with NetCDF-4 snapshots• Consistency checker

If one modifies dim info in an augmented file, file becomes inconsistent.

• Testing, Documentation

04/19/23 40Annual briefing to ESDIS

Thank You!

04/19/23 41Annual briefing to ESDIS

Acknowledgements

This work was supported by the Cooperative Agreement with the National Aeronautics and Space Administration (NASA) under NASA grant NNX06AC83A and NNX08A077A.

Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA.

04/19/23 42Annual briefing to ESDIS

Questions/comments?

04/19/23 43Annual briefing to ESDIS

Backup slides

04/19/23 44Annual briefing to ESDIS

Role of HDF4-OPeNDAP handler

OPeNDAP Clients OPeNDAP HDF-EOS2

HDF4-OPeNDAP handler

04/19/23 45Annual briefing to ESDIS