HDFHDF

Dimension Scales in HDF5Dimension Scales in HDF5 HDF-EOS Workshop IX HDF-EOS Workshop IX

San Francisco, CASan Francisco, CANovember 30 - December 2, 2005 November 30 - December 2, 2005

Pedro Vicente NunesPedro Vicente NunesTHG/NCSATHG/NCSA

Champaign-Urbana, ILChampaign-Urbana, IL

HDFHDF

- 2 - HDFHDF

• Overview and definition of Dimension Scales

• Dimension Scales Metadata

• Dimension Scales API functions and code example

- 3 - HDFHDF

C codeC code

• Example: I have an array of 12 elements

My array data of 12 elements:

int dset_data[12] = {1,2,3,4,5,6,7,8,9,10,11,12};

- 4 - HDFHDF

My spatial domainMy spatial domain

• But the spatial phenomenon I am trying to model has different width and height definitions

•

• X

Y

- 5 - HDFHDF

My array data of 12 elements:int dset_data[12] = {1,2,3,4,5,6,7,8,9,10,11,12};

My X axis spatial domain:float X[4] = {10,20,50,100};

My Y axis spatial domain:double Y[3] = {10,20,30};

C codeC code

- 6 - HDFHDF

Dimension Scales in HDF4Dimension Scales in HDF4

• SD - Multifile Scientific Data Interface

• API functions– SDgetdimscale – SDsetdimscale – SDdiminfo

- 7 - HDFHDF

Save my arrays as HDF5 datasets with the High Level HDF5 API function

H5LTmake_dataset_int

(fid,“My data",rank,dims, dset_data);

HDF5 APIHDF5 APIC codeC code

- 8 - HDFHDF

H5LTmake_dataset_int

(fid,“X axis",rankx,dimsx, X);

H5LTmake_dataset_int

(fid,“Y axis",ranky,dimsy, Y);

HDF5 APIHDF5 APIC codeC code

for the X and Y arraysfor the X and Y arrays

- 9 - HDFHDF

/* get the DS dataset id and dataset*/

did = H5Dopen(fid,” My data”);

dsid = H5Dopen(fid,” X axis”);

/* attach the “ X axis” dimension scale to “My data" at dimension 0 */

H5DSattach_scale(did,dsid,DIM0);

HDF5 APIHDF5 APICall H5DSattach_scale functionCall H5DSattach_scale function

- 10 - HDFHDF

/* get the DS dataset id and dataset*/

did = H5Dopen(fid,” My data”);

dsid = H5Dopen(fid,” X axis”);

/* attach the “ X axis” dimension scale to “My data" at dimension 0 */

H5DSattach_scale(did,dsid,DIM0);

Dimension Scales : HDF5 APIDimension Scales : HDF5 APICall H5DSattach_scale functionCall H5DSattach_scale function

- 11 - HDFHDF

HDF Explorer: Tree ViewHDF Explorer: Tree View

- 12 - HDFHDF

HDF Explorer: Grid ViewHDF Explorer: Grid View

- 13 - HDFHDF

HDF Explorer: Map ViewHDF Explorer: Map View

- 14 - HDFHDF

What is a Dimension Scale (in HDF5) ?What is a Dimension Scale (in HDF5) ?

• A HDF5 dataset. • With additional metadata that identifies the

dataset as a Dimension Scale• Typically Dimension Scales are logically

associated with the dimensions of HDF5 Datasets

• The meaning of the association is left to applications.

- 15 - HDFHDF

Example: 3D datasetExample: 3D dataset

Dataset: 3D Array with Dataset: 3D Array with 5 x 7 x 105 x 7 x 10

dimensionsdimensions

3 Dimension Scales Datasets3 Dimension Scales Datasets

5

7

10

- 16 - HDFHDF

• Relationships between dataset dimensions and their corresponding dimension scales are not to be directly maintained or enforced by the HDF5 library

• Few restrictions as possible on the use of dimension scale dataset

• A dimension scale is not required to be a 1-D array, or to have a specific datatype

OverviewOverview

- 17 - HDFHDF

• A dataset dimension can have more than 1 associated dimension scale

• A Dimension Scale can be shared by two or more Dataset dimensions

- 18 - HDFHDF

Example: 3D datasetExample: 3D dataset

Dataset: 3D Array with Dataset: 3D Array with 5 x 7 x 105 x 7 x 10

dimensionsdimensions

Several Dimension Scales Several Dimension Scales DatasetsDatasets

- 19 - HDFHDF

MetadataMetadata

• The following dataset attributes are used to describe dimension scale datasets:– Attribute named “CLASS” with the value

“DIMENSION_SCALE”– Optional attribute named “NAME”– Attribute references to any associated Dataset

- 20 - HDFHDF

• When the Dimension Scale is associated with a dimension of a Dataset, the association is represented by attributes of the two datasets.

- 21 - HDFHDF

- 22 - HDFHDF

• In the Dataset, the DIMENSION_LIST attribute is an array of object references to scales (Dimension Scale Datasets), and in the Dimension Scale Dataset the REFERENCE_LIST is an array of object references to Datasets.

- 23 - HDFHDF

- 24 - HDFHDF

- 25 - HDFHDF

- 26 - HDFHDF

APIsAPIs

• Functions for creating and using Dimension Scales are implemented as high level functions

• Other High Level APIs include – Image– Table– Lite

• See http://hdf.ncsa.uiuc.edu/HDF5/hdf5_hl/

- 27 - HDFHDF

API FunctionsAPI Functions

- 28 - HDFHDF

• Name: H5DSset_scale • Signature: • herr_t H5DSset_scale(hid_t dsid, char *dimname)• Purpose:

– Convert dataset dsid to a dimension scale, with optional name, dimname.

• Description: – The dataset dsid is converted to a Dimension Scale dataset.

Creates the CLASS attribute, set to the value "DIMENSION_SCALE" and an empty REFERENCE_LIST attribute. If dimname is specified, then an attribute called NAME is created, with the value dimname.

- 29 - HDFHDF

• Name: H5DSattach_scale • Signature: • herr_t H5DSattach_scale(hid_t did, hid_t dsid,

unsigned int idx) • Purpose:

– Attach dimension scale dsid to dimension idx of dataset dsid.

• Description: – Define Dimension Scale dsid to be associated with

dimension idx of Dataset did. Entries are created in the DIMENSION_LIST and REFERENCE_LIST attributes.

- 30 - HDFHDF

– Name: H5DSdetach_scale – Signature: – herr_t H5DSdetach_scale(hid_t did, hid_t dsid,

unsigned int idx) – Purpose:

• Detach dimension scale dsid from the dimension idx of Dataset did.

– Description: • If possible, deletes association of Dimension Scale dsid

with dimension idx of Dataset did. This deletes the entries in the DIMENSION_LIST and REFERENCE_LIST attributes

- 31 - HDFHDF

• Name: H5DSis_attached

• Signature:

• htri_t H5DSis_attached(hid_t did, hid_t dsid, unsigned int idx)

• Purpose: – Verifies if a dimension scale is attached to a

dataset

• Description: – Report if dimension scale dsid is currently attached

to dimension idx of dataset did.

- 32 - HDFHDF

• Name: H5DSiterate_scales • Signature: • herr_t H5DSiterate_scales(hid_t did, unsigned dim,

int *idx, H5DS_iterate_t visitor, void *visitor_data) • Purpose:

– Iterates the operation visitor through the scales attached to dimension dim.

• Description: – H5DSiterate_scales iterates over the scales attached to

dimension dim of dataset dset. For each scale in the list, the visitor_data and some additional information, specified below, are passed to the visitor function.

- 33 - HDFHDF

• Name: H5DSset_label

• Signature:

• herr_t H5DSset_label(hid_t did, unsigned int idx, const char *label)

• Purpose: – Set label for the dimension idx of did to the value

label.

• Description: – Sets the DIMENSION_LABEL_LIST for

dimension idx of dataset did.

- 34 - HDFHDF

• Name: H5DSget_label

• Signature:

• ssize_t H5DSget_label(hid_t did, unsigned int idx, char *label, size_t *size)

• Purpose: – Read the label for dimension idx of did into buffer

label.

• Description: – Returns the value of the

DIMENSION_LABEL_LIST for dimension idx of dataset did, if set.

- 35 - HDFHDF

• Name: H5DSget_scale_name

• Signature:

• herr_t H5DSget_scale_name(hid_t did, char *name, size_t *size)

• Purpose: – Read the name of scale did into buffer name.

• Description: – Read the value of the NAME attribute for scale

did. Reads up to size characters into name.

- 36 - HDFHDF

• Name: H5DSis_scale

• Signature:

• htri_t H5DSis_scale(hid_t did)

• Purpose: – Determines whether dset is a Dimension Scale.

• Description: – H5DSis_scale determines if did is a Dimension

Scale, i.e., has CLASS="DIMENSION_SCALE").

- 37 - HDFHDF

• Name: H5DSget_num_scales

• Signature:

• int H5DSget_num_scales(hid_t did, unsigned int idx)

• Purpose: – Determines how many Dimension Scales are

attached to dimension idx of did.

• Description: – H5DSget_num_scales determines how many

Dimension Scales are attached to dimension did of dataset dset.

- 38 - HDFHDF

Code exampleCode example

• #include "H5DS.h"• #include "H5LT.h"

• #define RANK 2• #define DIM_DATA 12• #define DIM1_SIZE 3• #define DIM2_SIZE 4• #define DIM3_SIZE 2• #define DIM0 0• #define DIM1 1• #define DIM2 2

• #define DS_1_NAME "ds_a_1"• #define DS_2_NAME "ds_a_2"

- 39 - HDFHDF

• int main(void)• {

• hid_t fid; /* file ID */• hid_t did; /* dataset ID */• hid_t dsid; /* DS dataset ID */• int rank = RANK; /* rank of data dataset */• int rankds = 1; /* rank of DS dataset */• hsize_t dims[RANK] = {DIM1_SIZE,DIM2_SIZE}; /* size of data

dataset */• int buf[DIM_DATA] = {1,2,3,4,5,6,7,8,9,10,11,12}; /* data of data dataset */• hsize_t s1_dim[1] = {DIM1_SIZE}; /* size of DS 1 dataset */• hsize_t s2_dim[1] = {DIM2_SIZE}; /* size of DS 2 dataset */• int s1_wbuf[DIM1_SIZE] = {10,20,30}; /* data of DS 1 dataset */• int s2_wbuf[DIM2_SIZE] = {100,200,300,400}; /* data of DS 2 dataset

*/

- 40 - HDFHDF

• /* create a file using default properties */• if

((fid=H5Fcreate("myfile.h5",H5F_ACC_TRUNC,H5P_DEFAULT,H5P_DEFAULT))<0)

• goto out;

• /* make a dataset */• if (H5LTmake_dataset_int(fid,"dset_a",rank,dims,buf)<0)• goto out;

• /* make a DS dataset for the first dimension */• if (H5LTmake_dataset_int(fid,DS_1_NAME,rankds,s1_dim,s1_wbuf)<0)• goto out;

• /* make a DS dataset for the second dimension */• if (H5LTmake_dataset_int(fid,DS_2_NAME,rankds,s2_dim,s2_wbuf)<0)• goto out;

- 41 - HDFHDF

• /*-------------------------------------------------------------------------• * attach the DS_1_NAME dimension scale to "dset_a"• *-------------------------------------------------------------------------• */

• /* get the dataset id for "dset_a" */• if ((did = H5Dopen(fid,"dset_a"))<0)• goto out;

• /* get the DS dataset id */• if ((dsid = H5Dopen(fid,DS_1_NAME))<0)• goto out;

• /* attach the DS_1_NAME dimension scale to "dset_a" at dimension 0 */• if (H5DSattach_scale(did,dsid,DIM0)<0)• goto out;

• /* close DS id */• if (H5Dclose(dsid)<0)• goto out;

- 42 - HDFHDF

• /*-------------------------------------------------------------------------• * attach the DS_2_NAME dimension scale to "dset_a"• *-------------------------------------------------------------------------• */

• /* get the DS dataset id */• if ((dsid = H5Dopen(fid,DS_2_NAME))<0)• goto out;

• /* attach the "ds2" dimension scale to "dset_a" as the 2nd dimension */• if (H5DSattach_scale(did,dsid,DIM1)<0)• goto out;

• /* close DS id */• if (H5Dclose(dsid)<0)• goto out;

• /* close */• H5Fclose(fid);

- 43 - HDFHDF

- 44 - HDFHDF

- 45 - HDFHDF

- 46 - HDFHDF

RFC documentRFC document

• http://hdf.ncsa.uiuc.edu/RFC/

/H5DimScales

- 47 - HDFHDF

Acknowledgements Acknowledgements • Special thanks to Bob McGrath (NCSA)• This presentation is based upon work supported in

part by a Cooperative Agreement with the National Aeronautics and Space Administration (NASA) under NASA grant NNG05GC60A.  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA. 

• Other support provided by NCSA and other sponsors and agencies:

• http://hdf.ncsa.uiuc.edu/acknowledge.html