Development of a Community Hydrologic Information

System

David G TarbotonJeffery S Horsburgh, David R. Maidment (PI),

Tim Whiteaker, Ilya Zaslavsky, Michael Piasecki, Jon Goodall, David Valentine, Thomas

Whitenack

dtarb@usu.edu

SupportEAR 0622374

CUAHSI

HISSharing hydrologic data

http://his.cuahsi.org/

Outline

• CUAHSI and HIS data publication system• WaterML and WaterOneFlow web services• Observations data model (ODM)• Data publication system• Analysis using HIS• Implementation• Performance

Consortium of Universities for the Advancement of Hydrologic Science, Inc.

An organization representing more than one hundred United States universities, receives support from the National Science Foundation to develop infrastructure and services for the advancement of hydrologic science and education in the U.S.

http://www.cuahsi.org/

• 110 US University members

• 6 affiliate members• 12 International

affiliate members (as of March 2009)

What is the CUAHSI HIS?

An internet based system to support the sharing of hydrologic data comprising databases connected using the internet through web services as well as software for data discovery, access

and publication.

Web services and SchemaWaterOneFlow and WaterML

to transmit hydrologic data in a standard way

GetSitesGetSiteInfoGetVariableInfoGetValues

Discovery

Hydroseek

Analysis

MATLAB, Excel, GIS, R, …

Modeling

HydroLink + OpenMI

Access

HIS Desktop

Observatories publication and archival

of field data

store and share hydrologic data

HIS Server

ODM

Data repository Service Registry

catalog hydrologic

data services and metadata

HIS Central3rd-Party Servers

include data from others

• Set of query functions • Returns data in WaterML

NWIS Daily Values (discharge), NWIS Ground Water, NWIS Unit Values (real time), NWIS Instantaneous Irregular Data, EPA STORET, NCDC ASOS, DAYMET, MODIS, NAM12K, USGS SNOTEL, ODM (multiple sites)

Slide from David Valentine

WaterML and WaterOneFlowWaterML is an XML language for communicating water dataWaterOneFlow is a set of web services based on WaterML

WaterML design principles• Goal - capture semantics of hydrologic observations

discovery and retrieval• Role - exchange schema for CUAHSI web services• Driven by

– Hydrologists (community review)– ODM– USGS NWIS, EPA STORET, Academic Sources

• Conformance with Open Geospatial Consortium standards. http://www.opengeospatial.org/

• For XSD pros, the WaterML schema is athttp://his.cuahsi.org/wofws.html

Slide from David Valentine

Values

• Each data value recorded in value element• Timestamp, plus metadata for the value,

recorded in element’s attributes

ISO Time valuequalifier

Slide from David Valentine

What are the basic attributes to be associated with each single data value and

how can these best be organized?

Value

DateTime

Variable

Location

Units

Interval (support)

Accuracy

Offset

OffsetType/ Reference Point

Source/Organization

Censoring

Data Qualifying Comments

Method

Quality Control Level

Sample Medium

Value Type

Data Type

CUAHSI Observations Data ModelStreamflow

Flux towerdata

Precipitation& Climate

Groundwaterlevels

Water Quality

Soil moisture

data

• A relational database at the single observation level (atomic model)

• Stores observation data made at points

• Metadata for unambiguous interpretation

• Traceable heritage from raw measurements to usable information

• Standard format for data sharing

• Cross dimension retrieval and analysis

Space, S

Time, T

Variables, V

s

t

Vi

vi (s,t)“Where”

“What”

“When”

A data value

Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), A Relational Model for Environmental and Water Resources Data, Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.

CUAHSI Observations Data Model http://his.cuahsi.org/odmdatabases.html

Water Chemistry from Laboratory Sample

Stage and Streamflow Example

Base StationComputer(s)

Telemetry Network

Sensors

Query, Visualize, and Edit data using ODM Tools

Excel Text

ODMDatabase

ODM Data

Loader

Streaming Data

Loader

GetSitesGetSiteInfoGetVariableInfoGetValues

WaterOneFlowWeb Service

WaterML

DiscoveryHydroseek

AccessAnalysis

GISMatlabSplus

RIDL

JavaC++VB

Water Metadata Catalog

Harvester

Service Registry Hydrotagger

HIS Central

HydroExcelHydroGetHydroLink

HydroObjects

ODM

ODM

Contribute your ODM

CUAHSI HIS Data Publication System

http://his.cuahsi.org

Dynamic controlled vocabulary moderation system

Local ODMDatabase

Master ODM Controlled Vocabulary

ODM Website

ODM ControlledVocabulary Moderator

ODM Data Manager

ODMControlled Vocabulary

Web Services

ODM Tools

Local Server

XMLXML

http://his.cuahsi.org/mastercvreg.html From Jeff Horsburgh

Direct analysis from your favorite analysis environment. e.g. Excel, MATLAB

Direct analysis from your favorite analysis environment. e.g. Matlab

% create NWIS Class and an instance of the classcreateClassFromWsdl('http://river.sdsc.edu/wateroneflow/NWIS/DailyValues.asmx?WSDL');WS = NWISDailyValues;% GetValues to get the datasiteid='NWIS:02087500';bdate='2002-09-30T00:00:00';edate='2006-10-16T00:00:00';variable='NWIS:00060';valuesxml=GetValues(WS,siteid,variable,bdate,edate,'');

1920 1930 1940 1950 1960 1970 1980 1990 2000 20100

0.5

1

1.5

2

2.5x 10

4

cfs

Daily Discharge NEUSE RIVER NEAR CLAYTON, NC

Federal water data available through HIS Central (http://river.sdsc.edu)

• Operational– USGS

• Daily Values• Instantaneous Values• Irregular Data• Groundwater

– EPA Storet – MODIS– NCEP North American

Model– DayMet

• Development– NCDC– SNOTEL

• 11 WATERS Network test bed projects• 16 ODM instances (some test beds have more than one ODM

instance)• Data from 1246 sites, of these, 167 sites are operated by WATERS

investigators

National Hydrologic Information ServerSan Diego Supercomputer Center

HIS Implementation in WATERS Network Information System

Prototype Texas HIS• TWDB is supporting a small project at

University of Texas to start building a prototype Texas Hydrologic Information System

Texas Hydrologic Information Server (at TNRIS)Texas Observations Catalogs and some state water datasets

HIS servers atdata sources

(State agencies,River authorities,

Water Districts, Cities,Counties….)

Web Services

Intermountain Constellation of Experimental WATERsheds (ICEWATER)

The Inland Northwest Research Alliance (INRA) Water Research Consortium is establishing a HIS Network to share experimental

watershed data to address regional water resources

challenges

Great Salt Lake Information SystemA common window for accessing, viewing and downloading

data and information in the Great Salt Lake Basin

State AgenciesLocal Organizations Federal agenciesUniversities

Conclusions• Data Storage in an Observations Data Model

(ODM)• Data Access through internet-based Water Data

Services using a consistent data language, called WaterML

• Data Indexing through a National Water Metadata Catalog

• Data Discovery through federated map and thematic keyword search system

The combination of these capabilities creates a common window on water observations data for the United States unlike any that has existed before.

HIS Team and Collaborators• University of Texas at Austin – David Maidment, Tim Whiteaker, Ernest

To, Bryan Enslein, Kate Marney• San Diego Supercomputer Center – Ilya Zaslavsky, David Valentine,

Tom Whitenack• Utah State University – David Tarboton, Jeff Horsburgh, Kim

Schreuders, Justin Berger• Drexel University – Michael Piasecki, Yoori Choi• University of South Carolina – Jon Goodall, Tony Castronova• Idaho State University – Dan Ames• CUAHSI Program Office – Rick Hooper, David Kirschtel, Conrad Matiuk• WATERS Network – Testbed Data Managers• HIS Standing Committee • USGS – Bob Hirsch, David Briar, Scott McFarlane• NCDC – Rich Baldwin

HIS Overview Report• Summarizes the

conceptual framework, methodology, and application tools for HIS version 1.1

• Shows how to develop and publish a CUAHSI Water Data Service

• Available at:

http://his.cuahsi.org/documents/HISOverview.pdf