data interoperability in the hydrologic sciences the cuahsi hydrologic information system
DESCRIPTION
Data Interoperability in the Hydrologic Sciences The CUAHSI Hydrologic Information System. David Tarboton, David Maidment, Ilya Zaslavsky, Dan Ames, Jon Goodall , Rick Hooper, Jeff Horsburgh, David Valentine, Tim Whiteaker, Tom Whitenack Kim Schreuders. CUAHSI HIS Sharing hydrologic data. - PowerPoint PPT PresentationTRANSCRIPT
Data Interoperability in the Hydrologic Sciences
The CUAHSI Hydrologic Information System
David Tarboton, David Maidment, Ilya Zaslavsky, Dan Ames, Jon Goodall, Rick Hooper, Jeff Horsburgh, David Valentine, Tim Whiteaker, Tom Whitenack
Kim SchreudersCUAHSI
HISSharing hydrologic data Support EAR 0622374
Consortium of Universities for the Advancement of Hydrologic Science, Inc.
An organization representing more than one hundred United States universities, receives support from the
National Science Foundation to develop infrastructure and services for the advancement of hydrologic science and
education in the U.S. http://www.cuahsi.org/
Hydrologic Information SystemHydrologic Measurement Facility
Hydrologic ModelingHydrologic Education Outreach
Partners:Academic: many research watersheds, CZO…Government: USGS, EPA, NCDC, NWS, state and localCommercial: Microsoft, ESRI, KistersInternational: Australia, UKStandardization: OGC, WMO (Hydrology Domain WG); adopted by USGS, NCDC, Army Corps of Eng.• An online distributed system to support the sharing of
hydrologic data from multiple repositories and databases via standard web service protocols
• Software for data publication, discovery, access and integration
What is the CUAHSI HIS?UT-Austin, SDSC/UCSD, Utah State U, Idaho State U, Drexel U, U of So. Carolina, CUAHSIPI: D. R. Maidment (UT-Austin)
HydroServer• Publication of point observations and geospatial
datasets• Distributed data with local control for data publishers
Data Discovery and Integration
Data Publication Data Analysis and Synthesis
HydroCatalog
HydroDesktopHydroServer BindData ServicesWater Data ServicesSpatial Data Services
Publish
Metadata Services
Service Registration
Catalog Harvesting
FindMetadata Search
Service and Data Theme Metadata
Data Carts
Ongoing Data Collection
Data presentation, visualization, and analysis through Internet
enabled applications
Internet ApplicationsPoint Observations Data
Historical Data Files
GIS Data
HydroServer
ODM Database
GetSitesGetSiteInfoGetVariableInfoGetValues
WaterOneFlowWeb Service
WaterML
OGCWMS, WFS, WCS
• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A qualifier is a symbol that provides additional information about the value
Data Service
Network
{Value, Time, Qualifier}
NWIS Daily Values
NWIS Sites
San Marcos River at Luling, TX
Discharge, stage (Daily or instantaneous)
18,700 cfs, 3 July 2002
Sites
Variables
Observation
Observations-Network Information Model
Observations Data Model (ODM)
Soil moisture
data
Streamflow
Flux tower data
Groundwaterlevels
Water Quality
Precipitation& Climate
• A relational database at the single observation level• Metadata for unambiguous interpretation• Traceable heritage from raw measurements to usable
information• Promote syntactic and semantic consistency • Cross dimension retrieval and analysis
Horsburgh, J. S., D. G. Tarboton, D. R. Maidment, and I. Zaslavsky (2008), A relational model for environmental and water resources data, Water Resources Research, 44, W05406, doi:10.1029/2007WR006392.
• Set of query functions • Returns data in WaterML
WaterML and WaterOneFlowWaterML is an XML language for communicating water dataWaterOneFlow is a set of web services based on WaterML
Manager Interface
Browser Interface
Service Interface
MapWebsite
Time Series Analyst Website
HydroServer Website
Capabilities Web Service
HydroServer Capabilities Database
Capabilities Configuration
Tool
ObservationData Model
ObservationData Model
WaterOneFlow Web Service
WaterOneFlow Web Service
WaterOneFlow Web Service
ObservationData Model
Spatial Data Services
Geo SpatialDatabase
Spatial Data File
HydroServerView server portal
Publish point data Publish spatial data
View spatial data
View point data
Configure server
Publish metadata
Load and Edit data
ODM Tools and Data Loaders
HydroServer
HIS Central - HydroCatalog• Central metadata catalog supporting data
discovery
Data Discovery and Integration
Data Publication Data Analysis and Synthesis
HydroCatalog
HydroDesktopHydroServer BindData ServicesWater Data ServicesSpatial Data Services
Publish
Metadata Services
Service Registration
Catalog Harvesting
FindMetadata Search
Service and Data Theme Metadata
Data Carts
Map integrating NWIS, STORET, & Climatic Sites 65+ public web services
13,000+ variables1.96+ million sites23.3 million observation time seriesReferencing 5+ billion data values
HIS Central HydroCatalog Content
Available via HIS Centraldiscovery services
Available via GetValues requests
Metadata for most services are harvested weekly
Data Publisher Interface
Data Service
InterfaceSearch Client Interface
Web Service Registry Website
Metadata Harvester
Metadata catalog
database
Search Web Service
HydroCatalog
Validation Services
Monitoring Services
Semantic Annotation
Website
Logging Service
Validate WaterOneFlow
Services
Collect Metadata
Register, View and Manage Services
View and Manage Ontology Mappings
Provide Client Search over Services and Observation
MetadataLog service
use
Get Concepts and Mappings
Monitor Service
Hydrologic Ontology
Interactively test WaterOneflow Services
Ontology Web Service
HydroCatalog
• A web service application programmers interface to the metadata catalog• Semantic annotations and a domain ontology aid in discovery
HydroDesktop• Discover, download, visualize, and analyze
hydrologic data
Data Discovery and Integration
Data Publication Data Analysis and Synthesis
HydroCatalog
HydroDesktopHydroServer BindData ServicesWater Data ServicesSpatial Data Services
Publish
Metadata Services
Service Registration
Catalog Harvesting
FindMetadata Search
Service and Data Theme Metadata
Data Carts
Thematic keyword search
Integration from multiple sources
Search on space and
time domain
HydroDesktop – Data Access and Analysis
UserInterface
HydroDesktopCore & Shared Libraries
Table View
Graph View
HydroRHydro
Modeler
Help
Plugins
Application Settings
XML
Export to CSV
Fetch BasemapDotSpatial
Library
WaterOneFlowWeb Service
WaterML1.01.1
HydroCatalogSearch & Ontology
Web Service
XML
Search
Project File
Project File
Project File Metadata
Cache DB
Metadata Cache DB
Metadata Cache DB
Data Repository
DB
Data Repository
DB
Data Repository
DB
Object ModelSite
VariableSeries
Method
ThemeDataValue
…
Database Abstraction Layer
Data Repository Manager
Metadata Cache
Manager
Search
• Interact with user
• Interact with other programs
• Get from/ save to disk
OtherLibraries
HydroDesktop
Collaborative Open Source Development
• HydroServer – http://hydroserver.codeplex.com• HydroDesktop – http://hydrodesktop.codeplex.com• HydroCatalog – http://hydrocatalog.codeplex.com
• Source Code• Downloads• Discussion Forums• Issue Tracker
Toward a Standard SOA for Water Data
• Operationalizing the CUAHSI HIS• Encourage broad implementation by software
providers and broad uptake by data publishers• Determine optimal web service interfaces and
data encodingsCatalog
DesktopServerBind????
Publish???? Find????
Open Geospatial Consortium Web Service Standards
• Map Services
• Web Map Service (WMS)• Web Feature Service (WFS)• Web Coverage Service
(WCS)• Catalog Services for the Web
(CS/W)
• Observation Services
• Observations and Measurements Model
• Sensor Web Enablement (SWE)
• Sensor Observation Service (SOS)
These standards have been developed over the past 10 years ….…. by 400 companies and agencies working within the OGC
OGC Hydrology Domain Working Group evolving WaterML into an International Standardhttp://www.opengeospatial.org/projects/groups/waterml2.0swg
Summary• CUAHSI HIS
– Cyberinfrastructure for managing and publishing observational data– Overcomes syntactic and semantic heterogeneity using a standard data model,
transmission language, controlled vocabularies, and ontology– End-to-end infrastructure - from publication to discovery, access, and analysis– Supports a national network of academic partners, and links to many important
federal water data repositories– Maintains national registry of services with searchable metadata– Already deployed at multiple locations, leveraged by other projects, free and
open source• New Developments
– HydroDesktop and associated plugins for data discovery, download, and analysis– WaterML 2.0 is an emerging OGC standard language for communicating water
observations data from academic and government sources– New OGC Concept Development Study to define best practices for web service
interfaces and data encodings