recent developments and trends in network access to oceanographic in-situ data nancy soreide,...
TRANSCRIPT
Recent developments and trends in Network Access
to Oceanographic In-situ Data
Nancy Soreide, NOAA/PMELNancy Soreide, NOAA/PMEL
John “Oz” Osborne, NOAA/PMEL - OceanAtlas Software John “Oz” Osborne, NOAA/PMEL - OceanAtlas Software Joe Sirott, NOAA/PMEL – Sirott and AssociatesJoe Sirott, NOAA/PMEL – Sirott and Associates
Tiffany Vance – NOAA/AFSCTiffany Vance – NOAA/AFSCNazila Merati – NOAA/PMEL – JISAONazila Merati – NOAA/PMEL – JISAO
Christopher Moore – NOAA/PMEL - JISAOChristopher Moore – NOAA/PMEL - JISAO
American Meteorological Society MeetingJanuary 29-Feb 3, 2006, Atlanta, GA
What is OPeNDAP?
OPeNDAP (Open source Project for a OPeNDAP (Open source Project for a Network Data Access Protocol) is a Network Data Access Protocol) is a protocol for remotely accessing scientific protocol for remotely accessing scientific datadata
Uses HTTP as transportUses HTTP as transport Endorsed as a standard by IOOS Data Endorsed as a standard by IOOS Data
Management and Communications Management and Communications (DMAC) committee(DMAC) committee
Joe Sirott
But…
OPeNDAP in-situ data is poor stepchild of OPeNDAP in-situ data is poor stepchild of OPeNDAP gridded dataOPeNDAP gridded data
Few desktop clients support itFew desktop clients support it Even fewer Web apps support itEven fewer Web apps support it Some recent progress (GrADS, Java Some recent progress (GrADS, Java
OceanAtlas) but change is slowOceanAtlas) but change is slow
Joe Sirott
What is Dapper?
Web server that provides distributed access to Web server that provides distributed access to in-situ data via OPeNDAP protocolin-situ data via OPeNDAP protocol
Clients include GrADS, Matlab, LAS, NcBrowse, Clients include GrADS, Matlab, LAS, NcBrowse, Java Ocean Atlas, DChartJava Ocean Atlas, DChart
Available data:Available data: Realtime global Argo profiles Realtime global Argo profiles NDBC Time SeriesNDBC Time Series WOCE Sea Level WOCE Sea Level PMEL in-situ data (EPIC database)PMEL in-situ data (EPIC database) Coming: TAO & NODC World Ocean Database 2001Coming: TAO & NODC World Ocean Database 2001
All components are open-sourceAll components are open-source
Joe Sirotthttp://www.epic.noaa.gov/epic/software/dapper/
What does Dapper do?
Efficiently aggregates thousands of in-situ data Efficiently aggregates thousands of in-situ data files (>2000000 for WODB2001)files (>2000000 for WODB2001)
Handles several in-situ data formats:Handles several in-situ data formats: Argo GDAC netCDFArgo GDAC netCDF PMEL EPIC netCDFPMEL EPIC netCDF COARDS compliant netCDFCOARDS compliant netCDF
Supports time series or profilesSupports time series or profiles Supports space/time queriesSupports space/time queries Based on HPCC funded Climate Data Portal Based on HPCC funded Climate Data Portal
(Soreide, Zhu, Denbo)(Soreide, Zhu, Denbo)
Joe Sirotthttp://www.epic.noaa.gov/epic/software/dapper/
Dapper architectural overview
Uses relational database to store Uses relational database to store metadata and coordinate boundaries for metadata and coordinate boundaries for each profile/time series fileeach profile/time series file
Data maintained in original formatData maintained in original format All data delivered as OPeNDAP All data delivered as OPeNDAP
sequencessequences
Joe Sirotthttp://www.epic.noaa.gov/epic/software/dapper/
Dapper architecture diagram
CORBA (IIOP)
Climate Data Portal (CDP)
MySQL
Java netCDFlibrary
netCDFprofile
netCDFprofile
Dapper HTTP Server
CDPservice
netCDFservice
OPeNDAP clients
OPeNDAP Protocol(HTTP)
Database loader
Joe Sirotthttp://www.epic.noaa.gov/epic/software/dapper/
DChart
Browser based in-situ OPeNDAP clientBrowser based in-situ OPeNDAP client Based on FY2004 HPCC funded project Based on FY2004 HPCC funded project
for shipboard display of cruise datafor shipboard display of cruise data AJAX based user interface AJAX based user interface
AJAX is used by Google MapsAJAX is used by Google Maps
Joe Sirott
DChart Design Goals
Highly responsive browser based user Highly responsive browser based user interfaceinterface
Low learning curveLow learning curve Rapid previewing and downloading of Rapid previewing and downloading of
Dapper datasetsDapper datasets Highly interactive map for region selectionHighly interactive map for region selection No Java applets or proprietary plug-insNo Java applets or proprietary plug-ins
Joe Sirott
Ocean Data Navigator(aka DCHART)New!
New!
Joe Sirotthttp://dapper.pmel.noaa.gov/dchart
Ocean Data Navigator(aka DChart)New!
New!
Joe Sirotthttp://dapper.pmel.noaa.gov/dchart
AJAX
The Good, the BadThe Good, the Badand the Uglyand the Ugly
Joe Sirott
Good stuff
Much more responsive user interfaceMuch more responsive user interface Desktop application functionality in familiar Desktop application functionality in familiar
Web browser interfaceWeb browser interface Client responsible for state managementClient responsible for state management Zero effort software updates (for users!)Zero effort software updates (for users!) ““Cool” factorCool” factor
Joe Sirott
Bad stuff Requires “modern” browserRequires “modern” browser
DChart support currently limited to IE 6, Firefox DChart support currently limited to IE 6, Firefox 1.0.x, Mozilla1.0.x, Mozilla
Covers > 90% of users and alternative would be Covers > 90% of users and alternative would be downloading desktop app downloading desktop app
Debugging is difficultDebugging is difficult A lot for developers to learn:A lot for developers to learn:
XML,XPath,XSLT,JavaScript, CSS, DHTML, XML,XPath,XSLT,JavaScript, CSS, DHTML, DOM…DOM…
Libraries/frameworks immatureLibraries/frameworks immature Maintenance and testing difficultMaintenance and testing difficult
Joe Sirott
And the ugly…
Browser incompatibilities still an issue:Browser incompatibilities still an issue: GET limited to 2048 characters in IEGET limited to 2048 characters in IE IE window coordinate system different than FirefoxIE window coordinate system different than Firefox IE memory leaks IE memory leaks
Legacy ActiveX reference countingLegacy ActiveX reference counting Transparent images with alpha channelTransparent images with alpha channel
Disappearing DOM nodes with XSLT in IEDisappearing DOM nodes with XSLT in IE
Ugly kludges required:Ugly kludges required: GET/POST through Iframe to retain historyGET/POST through Iframe to retain history
Joe Sirott
Lessons Learned
Desktop applications are moving to the Desktop applications are moving to the browser.browser.
It’s easier to move data to servers than It’s easier to move data to servers than servers to data.servers to data.
The response time of the back end is The response time of the back end is critical in AJAX applications.critical in AJAX applications.
It takes a lot more time to develop the It takes a lot more time to develop the back end than the user interface. back end than the user interface.
Joe Sirott
Java OceanAtlasComprehensive Profile Data Analysis for the DesktopComprehensive Profile Data Analysis for the Desktop
Data types:–Supports many profile data formats (bottle and CTD data)–Argo–Distributed data (OPeNDAP/dapper)–Gridded netCDF data–Time series
Graphics:–Property-property, profile, contour, “station” value, and map plots–Overlay section contour plots–Contour plots on maps–Color plots by any parameter or metadata (date/time, longitude, or latitude) –Color map symbols by station calculation values–Presentation quality graphics and choice of fonts
Calculations:–Large selection of standard oceanographic calculations–Station calculations: mixed-later depth, interpolation to surface, neutral surface, extrema, statistics–Section calculations: section difference and mean cast
John Osborne
Java OceanAtlas
10°N Residuals O2: WOCE vs. World Ocean Atlas
Surface O2
John Osborne
OceanGISIntegration of GIS and Java tools
Read simple shape filesRead simple shape files Read ocean data from dapperRead ocean data from dapper Read model outputRead model output Rotate/pan/zoomRotate/pan/zoom Hyperlink objects and 3D “picking”Hyperlink objects and 3D “picking” Switch from 2D projections to 3DSwitch from 2D projections to 3D
GIS toolsGIS tools ArcObjects, ArcGIS, ArcIMS, ArcObjects, ArcGIS, ArcIMS,
ArcGIS Engine ArcGIS Engine Java toolsJava tools
MapObjects Java, Intra-Layer MapObjects Java, Intra-Layer Calculator, VTK (Java wrapper), Calculator, VTK (Java wrapper), Java 3D, Oceanographic tools Java 3D, Oceanographic tools (Java), GeoTools(Java), GeoTools
Tiffany Vance, Christopher Moore, Nazila Merati
Accessing ocean profile sensor data fromdatabase (selected profiles in blue)
OceanGIS Dapper Data Access
Tiffany Vance, Christopher Moore, Nazila Merati
CTD cast location(light blue)
Mixed-layer depth(dark blue)
Tiffany Vance, Christopher Moore, Nazila Merati
Resulting ocean profile sections analyzedwith GIS statistical analysis tools
Tiffany Vance, Christopher Moore, Nazila Merati
Summary
Dapper – OPeNDAP server for ocean observations dataDapper – OPeNDAP server for ocean observations data DChart – AJAX Web client for dapperDChart – AJAX Web client for dapper Java OceanAtlas – Desktop java client for dapperJava OceanAtlas – Desktop java client for dapper OceanGIS – uses dapper for access to ocean profilesOceanGIS – uses dapper for access to ocean profiles Other dapper clients – ncBrowse, GrADS, LASOther dapper clients – ncBrowse, GrADS, LAS
More information:[email protected]