ogc survey for dmac st

OGC Survey for DMAC ST

Luis Bermudez Director of Interoperability Certification

[email protected]

January 18th, 2011Washington DC

1

mailto:[email protected]

!5 Responses

• NASA JPL• IMOS• ASA• OBIS• EDAC• USGS• FWC• WHOI

• NOAA / IOOS• DNR – MD• NOAA/NODC• BOEING• SCCOOS• NERACOOS• OBIS-USA

NASA Jet Propulsion Laboratory (JPL)

• Presenter Name : Michelle Gierarch• The Jet Propulsion Laboratory is the lead U.S.

center for robotic exploration of the solar system, and conducts major programs in space-based Earth sciences.

• Are OGC Web Services a standard technology in your organization? I– We are trying to be compatible with OGC standards

such as WMS, WCS, and WFS, but they are not our default "standard" protocol.

OGC Relation

• Are you a member (yes or no) - YES• If so, which forums do you participate?

– YES .. But no specifics

OGC Standards in your organization

• OGC standards being used– Lunar Mapping and Modeling Project (LMMP), uses

WMS/WCS/WFS– WMS to provide rendered image(png and jpeg) from L2 and L3

data to our various tools.– WCS ( this is on its way. We have services already that are ready

to use this protocol. ) – WFS to search and provide information about natural events like

hurricane…• Planned OGC standards to be used in the next three years:

– have plan to use OGC standards where applicable on all of our web services.

Pros ans Cons of OGC standards

• PRO: - Basic profiles of various OGC standards do insure a useful compatibility level between applications

• PRO: - Use of OGC standards fosters collaboration with other users

• CON: - Small community of OGC participants leads to a proliferation of rapidly evolving specifications of little use, sometimes even overlapping ones

Title of Organization

• Presenter Name : Roger Proctor• 1 sentence about the organization: Australian

Ocean Data Network / Integrated Marine Observing System

• Are OGC Web Services a standard technology in your organization ? Yes

OGC Relation

• Are you a member (yes or no) AODN/IMOS is a member through the Australian Ocean Data Centre Joint Facility (AODCJF)

• If so, which forums do you participate? Public?

OGC Standards in your organization• OGC standards being used

– Here are some of the ways in which we use OGC web services:– We harvest metadata via WCS, WMF, WFS & SOS getCapabilities documents (for IMOS

and AODN observations).– We support CSW through Geonetwork for search access via the IMOS/AODN portal, and

potentially other clients.– SOS is also used to publish data for the South Esk sensor web project.– GeoServer is used to publish WMS/WFS (for use by the IMOS/AODN portal).– some IMOS data is published via THREDDS, which implements WMS (ncWMS) and WCS.– We will generate KML.– We use SensorML for the South Esk project, and there is talk of a sensor repository that

will support SensorML.

• Planned OGC standards to be used in the next three years:– Non-specific, see answer to next question.

Non OGC standards

• Which non OGC standards are you using and why ?– We use or consider many standards such as OPeNDAP,

REST services, OAI-PMH, StarFL, ISO metadata standards etc. Sometimes these overlap or compete with OGC standards. We try to adopt the most widely used standards (where possible). We are, nevertheless, supportive of the OGC standards and would like to see these improved and further adopted in our organisation where appropriate.

Challenges

• List of Challenges– In general, the standards don't always fit our

needs. I don't think there's and easy solution as making the standards more general isn't necessarily useful either. The OGC standards are also sometimes complicated and not consistently implemented, which reduces interoperability.

ASA

• Presenter Name : Eoin Howlett• 1 sentence about the organization:Private consulting company specializing in data

management solutions for the ocean, atmospheric and GIS community

• Are OGC Web Services a standard technology in your organization ?

Yes

OGC Relation

• Are you a member (yes or no)Yes• If so, which forums do you participate?Actively participated in NetCDF standards,

some participation in a metocean IE.


• OGC standards being used– WMS, WCS, WFS, SWE, KML, NetCDF

• Planned OGC standards to be used in the next three years:– WCS, WPS, WCPS, WMTS– OGC Web Map Service – Proposed Animation

Service Extension

Non OGC standards

• Which non OGC standards are you using and why ?

Custom JSON for responses of WMS GetFeatureInfo, Custom NetCDF conventions, Custom cataloging solutions, ESRI products (Geodatabases, shapefiles), Postgres/PostGIS, SpatialLite, and Oracle spatial databases, ERDDAP no-standard response formats.

Challenges

• Complexity and non-homogenous specifications

• Client tools to support OGC services• Performance (practicality of some standards)• Import into ESRI products a hassle. We end

up coming up with a custom solution.• Lack of testing suite to test developed OGC

services

WMS (with GetFeatureInfo)

WMS and SOS

NetCDF and WMS

OBIS

• Presenter Name : Edward Vanden Berghe (Mark Fornwall)

• An on-line, open-access, globally-distributed network of systematic, ecological, and environmental information systems


OGC Relation

• Are you a member (yes or no) - no• If so, which forums do you participate?


• OBIS web site (www.obis.org) is built around GeoServer, so has OGC services built-in.

• It provides data via WFS and WMS, and KML

Non OGC standards

• Which non OGC standards are you using and why ?– GCMD for metadata (also FGDC within OBIS USA); – DiGIR for data exchange protocol; Darwin Core for

exchange format.

Challenges

• getCapabilities: We haven’t allowed direct public access to the OGC capabilities of our GeoServer yet, as this would 1/ make it difficult for us to monitor use, and 2/ we're afraid that users are going to swamp our servers with data requests that are to broad; so we wanted to build a layer between OGC and the user, trying to log activity, and stop overly broad requests.

Challenges

• Too many layers: One of the problems we were facing while building the web site is that we have too many layers - each taxon is a layer, and there are several 1000s of those; so we can't predefine them, we have to be able to query the database on the fly. We worked with Chris Holmes and his people at The Open Plan Project to extend the functionality of GeoServer. It has worked very well.

Earth Data Analysis Center

• Presenter Name : Karl Benedict ([email protected])

• EDAC is an applied geospatial technologies center affiliated with the University of New Mexico


OGC Relation

• EDAC is a University member of OGC• Thus far we have primarily lurked in the

various forums looking for indications of where the standards are heading to facilitate our project planning.


• OGC standards being used– WMS/Time-enabled WMS, WFS, WCS, CSW, KML– Purpose

• Geospatial Clearinghouse (http://rgis.unm.edu)• Core application infrastructure: data access & exchange, web mapping• Content delivery to non-geo specialists

• Pros & Cons– Pros: broad support in a wide variety of client platforms and

programming frameworks/libraries– Cons: server and client implementations remain uneven in their

capabilities, stability, and performance. • Planned OGC standards to be used in the next three years:

– WPS

Non OGC standards• Which non OGC standards are you using and why ?

– OPeNDAP – as an alternative access model for data published through the THREDDS server platform

– OAI-PMH – metadata harvesting with other systems for repurposing of metadata also published via CSW

– X39.50 – additional catalog service protocol provided by the GeoNetwork platform being used for catalog services

– HTTP (REST) – general web services “standard” for functionality not directly supported by OGC and other standards

– ISO19115 (and related standards: 19115-2, 19139) – metadata content model and XML encoding

Challenges

• Slow uptake & understanding by end-user community => services as infrastructure that end-users don’t necessarily know is there. Need for tools and wrappers for services as the elements that clients interact with.

• Incomplete/unstable implementation by some data providers (e.g. incorrect representation of time parameters in time-enabled WMS)

• Need for expanded parameterization within KML for more efficient time support (i.e. adopt ISO model used in time-enabled WMS)

WMS / Time-enabled WMS

• Dust Forecast Products• Publication of hourly dust forecast products for the

greater Southwest region of the US in a variety of formats– Animated GIFs for direct linking in web pages– Custom web mapping clients– KML files for visualization in Google Earth

• Issues– Limited direct client support for time-enabled WMS requires

developing custom conversion tools and application interfaces

WCS

• Data exchange for distributed environmental modeling

• Publication of meteorological forecast products and low-resolution dust forecasts for use in setting boundary conditions and parameterizing high-resolution dust models

• Issues– No significant issues. As this is an automated

system, client support was not a large issue.

WMS => KML• Automated re-packaging data published as WMS into KML

for streamlined access and delivery to non-geospatial users• Developed scripted converters from WMS capabilities XML to

KML wrapper for WMS map images– XSLT for WMS -> KML– Python script for time-enabled WMS - > KML

• Issues– Temporal dimension in capabilities XML files published by some

services do not comply with OGC model– XSLT method of conversion of time-enabled WMS -> KML exposed

some recursion limitations in traditional XML processing tools

US Geological Survey

• Presenter Name : Rich Signell• USGS Mission: “Science for a Changing World”• Are OGC Web Services a standard technology

in your organization ? Yes.

OGC Relation

• The USGS is an OGC Member• Architecture DWG (Arch DWG) Lead, Doug Nebert,

US Geological Survey (USGS)• Catalog DWG (Cat DWG) Lead, Doug Nebert, US

Geological Survey (USGS)• cf-netcdf-1.0.swg: Rich Signell• Hydro Domain Working Group: Nate Booth,

Roland Viger• Sponsors OGC Interoperability Program initiatives

http://www.opengeospatial.org/projects/groups/archdwg

http://www.opengeospatial.org/projects/groups/catdwg


• OGC standards being used– WMS , WCS, SOS, WPS, NetCDF encoding

• Planned OGC standards to be used in the next three years:– CF

Non OGC standards

• Which non OGC standards are you using and why ? OpenDAP + CF Conventions , because OGC standards cannot yet represent 4D model output in a standard way.

• ESRI REST services• OBIS

Challenges

• WMS: tiling standard: is it standard yet?• WCS: not many clients, not sure how it’s

progressing. Can we progress to non-uniform data?

• WPS: would love to have a hands-on-workshop, perhaps led by USGS CIDA (Nate Booth, Dave Blodgett, Tom Kunicki)?

OGC WMS

• We use the Godiva2 web client to access extended WMS (ncWMS) all the time as a quick browse of model (and other gridded data) results

OGC WPS, WCS and OPenDAP/CF

• USGS GeoData Portal (GDP): http://cida.usgs.gov/climate/gdp/

• This is a very cool interactive climate app build by USGS CIDA folks, using WPS, WCS, OPeNDAP/CF, THREDDS

Florida Fish and Wildlife Conservation Commission

• Presenter Name : Kathleen OKeife• 1 sentence about the organization: Florida

Fish and Wildlife Conservation Commission, a state agency.

• Are OGC Web Services a standard technology in your organization ? Yes, we use several of the standards.

OGC Relation

• Are you a member (yes or no) No.• If so, which forums do you participate?


• OGC standards being used– WMS, WFS, KML, XML, REST

• Planned OGC standards to be used in the next three years:– Perhaps we will add GML

Non OGC standards


Do not know. Cannot answer for the whole agency.

Challenges

• List of ChallengesFrom a GIS perspective, the standards are an

assistance.

Martha’s Vineyard Coastal Observatory (Woods Hole Oceanographic Institution - WHOI)

• Presenter Name : Janet Fredericks

• MVCO provides real-time observations in support of sponsored research. Data are freely and openly available to the public. WHOI is a non-profit research organization primarily funded by NSF/ONR/NOAA/NASA and other federal funding, as well as private grants.

• OGC Web Services are a standard technology in our organization.

OGC Relation

• WHOI has recently become an associate member (NPO)

• If so, which forums do you participate?


• OGC standards being used (for example)– MVCO – Sensor Web Enablement– BCO-DMO - WMS– Upper-Ocean Processes Group –

NetCDF/THREDDS• Planned OGC standards to be used in the next

three years:

Non OGC standards

• Which non OGC standards are you using and why ? W3C – OWS/RDF

For example: The SensorML description files reference OWS ontologies (registered terms). This enables data aggregation centers to map our definitions to others (for example my spiketest to your outlier or my qcflag-good (value=0) to your good-flag (value=3)

Challenges• List of Challenges – time-consuming and tedious metadata Most standards implementations are geared towards discovery and transport of observations. Implementation of data quality assessment capabilities tends to make descriptions complex, creating a hurtle that no one seems willing to take – we need to develop tools that automate the generation of metadata in a standards-based (OGC) framework and enable the discovery and assessment of meta-data, along with observational data delivery. Why aren’t we grasping this unprecedented opportunity to provide a solid foundation in our evolving earth observing systems?

Sensor Web Enablement (1.0) SensorML for DescribeSensor

O&M for GetObservation

• Purpose: enable dynamic quality assessment of real-time oceanographic data

• Noteworthy: http://q2o.whoi.edu/node/129 model for the description of how an observation came to be from observed properties including: capabilities, characteristics and provenance of sensor(s), deployment (s) and processing (process descriptions and qc tests and results).Next steps:• Need to infuse/harmonize ISO 11915-2 and update to SWE 2.0 • what we call content-rich – other’s call complicated! Cultural hurdle!• Need to get someone (NDBC?) to ingest and utilize the capability (The

capability needs to be exercised.)

http://q2o.whoi.edu/node/129

NOAA / IOOS

• Presenter Name : Derrick Snowden• 1 sentence about the organization: Interagency

effort to develop a system of systems for end to end management of ocean information addressing data collection through product delivery.

• Are OGC Web Services a standard technology in your organization ? Emerging standard and usage is increasing but not yet operational.

OGC Relation

• Are you a member (yes or no) Yes (NOAA is the member)

• If so, which forums do you participate?– SWE WG – SensorML WG


• OGC standards being used– SOS primarily, also WMS/WCS– SWE/O&M– CF-netCDF, netCDF-4 Classic

• Planned OGC standards to be used in the next three years:– CS/W,

Non OGC standards

• Which non OGC standards are you using and why?– We use technologies based upon NetCDF but

which themselves are not OGC standards– Global Telecommunications System, operational

mandate.

Challenges• List of Challenges

– New and unproven technologies require extensive experimentation and concept testing

– Lack of deployed systems to judge – Complex and interwoven standards are often difficult

to interpret and understand, again, more pragmatic experience and documentation. OGC Standards, particularly those written in the new style, are not easy to read and understand the big picture. How are these useful to me today?

– Lack of clients, particularly those that allow OGC web services to be used from within common tools.

OGC Standard (and versions)• SOS 1.0 (GML, O&M 1.0, SensorML1.0.1, moving toward SWE 2.0, )• Purpose (data type): Web Service providing access to, primarily, in

situ ocean data (moored buoys, gliders, shore stations, vertical profile or sounding data)

• Noteworthy (cool things you have done with it)• Issues: Many, primarily listed on challenges page. Difficulty in

implementation also inhibits testing for some of the other issues that will likely pop up in the future.– What are the throughput limitations (kbps) and are any inherent in the

service or completely related to implementation?– How can we encourage interaction with instrument providers? SensorML

direct from the sensor or manufacturer etc?– Need for lightweight profiles, but not too light.

Maryland Department of Natural Resources

Maryland Department of Natural Resources

• Presenter Name : Mark Trice• 1 sentence about the organization:

– The Department of Natural Resources leads Maryland in securing a sustainable future for our environment, society, and economy by preserving, protecting, restoring, and enhancing the State’s natural resources.

• Are OGC Web Services a standard technology in your organization ? NO

OGC Relation

• Are you a member (yes or no) -NO• If so, which forums do you participate?


• OGC standards being used– None

• Planned OGC standards to be used in the next three years:– Maybe .. By making our data available through

the MARACOOS data assets map/portal

Non OGC standards

• We collect nearly 10M records of tidal water quality data each year. Much of this data is a deliverable to the EPA Chesapeake Bay Program and the submission format is via a web upload of monthly data sets in a EPA format that dates back to the 1980’s. Some of our newer data via automated data sondes is made available through our website www.eyesonthebay.net, but currently is not available via services.

NOAA/NODC

• Presenter Name : Krisa M. Arzayus• 1 sentence about the organization: The

Nation’s archive for oceanographic data• Are OGC Web Services a standard technology

in your organization ?Yes. Most of the software for data access that

are standard part of NODC operations like THREDSS, Hyrax and now Geoportal incorporate OGC web service standards.

OGC Relation

• Are you a member (yes or no)• If so, which forums do you participate?Yes, NOAA is a member of OGC (see

http://www.opengeospatial.org/ogc/members)NODC follows most of the forums.

http://www.opengeospatial.org/ogc/members


• OGC standards being used– WMS, WCS, CSW, KML and also Filter

Encoding(this is really part of other standards)• Planned OGC standards to be used in the next

three years:– NetCDF-CF and Sensor Observation Service (SOS) ?

Also, DAP if it becomes OGC standard.

Non OGC standards• Which non OGC standards are you using and why ?Probably the most important standards that we use are http and ftp since I guess

users understand these the best. Most of the web service standards are built on top of these standards. Example, OpenDAP. We use OpenDAP, since it is designed for accessing scientific data – grids & sequences, which is what we have.

As far as data format standards, we use NetCDF/HDF which are sort of becoming de-facto standards for complex data. But NODC does not impose any restriction on the originator format and so we probably have data in too many standard formats. We don’t necessarily “use” the standards.

All the code lists and digital standards for Oceanographic data developed by oceanographic community. I don’t know all of them.

We use the Open Archival Information System standard (OAIS) to model our archive on.

Last, but not the least, FGDC and ISO standards for metadata representation.

Challenges

• List of ChallengesRapid evolution of some of the OGC standards poses a

problem for adoption.Standards for observational data is somewhat lacking

with OGC, at least until recently. OGC standards for vector data are too complex for observational data which are generally just point data.

OGC Standard (and versions)

• Purpose (data type)• Noteworthy (cool things you have done with

it)1) NODC Geoportal2) Environmental Response Management Application Gulf Response (to Deepwater Horizon)- it is a web-based GIS tool to assist emergency responders and resource managers.http://gomex.erma.noaa.gov/

• Issues

http://gomex.erma.noaa.gov/

BOEING

• Presenter Name : Steve Uczekaj and Rick Blair• 1 sentence about the organization: Boeing• Are OGC Web Services a standard technology

in your organization ? – YES

OGC Relation

• Are you a member (yes or no)– Yes

• If so, which forums do you participate?


• OGC standards being used– SOS– WMS

• Planned OGC standards to be used in the next three years:– WMTS ..

Non OGC standards

• Which non OGC standards are you using and why ?– ERDDAP

• Efficiency for Large Data Sets• Data Aggregation from different sources

– Thredds• Handles Large Data Sets• Streaming protocol• Wide spread use

Challenges

• List of Challenges– Data Encoding Standards – Biggest one from My

perspective. No Standard way to encode the data from SOS

– Need a way to define the interface to SOS service. • What Data Encoding• What semantics for specific operations• What semantics fo “standard” data fields• Etc.

– Need a ‘Streaming” type of data service suitable for large data sets such as history, profiles, and transects.

Southern California Coastal Ocean Observing System (SCCOOS)

• Presenter Name : Julie Thomas (SCRIPPS)• SCCOOS brings together coastal observations in

the Southern California Bight to provide information necessary to address issues in climate change, ecosystem preservation and management, coastal water quality, maritime operations, coastal hazards and national security.

• Are OGC Web Services a standard technology in your organization ? Depends on the data

OGC Relation

• Are you a member (yes or no) – YES? Via UCSD

• If so, which forums do you participate?– UCSD participate in various forums


• OGC standards being used– Those supported by THREDDS and NetCDF to

serve HFR as one of the DACs

Non OGC standards

• Which non OGC standards are you using and why ?– For Wave Data we used an NDBC XML file via FTP.

Challenges

• No standard for NetCDF wave spectra. ERDAP, NODC and CDIP have their own formats.

NERACOOS

• Presenter Name : Eric Bridger• Northeast regional IOOS component.• Are OGC Web Services a standard technology

in your organization ? Yes. As part of the IOOS and preceding IOOS we have been using OGC standards for quite a few years.

OGC Relation

• Are you a member: Yes. Via Gulf of Maine Research Institute.

• If so, which forums do you participate? SOS, SWE, OWS, WMS


• OGC standards being used– WMS– SOS– SWE 2.0– WFS– WCS– KML

• Planned OGC standards to be used in the next three years:– WMTS– SOS 2.0

Non OGC standards


• THREDDS• NetCDF• EPA Exchange Network

Challenges

• List of Challenges– Evolving changes to standards, keeping abreast– Community agreements on which to utilize– Lack of tools for non-technical implementors

OBIS-USA

• Presenter Name : OBIS-USA• 1 sentence about the organization: OBIS-USA

aggregates marine biogeographic data (presence-absence-abundance and tracking) from US gov, university, museum and non gov sources, and serves this data interoperably in standard formats for applications.


OGC Relation

• Are you a member (yes or no) No• If so, which forums do you participate?


• OGC standards being used– WMS (GeoServer-based)

• Planned OGC standards to be used in the next three years:– WFS

Non OGC standards

• Which non OGC standards are you using and why ?– FGDC/ISO– GCMD– Darwin Core– ERDDAP (OpenDAP)– DiGIR/Tapir

Challenges

• List of Challenges– one challenge is simply staff time to implement and maintain

services– a secondary challenge is server performance: some services

consume a lot of CPU to generate results; or consume a lot of storage space for cache

• Less challenging:– software cost is rarely a challenge because solutions are open-

source– given some configuration and management cost, source data and

OBIS-USA standards/processes make data compatible with geospatial services such as WMS

WMS

• Purpose (data type) Serving Marine BioGeographic occurrence data from many different datasets in a standard interoperable format

• Noteworthy (cool things you have done with it) Providing data from 150 datasets to Federal data aggregators/portals (Data.gov) without having to repeat 150 different configurations; a single configuration in WMS was able to serve all OBIS-USA standard data

• Issues We are evaluating additional detail from OBIS-USA that we would like to serve. We may need to implement WFS in addition to WMS; we are uncertain until we do the detailed implementation if this will satisfy all requirements.

Next Steps

• [LB] Summarize the challenges related to OGC standards and action items related to those challenges.

• [LB] Send it to the DMAC ST for review• [LB] Follow up on action items.

ogc survey for dmac st

Documents