ogc-epri conflation webinarogc ® difficulty of conflation depends on many factors • complexity of...

®

OGC-EPRI Conflation Webinar

Carl Reed, PhD CTO

Open Geospatial Consortium October 16, 2014

© 2013 Open Geospatial Consortium

OGC ®

Copyright (c) 2014 Open Geospatial Consortium http://www.opengeospatial.org.

Who am I?

• I am a geographer who could not get the hang of manual cartography so in 1969 I started to use computers to make maps and have been a geospatial tech geek ever since.

OGC ®

The Power of Geography

• The great integrator, the power of map overlay, mashups, etc

© 2014 Open Geospatial Consortium 3

geosphere.gsapubs.org

Lancaster University

Bermuda Conservation

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=y2y4qSXK2FqPVM&tbnid=l1nluMzjsngNeM:&ved=0CAYQjB0&url=http://geosphere.gsapubs.org/content/2/4/236/F2.expansion.html&ei=o-ouVO2-KoKOyASvmILwBQ&bvm=bv.76802529,d.aWw&psig=AFQjCNETN00ngnsgSz89bLtm7YcbhS8gfg&ust=1412447216020726

OGC ®

However

• Different spatial reference systems • Different Accuracies • Different Scales • Different semantics • Different classification systems • Incompleteness of source materials • Collected for different use cases • And the list goes on


OGC ®

The “simple case”

• Use case for emergency response: Create a seamless digital map of the transportation network between two counties in two adjacent states – Different road classifications – Different street names – Edge match issues – Scale differences


OGC ®

And another simple case

• Same theme but from two different sources – conflate the two datasets to create a higher quality product.


University of California Santa Barbara

OGC ®

Conflation: Definition

• The process of unifying two or more separate datasets, which share certain characteristics, into one integrated all-encompassing result (OGC)

• Combining map data from separate sources to create data that is better than either source on its own (OpenStreetMap wiki)

• The process of combining geographic information from overlapping sources so as to retain accurate data, minimize redundancy, and reconcile data conflicts. – Longley, Paul A. et al (2001). Geographic Information Systems and

Science


OGC ®


OGC ®

Many aspects of conflation

• Vertical

• Horizontal – Edge match

• Attributes – Semantics – Vocabularies – Ontologies – Data models

• Visualization


OGC ®

Difficulty of conflation depends on many factors

• Complexity of representation • Volume and accuracy of the datasets involved. • Specifically, incompleteness and inaccuracy of the original

datasets, different reference systems, distinct generalizations and representations of reality, semantic issues of terminology and classification, various scales, and different purposes, as well as various time frames – Goodchild and Raubal, 2009


From Geoinformatics (Digitális Tankönyvtár) www.tankonyvtar.hu

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=qPhdAiX5pjp3OM&tbnid=rbfXV19dQDRkiM:&ved=0CAYQjB0&url=http://www.tankonyvtar.hu/hu/tartalom/tamop425/0032_terinformatika/ch02.html&ei=pOQ2VMsFw6_IBNeFgbgJ&bvm=bv.76943099,d.aWw&psig=AFQjCNF8SOU4RF29oTI7KFRjtruzi9Gtuw&ust=1412970012949390

OGC ®

Solution requires

• Metadata!

• Agreement on conflation rules – Coordinate reference system – Quality – Feature matching – Geometry matching – Attribute Matching – Selection rules – Accuracy


®

OGC Standards Relevant to Conflation


OGC ®

OGC Web Services Standards

© 2014, Open Geospatial Consortium

Web Map Service (WMS) Web Feature Service (WFS) Web Coverage Service (WCS) Catalogue (CSW) Geography Markup Language (GML) Web Processing Service (WPS) Web Coverage Processing Service GeoSPARQL

The GeoWeb is enabled by OGC standards:

The Geospatial Web is about the complete integration and use of location at all levels of the internet and the web.

Dr. Carl Reed CTO OGC

OGC ®

Copyright (c) 2014, Open Geospatial Consortium 14

OGC Web Map Service (WMS) Standard

• Simple HTTP interface for requesting geo-registered map images from one or more distributed geospatial databases. A WMS request defines the geographic layer(s) and area of interest to be processed. The response is one or more geo-registered map images (returned as JPEG, PNG, etc) that can be displayed in any browser application.

– Thousands of implementations

• Easy to implement – http://clearinghouse1.fgdc.gov/scripts/ogc/ms.pl?version=1.1.1&

request=map&srs=EPSG:4326&bBox=-180,-90,180,90& width=400&height=200&format=JPEG&styles=BLACK& layers=boundary,coastline,elevation,lakes,rivers&

• An ISO Standard

Presenter

Presentation Notes

Slide 21: The OpenGIS Web Map Server Specification (WMS) is a set of interface specifications that provide uniform access by Web clients to maps rendered by map servers on the Internet. WMS supports overlay of image files to create a composite map. For example, a WMS client may “getMap” from multiple WMS servers using the same: bounding box (bBox) Spatial Reference System (srs) output size (width and height) Use of the “transparency” parameter is used to control the ability to display a picture of a road map on top of an satellite image.

OGC ®

Copyright (c) 2014, Open Geospatial Consortium 15

Web Feature Service Interface Standard

• Specifies the behaviour of a service that provides transactions on and access to geographic features in a manner independent of the underlying data store. The standard specifies discovery operations, query operations, locking operations, transaction operations and operations to manage stored parameterized query expressions.

http://www.ibm.com/developerworks/data/library/techarticle/dm-0810kleoppel/

http://nationalmap.gov/framework.html

Presenter

Presentation Notes

The Web Feature Server Interface Specification (WFS) describes data manipulation operations on OpenGIS Simple Features (e.g., points, lines, and polygons) so that servers and clients can communicate at the feature level. A WFS request – like those supported in many GIS and RDBMS packages – consists of a description of the query and data transformation operations that are to be applied to WFS enabled spatial data warehouses on the Web. The request is generated on the client and is posted to a WFS server. The WFS Server reads and executes the request, returning the result in a feature set as GML. A GML enabled client then can use the feature set.

OGC ®

Multiple thematic

data layers

GetFeature request:

Web Feature Service (WFS) gets operable feature data from multiple servers

Cities

Borders Elevation

Each layer is data, not merely a view:

Country is: _ Name: Italy _ Population: 57,500,000 _ Area: 301,325 sq km . . .


OGC ®

Geography Markup Language: Representing Geographic Features

Another Information Community’s Schema Highway is: _Pavement thickness _Right of way _Width …. Cell transm. Platform is: _Location _No. of antennas _Elevation ….

One Information Community’s Schema Road is: _Width _Lanes _Pavement type …. Cell tower is: _Owner _Height _Licensees ….

(an instance of Road in one IC’s schema)

Mayberry’s Cell Tower

(an instance of Cell Transm. Platform in another IC’s schema)

Mayberry Road

GML Support for complex geometries, spatial and temporal reference systems, topology, units of measure, metadata, feature and coverage

visualization. Backward compatible

GML defines a data encoding in XML that allows geographic data and its attributes to be moved between disparate systems Version 3.3 advances interoperability on all fronts!!


OGC ®


Web Processing Service 1.0

• Designed to standardize the way that geospatial calculations, such as polygon overlay, are made available to the Internet. WPS can describe any calculation (i.e. process) including all of its inputs and outputs, and trigger its execution as a Web Service.

• Supports simultaneous exposure of processes via GET, POST, and SOAP

• Integrated with Hadoop and MapReduce for Big Data analytics

OGC ®

OGC Web Processing Service (WPS)

WPS GetCapabilities Execute DescribeProcess

Algorithms Repository

…

…

Algorithm 1

Data Handler Repository

…

…

Data Handler A

Communication over the web using HTTP

WPS-client

Web Processing Service


OGC ®

Simple Symbology Conflation Display Feature Data with different Symbols

…

Fire Incident

Commercial Facility Fire

Forest Fire

Grassland Fire

Hotspot Fire

Unknown Friendly Neutral Hostile Violent Activities: Arson Fire

Map Viewer Client

Map Viewer Client

Features (GML) Maps (GIF,PNG,JPG) Metadata (XML) Styles (SLD), Symbols (CGM,SVG)

Emergency Management Data Sources (Regional, International, National, State, Local)

Transportation

Cadastral

Incidents

WFS WMS CSW CSW

Intelligence

Critical Infrastructure

Population

Cultural Features

Env. Conditions

Emergency Management Symbol Sets

Emergency Management

Maps

User Community “A” User Community “Y”


®

OGC Activities Related to Conflation


OGC ®

The OGC Interoperability Program (IP)

• A global, collaborative, hands-on engineering, prototyping and testing designed to rapidly deliver – Running code implementations – Engineering Reports – Change Requests – Demonstration in real world scenarios

• Sponsors and Participants work together. – Sponsors provide requirements, use / business cases and funding

– Participants work with sponsors to define and/or refine standards to solve a given interoperability problem

Copyright © 2014 Open Geospatial Consortium

OGC ®

Types of OGC-IP Initiatives

OGC Testbed

OGC Interoperability

Experiment

OGC Pilot

Purpose Develop new standards & refine existing specs

Refine & extend existing standards

Test existing standards in operational environment

Project Management

OGC IP Team OGC Members OGC IP Team

Sponsorship Yes No Yes

Participation OGC Members OGC Members & approved non-Members

Members & operational partners

The OGC Interoperability Program (OGC Document 05-127r5) http://portal.opengeospatial.org/files/?artifact_id=45656


OGC ®

GOS-TP Goal & Objectives Completed 2002

• Implement a vertical slice of GOS, testing the assumptions made in the broader GOS objectives using the Road Transportation theme.

• Primary Objectives: – Test assumption that Unified

Modeling Language (UML) model can be successfully expressed as a physical schema

– Test assumption that Federal, State, Local, and/or Tribal data can be translated to a schema different from their local, native schema

– Pilot a portal-based system implementing metadata, data access (including translation/semantic mediation), and web mapping

UML Model

UGAS Tool

GML Application Schema (SchemaGlobal)

GML Application Schema (SchemaLocal)

Oregon Node

California Node CIPI/Other

Nodes

“User” Site

DOT Portal Node

Jackson County Node

Siskiyou County Node

OGC ®

GOS-TP Operational Context

Oregon Node

California Node

CIPI/Other Nodes

“User” Site

DOT Portal Node

Jackson County Node


Client Browser

Registry & Portrayal Services

Data Services

OGC ®

DOT Portal Node

GOS-TP Technical Architecture

Oregon Node

WFS-X

WMSSLD

Jackson County Node

WFS-X

WMSSLD


WFS

WMSSLD

WOS

California Node

WFS

WMSSLD

WRS WRS CG

WMS CG

WFS CG

WOS CG

User Browser

WFS Web Feature Service WFS-X Web Feature Service, Translating WMS Web Map Service SLD Styled Layer Descriptor WRS Web Registry Service WOS Web Object Service

Acronyms

OGC ®

OWS 5 Conflation Study

• Describes the process of conflation, outlines a framework for conflation and conflation rules services within a service oriented architecture, and describes the implementation of conflation services during the testbed


OGC ®

Semantic Mediation OGC Web Services Phase 9 Test bed (2012)

California National Guard

Monterey Airport Field operator

NGA model - Local Topographic

Data Store (LTDS)

Prefers USGS model - The National Map

(TNM)

Prefers

Mediate


OGC ®

Semantic Mediation Operational Prototype Development in OWS-9



OGC ®

OWS-9 Cross Community Interoperability (CCI) Conflation with Provenance

• Describes the architecture of a WPS capable of conflating two datasets while capturing province information about the process. Also provides information about defining and encoding conflation rules and about encoding provenance information using the W3C PROV recommendation.


OGC ®

OWS-10 Cross-Community Interoperability (CCI) 2013

• Increase Geospatial community interoperability by building on CCI OWS-9 work in semantic mediation, volunteer geographic information (VGI), provenance and data quality, and Global Gazetteer. Explore the potential of interoperability in the hydrology domain utilizing semantic mediation and ontologies to more easily share and visualize geospatial data.


OGC ®

Virtual Global Gazetteer (attribute conflation)

• The access to linked data information provided by these gazetteers was achieved by GeoSPARQL enabling these gazetteers using semantic mapping components (provided by Image Matters LLC) mapping RDBMS and WFS data to knowledge representation (RDF) on the fly


Presenter

Presentation Notes

The Virtual Global Gazetteer integrates two gazetteers – a copy of the USGS gazetteer containing domestic names and a copy of the NGA gazetteer containing non-domestic names (hosted by Interactive Instruments) and provides the capability to link to additional local gazetteers and linked data information, allowing a user to retrieve extended information on locations selected from either of the initial gazetteers. The access to linked data information provided by these gazetteers was achieved by GeoSPARQL enabling these gazetteers using semantic mapping components

OGC ®

Cross Community Interoperability (CCI) Hydro Model Interoperability

• Defining best practices for supporting interoperability among the National Hydrographic Network (NHN) of Canada, the National Hydrographic Dataset Plus (NHD+) of United States, and the OGC HY_Features model developed and proposed by the World Meteorological Organization (WMO)


Hydro Model Service

Hydro Data Services

Xlink-enabled Mediation Service

OWL-enabled Meidation Serivce

WS

DL

(Web

se

rvic

e op

erat

ions

)

SP

AR

QL/

Geo

SP

AR

QL

endp

oint

Hydro Mediation Services

OG

C W

PS

FCU.GIS Hydro Model Web Test

ClientPyxis

NHNSQ

L

NHD+SQ

L

NHN

OG

C W

FS

NHD+

OG

C W

FS

Retrieval /Storage

Transaction

Mediation (xlink inference

/semantic mediation)

Hydro Modeling Clients

Invocation /Data retrieval

Data retrieval

Data retrieval

Presenter

Presentation Notes

Data models used by different hydrology modelling applications were investigated for semantically data integration by using a common hydrologic feature model – HY_Features

OGC ®

Architecture


®

An example of non-OGC Conflation Research and

Activities


OGC ®

Widespread sharing and seamless integration of distributed urban geospatial data: Los Angeles test bed


Presenter

Presentation Notes

A Geospatial Cyberinfrastructure for Urban Economic Analysis and Spatial Decision-Making Wenwen Li 1,*, Linna Li 2, Michael F. Goodchild 2 and Luc Anselin 1 In this paper, we report on our efforts to design and develop a geospatial cyberinfrastructure (GCI) for urban economic analysis and simulation. This GCI provides an operational graphic user interface, built upon a service-oriented architecture to allow (1) widespread sharing and seamless integration of distributed geospatial data; (2) an effective way to address the uncertainty and positional errors encountered in fusing data from diverse sources; (3) the decomposition of complex planning questions into atomic spatial analysis tasks and the generation of a web service chain to tackle such complex problems; and (4) capturing and representing provenance of geospatial data to trace its flow in the modeling task. The Greater Los Angeles Region serves as the test bed.

®

Where could EPRI and OGC collaboration have significant

impact on conflation?


OGC ®

Smart Cities/Energy

• Use Case: Many Smart City and Smart Energy workflows require integration of crowd sourced, in situ-sensor, and other near real time sources of new content (satellite or UAV) with existing geospatial data sources to enhance decision making and provide real time situational awareness. – Response to natural disasters – Reduce energy consumption – Increase effectiveness of planning and maintenance – Integration of alternative energy sources – Etc.


OGC ®

OGC Interoperability Initiatives

• Rapidly capture requirements

• Marshall technology resources (EPRI and OGC members, data, software, etc)

• Agile and rapid development of “demonstratable” prototypes

• Engineering reports, webinar, videos, etc

• All in 6 to 9 months


®

Example use cases from the Energy Sector


OGC ®

Solar Energy Production Potential Analysis

Source : LGV Hamburg, Fa. simuPLAN

Presenter

Presentation Notes

OGC ®

Electric Car Charging, Austria


OGC ®

Wind data visualisation example with UAV camera recorded ground overlay

OGC ®

3d and sensors Common Scents – City Sense

• Workflow of crowd sourced sensor input, Open Street Map, data fusion, and modelling that generates a 3d visualization. Uses numerous OGC standards


Presenter

Presentation Notes

They implemented the real-time data integration component in a custom datastore for the open-source server GeoServer 4 . This solution offers two main advantages: at first, data are fused on-the-fly in a highly dynamic, fast and parallelised process. At second, GeoServer provides standardised WFS, WMS and WCS outputs, as described above, which allows for simple integration into analysis and visualisation software. After having set up the infrastructure described above, they implemented two GIS analysis mechanisms. The first analysis component performs a spatial Inverse Distance Weighting (IDW) interpolation on temperature measurements, which will be used in further research efforts for correlation operations with emission distribution or traffic emergence, and for the detection of urban heat islands. The basic heat map of the GPS traces and the output of the interpolation process – a navigable 3D map – are shown in Figure 4.

OGC ®

Netherlands: Energy Neutral Cities project

• OGC CityGML & NetCDF for energy neutral cities – http://www.3dpilot.nl/?p=92


Presenter

Presentation Notes

Urban areas are registered as a collection of objects carried by vectors in city information models like CityGML. The geological (geotechnical, geohydraulic, geochemical, geophysical) subsurface as a continuous medium is normally modelled on a regular (in oil and gas also on non-regular) raster. Raster data can be conveyed by means of a.o. in NetCDF data format. The data are hooked up to the raster points or are centered between raster points. The properties on the raster points are considered representative for box shaped volumes or voxels.

®


Thank you for your attention

Carl Reed [email protected]

ogc-epri conflation webinarogc ® difficulty of conflation depends on many factors • complexity of...

Documents