restful ogc services
DESCRIPTION
Presentation to the OpenGeoSpatail Consortium Technical Committee Meeting In Boulder Colorado, 9/17/2007TRANSCRIPT
DRAFT Aug 13, 2007
RESTful OGC Services
User-Centric Resource Oriented Architecture (ROA) Approach
Pat Cappelaere, [email protected] Derezinski, [email protected] Mettraux, [email protected] Swenson, [email protected] Turner, Mapufacture
1
Version 4
OGC Services
REST (Classic)GetCapabilitiesGetFeasibilitiesGetObservationDescribeTaskingSubmit...
SOAPful RESTful
RPC/HybridMandatory
Option
Why?
[G]EO-Web 2.0GEO - GEOSS
Movie http://blog.geobliki.com
[G]EO-Web 2.0GEOSS Use-Case South Africa
Transmission Power Grid (Red)
[G]EO-Web 2.0
Fire Season MODIS Fires (every 4 hours)Population Overlay
Power Grid
[G]EO-Web 2.0Subscription
Notify Me: PowerLineThreat within 10km (6nm)
in my area of interest (BBOX)use Twitter, IM and SMS
[G]EO-Web 2.0Alert
User Receives AlertPowerLineThreat within 10km (6nm)
in BBOXuse email, Twitter, IM and SMS
User clicks on url
[G]EO-Web 2.0Feed Publication / Aggregation
[G]EO-Web 2.0KML Feed Visualization in GE
Threat:Fire at 3nmlat=xxxlong=xxx
Do You Want to Image?
[G]EO-Web 2.0
Custom Data Product To The DeskTop
[G]EO-Web 2.0Problem Statement
• Many National & International Organizations
• Scarce Resources
• Money
• Time
• Engineering Resources
• We Need All Of the OGC Services
Market Positioning
RESTful SOAPful
Web Services
Service Oriented Architecture
Market Big IT InfrastructureNO IT Infrastructure
Mass Market Corporate 100 America
Users Neo-Geographers Corporate Developers/Engineers
Capability 80% 100%
Cost 10% 100%
Time 10% 100%
Risk Low HIgh
Technical Differences
RESTful SOAPful
Discovery
Expose Resourcesand Attributes
Expose Functionsand Inputs/Outputs
WADLAPP service Document WSDL
Spec Size Very small if existent Very Large
Everything is a URL
Language Java, C++. .NETHtml / Javascript / Ruby
Framework Ruby on-Rails EJB
Schemas non-existent extensive
Security WS-Security (Geo-DRM)OpenID/PKI, OpenAuth...
SOAP - WSDL ...
xsd wsdl
common 0.3.20 9
context 1 2
csw 2.0.2 6 6
ebxml-regrep 3 1
filter 1.2 4
gml 3.2 94
hma 0.95 5
ols 1.1 11
ows 1 9
phr 0.81 2
sld 1 1
swecommon 1.0.3 2
wcs 1.0.2 7
wfs 1.2 2 5
wms 1.3 3
wrs 1.0. 4
wsdl 4
xlink 1.0. 1
xml 2
Total 165 15
Unique Opportunity
For System-Level Architectural Change
RESTful Services For ROA
• http://www.oreilly.com/catalog/9780596529260/index.html
HTTP with Five Operations: HEAD, GET, PUT, POST and DELETE
17
“to codify the folklore, define what’s been left undefined, and try to move past the theological arguments.”
Everything is a url!
Other Technologies For Successful ROA
• Publishing Formats: GeoRSS/Atom, KML
• Atom Publishing Protocol (APP)
• GData
• OpenSearch
• Security
Resource URI Patterns and Nested Resources
• Resource Collections Are Plural
• GET /recipes
• GET /recipes/{id}
• GET /recipes/{name}
• Nested Collections (recipes have many ingredients)
• GET /recipes/{id}/ingredients/
• GET /recipes/{id}/ingredients/{id}
• same as: GET /ingredients/{id | name}
Retrieve all available ingredients for that particular recipe
Retrieve specific ingredient for that particular recipe
Retrieve specific ingredient by id or name
Note: Last 2 GET’s return the same data. BUT first one keeps track of context. Try to backtrack the URL to see what you would get
Retrieve all available recipes
Retrieve specific recipe by id
Retrieve specific recipe by name
19
APP: Atom Publishing Protocol Format
• The Atom Publishing Protocol is an application-level protocol for publishing and editing Web Resources using HTTP [RFC2616] and XML 1.0 [REC-xml]. The protocol supports the creation of Web Resources and provides facilities for:
* Collections: Sets of Resources, which can be retrieved in whole or in part. * Services: Discovery and description of Collections. * Editing: Creating, editing, and deleting Resources.
draft-ietf-atompub-protocol-17.txt
http://bitworking.org/projects/atom/draft-ietf-atompub-protocol-17.html
Geo/Atom In - Geo/Atom Out
Google GData
• The Google data APIs ("GData" for short) provide a simple standard protocol for reading and writing data on the web.
• GData uses either of two standard XML-based syndication formats: Atom or RSS. It also has a feed-publishing system that consists of the Atom publishing protocol plus some extensions (using Atom's standard extension model) for handling queries (including geoqueries)
• Items of type Products containing the terms digital and camera in the area of San Francisco: digital camera [item type: products] [location: @"San Francisco, CA" + 50mi]
snippets/-/products?bq=digital+camera+%5blocation:@%22San+Francisco,CA%22%2b50mi%5d
OpenSearch
• OpenSearch 1.1
• OpenSearch description document• Discoverable by Browser• Geo Extension Draft
http://www.opensearch.org
Security
• OpenID 2.0 + Auth or PKI
• OpenAuth (AOL)
• That’s another topic of discussion!
Goal
Consistent User Experience Across All OGC Services
What OGC Services Are We Looking At?
• Workflow Chaining Service (WfCS)
• Sensor Planning Service (SPS)
• Web Processing Service (WPS)
• Web Coordinate Transfer Service (WCTS)
• Web Feature Service (WFS)
• Web Notification Service (WNS) or OGC Publish/Subscribe (OPS-B)
• Sensor Observation Service (SOS)
• WCS (Steven Keens, PCI) 25
WfCS
• It Is The Most Complex Service, Let’s Do It First
• It Requires Asynchronous Services
• Patterns Can Be Used For SPS, WPS And WCTS
• Similar Patterns Can Be Inferred For WFS & SOS
• It Can Use WNS Or OPS-B
26
WfMC Reference Model
27
Workflow Chaining Service (WfCS)
XPDL 2.0:
Serialization Mechanism for the Business Process Modeling Notation (BPMN 1.0)
http://www.wfmc.org/
28
Workflow Interoperability
BPMN 1.0XPDL
2.0
BPEL
SensorML
WorkflowEngines
API
Level 0 Level 1WfCSLevel 2
ModelPersistenceLayer
GUILayer
WorkflowInternalRepresentation
OpenWFE
Offline Real-time
WfCS Resources
• Users Are Resources, OpenID is Their URI.
• Note: You Always Know Your User and Have Access to Her Profile Information
• Users Create Workflows.
• Each Workflow Can Have one or more Definitions (or version).
• A Workflow Definition Orchestrates Activities Handled By Participants
• A Specific Workflow Is Instantiated (or Launched).
• A Particular Instance Has Traces (history) Related To Its Various Activities (start/end).
29
Think of a Workflow as a blueprint that goes through the versioning process as time goes by. A Workflow is a mere entry in a registry to maintain its name consistent over time. The definitions are the modified versions of the flow as designed by the engineer and uploaded to the server.These different versions can be enabled, disabled and eventually deleted.
Another to think of it is that a workflow is akin to a catalog entry with one or more definitions attached to it as different versions
WfCS Resources Are:
•Users
•Workflows
•Definitions
• Instances
•Activities
•Traces
•Participants
Discovery
• WADL
• APP Service Document
• http://geobpms.geobliki.com/wfcs/app
Web Application Description Document: WADL
https://wadl.dev.java.net/
GET /wfcs.wadl
http://weblogs.java.net/blog/mhadley/archive/2005/05/introducing_wad.html
Returns the WADL document for workflow resource
32
REST Equivalent of OGC GetCapabilitiesor WSDL
Resource Metadata
• HEAD /wfcs/workflows
• GET /wfcs/workflows.metadata
• GET /wfcs/instances.metadata
• GET /wfcs/definitions.metadata
See Google GData
33
Retrieving Workflows
• GET /wfcs/workflows
• GET /wfcs/workflows.atom
• GET /wfcs/workflows/11
• GET /wfcs/workflows/get_sa_threats
Retrieve all available workflows
Retrieve all available workflows in atom output
Retrieve specific workflow by id
Retrieve specific workflow by name
34
Threat Generation Flowchart
AcquireMODIS
GenerateThreats
GetCloud
Coverage
GetFeasibilities
EO-1SPS
ASTERSPS
SPOT-5SPS
PublishThreats WFS-B
GeneratePolyons
BPMN Workflow
Workflow Definitions
• Uses XPDL 2.0 [generated from an external GUI]
• GET /wfcs/workflows/get_sa_threats/definitions
• GET /wfcs/workflows/get_sa_threats/definitions/v1_0
• To create/upload a new definition to be versioned:
• POST /wfcs/workflows/get_sa_threats/definitions
With XML Payload
37
Starting Workflow Instance
POST /wfcs/workflows/get_sa_threats/instances
<entry xmlns:g='...' xmlns='http://www.w3.org/2005/Atom'> <title>Get SA Fire Threats</title> <category term='sa,fire,threats' scheme='http://geopbms/1.0'/> <content>Get South African Fire Threats workflow</content> <g:item_type>instances</g:item_type> <g:command>start</g:command> <g:context_data> <scenario>2</scenario> <all_threats>data/sa_all_threats.kml</all_threats> <final_threats>data/sa_final_threats.kml</final_threats> <feed_file>data/sa_threat_feed_file.xml</feed_file> <send_to_draper>0</send_to_draper> </g:context_data></entry>
Launch specific workflow by name, and use latest enabled version
38Instance Location is returned
Dealing With Workflow Instances
• GET /wfcs/instances.atom
• Returns Atom Feed
• Returns Instance Attributes, Status, ContextData and ResultData
39
<entry> <title>Get SA Fire Threats</title> <link href="http://geobliki.com/wfcs/instances/294" rel="edit"/> <link href="http://geobliki.com/wfcs/definitions/11" rel="alternate"/> <id>http://geobliki.com/wfcs/instances/294</id> <created>2007-09-12T14:15:55Z</created> <updated>2007-09-12T14:15:55Z</updated> <author> <name>cappelaere</name> </author> <content>Get South African Fire Threats workflow</content> <g:item_type>instances</g:item_type> <g:command>start now</g:command> <g:status>scheduled</g:status> <g:context_data> <feed_file>data/sa_threat_feed_file.xml</feed_file> <send_to_draper>0</send_to_draper> <all_threats>data/sa_all_threats.kml</all_threats> <final_threats>data/sa_final_threats.kml</final_threats> <scenario>2</scenario> </g:context_data> <g:result_data> </g:result_data> <g:traces> </g:traces></entry>
Modifying Workflow Instances
• PUT /wfcs/instances/294 <entry xmlns:g='http://geopbms/1.0' xmlns='http://www.w3.org/2005/Atom'>
<g:command>terminate</g:command>
</entry
• PUT /wfcs/instances/294<entry xmlns:g='http://geopbms/1.0' xmlns='http://www.w3.org/2005/Atom'>
<g:command>suspend</g:command>
</entry
• PUT /wfcs/instances/294<entry xmlns:g='http://geopbms/1.0' xmlns='http://www.w3.org/2005/Atom'>
<g:command>resume</g:command>
</entry
40
Sensor Planning Service (SPS) -- Old Way
• User can submit a task request and get an ID back
• Submit
• User can get status information about that task and change some task attributes
• GetStatus, UpdateRequest
• User can cancel request
• CancelRequest
• Service allows users to get access to files generated as a result of the task
• DescribeAccess
• Service allows users to check feasibilities
• GetFeasibility
41
RESTful SPS
• POST /wfcs/workflows/eo1-tasking/instances
• <entry>...</entry>
• GET /wfcs/instances/24.atom
• PUT /wfcs/instances/24.atom
• <entry>..</entry>
• DELETE /wfcs/instances/24.atom
• POST /wfcs/workflows/eo1-feasibility/instances
• <entry>...</entry>
• GET /wfcs/workflows/instances/25.atom
Creates new task, input parameters are defined in definition file
Retrieves Task Instance Information (status and result data if available)
Updates Task Instance Information
Cancels Task Instance
Submits a GetFeasility Request
Rerieves the GetFeasility Response in result_data
42
RESTful SPS - Page 2
• DescribeTasking
• GET /wfcs/workflows/eo1-tasking
• DescribeFeasibility
• GET /wfcs/workflows/eo1-feasibility
Retrieves XPDL for EO-1 Tasking
Retrieves XPDL for EO-1 GetFeasibility
43
WPS & WCTS
• They are processes which map directly to named workflows
44
WFS Page-1
• RESOURCES ARE THE FEATURES
• GET /wfs/app
• GET /wfs/{feature}.atom
• GET /wfs/{feature}.metadata
Retrieves WFS APP Service Document
Generates an atom feed with all features of that type
Generates Metadata document for that feature type
45
WFS with Transactions (WFS-T) Page 2
• POST /wfs/{feature}
XML Payload: <entry>...</entry>
• PUT /wfs/{feature}/{id}
XML Payload <entry>...</entry>
• DELETE /wfs/{feature}/{id}
46
WFS Filter Page-3
• Uses OpenSearch
• Keywords
• BBOX
• Timespan queries
• Create Your Custom OpenSearch Template
• Browser will discover it automatically
47
WNS / OPS-B
• Web Notification Service / OGC Publish/Subscribe
• Receives Atom Feeds
• Matches entries with user-based subscriptions
• Delivers custom feed using user-selected notification preferences
Resource Notification Service
• Every Resource Has An URI That Follows A Simple Pattern
• When A Resource Operation Takes Place, An Event Is Generated For That URI
• You Can Subscribe For Those Events
• For That Specific Resource
• For That Specific Collection
49
Subscriptions
POST /wns/subscriptions
Examples
POST /wns/subscriptions
<path>uri</path>
uri = /wfcs/workflows/eo1-tasking
uri = /wfcs/definitions/*
uri = /wfcs/instances/*
uri = /wfcs/instances/*/traces/*
Subscribe to all workflow events (CRUD)
Subscribe to all workflow definitions events (CRUD)
Subscribe to all workflow Instances events (CRUD)
Subscribe to specific workflow event by name
Subscribe to specific workflow instance trace events
use DELETE to unsubscribe 50
To create a new subscription
Notifications
POST /wns/notifications
<email_address>[email protected]</email_address>
<im_address>[email protected]</im_address>
<twitter_address>cappelaere</twitter_address>
PUT /wns/notifications
<email>disabled | enabled</email>
<im>disabled | enabled</im>
<twitter>disabled | enabled</twitter>
<notifications>enabled | disabled </notifications>
51
Sensor Observation Service (SOS)
52
• GET /sos/app
• Sensor Collection
• GET /sos/sensors/
• GET /sos/sensors.atom
• GET /sos/sensors/{id|name}
• GET /sos/sensors/{id|name}.atom
Retrieve all available sensors (html)
Retrieve all sensors in atom format
Retrieve specific sensor by id or name
Retrieve specific sensor by id or name in atom format
SOS (cont)
• Getting Observations
• GET /sos/observations[.atom]
• GET /sos/observations.metadata
• GET/sos/observations/{id|name}[.atom]
• GET /sos/observations/{id|name}.metadata
• GET /sos/observations/{id|name}/offering/{id|name}.{format}
• GET /sos/observations/{id|name}/offering/{id|name}.metadata
Retrieve all observations html/atom
Retrieve all observations metadata
Retrieve a specific observation by id or name html/atom
Retrieve specific sensor by id or name in atom format
Retrieve specific offering by id/name in a specific format (.tiff/.tar.gz/.jpg, etc)
Retrieve specific offering by id/name meta data
SOS Search, Exactly like WFS’s Search
• Uses OpenSearch
• Keywords
• BBOX
• Timespan queries
• Create Your Custom OpenSearch Template
• Browser will discover it automatically
SOS-T
• PUT /sos/observations/{name}
XML Payload <xml>...</xml>
• PUT /sos/observations/{name}/offering/{name}
XML Payload <xml>...</xml>
• DELETE /sos/observations/{id|name}
• DELETE /sos/observations/{id|name}/offerings/{id|name}