sensor web enablement (swe) and sensorml august 2008

Mike Botts – August 2008 1

Sensor Web Enablement (SWE)and

SensorML

August 2008

Mike Botts

[email protected]

VAST Team - NSSTC

University of Alabama in Huntsville


The “Big Hammer” Approach to Sensor Fusion (circa 1991)

Mike Botts – May 2008 3


Benefits and Challenges of Standards

• Ideally the benefits of standards are that they:– Enable interoperability of data and information– Enable development and thus availability of tools (commercial and open-source) for

meeting a variety of needs– Enable supply of commodity hardware/software from a variety of vendors– Enable rapid deployment … plug-and-play– Allow one to focus on the real job and not the technology

• Some possible challenges with standards are:– Standard may not exist when you need it and developing them is too slow for your

needs– Buying into a standard too early can result in

• Choosing a losing standard (e.g. Betamax, HD-DVD)• Investment of effort that you may not be able to afford • Tools don’t exist to realize immediate benefits

• Some successful standards– WWW (html, httpd, TCP-IP, etc)– USB, – Microsoft Windows, Linux, and MacOS– jpeg, mpeg

• The best standards are those we don’t need to think about anymore


Standards Buy-In Decision Chart

Relative return

on effort

Time

Standards-based

Do-it-yourself

early implementer

sweet-spot implementer

– Do-it-yourself may

result in faster but

limited return

– Return on standards

tends to

increase with time

– Early implementers may see

delayed return on investment

– Sweet-spot implementers may reap

maximum benefit for least effort


Open Geospatial Consortium

• The Open Geospatial Consortium, Inc (OGC) is an international industry consortium of 334+ companies, government agencies and universities participating in a consensus process to develop publicly available interface specifications and encodings.

• Open Standards development by consensus process

• Interoperability Programs provide end-to-end implementation and testing before spec approval

• Standard encodings (e.g. GML, SensorML, O&M, etc.) – Geography Markup Language (GML) – Version 3.2– Style Layer Description language (SLD) – SensorML– Observations and Measurement (O&M)

• Standard Web Service interfaces; e.g.:

– Web Map Service (WMS)

– Web Feature Service (WFS)

– Web Coverage Service (WCS)

– Catalog Service– Open Location Services – used by communications and navigation industry

– Sensor Web Enablement Services (SOS, SAS, SPS)

Helping the World to CommunicateGeographically

Basic DesiresBasic Desires

• Quickly discover sensors and sensor data (secure or public) that can meet my needs – location, observables, quality, ability to task

• Obtain sensor information in a standard encoding that is understandable by me and my software

• Readily access sensor observations in a common manner, and in a form specific to my needs

• Task sensors, when possible, to meet my specific needs

• Subscribe to and receive alerts when a sensor measures a particular phenomenon


Decision Support Tools

- vendor neutral- extensive

- flexible- adaptable

Heterogeneous sensor network

In-Situ monitors

Bio/Chem/RadDetectorsSurveillance

AirborneSatellite

- sparse- disparate

- mobile/in-situ- extensible

Models and Simulations

- nested- national, regional, urban- adaptable- data assimilation

M. Botts -2004

Sensor Web Enablement

- discovery- access- tasking- alert notification

web services and encodings based on Open

Standards(OGC, ISO, OASIS, IEEE)

Sensor Web Enablement FrameworkSensor Web Enablement Framework


Background -1-Background -1-

SensorML initiated at University of Alabama in Huntsville: NASA AIST funding

OGC Web ServicesTestbed 1.1:

• Sponsors: EPA, NASA, NIMA• Specs: SOS, O&M, SensorML• Demo: NYC Terrorist• Sensors: weather stations, water quality

OGC Web ServicesTestbed 1.2:

• Sponsors: EPA, General Dynamics, NASA, NIMA• Specs: SOS, O&M, SensorML, SPS, WNS• Demo: Terrorist, Hazardous Spill and Tornado• Sensors: weather stations, wind profiler, video, UAV, stream gauges1999 - 2000 2001

• Specs advanced through independent R&D efforts in Germany, Australia, Canada and US

• Sensor Web Work Group established• Specs: SOS, O&M, SensorML, SPS, WNS, SAS• Sensors: wide variety

2002 2003-2004


Background -2-Background -2-OGC Web Services

Testbed 3.0:

• Sponsors: NGA, ORNL, LMCO, BAE • Specs: SOS, O&M, SensorML, SPS, TransducerML• Demo: Forest Fire in Western US• Sensors: weather stations, wind profiler, video, UAV, satellite

SAS Interoperabilty Experiment

SWE Specifications toward approval:

SensorML – V0.0TransducerML – V0.0SOS – V0.0SPS – V0.0O&M – Best PracticesSAS – Best Practices

2005

OGC Web Services Testbed

4.0:

•Sponsors: NGA, NASA, ORNL, LMCO • Specs: SOS, O&M, SensorML, SPS, TransducerML, SAS• Demo: Emergency Hospital• Sensors: weather stations, wind profiler, video, UAV, satellite2006

OGC Web ServicesTestbed 5.1

•Sponsors: NGA, NASA, •Specs: SOS,SensorML•Demo: Streaming JPIP of Georeferenceable Imagery; Geoporocessing Workflow•Sensors: Satellite and airborne imagery

2007


Brief Video InterludeBrief Video Interlude

• Discovery of SWE services using registry

• Access and portrayal of SWE Observation data


SWE SpecificationsSWE Specifications

• Information Models and Schema

– Sensor Model Language (SensorML) for In-situ and Remote Sensors - Core models and schema for observation processes: support for sensor components, georegistration, response models, post measurement processing

– Observations and Measurements (O&M) – Core models and schema for observations

– TransducerML – adds system integration and multiplex streaming clusters of observations

• Web Services– Sensor Observation Service - Access Observations for a sensor or sensor

constellation, and optionally, the associated sensor and platform data– Sensor Alert Service – Subscribe to alerts based upon sensor observations– Sensor Planning Service – Request collection feasibility and task sensor system for

desired observations– Web Notification Service –Manage message dialogue between client and Web

service(s) for long duration (asynchronous) processes– Sensor Registries – Discover sensors and sensor observations


StatusStatus

• Current specs are in various stages– SensorML (and SWE Common) – Version 1.0.1 – TransducerML – Version 1.0– Observations & Measurement – Version 1.0– WNS – Request for Comments– SOS – Version 1.0– SPS – Version 1.0 – SAS – Being released for vote

• Approved SWE standards can be downloaded:– Specification Documents: http://www.opengeospatial.org/standards – Specification Schema: http://schemas.opengis.net/


What is SensorML?

• XML encoding for describing sensor processes

– Including sensor tasking, measurement, and post-processing of

observations

– Detectors, actuators, sensors, etc. are modeled as physical processes

• Open Standard –

– Approved by Open Geospatial Consortium in 2007

– Supported by Open Source software (COTS development ongoing)

• Not just a metadata language

– enables on-demand execution of algorithms

• Describes

– Sensor Systems

– Processing algorithms and workflows


Why is SensorML Important?

• Importance:

– Discovery of sensors and processes / plug-n-play sensors – SensorML is

the means by which sensors and processes make themselves and their

capabilities known; describes inputs, outputs and taskable parameters

– Observation lineage – SensorML provides history of measurement and

processing of observations; supports quality knowledge of observations

– On-demand processing – SensorML supports on-demand derivation of

higher-level information (e.g. geolocation or products) without a priori

knowledge of the sensor system

– Intelligent, autonomous sensor network – SensorML enables the

development of taskable, adaptable sensor networks, and enables higher-level

problem solving anticipated from the Semantic Web


SensorML Processes

Physical ProcessesNon-Physical Processes

Atomic Processes

Composite Processes

Processes that are considered Indivisible either by design or necessity

Processes that are composed of other processes connected in some logical manner

Processes where physical location or physical interface of the process is not important (e.g. a fast-Fourier process)

Processes where physical location or physical interface of the process is important (e.g. a sensor system)


Example Atomic Processes

• Transducers (detectors, actuators, samplers, etc.)

• Spatial transforms (static and dynamic)

– Vector, matrix, quaternion operators

– “Sensor models”

• scanners, frame cameras, SAR

• polynomial models (e.g. RPC, RSM)

• tie point model

– Orbital models

– Geospatial transformations (Map projection, datum, coordinate system)

• Digital Signal Processing / image processing modules

• Decimators, interpolators, synchronizers, etc.

• Data readers, writers, and access services

• Derivable Information (e.g. wind chill)

• Human analysts


Example Composite Processes

• Sensor Systems, Platforms

• Observation lineage

– from tasking to measurement to processing to analysis

• Executable on-demand process chains:

– geolocation and orthorectification

– algorithms for higher-level products

• e.g. fire recognition, flood water classification, etc.

– Image processing, digital signal processing

• Uploadable command instructions or executable processes


Status of SensorML and SWE Common

– SensorML history

• Influenced by interoperability challenges for satellite sensors at NASA

• Started at UAH in 1998 under NASA AIST funding; brought into OGC in 2000

• Approved as Public Discussion Paper (2002)

• Approved as Recommended Paper (2004)

• OGC 05-086 approved as Best Practices Document in Bonn (Nov 2005)

• OGC 05-086r3 approved as Version 0.0 Technical Specification in July 2006

• OGC 07-000 approved as Technical Specification Version 1.0 on June 23, 2007

– Current: document (OGC 07-000)

• Approved Version 1.0 of SensorML and SWE Common data types

• Official document available at OGC ( http://www.opengeospatial.com )

• Official Reference Schema resides online at http://schemas.opengis.net/

• Doc and schema also available at http://vast.uah.edu/SensorML

Mike Botts – April 2008

Where and how can SensorML processes

be used?


SensorML provides metadata suitable for discoveryof sensors and processes

Find all remote sensor systems measuring in the visible spectral range with

ground resolution less than 20m.


Discovery Based on SensorML

Credit: Northrop Grumman

PulseNet Project


Specific Discovery Needs

• Unique resource ids used throughout SWE;

– sensor centric example:

• Find sensors that can do what I need (returns id=“urn:ogc:id:sensor:123”)

• Get me a full description of this sensor urn:ogc:id:sensor:123

• Now, find a service (SPS) that lets me task this sensor urn:ogc:id:sensor:123

• Find all services (SOS) where I can get observations from this sensor

urn:ogc:id:sensor:123

• Find all processes that can be applied to this sensor output to generate the

information I require

• Catalog profiles for each need:

– SPS, SOS, SAS services

– sensors and processes

– observations

– terms (either through dictionaries or ontologies)


Need for Term Definitions used in SensorML and SWE

• Observable properties / phenomena / deriveable properties (“urn:ogc:def:property:*)

– temperature, radiance, species , exceedingOfThreshold, earthquake, etc.

– rotation angles, spectral curve, histogram, etc.

• Capabilities, Characteristics, Interfaces, etc. (“urn:ogc:def:property:*”)

– Width, height, material composition, etc.

– Ground resolution, dynamic range, peak wavelength, etc.

– RS-232, USB-2, bitSize, baud rate, base64, etc.

• Sensor and process terms (“urn:ogc:def:property:*”)

– IFOV, focal length, slant angle, etc.

– Polynomial coefficients, matrix, image, etc.

– QA/QC terms and tests

• Identifiers and classifiers (“urn:ogc:def:identifierType:*; urn:ogc:def:identifier:*” )

– Identifiers – longName, shortName, model number, serial number, wingID, missionID, etc.

– Classifiers – sensorType, intendedApplication, processType, etc.

• Role types (“urn:ogc:def:role:*”)

– Expert, manufacturer, integrator, etc.

– Specification document, productImage, algorithm, etc.

• Sensor and process events (“urn:ogc:def:classifier:eventType:*”)

– Deployment, decommissioning, calibration, etc.


Supports description of Lineage for an Observation

Observation

SensorML

Within an Observation, SensorML can describe

how that Observation came to be using the

“procedure” property


On-demand processing of sensor data

Observation

SensorML processes can be executed on-demand to

generate Observations from low-level sensor data

(without a priori knowledge of sensor system)

SensorML


SensorML – Sensor Systems

Mike Botts, Alexandre Robin, Tony Cook - 2005

Sensor 1Scanner

System - Aircraft

Sensor 2IMU

Sensor 3GPS

IR radiation

Attitude

Location

Digital Numbers

Pitch, Roll, Yaw Tuples

Lat, Lon, Alt Tuples


AIRDAS UAV Geolocation Process Chain Demo

AIRDAS data stream (consisting of navigation

data and 4-band thermal-IR scan-line data)

AIRDAS data stream geolocated using

SensorML-defined process chain

(software has no a priori knowledge of

sensor system)


On-demand processing of higher-level products

Observation

SensorML

SensorML processes can be executed on-

demand to generate higher-level Observations

from low-level Observations (e.g. discoverable

georeferencing algorithms or classification

algorithms)

Observation


On-demand Geolocation using SensorML

AMSR-E SSM/I

Cloudsat LIS

TMI

TMI & MODIS footprints

MAS

Geolocation of satellite and airborne sensors using SensorML


Clients can discover, download, and execute SensorML process chains

For example, Space Time Toolkit is designed

around a SensorML front-end and a Styler back-

end that renders graphics to the screen

SensorML

OpenGL

SensorML-enabled Client (e.g. STT)

Stylers

SLD

SOS


Incorporation of SensorML into Space Time Toolkit

Space Time Toolkit being retooled to be SensorML process chain executor + stylers


SensorML-Enabled Web Services


SensorML can support generation of Observations within a Sensor Observation Service (SOS)

Observation

SensorML

For example, SensorML has been used to

support on-demand generation of nadir tracks

and footprints for satellite and airborne sensors

within SOS web services

SOS Web Service

request


Incorporation of SensorML into Web Services

SensorML process chains have been used to drive on-demand data within services

(e.g. satellite nadir tracks, sensor footprints, coincident search output)


SensorML can support tasking of sensors within a Sensor Planning Service (SPS)

SensorML

For example, SensorML will be used to support

tasking of video cam (pan, tilt, zoom) based on

location of target (lat, lon, alt)

SPS Web Service

request


SPS control of Web Cam


Dual VASTCAMS (1 & 2)


3D Reconstruction of Storms using SensorML

3D Reconstruction of storms

Multi-directional views


SensorML for Portrayal


SWE Visualization Clients can render graphics to screen

SensorML

SOS

OpenGL

SensorML-enabled Client (e.g. STT)

Stylers

SLD


SWE Portrayal Service can “render” to various graphics standards

SensorML

For example, a SWE portrayal service can utilize

a SensorML front-end and a Styler back-end to

generate graphics content (e.g. KML or Collada)

SOS

KML

SWE Portrayal Service

Google Earth Client

Stylers

SLD

Collada

Mike Botts – March 2008 47

SensorML to Google Earth (KML – Collada)

AMSR-E SSM/I

LIS

TMI

MAS


SensorML Support Activities


SensorML Navigation


Current Tool Development and Support for SensorML

• SensorML Forum – mail list for SensorML discussion (300+ active members from

various backgrounds) https://lists.opengeospatial.org/mailman/listinfo/sensorml

• Open Source SensorML Process Execution Engine – Along with open-source

process model library, provides execution environment for SensorML described

algorithms

• Open Source SensorML editor and process chain development client – on-

going development of tools to allow human-friendly editors for SensorML

descriptions

• SensorML-enabled decision support client – Open source Space Time Toolkit is

SensorML-enabled and will be available to discover, access, task, and process

sensor observations; use as is or as template for COTS development

• SensorML white papers and tutorials – being written and released on an array of

SensorML topics

– Describing a Simple Sensor System

– Creating New Process Models;


SensorML Process Editors

Currently, we diagram the process (right top) and then type the XML version; soon the XML will be generated from the diagram itself (right bottom)

Currently, SensorML documents are edited in XML (left), but will soon be edited using human friendly view (below)


SensorML Table Viewer

• Will provide simple view of all data

in SensorML document

• Web-based servlet or standalone;

upload SensorML file and see view

• Ongoing effort: initial version in May

2008

• Future version will support

resolvable links to terms, as well as

plotting of curves, display of images,

etc


Simple SensorML Forms for the Mass Market

User fills out simple form with manufacturer name and model number, as well as other info. Then detailed SensorML generated.


Where are SWE and SensorML currently being used?


Example Known Activities using SWE -1-Example Known Activities using SWE -1-

• Community Sensor Models (NGA)

– SensorML encoding of the CSM; CSM likely to be the ISO19130 standard

• Multi-Intelligence Metadata Standards (DIA) –

– MASINT committed to SensorML and SWE as future direction

– Conducting a Sensor Standards gap analysis on SensorML, 1451, and other CBRNE sensor standards

(DIA, ORNL, JPEO, UAH, NIST)

• OGC OWS5.1 Georeferenceable Imagery (NGA/NASA)

– demonstrating use of SensorML within JPEG2000 and JPIP for support of geolocation of streaming

imagery (30 GB imagery)

– demonstrating SWE workflow for processing data

• Oak Ridge National Labs SensorNet

– funded project is adding SWE support in SensorNet nodes for CBRNE threat monitoring

– Demonstrating SensorNet/SWE for North Alabama (SMDC, DESE, UAH, ORNL)

• Northrop Grumman IRAD (NGC TASC)

– demonstrated end-to-end application of SensorML/SWE for legacy surveillance sensors in field [PulseNet]

• Empire Challenge 08 (NGA – Seicorp - JFCOM)

– Testing and demonstrating SWE services and SensorML encodings for sensor observation processing

and integration in 2008 demonstration event; initial testing was in EC 07



• European Space Agency

– developing SensorML profiles for supporting sensor discovery and processing within the European

satellite community

– establishing SPS and SOS services for satellite sensors

• Canadian GeoConnections Projects

– using SensorML in water monitoring network

• Sensors Anywhere (SAny) and OSIRIS

– Using SensorML and SWE within several large European Union sensor projects

• Marine Metadata Initiative, OOSTethys, Q2O

– Testing and demonstrating SensorML and SWE in oceans monitoring

– Developing SensorML models and encodings for supporting QA/QC in ocean observations

• NASA ESTO

– funded 30 3-year projects on Sensor Webs; 5 SBIR topics with SensorML and Sensor Web called out

(including 2 at NSSTC)

• Hurricane Missions (NASA MSFC)

– testing SensorML for geolocation and processing of satellite and airborne sensors, and SWE for

access to observations

– Investigating further use for future mission monitoring



• Others

– SPOT Image converting DIMAP format to SensorML

– Landslide monitoring in Germany

– Water quality monitoring in Europe

– Mining and water management in Australia

– Building monitoring in Australia

– SWE being investigated within Office of Under Secretary of Defense/Intelligence (OSD/I)

– SWE a part of GEOSS activity

– SWE a part of CEOS effort

– Recent Air Force Research Lab SBIR mentioned Sensor Web focus

– Recent Department of Homeland Security SBIR specifically cited SensorML and SWE

– Vaisala Weather Station manufacturer recently joined OGC and started to create SensorML

descriptions of sensors


Relevant Links

Open Geospatial Consortium Standard Documents

http://www.opengeospatial.org/standards

OGC Approved Schema

http://schemas.opengis.net/

Sensor Web Enablement Working Group

http://www.opengeospatial.org/projects/groups/sensorweb

SensorML information

http://vast.uah.edu/SensorML

SensorML Public Forum

http://mail.opengeospatial.org/mailman/listinfo/sensorml