SECURE: Semantics Empowered resCUe enviRonmEnt

demo @ SSN-ISWC2011

P. Desai, C. Henson, P. Anandtharam, A. ShethOhio Center of Excellence in Knowledge-Enabled

Computing (Kno.e.sis),Wright State University, Dayton, Ohio

Semantic Sensor Web @ Kno.e.sis

Introduction

• Timely response of first responders is crucial in rescue operations.

• First responders inundated with streams of data from sensors (machine + citizen).– “Emergency responders have to assimilate large amounts of

information in very short periods of time” [Cowlard et. al].• Streams of data when interpreted with domain

knowledge, results in abstractions.• Abstractions (intuitive to humans) makes first

responders respond quickly in rescue environments.

Cowlard, Adam and Jahn, Wolfram and Abecassis-Empis, Cecilia and Rein, Guillermo and Torero, José, Sensor Assisted Fire Fighting, In the Journal of Fire Technology, Volume 46, pp. 719-741, 2010

– Environment ignorant• Machines without any sensors

Motivation

http://www.familycourtchronicles.com/philosophy/dissonance/remote-control-car.jpg

– Environment sensing• Machines with sensors

Motivation

Photo courtesy NASAThe autonomous Urbie is designed for various urban operations,

including military reconnaissanceand rescue operations.

– Environment comprehending• Machine with sensors + perceiving background

knowledge + comprehending background knowledge

Motivation

http://www.nytimes.com/2010/10/10/science/10google.html

Traffic signals

pedestriansand otherson roads.

Stimuli

Speed restrictions

Google’s car that wonthe DARPA challenge.

• Building a rescue robot (mobile-platform) with many sensors.

• Data Collection and annotation using SSN ontology.

• Analysis to be carried out for situational awareness using perception ontology [2].

• Visualization of the emergency situation in terms of abstractions.

Project Focus

System Architecture

Robot (Mobile Platform) With Sensors

Paper on Fire

Data Collection

Annotation Visualization

Perceptual Analysis

Events in environment

• Collection of data from sensors on the robot.– Position data: Observation form position sensors.– Sensor data: Observation from environment

sensors.• Annotation of raw sensor data. – Use of SSN ontology which has concepts to

describe sensors and their observations.

Data Collection + Annotation

Raw Sensor DataRobot (Mobile Platform) With Sensors

Annotated Data (triple

store)

Position data

Paper on Fire AnnotatedData Stream

PositionData Stream

Sensor Data (CO2, Temperature, IR, CO

data.)

• Perceptual ontology used to derive abstractions from annotated sensor data.

• Domain knowledge is used to derive these abstractions.

Perceptual Analysis

Images: http://static3.depositphotos.com/1001416/130/i/950/depositphotos_1304999-Sheet-of-the-old-scorched-paper-and-fire.jpghttp://www.firesystems.net/images/portable-fire-extinguishers/types-1.jpghttp://www.blogcdn.com/www.engadget.com/media/2007/02/irobot-packbot-510.jp

Perceptual Reasoning

Domain Knowledge

Abstraction Stream

AnnotatedSensor Data

• Visualization serves as a dashboard for presenting real-time:– Raw sensor data– Position Data– Derived abstractions– Video of the robot

Visualization

Demo

SECURE OnlineDemo:http://www.youtube.com/watch?v=smu9mPFFyNs

Local

• Robot with perceptual abilities give out abstractions that are intuitive to humans.

• Demonstrated a real-time physical system that uses domain knowledge to process heterogeneous sensor data.

• Demonstrated visualization of events (as abstractions) in an emergency situation in real-time.

Conclusions

[1] Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth, Pascal Hitzler, 'Representation of Parsimonious Covering Theory in OWL-DL,' In: Proceedings of the 8th International Workshop on OWL: Experiences and Directions (OWLED 2011), San Francisco, CA, United States, June 5-6, 2011.

[2] Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web. Applied Ontology, 2012. (accepted).

Demos, Papers and more at: http://semantic-sensor-web.com

Semantic Sensor Web @ Kno.e.sis

References