middleware platform for sentient computing applications

CO-operating Real-time senTient objects:architecture and EXperimental evaluation

Project IST-2000-26031

Middleware Platform for Sentient Computing Applications

Computing Department, Lancaster University, UK

Thirunavukkarasu Sivaharan, Maomao Wu, Gordon Blair, Adrian Friday, Paul Okanda.

2nd MiNEMA Closed Workshop@ Lancaster, 1st Dec 2004

Lancaster University2


Overview of Presentation IntroductionSentient ObjectsResearch Challenges &

Component Frameworks Middleware ArchitectureSentient Vehicle DemonstratorConclusions



Introduction(2)

EU FET Project : CORTEX– Universidade de Lisboa (Portugal)– Lancaster University (United Kingdom)– Trinity College (Ireland)– Universität Ulm (Germany)

Aims– Middleware support for constructing distributed mobile

proactive applications based on real-time sentient objects– Proposes sentient object model to support the construction

of mobile, context aware, decentralised ,autonomus ,proactive and collaborative applications such as intelligent vehicles and smart buildings.

– A middleware for networked embedded systems



Sentient Object Model(1) Sentient Object Model

– System consists of environment and a set of sentient objects

– Sentient objects are capable of independently sensing the environment, derive context and infer autonomous actions

– Sentinet objects communicate using event channels to establish higher level context and thus cooperate with each other

SentientObjectC P

Ev

en

tC

on

su

mp

tio

n

Ev

en

tP

rod

uc

tio

n



Sentient Object(2)

Consume Produce

Sensory Capture

and Fusion

Context Representation

Inference Engine

Sensor

Sensor

Actuator

Actuator

Sentient Object

Event



Autonomous sentient vehicle application in MANET

Autonomous navigation of vehicles from a source to destinations

Cooperating vehicles in MANETContext aware vehicles



Some of the research challenges addressed

Suitable Communication Model for MANETRouting in mobile ad-hoc environmentContext-awareness End-to-End QoS and Fail safetyRun time and deployment time reconfigurations



Component Framework based Reflective Middleware

Publish-Subscribe Component framework (CF)Multicast CFContext CFResource Management CF



Why Component Framework based Middleware Platform?Middleware is engineered as family of Component

frameworks (CF) using Reflection and component technology

Each CF addresses specific research areasComponent Frameworks are highly configurable and

dynamically reconfigurable (with the granularity of a component)

Clear separation of concernsAdaptable to diversity of CORTEX applicationsReduction of memory footprint CFs are implemented using Lancaster’s OpenCOM

reflective component technology



Middleware Architecture

WLAN 802.11b (ad-hoc), Windows CE

Payload Channel TCB control channel

Group Communication CF-( Ad-hoc Multicast )

Publish-Subscribe CF- (for MANET)

Programming InterfacesMIDDLEWARE

SentientObjects

SentientObjects

Context CF-Sensor Fusion

Inference Engine

Timely Computing Base

Middleware Configuration for MANET



Publish-Subscribe CF(1)Communication model inspired by STEAM Implicit event modelSender & receiver based event filteringSubscription Language supports subject, content &

context based event filteringSupports distance based context filtering & extensible to

other contextsXML based generic eventsEvents transported via selectable Multicast protocol



Publish-Subscribe CF(2)

Subscriber

SOAP Messaging

Filter

SOAPtoMulticast

Multicast

IFilter

IMulticast

ISOAPMessaging

ISubscribe

ISOAPTransport

Publisher Notifier

IApplicationNotify

Filter

IFilter

Receptacle

Interface

IPublish

Dispatcher

IDispatch



Multicast CF

Underlying event Routing Protocol is based on multicast

The multicast protocol for ad-hoc networks is a probabilistic, stateless and multi-hop protocol

We offer this service in the form of a component framework.

Shared memory based IP

MulticastProbabilistic Multicast



Context CF (1)Sensor capture and fusion

– Multivariate Gaussian modelling– Bayesian networks– Dead-reckoning

Inference engine– A program that reasons about a set of rules (a knowledge base)

in order to derive an output.– The knowledge is encoded as a set of production rules, contexts

are represented as “fact”.– CLIPS – C Language Integrated Production System, its internal

implementation is based on RETE net.



Context CF (2)

CLIPS rule sample

The paradigm facilitates uniform treatment of both context and QoS– Rules to trigger adaptations and actuations based on changes in

measure of QoS data

CLIPS DLL and OpenCOM component for WinXP and WinCE

(defrule rule-obstacle-near "CLIPS rule for obstacle near" (car-id (id ?id)) ?f1 <- (obstacle (distance near)) => (retract ?f1) (publish ?id stop))



End-to-End QoS Management and Fail Safety- Timeliness requirement

How can this be achieved?– Enforcing timely perceptions of the environment and

timely actuations on it.– Which means timely event delivery and awareness of

QoS of the event channels used for inter-sentient object communication

The key issue in uncertain and highly dynamic environments is that timing bounds for distributed actions may be violated because of timing failure



End-to-End QoS Management and Fail Safety-Timeliness Requirements We model the uncertainty of timely event dissemination via event

channels using a dependable timing failure detection service. This service is provided by University of Lisboa’s Timely Computing

Base (TCB) TCB facilitates to construct distributed event channels with timing

bound specification This enables publisher or subscriber to be aware of the timing

failures of event channels Thus providing awareness of timing failure probability for a given

required coverage Fail safety is achieved by switching to fail-safe state as soon as QoS

specifications are violated.



Autonomous Sentient Vehicles Demonstrator

Two Sub problems– Cooperative behaviour without human control– Autonomous vehicle navigation from a given source to

pre-determined destinationVehicles Objectives

– Travel along a given path( virtual circuit-VC) defined by set of GPS waypoints and bearings.

– Every vehicle that travels on the VC cooperate with other vehicles to avoid collisions and travel safely

– Obey external roadside traffic lights.– Give way to pedestrians who cross the road.



OC CLOSE( 4m)OC BEHIND OC FAR(4- 10m)

Location aware Cooperating Sentient Vehicles

Car publishes

on Carcontrol

channel: Event

Packet: <car

status,

Location>Car

subscribes to:

CarControlC

hannel &

Receives

events from

other cars

Car

subscribes to:

CarControlC

hannel &Receives

events from

other cars

Car publishes on Carcontrol

channel: Event Packet: <car

status, Location>

IEEE 802.11b(ad-hoc)---Event Channel---CarControlChannel

Satellites

Car A Car B

OC – Other car

OC VERY FAR

Other car’s location context w.r.t car A

OC BEHIND

Other Car’s location context w.r.t Car B

4m

OC CLOSE



Pedestrian detection

Obstacle Sensing Service: Consumes raw ultrasonic sensor data and fuses using a suitable algorithm (reliable, timely-unreliable, Gaussian, …) to derive higher level obstacle distance context such as NEAR , FAR , NOOBJECT.

Ultra sound waves

Ultrasonic sensors

Non event publishing obstacle



Component

Example: The Car Sentient Object & Context CF

Component

Location Sensing Service

GPS Fusion 1

GPS Fusion 2

Obstacle Sensing Service

Ultrasonic Fusion 1 Ultrasoni

c Fusion 2

Direction Sensing Service

Compass Fusion 1

Inference Service

CLIPS Inference Engine

Ultrasonic sensor

GPS sensor

Sentient object

Digital Compass sensor

Consume Produce

SpeedActuator

SteerActuator

Sentient object

receptacle

Interface



Sentient Vehicle Test Bed



Cont’d



Demo Settings



Waypoint 1

Waypoint 2 Waypoint 3

Waypoint 4

Traffic Light

Virtual Circuit



Demo Video



Concluding Remarks The sentient object model

– has proved to be valuable programming abstraction for the development of real-time, cooperative, context-aware applications.

The component-Framework based Middleware approach– offers benefits of flexible configuration and reconfiguration of the

middleware components The middleware architecture

– also provides the management of non-functional concerns such as timeliness and reliability properties.

Our middleware is reusable– we are keen to investigate the generality of our approach by applying our

middleware to other application domains involving embedded autonomous components.



Thank You

Questions

middleware platform for sentient computing applications

Documents