Middleware Services for Sensors Systems in Dynamic Data-driven Oil Applications

Manish ParasharThe Applied Software Systems Laboratory

ECE/CAIP, Rutgers Universityhttp://www.caip.rutgers.edu/TASSL

(Ack: NSF, DoE)

Outline

• Pervasive Grid Environments and Data-driven Management of Subsurface Geosystems– The Instrumented Oil Field

• Project AutoMate and Meteor: Middleware Services for Sensors Systems

• Deployments and Evaluations

• Concluding Remarks

Pervasive Computational Ecosystems and Dynamic Data Driven Applications

Components dynamically composed.

“WebServices”discovered &

invoked.

Resources discovered, negotiated, co-allocated on-the-fly. components

deployed

Experts query, configure resources

Experts interact and collaborate using ubiquitous and

pervasive portals

Applications& Services

Model A

Model BLaptop

PDA

ComputerScientist

Scientist

Resources

Computers, Storage,Instruments, ...

Data Archive &Sensors

DataArchives

Sensors, Non-Traditional Data

Sources

Experts mine archive, match

real-time data with history

Real-time data assimilation/injection

(sensors, instruments, experiments, data

archives),

Automated mining & matching

Components write into the archive

Experts monitor/interact with/interrogate/steer models (“what if”

scenarios,…). Application notifies experts of interesting events.

Pervasive Grid Environments – Unprecedented Opportunities• Pervasive Grids Environments

– Seamless, secure, on-demand access to and aggregation of, geographically distributed computing, communication and information resources

• Computers, networks, data archives, instruments, observatories, experiments, sensors/actuators, ambient information, etc.

– Context, content, capability, capacity awareness– Ubiquity and mobility

• Knowledge-based, information/data-driven, context/content-aware computationally intensive, pervasive applications– Symbiotically and opportunistically combine services/computations, real-

time information, experiments, observations, and to manage, control, predict, adapt, optimize, …

• Crisis management, monitor and predict natural phenomenon, monitor and manage engineering systems, optimize business processes

• A new paradigm ?– seamless access

• resources, services, data, information, expertise, …– seamless aggregation– seamless (opportunistic) interactions/couplings

Data-driven Management of Subsurface Geosystems: The Instrumented Oil Field (with UT-CSM, UT-IG, OSU, UMD, ANL)

Detect and track changes in data during production.Invert data for reservoir properties.Detect and track reservoir changes.

Assimilate data & reservoir properties intothe evolving reservoir model.

Use simulation and optimization to guide future production.

Data Driven

ModelDriven

Optimize

• Economic revenue

• Environmental hazard

• …

Based on the present subsurface knowledgeand numerical model

Improve numerical model

Plan optimal data acquisition

Acquire remote sensing data

Improve knowledge of subsurface to

reduce uncertainty

Update knowledge of modelMan

agem

ent d

ecis

ion

START

Dynamic Decision Dynamic Decision SystemSystem

Dynamic DataDynamic Data--Driven Driven Assimilation Assimilation

Data assimilation

Subsurface characterization

Experimental design

Dynamic, Data Driven Reservoir Management

LandfillsLandfills OilfieldsOilfields

ModelsModels SimulationSimulation

DataDataControlControlUndergroundPollution

UndergroundPollution

UnderseaReservoirsUnderseaReservoirs

Vision: Diverse Geosystems – Similar Solutions

Management of the Ruby Gulch Waste Repository (with UT-CSM, INL, OU)

– Flowmeter at bottom of dump– Weather-station– Manually sampled chemical/air

ports in wells– Approx 40K measurements/day

• Ruby Gulch Waste Repository/Gilt Edge Mine, South Dakota – ~ 20 million cubic yard of

waste rock– AMD (acid mine drainage)

impacting drinking water supplies

• Monitoring System– Multi electrode resistivity system

(523)• One data point every 2.4

seconds from any 4 electrodes – Temperature & Moisture sensors

in four wells

“Towards Dynamic Data-Driven Management of the Ruby Gulch Waste Repository,” M. Parashar, et al, DDDAS Workshop, ICCS 2006, Reading, UK, LNCS, Springer Verlag, Vol. 3993, pp. 384 – 392, May 2006.

Pervasive Grid Applications – Unprecedented Challenges: Uncertainty

• System Uncertainty– Very large scales– Ad hoc structures/behaviors

• p2p, hierarchical, etc, architectures– Dynamic

• entities join, leave, move, change behavior

– Heterogeneous• capability, connectivity, reliability,

guarantees, QoS– Lack of guarantees

• components, communication– Lack of common/complete

knowledge• number, type, location, availability,

connectivity, protocols, semantics, etc.

• Information Uncertainty– Availability, resolution, quality of

information– Devices capability, operation,

calibration– Trust in data, data models – Semantics

• Application Uncertainty– Dynamic behaviors

• space-time adaptivity– Dynamic and complex couplings

• multi-physics, multi-model, multi-resolution, ….

– Dynamic and complex (ad hoc, opportunistic) interactions

– Software/systems engineering issues

• Emergent rather than by design

Pervasive Grid Computing – Research Issues, Opportunities• Programming systems/models for data integration and runtime self-

management– components and compositions capable of adapting behavior, interactions

and information– correctness, consistency, performance, quality-of-service constraints

• Content-based asynchronous and decentralized discovery and access services– semantics, metadata definition, indexing, querying, notification

• Data management mechanisms for data acquisition and transport with real time, space and data quality constraints– high data volumes/rates, heterogeneous data qualities, sources – in-network aggregation, integration, assimilation, caching

• Runtime execution services that guarantee correct, reliable execution with predictable and controllable response time– data assimilation, injection, adaptation

• Security, trust, access control, data provenance, audit trails, accounting

Project AutoMate: Enabling Autonomic Applications

• Conceptual models and implementation architectures – programming systems based on popular programming models

• object, component and service based prototypes– content-based coordination and messaging middleware– amorphous and emergent overlays

• http://automate.rutgers.edu

Rudder

Coordination

Middlew

are

Sesam

e/DA

IS P

rotection Service

Autonomic Grid Applications

Programming SystemAutonomic Components, Dynamic Composition,

Opportunistic Interactions, Collaborative Monitoring/Control

Decentralized Coordination EngineAgent Framework,

Decentralized Reactive Tuple Space

Semantic Middleware ServicesContent-based Discovery, Associative Messaging

Content OverlayContent-based Routing Engine,

Self-Organizing Overlay

Ont

olog

y, T

axon

omy

Met

eor/S

quid

Con

tent

-bas

edM

iddl

ewar

e

Acc

ord

Prog

ram

min

gFr

amew

ork

Project Meteor: Middleware Stack

• A self-organizing tiered overlay network

• Content-based routing engine (Squid)– Flexible content-based routing and

querying with guarantees and bounded costs

– Decentralized information discovery

• Associative Rendezvous Messaging

– Symmetric post programming primitive– Content-based decoupled interaction

with programmable reactive behavior– Decentralized content-based in-

network aggregation (assimilation)

• Service-based API (WS)

AR messagingAR messaging

Content-based Routing

Self-organizing OverlaySelf-organizing Overlay

Pervasive Grid Environment

Pervasive Application

Agg

rega

tion

Agg

rega

tion

Project Meteor: Associative Rendezvous

• Content-based decoupled interaction with programmable reactive behaviors– Messages - (header, action, data)

• Symmetric post primitive: does not differentiating between interest/data

– Associative selection• match between interest and data

– Reactive behavior• Execute action field upon matching

• Decentralized in-network aggregation – Tries for back-propagating and

aggregating matching data items• Supports WS Notification standard

profile credentials

message context

TTL (time to live)

Action

[Data]

Header

• store• retrieve• notify• delete

Profile = list of (attribute, value) pairs:Examples:- Post <(sensor_type, temp.), (latitude, 10), (longitude, 20)>- Post <L, speed, 3600>, retrieve(AVERAGE, 300)

C1

C2

post (<p1, p2>, store, data)

notify_data(C2)

notify(C2)

(1)

(3)

(2) post(<p1, *>, notify_data(C2) )

<p1, p2> <p1, *>

match

Implementation/Deployment Overview

• Current implementation builds on JXTA– SquidTON, Squid, Comet and

Meteor layers are implemented as event-driven JXTA services

• Deployments include– Campus Grid @ Rutgers– ORBIT wireless testbed (400

nodes)• 400 802.11 radio nodes with

802.11 wireless connections

– PlanetLab wide-area testbed• At least one node selected

from each continent

ORBIT Testbed for Evaluation of Next Generation Wireless Networks: Phase 1: Indoor Grid

80 ft ( 20 nodes )

70 ft

m (

20 n

odes

)

Control switch

Data switch Application Servers

(User applications/ Delay nodes/

Mobility Controllers / Mobile Nodes)

Internet VPN Gateway / Firewall

Back-end servers

Front-endServers

Gigabit backboneVPN Gateway to

Wide-Area Testbed

SA1 SA2 SAP IS1 IS2 ISQ

RF/Spectrum Measurements Interference Sources

80 ft ( 20 nodes )

70 ft

m (

20 n

odes

)

Control switch

Data switch Application Servers

(User applications/ Delay nodes/

Mobility Controllers / Mobile Nodes)

Internet VPN Gateway / Firewall

Back-end servers

Front-endServers

Gigabit backboneVPN Gateway to

Wide-Area Testbed

SA1 SA2 SAP IS1 IS2 ISQ

RF/Spectrum Measurements Interference Sources

ORBIT Phase 2: Field Trial

• Requires ruggedized outdoor ready equipment (suggesting of the shelf technologies)

• Standard nodes used in dual role of mobile AP/mobile nodes deployed on busses

• Where possible connected to wired infrastructure; otherwise use of second radio interface for mesh type networking/wide area access

ORBIT: Hardware

512 MBRAM

Gigabit Ethernet(control)

GigabitEthernet

(data)

Intel/AtherosminiPCI802.11a/b/g

22.1Mhz

1 Ghz

pwr/resetvolt/temp

20 GBDISK

Serial Console110

VAC

RJ11 NodeIdBox+5v standby

PowerSupply

CPUVIA

C3 1Ghz

Intel/AtherosminiPCI802.11a/b/g

BluetoothUSB

CPURabbit Semi

RCM3700

10 BaseTEthernet

(CM)

Experimental Evaluation

• Data provided by Adolfo/Hector

• Aggregation services:– Complex queries issued from each of the three deployment environments

• Post(34-444,temp*, retrieve(count))

• Effectiveness of in-network aggregation– Aggregate range queries– Number of messages processed in the network

• Robustness in case of single peer failures

• Note that the performance of Squid and AR have been separately evaluated

Scalability of the Aggregation Services

Scalability of Meteor aggregation queries

0

500

1000

1500

2000

2500

3000

4 8 16 32 64

Number of peers

time

(ms) Wired LAN

Orbit Wireless Testbed

Wide-area PlanetLab

Aggregate query: post(<(100-300, 75-125), temperature>, retrieve(avg, 300))

• Experiences/issues with ORBIT deployment– Limitations of the MAC layer/protocol (802.11x)

• Channel interference, dropped messages, dropped beacons – Overlay structure is critical

• Hierarchical, multiplexed channels– Message sizes and volumes impact latency

• Average latency on 200 nodes (dedicated): ~2-3 seconds

Effectiveness of In-Network Aggregation

• Simulation results– 500 nodes

• AR aggregate message:– Post(<100-230, 35-120>,

retrieve(avg))

• Fig (a) number of messages with/without in-network aggregation

• Fig (b) actual number of messages per peer

Robustness of Aggregation Service

0

10

20

30

40

50

60

70

20 60 100

140

180

220

260

300

340

380

Time (milliseconds)

Num

ber o

f inc

lude

d re

sults

Time to aggregate all

Single node failure

Aggregate trie reconstruction

Reconstructed trie

• Assume single node failure– An intermediate node

fails at about 205 time ticks

– On-going aggregation operations are lost at this node

• About 100 ms required to correct the aggregation trie in a LAN environment

Conclusion

• Pervasive Computational Ecosystems & Next Generation Scientific Investigation– Knowledge-based, data and information driven, context-aware,

computationally intensive– Project AutoMate: Enabling Autonomic computational science

• Meteor middleware for sensor-based pervasive Grid environments– Unified programming abstractions for content-based decoupled interactions

and decentralized in-network aggregation• Recurrent and spatially constrained queries• Scalable and efficient trie-based design and implementation

– Deployments on LAN, Orbit and PlanetLab• Scalability, performance, robustness

• More Information, publications, software– www.caip.rutgers.edu/~parashar/– parashar@caip.rutgers.edu