ExScal Breakout Session

Anish Arora

ExScal Team

June 2004

2

Objective of Extreme Scaling

• To create a robust 10,000 node autonomous ad hoc sensor network in an operationally relevant environment for demonstrating the capabilities in the monitoring and protection of long linear structures

• Analysis: Robust 10,000 node autonomous ad hoc sensor network Not

a toy demonstration, two orders of magnitude larger than before

Operationally relevant environment Not a laboratory demonstration, not a simulation, it will be in a real system

Demonstrating the capabilities Okay, not a real fieldable system, but having all the capabilities for the mission

Monitoring and Protection The network detects, tracks, and classifies intruders

Long linear structures Pipelines, borders, …

3

Candidate applications

Colombian Pipeline

Kansas Pipeline

Northern border control

Ideal environmentals2D detection of mounts and dismounts

Kansas, but canopyprecludes aerial techniques

Rain forest –Mountains - WaterMaximum environmental challenges

4

Convoy protection 1D

Convoy protection 2D

Candidate applications

IEDHide Site

Detect anomalous drivingpatterns on road

Detect anomalous activityalong roadside

5

Pipeline

Pump Station

Security Zone

20 km

1 km

Detection and tracking of personnel

Detection and tracking of vehicles

Detection of unknowns

?

Reaction Force

Canonical problem and solution

6

Outline

• Timeline• Xsm schedule

• Workplan

• Metrics• Site and logistics• Deployment• Monitoring & Management• Visualization • Simulation & Testbed• Beyond extreme scaling:

• Other proposed ExScal experiments for December ?

• What to measure ?

7

Checkpoint Status

-Status on April 30th 2004 (actual):- XSM prototype integration

- Tier-2 Stargate relay

- Tier-3 display

- Topology: 12 XSM, double-line, 10m apart

- Intruder model: Diesel golf-cart, diesel truck

- Detection: Acoustic, infra-red and magnetometer

- Classification: Acoustic-based- Integrated services: routing and time-sync at Tier-1

- Terrain and environment: fallow field, minor growth, wind

- Packaging: Prototype wind shields for XSMs

- Standalone demonstrations of reprogramming, localization

8

ExScal is behind schedule!

-Status on April 30th 2004 (planned):- 100 XSM Motes, 2 Stargates, Tier 3 visualization

- Packaged nodes

- Topology: 180x36m

- Intruder model: Dismounts 2-5kmph , vehicles 5-40kmph Steady-state: 4 intruders in network Separation: 50m Frequency: 1-4/minute Straight-line motion with variable speed

- Detection: Acoustic, infra-red, magnetometer- Classification: Dismounts, vehicles

- Tracking: Continuous, within 5-10m- Integrated self-localization and reprogramming at Tier 1

-And we’re not here even today !

9

Extreme Scaling Project Plan (Dec 2003)

ActivityPrimary

Performer

System AnalysisDefine Operational ScenarioDefine Operational/Technical Rqmts.Document System ArchitectureDocument Performance Metrics

DARPADARPAOSUOSU

System DesignDesign System HardwareDesign System Software

CrossbowOSU

System DevelopmentEnhance XSM Algorithms/SoftwareEnhance Relay Node SoftwareEnhance Display Unit Software

DevelopersDevelopersDevelopers

System ProductionXSM ProductionRelay Node ProductionDisplay Unit Production

CrossbowCrossbowOSU

System IntegrationIntegrate XSM / Relay Node / Display Unit Components

OSU

System Testing/DemonstrationSystem Component TestingPreliminary Integration TestingFinal Integration TestingSystem Demonstrations

DevelopersOSUOSUOSU

Program ManagementExtreme Scaling WorkshopIPT MeetingsStatus Conference Calls

DARPADARPADARPA

Dec. ’03

Jan. ’04

Feb. ’04

Mar. ’04

Apr. ’04

May ’04

Jun. ’04

Jul. ’04

Aug. ’04

Sep. ’04

Oct. ‘04

Build 1 DeliveredBuild 1

DeliveredBuild 2

DeliveredBuild 2

DeliveredBuild 3

DeliveredBuild 3

Delivered

Component Demo

Integrated Demo #1

Ful dress rehearsa

l

Integrated Demo #2

Final demo December 9, 2004Final demo December 9, 2004

10

Progress thus far weighed down by XSM design

Main delay: XSM design, validation, and delivery

First test units : xBow schedule slipped from Feb 25th to March 25th

then found driver & testing issues and design flaws:

incorrect spec for PIR signal conditioning mag circuits (circuit error, needed hardware patch)

no packaging yet, Packaging not dealt with, so wind issues could only get signal chains working by April 25th

Following April IPT joint specification revision, design review, schematics review, test

framework development contract issues

11

Delivery schedule

XSM:• July 15: Test units

• August 6: 200 XSM

• September 3 : 1500 XSM

• end October : rest

Stargates:

• expected late July

12

Work-plan (locations, dates, and goals)

OSU

4/30/04

OSU

7/31/04

OSU8/30/04

OSU

9/30/04

WPAFB

10/30/04

Eglin-1

11/16/04

Eglin-Final

12/14/04

XSM prototype

PIR, Magnetic sensors

Acoustic detection & classification

Tier-1 Routing, Time-sync; Tier 2 Relay

Stand-alone re-programming, localization

Redesigned XSM validated

XSM Enclosure

All partner components integrated

Small-topology tier-2 network

100 XSM stress-test:

variability and fault tolerance

1000 XSM simulation

150 Stargate Tier-2 expts.

400 XSM:

Confidence in localization & reprogramming (400XSM)

Alternative topology test

Deployment by OSU, with AFRL parti-cipation

Remote deployment (1000 XSM, 1 km)

275 Stargate expts.

Confidence in application

Site logistics

Dry run at complete scale (10, 000 XSM, 10 km x 500m)

Final demo

13

Current focus

-Integrated baseline- 3 signal chains- Routing, time-sync, localization, reprogramming, visualization

-Tier 2 scale

-Application refinement:- Multi-modal fusion- Probabilistic finer-grained classification,

- Based on harmonic analysis, trajectory analysis

-Power management

-System monitoring and management

-Deployment and logistics

14

Outline

• Site and logistics• Deployment• Timeline

• Xsm schedule

• Workplan

• Metrics• Monitoring & Management• Visualization • Simulation & Testbed• Beyond extreme scaling:

• Other proposed ExScal experiments for December ?

• What to measure ?

15

Metrics

-Coverage: 10km x 500m

-Sensors: 9500 XSM, 275 Stargates

-Deployment: < 1 day

-Startup: Desired 2 min- Current target 10 min

-Shutdown: 1 min

-Collection: < 1 day: 10-20 personnel + vehicles

-Reprogramming: Desired 10 min for 128KB

16

Metrics (contd.)

-Reliable communication range: - Tier-1: < 30m - Tier-2: < 500m - Tier-2-Tier-3: < 5 km

-Detection: PFA: < 0.005

- Range: >10m

-Classification: PFA: < 0.01 for vehicle,

- < 0.04 for dismount -Latency: Detection: < 2s Classification: < 10s

-Accuracy: Tracking: - continuous within thick region, - bounded-box in thin region

17

Metrics (contd.)

-Robustness:-random failure: < 10%

-byzantine failure: < 3%

-network: loss of 1 Stargate per 1000 nodes

-Endurance: Tier 1, Tier-2: 14 days at 10hrs/day, 10 events/day

-Time synchronization: Within 1ms

-Visualization: Correlated with landmarks

18

Outline

• Site and logistics• Deployment• Timeline

• Xsm schedule

• Workplan

• Metrics• Monitoring & Management• Visualization • Simulation & Testbed• Beyond extreme scaling:

• Other proposed ExScal experiments for December ?

• What to measure ?

19

Management

• Handle mode switching between application phases

e.g. data transfer from one mode to next (e.g. node coordinates &

potential parents after localization phase)

• Scheduler services to allow exclusive access to mote resources

for better performance

e.g. suspend application during reprogramming

• Managing RAM usage across different services

localization + sensing application > 4K RAM

20

Management

• Tuning application parameters

Defined interfaces to allow components to expose tunable parameters

Piggyback parameter updates on routing control messages Network friendly Reliable and self stabilizing

Use tools e.g. config (UCB) to automate code generation, component wirings, etc

21

Monitoring

Detecting system-level deadlocks and livelocks Watchdog timer Periodically post a task and check if it executes

• Monitoring component liveness through heartbeats

• Grenade timer Forced reset to allow access to protected code Recover from incorrectly downloaded, corrupted or misbehaving

programs

• Monitoring system and component level invariants

22

Outline

• Site and logistics• Deployment• Timeline

• Xsm schedule

• Workplan

• Metrics• Monitoring & Management• Visualization • Simulation & Testbed• Beyond extreme scaling:

• Other proposed ExScal experiments for December ?

• What to measure ?

23

Visualization

• Bird’s eye view of the entire test area

• Close-up 3D view of a selectable portion of the test area, with ability to rotate, zoom and pan across the display

• Real time display of network topology, target locations, tracks etc.

• Ability to freeze, playback, etc.

• Display of diagnostic/informational parameters

24

2D view of the entire test area showing Tier 2

nodes

Close up 3D view with Tier 1 and

Tier 2 nodes

Motes detecting targets

Mote topology info

Target with red trail

Tier 2 nodes

Toggle switches for viewing various features

on demand

Control Panel for Start/Stop,

Playback, etc.

Summary info about the

experiment

Text messages for diagnostics

25

Outline

• Site and logistics• Deployment• Timeline

• Xsm schedule

• Workplan

• Metrics• Monitoring & Management• Visualization • Simulation & Testbed• Beyond extreme scaling:

• Other proposed ExScal experiments for December ?

• What to measure ?

26

Kansei testbed

Experience

progressive increase in scale : from 10 100 10,000 nodes concomitant increase in component depth and interaction

complexity comcomitant observations of failure of components fail to meet the

demands of higher scale

progressive increase in deployment, reliability, & robustness, issues: complex failure modes are emerging limited node and network capacity imply nontrivial event loss byzantine nodes

simulations have thus far been inadequate to validate solutions, experiments have been necessary and time consiuming

27

Kansei testbed module

28

Testbed addresses several needs

• Scalability Modular model railroad construction materials

• Fidelity Static and mobile sensor nodes, pursuers, evaders, environmental

stimuli• Usability

MoteLab with enhancements Open to larger community

• Repeatability Simply rerun previously uploaded programs, network topology, and

agent actions/trajectories• Out-of-Band Reporting

StarGate with 802.11 Card for out of band communications Overhead camera and microphone for videotaping experiments

29

Outline

• Site and logistics• Deployment• Timeline

• Xsm schedule

• Workplan

• Metrics• Monitoring & Management• Visualization • Simulation & Testbed• Beyond extreme scaling:

• Other proposed ExScal experiments for December ?

• What to measure ?

30

Proposed experiments following demo

• MIT : co-dependent localization and tracking

• Mitre :

• ARL ?

• Others ?

• Q: Support