deokwoo jung march 18, 2010 towards cooperative localization of wearable sensors using...

March 18, 2010 Deokwoo Jung

Towards Cooperative Localization of Wearable Sensors using

Accelerometers and Cameras

Deokwoo Jung(with Thiago Teixeira and Andreas Savvides)

[email protected] Networks & Applications Lab

Yale University


Indoor Localization• Indoor localization is an essential technology for many applications

– Security Application, Assisted Living, Life logging System etc

• Indoor localization comparison

Cricket [Priyantha.et.al ,00,04]

RADAR[Bahl.et.al,00]

Surroundsense[Azizyan.et.al,09]

Our System

PrecisionPhysical Location

(cm)Physical Location

(<5 meter)Logical Location

(>5 meters)

Physical Location

(cm)

Mobile device

Customized Sensor NodeWLAN Card -

LaptopMobile Phone

Mobile Phone/ Wearable Sensor

Infra- structure

Ultrasound Beacon Nodes on Ceiling

WLAN APs + RF fingerprint data

base

GSM Network + Ambient signal

databaseNetworked cameras

Sensing modality

Ultrasound RF signal Ambient signals-

Light, sound, colorHuman walking motions


Cooperative Localization Approach• Our Approach

– Localizing mobile phones by combining their built-in accelerometer (human motion) and infrastructure camera (human centroids)

• Why Human Centroids and Human Motion ?– They are Complementary to each other

Wearable inertial sensors (Human Motion) Camera (Human Centroid)

ID tracking Accurate - node address Difficult – feature extraction

Location Positioning Difficult –walking orientation and distance estimation Accurate –background subtraction


Sensing Modeling and Approach

• Human Walking Model

– ∫ ∫Accel. ≠ Walking distance

– The law of movement of human body by complex kinetics

– Inverted pendulum model of human gait.

– The body center of mass (BCOM) oscillates in the z direction

– as the person moves forward (y direction).


Statistical Analysis of Sensor Data

• Intuition : BCOM follows a sinusoidal pattern– Velocity of Body α Standard Deviation of Vertical Acceleration

• A correlation coefficient for the similarity measure between accelerometer and camera data

• Experiment

-0.5 0 0.5 1 1.5 20

0.1

0.2

0.3

0.4

az ( g)

Distrib

ution

V=0.61 m/sec

-0.5 0 0.5 1 1.5 20

0.1

0.2

0.3

0.4V=0.76 m/sec

-0.5 0 0.5 1 1.5 20

0.1

0.2

0.3

0.4V=0.91 m/sec

-1 0 1 20

0.1

0.2

0.3

0.4V=1.67 m/sec

= 0.0914 = 0.1294

= 0.1943 = 0.4537

0 0.5 1 1.5 20

0.1

0.2

0.3

0.4

0.5

Walking Velocity, m/sec

Stan

dard

devia

tion o

f az

az

= 0.36*v - 0.13

Sample data linear regression


Tracking Algorithm

• A camera extracts only centroid information – Privacy Preserving and Cheap

• A simple tracking algorithm computes a speed of anonymous centroids– associates human centroids in consecutive frames based on their

distances.

• Problem: Many possible ambiguous associations


Path Disambiguation Problem

• Path Ambiguity Problem in Human Centroid Tracking– A tracker associates one object with more than two objects

in two consecutive image frames when two or more objects come close to each other.


Disambiguation Algorithm

• Path Disambiguation as Non-linear Optimization Problem– Find a set of association hypotheses to maximize a matching rate,– The number of correct ID matchings between accelerometers

and centroids

• Develop a search algorithm in a tree structure – A leaf node: a hypothesis of path segmentations– Three stage pruning algorithm

• Sub-tree evaluation, • Classification and Pruning, • Reconstruction

N

iK

Nji T

TK

jiIN

ET 1

1,,1,,

,,|,maxarg11

max1


Clustering and Pruning in Hypothesis Tree

• Hypothesis Quality Metric : – how credible a given path hypothesis is compared to others?– Correlation Coefficient Distance metric

• D(ρ|H) = |E(ρ, e0|H) − E(ρ, e1|H)|

Wrong Hypothesis Correct Hypothesis

Accelerometers

6.04.03.04.0

5.05.04.03.0

5.06.03.02.0

4.04.02.05.0

2H

Cen

troi

d t

race

s 2

8.02.01.01.0

1.09.02.01.0

2.01.03.07.0

1.01.08.01.0

1H

Cen

troi

d t

race

s 1

Accelerometers

-0.4 -0.2 0 0.2 0.4 0.6 0.8 10

0.002

0.004

0.006

0.008

0.01

0.012

Correlation Coefficient, x

Dis

trib

utio

n

P( =x| Correct Matching )

P( =x | Incorrect Matching )

-0.4 -0.2 0 0.2 0.4 0.6 0.8 10

0.01

0.02

0.03

0.04

0.05

0.06

Correlation Coefficeint , x

Dis

trib

utio

n

Pr(=x| Incorrect Matching)

Pr(=x| Correct Matching)

D(ρ|H1) D(ρ|H2)


Clustering and Pruning in Hypothesis Tree

• Leaf Clustering, Pruning, and Path Reconstruction– Clusters the leaf nodes into groups and prunes the subset of

groups with lower metric values.

– When only one leaf is left, reconstructs the matching sequence

Tree Pruning Algorithm

8.02.05.01.0

1.09.02.01.0

2.01.03.07.0

1.04.08.01.0

1H

Cen

troi

d t

race

s 1

Accelerometers

8.02.05.01.0

1.09.02.01.0

2.01.03.07.0

1.04.08.01.0

1H

Cen

troi

d t

race

s 1

AccelerometersAccelerometersAccelerometers

8.05.01.04.0

7.07.07.03.0

6.08.06.02.0

5.07.02.08.0

2H

Cen

troi

d t

race

s 2

Accelerometers

8.05.01.04.0

7.07.07.03.0

6.08.06.02.0

5.07.02.08.0

2H

Cen

troi

d t

race

s 2

0 10 20 30 40 50 600

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Time (sec)

Cor

rela

tion

Coe

ffic

ient

Dis

tanc

e, D

()

Perfect centroid trace estimation, HGround Truth

No path ambiguity Path ambiguities are accumulating


Performance Evaluation via Experiment & Simulation

• Experiment Setup– A ceiling mounted camera (12ft) with

a Intel iMote2 node • Computes the centroid position of a p

erson, 15 times per second.

– A wearable sensor node with an Analog Devices ADXL330 accelerometer on the person’s waist

• Collecting body acceleration data with 15Hz sampling rate.

• Transmitting its measurements to a computer (fusion center) via a Zigbee wireless link.

– People walk for 1 minute in a 5.4 m2 space.


Experiment Dataset

• Walking trajectory of 12 people collected from camera

0 2 40

1

2

3

Trace1

X ( meter)

Y (

met

er)

0 2 40

1

2

3

Trace2

0 2 40

1

2

3

Trace3

0 2 40

1

2

3

Trace4

0 2 40

1

2

3

Trace5

0 2 40

1

2

3

Trace6

0 2 40

1

2

3

Trace7

0 2 40

1

2

3

Trace8

0 2 40

1

2

3

Trace9

0 2 40

1

2

3

Trace10

0 2 40

1

2

3

Trace11

0 2 40

1

2

3

Trace12


Similarity Metric Performance • 100 % matching rate without path ambiguity

Standard Deviation of z-acceleration and velocity of BCOM over time for tracesBar Graph of Correlation Coefficient Matrix


Disambiguation Algorithm Performance

• The performance depends on the level of crowd in camera field of view. – Evaluate the performance using crowd density metric, the number of

pedestrians per area, m2, [Abishai.et.al, Pedestrian flow and level of service]

• Crowd Density Scenario

Scenario A: Normal flow B: Restricted Flow C: Dense Flow D: Very Dense Flow

Crowd Density

office building in business

hour

crowded shopping mall in weekend

Crowded weekend party

Subway station in Manhattan during the

rush hour

People / m2

<0.5 0.5~0.8 0.81~1.26 1.27~2

• If the crowd density > 2, the pedestrian flow is jammed, i.e. practically people’s movement appears to be static

• Our system is mainly targeting for the scenario A (free flow), i.e. people can walk around without much interaction.


Performance over complexity of scenario

• The number of tracking errors grows with polynomial order with crowd density (left)

• The matching performance of disambiguation algorithm for different crowd densities (right)

• The performance gap is widening as crowd density increases. – The performance becomes twice in the scenario D.

0.18 (1) 0.37 (2) 0.56 (3) 0.75 (4) 0.94 (5) 1.13 (6) 1.32 (7) 1.51 (8) 1.70 (9)

0

5

10

15

20

25

30

35

40

Crowd Density, people / m 2 (# people)

Ave

rag

e nu

mbe

r o

f cul

uma

tive

tra

ckin

g e

rror A. VERY LOW DENSITY

NORMAL FLOWB. LOW DENSITY RESTRICTIED FLOW

C. MODERATE DENSITY DENSE FLOW

D. HIGH DENSITY VERY DENSE FLOW

0.18 (1) 0.37 (2) 0.56 (3) 0.75 (4) 0.94 (5) 1.13 (6) 1.32 (7) 1.54 (8) 1.70 (9)

30

40

50

60

70

80

90

100

Crowd Density, people / m 2 (# people)

Ave

rag

e m

atc

hing

ra

te, %

Tracker OnlyDisambiguation algorithm

A B DC


Performance over disambiguation stages

• Matching rate improvement by disambiguation algorithm

0 10 20 30 40 50 600

20

40

60

80

100

time (sec)

mat

chin

g ra

te,

%

0 10 20 30 40 50 600

20

40

60

80

100

time (sec)

mat

chin

g ra

te,

%

0 10 20 30 40 50 600

20

40

60

80

100

time (sec)

mat

chin

g ra

te,

%


Localization System Demonstration

• Controlled experiments with 10 people walking scenario.

• The performance of disambiguation algorithm (right) is compared to the tracker-only localization (left).


Conclusion

• We presented a hybrid localization system using accelerometers and cameras.

• The proposed disambiguation algorithm operates reliably, degrading gracefully even for crowded scenarios

• The constraint of accelerometer position (waist) can be relaxed using additional inertial measurement sensors.

• Future work is to have a complete system implementation running on a mobile phone + More sensors


QUESTION ?

Thanks for your

interest!For more

information, please visit

http://pantheon.yale.edu/~dj92/

deokwoo jung march 18, 2010 towards cooperative localization of wearable sensors using...

Documents