indoor localization using place and motion signatures · indoor localization using place and motion...
TRANSCRIPT
![Page 1: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/1.jpg)
Indoor Localization using Place and Motion Signatures Jun-geun Park
Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick Roy Prof. Tommi Jaakkola
1
![Page 2: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/2.jpg)
Outline
• Motivation & prior work • Thesis contribution • Algorithms for organic indoor
localization (part I)
• Motion compatibility-based indoor localization (part II)
– Overview – Motion labeling – Route network generation – Trajectory matching
2
![Page 3: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/3.jpg)
Location, Location, Location
3
![Page 4: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/4.jpg)
GPS does not work indoors…
• Ultra-high frequency (1.57 GHz) signals of
GPS do not penetrate walls very well. • GPS does not provide enough precision
for room-grained location-based services.
4
![Page 5: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/5.jpg)
Early Approaches
Require instrumenting spaces with dedicated transceivers.
“Active Badge” (Infrared), 1992
MIT “Cricket” (RF+Ultrasound), 2000
5
![Page 6: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/6.jpg)
WiFi Localization
Survey environment to build WiFi fingerprint database.
0xa3b 0x5fe 0xbc4 0x6d2
333
334
335
337
Signal strength (dBm)
6
![Page 7: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/7.jpg)
WiFi Localization
0xa3b 0x5fe 0xbc4 0x6d2
333 -55 -82 -39 -85
334 -30 -65 -63 -45
335 -60 -55 -50 -73
337 -72 -31 -73 N/A
Surveyor
Survey environment to build WiFi fingerprint database.
Signal strength (dBm)
7
![Page 8: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/8.jpg)
WiFi Localization
0xa3b 0x5fe 0xbc4 0x6d2
333 -55 -82 -39 -85
334 -30 -65 -63 -45
335 -60 -55 -50 -73
337 -72 -31 -73 N/A
Signal strength (dBm)
(-31,-66,-60,-40) dBm “Where am I?”
Survey environment to build WiFi fingerprint database.
8
![Page 9: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/9.jpg)
WiFi Localization
0xa3b 0x5fe 0xbc4 0x6d2
333 -55 -82 -39 -85
334 -30 -65 -63 -45
335 -60 -55 -50 -73
337 -72 -31 -73 N/A (-31,-66,-60,-40) dBm “Where am I?”
Survey environment to build WiFi fingerprint database.
Signal strength (dBm)
9
![Page 10: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/10.jpg)
WiFi Localization
• Fingerprinting-based methods require
extensive, costly survey.
• Not suitable for large-scale, long-term location services.
10
![Page 11: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/11.jpg)
Extending the Horizon
Users “Organic” place signatures collected by end-users.
Motion signatures The shape and the type of a space define motions.
11
![Page 12: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/12.jpg)
Thesis Contributions
• Part I: Algorithms for Organic Localization – User prompting – Erroneous user input filtering – Device heterogeneity
• Part II: Motion Compatibility-Based Localization – Motion labeling – Route network generation – Trajectory matching
12
TODAY
“Users”
“Motion signatures”
![Page 13: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/13.jpg)
II. Motion Compatibility-Based Indoor Localization Modern smartphones are equipped with a variety of sensors.
Accelerometer
Gyroscope
Magnetometer
Barometer Proximity sensor
Light sensor
Acoustic sensor GPS
WiFi
Bluetooth Motion sensors
13
![Page 14: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/14.jpg)
Previous Work
“Indoor Pedestrian Navigation” Step 1: Sensors at a fixed, known position Step 2: Step-counting & heading estimation Step 3: Kalman filters / particle filters
Ascher et al., “Dual IMU Indoor Navigation with Particle Filter based Map-Matching on a Smartphone”, IPIN, 2010.
Foot-mounted IMU for “zero velocity update”
Rai et al., “Zee: Zero-Effort Crowdsourcing for Indoor Localization”, MobiCom, 2012
Particle filtering
14
![Page 15: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/15.jpg)
Motion Compatibility-Based Indoor Localization
• Human motions in indoor environments
are highly structured. • The shape and the type of a space imply
a motion signature. • Observed motions originating path
15
![Page 16: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/16.jpg)
Recovering Paths from Motions
Original path
16
![Page 17: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/17.jpg)
Recovering Paths from Motions
Original path 3-motion sequence
17
![Page 18: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/18.jpg)
Recovering Paths from Motions
Original path 3-motion sequence Many plausible paths
18
![Page 19: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/19.jpg)
Recovering Paths from Motions
Original path 7-motion sequence
19
![Page 20: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/20.jpg)
Recovering Paths from Motions
Original path 7-motion sequence One matching path
20
![Page 21: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/21.jpg)
Recovering Paths from Motions 1. “Path-compatibility”: Metric/topological/semantic constraints
over paths. 1. Less constraints higher ambiguity. 2. More constraints Lower ambiguity.
2. (Accurate) motion labeling 3. (Automatic) map generation. 4. Uncertainty & noise in inputs. 5. Salient features: vertical motions, “opening a door”…
21
![Page 22: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/22.jpg)
Recovering Paths from Motions
Motion labeling Sensor data stream Motion sequence
Route network generation Floorplan Route network
Trajectory matching Motion + map User path
22
![Page 23: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/23.jpg)
Motion Models
• Rest (Sitting)
• Standing • Straight Walk • Turn • Walking Ascent / Descent (on stairs) • Elevator Up / Down • Access (“Opening a door”,
“Pressing a button”)
23
![Page 24: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/24.jpg)
Recovering Paths from Motions
Motion labeling Sensor data stream Motion sequence
Route network generation Floorplan Route network
Trajectory matching Motion + map User path
24
![Page 25: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/25.jpg)
Motion Labeling
• CRF (Conditional Random Field) based sequence classifier, labeling motions at 3Hz.
• Accelerometer/gyroscope/barometer/magnetometer • Challenge: Different motion durations. • Solution:
– Multiple feature windows with varying widths for each feature. – CRF learns an “optimal” weight for each window. – +10~15% improvement over single-window cases.
25
![Page 26: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/26.jpg)
Motion Labeling Demo
26
Principal Lateral Vertical Auxiliary
Accel- eration
Angular velocity
Atm. pressure
Magnetic field
![Page 27: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/27.jpg)
Motion Labeling Performance
• 94% per-frame overall “accuracy” (F-measure)
• Confusions between “Door Open” “Button Press” “Sitting” “Standing”
27
![Page 28: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/28.jpg)
Recovering Paths from Motions
Motion labeling Sensor data stream Motion sequence
Route network generation Floorplan Route network
Trajectory matching Motion + map User path
28
![Page 29: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/29.jpg)
Route Network Generation
CAD drawings (AutoCAD DXF)
By BMG (building model generation) group
Floor plan XML documents Medial axis approximation
(using constrained Delaunay triangulations)
Route networks Linked through horizontal & vertical adjacencies
29
![Page 30: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/30.jpg)
Route Network Generation
Stata Center, 1st~4th floor
30
![Page 31: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/31.jpg)
Recovering Paths from Motions
Motion labeling Sensor data stream Motion sequence
Route network generation Floorplan Route network
Trajectory matching Motion + map User path
31
![Page 32: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/32.jpg)
Trajectory Matching
• Inputs
• Sequence labeling with HMM. States: Nodes/edges in the route network, parameterized with directions.
32
(5 sec.) (1 sec.) (15 sec.) (10 sec.)
![Page 33: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/33.jpg)
Trajectory Matching: Models
Example: Elevator Up (11 sec.)
33
![Page 34: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/34.jpg)
Trajectory Matching: Models
Example: Elevator Up (11 sec.) 1. Elevator (21, 23, 24)
34
![Page 35: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/35.jpg)
Trajectory Matching: Models
Example: Elevator Up (11 sec.) 1. Elevator 2. Up (23, 24)
35
![Page 36: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/36.jpg)
Trajectory Matching: Models
Example: Elevator Up (11 sec.) 1. Elevator 2. Up 3. 11 sec (24)
36
![Page 37: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/37.jpg)
Trajectory Matching: Algorithms
• Formulation as an HMM:
– Inference (“decoding a trajectory”) using standard algorithms (forward-backward, Viterbi…)
– Parameter estimation (hard-EM) • Walking speed constant, matching flexibility
parameters…
37
![Page 38: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/38.jpg)
Trajectory Matching Demo
38
1st
2nd
3rd
4th
Ground-truth trace Best estimate Uncertainty
![Page 39: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/39.jpg)
Trajectory Matching Evaluation (1/4)
Ensemble error: It takes 1+ min. of data after a “cold start.” to achieve an accurate user path estimate.
39
![Page 40: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/40.jpg)
Trajectory Matching Evaluation (2/4)
Salient motions: Distinctive motions (vertical
transitions, in particular) facilitate matching.
40
![Page 41: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/41.jpg)
Trajectory Matching Evaluation (3/4)
Prior information: Knowing the starting floor makes the matching faster.
41
65 sec.51 sec.
![Page 42: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/42.jpg)
Trajectory Matching Evaluation (4/4)
Computation time: Linear in the number of states (map size) & the length of the input (input size).
42
![Page 43: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/43.jpg)
Extending the Horizon, Further
• Any sensor data can form signatures. • Multiple trips / multiple users
– Inference on chains on networks
• Bootstrapping signatures
43
![Page 44: Indoor Localization using Place and Motion Signatures · Indoor Localization using Place and Motion Signatures Jun-geun Park Thesis Committee: Prof. Seth Teller (Advisor) Prof. Nick](https://reader034.vdocuments.net/reader034/viewer/2022042309/5ed7099262136e72fb7bb73f/html5/thumbnails/44.jpg)
Contributions
• Part I: Algorithms for Organic Localization – User prompting – Erroneous user input filtering – Device heterogeneity
• Part II: Motion Compatibility-Based Localization – Motion labeling – Route network generation – Trajectory matching
44
• Park et al., “Growing an Organic Indoor Location System”, MobiSys, 2010
• Park et al., “Implications of Device Diversity for Organic Localization”, INFOCOM, 2011
• Park et al., “Online Pose Classification and Walking Speed Estimation using Handheld Devices”, UbiComp, 2012
• Park et al., “Motion Compatibility for Indoor Localization”, submitted