decision-point panorama-based indoor navigation
DESCRIPTION
We present a novel user interface concept for indoor navigation which uses directional arrows and panorama images at decision points. The interface supports the mental model of landmark-based navigation, can be used on- and offline and is highly tolerant to localization inaccuracy.TRANSCRIPT
![Page 1: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/1.jpg)
Decision-Point Panorama-Based Indoor Navigation
Andreas Möller, Matthias Kranz, Luis Roalter, Stefan Diewald, Kåre Synnes
MCPT Workshop on Eurocast 2013, February 15, 2013
![Page 2: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/2.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
![Page 3: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/3.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Background and Motivation § Location information still the
most important contextual information § Indoor localization is a hot topic
and useful for a lot of locations: □ Airports □ Hospitals □ Conference venues □ Large environments
§ Various indoor localization methods possible: □ WLAN/cell-based localization □ Sensor-based localization □ Beacon-based localization □ Vision-based localization
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 4: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/4.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Vision-Based Localization
§ Principle □ Comparing of query images, taken with
camera, to reference images based on visual features
§ Advantages □ No infrastructure and augmentation of the environment needed □ Immune to disturbances (reflection/refraction…) □ Works with existing
(modern) hardware
Live features
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 5: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/5.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
![Page 6: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/6.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
HCI Perspective: Interaction Concept
§ Augmented Reality View for intuitiveness □ But: Consequence when location estimate
is inaccurate: wrong overlays!
§ Permanent re-localization during path required □ High computational effort □ Uncomfortable camera pose □ Not all scenes adequate for visual localization
(sometimes insufficient features for algorithm)
?
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 7: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/7.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Interaction Concept (cont.)
Instead: § Re-localization from time to time
(when possible) □ Estimation of intermediate position
with odometry (device sensors) □ More comfortable usage
§ “Virtual Reality” View □ Regularly show 360° panorama images
of the environment with embedded navigation instructions (those panoramas are already available from reference images)
□ More robust: user matches virtual and real world
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 8: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/8.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Initial User Study
§ Comparison of Augmented Reality and Virtual Reality
§ Simulation of different levels of accuracy
§ Survey of user preferences
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
See Full Paper: A. Möller, M. Kranz, R. Huitl, S. Diewald, L. Roalter A Mobile Indoor Navigation System Interface Adapted to Vision-Based Localization In: 11th Intl. Conf. on Mobile and Ubiquitous Multimedia (MUM 2012)
![Page 9: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/9.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Initial User Study: Summary
§ Panoramas provide better guidance when location estimate is inaccurate than Augmented Reality
§ But: Subjects experienced frequent changes (~2m) as irritating, as □ Shown locations are not always correct □ Locations sometimes do not look like in reality □ Frequent changes in the UI are visually straining
§ Subjects manage finding their goal, also when not looking at the screen all the time
§ How to improve? □ Reduce frequent visual changes □ Simplify match between panoramas and real world (problem of
misplaced panoramas?) 15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 10: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/10.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
![Page 11: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/11.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible (by swiping through list of route instructions)
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 12: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/12.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 13: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/13.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 14: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/14.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 15: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/15.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 16: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/16.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 17: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/17.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Decision-Point Based Navigation (DPBN)
§ Show only panoramas of decision points § Compliance with familiar mental model of self-orientation § Users can always re-localize and retrieve the panorama
of their current location § High error tolerance § Offline usage possible
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 18: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/18.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
![Page 19: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/19.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
User Study: Research Questions
§ RQ1: Does DPBN have an effect on efficiency? Are users as fast as with continuous panoramas?
§ RQ2: Is DPBN more convenient than continuous panoramas? What mode do users prefer? How well do they feel guided?
§ RQ3: What usage patterns can be identified? What do we learn for designing an “ideal” route description?
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 20: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/20.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
User Study – Quick Facts
3 conditions - Continuous panoramas (automatic *) - Decision points (automatic *) - Decision points (manual navigation)
3 routes (avoidance of learning effects) Measured data: time, user behavior, questionnaires 12 Participants, 75% male, 25% female, average 25 years Within-subjects design 75% own a smartphone, only one has experience with indoor navigation
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Source: iconsdb.com
* Wizard of Oz
![Page 21: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/21.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Results: RQ1
196 208 263
0
50
100
150
200
250
300
350
Continuous (automatic)
Decision-Point (automatic)
Decision-Point (manual)
Average Time per Mode
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Tim
e in
sec
onds
Difference not
significant
![Page 22: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/22.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Results: RQ2
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
I found the method pleasing to use.
Continuous (automatic)
Decision-Point (automatic)
Decision-Point (manual)
5: fully agree 1: fully disagree
- Median + Mean
![Page 23: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/23.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
I felt guided well to the goal.
Continuous (automatic)
Decision-Point (automatic)
Decision-Point (manual)
Results: RQ2
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
5: fully agree 1: fully disagree
- Median + Mean
![Page 24: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/24.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Which system would you choose?
Results: RQ2
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
75%
17%
8% Continuous (automatic) Decision-Point (automatic) Decision-Point (manual)
Panoramas of decision points are sufficient for
orientating
5: fully agree 1: fully disagree
![Page 25: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/25.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
0
10
20
30
40
20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340
Dis
tan
ce (
m)
Time (s)
Participant 12 - Path 1
swipe relocalize Δd position
Results: RQ3
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 26: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/26.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
-60
-50
-40
-30
-20
-10
0
10
20
30
40
50
60
20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340
Dis
tance
(m
)
Time (s)
Participant 2 - Path 2
swipe relocalize Δd position
Results: RQ3
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 27: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/27.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
-20
-10
0
10
20
30
20 40 60 80 100 120 140 160 180 200 220
Dis
tan
ce (
m)
Time (s)
Participant 5 - Path 3
swipe relocalize Δd position
Results: RQ3
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 28: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/28.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Results: RQ3 – Manual Mode
§ Average numbers □ 52.5 continuous locations (automatic, continuous mode) □ 10.3 decision points (automatic, decision-point mode) □ 15.3 manual swipes □ 9.3 times relocalized
§ Different usage patterns/strategies □ Showing “ahead” panoramas, no use of automatic re-localization □ Frequent use of re-localization, (sometimes only in parts of the
route) – equivalent to continuous panorama mode □ “Memorizing” of the entire route not observed, subjects only
looked at the “next” panorama location
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 29: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/29.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Overview
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
Motivation: Vision-Based Localization
UI Concept and First Evaluation
Decision-Point Based Navigation
User Study
Discussion and Future Work
![Page 30: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/30.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Discussion
§ Subjects were equally fast with DPBN as with continuous panoramas (RQ1)
§ Subjects felt guided better with continuous panoramas (RQ2) □ Due to more certainty whether they are on the right way □ Confirmed by frequent swipes and re-localizations in manual mode
§ DPBN is desirable from a technical point of view (reliability…), but acceptance must be increased
§ Requires stronger confirmation whether subjects are (still) on the right way
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 31: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/31.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
How could this be achieved? (Lessons learned) § Compromise between continuous mode and DPBN as it is now
§ Distance information § Verification when decision point was passed § Add intermediate landmarks in decision point mode □ Especially if decision point lies too far away (distance threshold)
§ Simplify the mapping of picture content to real environment □ Highlight significant objects in panorama
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 32: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/32.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Summary and Outlook
§ With Decision-Point Based Navigation, we have presented a novel interface for visual indoor navigation □ Very robust in case of localization inaccuracy □ Works even if localization fails (manual mode) □ Supports users’ mental model of navigation (landmarks)
§ Real-world study results □ DPBN as efficient as full panorama navigation □ DPBN needs more information to reduce users’ uncertainty
(landmarks, confirmations) to increase satisfaction § Future work □ See previous slide ;-) □ Identification of good candidates for (additional) key points
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 33: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/33.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Thank you for your attention! Questions?
www.vmi.ei.tum.de/team/andreas-moeller.html
This research project has been supported by the space agency of the German Aerospace Center with funds from the Federal Ministry of Economics and Technology on the basis of a resolution of the German Bundestag under the reference 50NA1107.
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 34: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/34.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
Paper Reference
§ Please find the associated paper at: https://vmi.lmt.ei.tum.de/publications/2013/MCPT2013-IndoorNav_preprint.pdf
§ Please cite this work as follows: § A. Möller, M. Kranz, L. Roalter, S. Diewald, Kåre Synnes
Decision-Point Panorama-Based Indoor Navigation In: 14th International Conference on Computer Aided Systems Theory (EUROCAST 2013), pp. 325-326, Las Palmas de Gran Canaria, Spain, February 2013
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
![Page 35: Decision-Point Panorama-Based Indoor Navigation](https://reader034.vdocuments.net/reader034/viewer/2022052619/555047e8b4c905b2788b4e14/html5/thumbnails/35.jpg)
Technische Universität München Institute for Media Technology Distributed Multimodal Information Processing Group
If you use BibTex, please use the following entry to cite this work:
15.2.2013 A. Möller, M. Kranz, L. Roalter, S. Diewald, K. Synnes
@INPROCEEDINGS{MCPT13IndoorNav, author = {Andreas M{\"o}ller and Matthias Kranz and Luis Roalter and Stefan Diewald}, title = {{Decision-Point Panorama-Based Indoor Navigation}}, booktitle = {14th International Conference on Computer Aided Systems Theory (EUROCAST 2013)}, editor = {Alexis Quesada-Arencibia and Jos\'{e} Carlos Rodriguez and Roberto Moreno-Diaz jr. and Roberto Moreno-Diaz}, year = {2013}, month = feb, pages = {325--326}, ISBN = {978-84-695-6971-9}, location = {Las Palmas de Gran Canaria, Spain}, }