tool support for prototyping interfaces

Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz

Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation

Andreas Möller1, Christian Kray2, Luis Roalter1, Robert Huitl1, Stefan Diewald1, Matthias Kranz3

1Technische Universität München, Germany

2University of Münster, Germany 3Luleå University of Technology, Department of Computer Science,

Electrical and Space Engineering, Luleå, Sweden

1st Workshop on Mobile Vision and HCI held at MobileHCI 2012, San Francisco


Background and Motivation

•  Location information still the most important contextual information

•  Indoor localization is a hot topic and useful for a lot of scenarios: –  Airports –  Hospitals –  Conference venues –  Large environments

•  Various indoor localization methods possible: –  WLAN/cell based localization –  Sensor based localization –  Beacon based localization –  Vision based localization

21.09.2012 Luis Roalter 2



Why use vision based localization?


Why vision based localization?

•  Use existing features in the environment

•  Modern devices are equipped with a camera and powerful processor

•  No additional augmentation needed




Which challenges will appear?


Challenges of Vision-Based Localization

•  No fixed device needed –  Use your own smartphone without any optional accessories –  Hold the device as you like (landscape, portrait, to the ground, to the wall)

•  Quality of visual features –  Repeating furniture in one hallway –  Wall of the same paint

•  Orientation of the user’s smartphone –  Features are typically found in eye height (windows, adverts, posters)

•  In a typical pose, user holds the device in a way that it points at the floor



Challenges in HCI

•  User interfaces need to support/address those problems: –  Deal with accuracy –  Corrective actions if not enough features are available

•  Hard to evaluate (would require already functional localization, but interfaces goes hand in hand with algorithms)

•  Little/no experience with developing user interfaces for vision-based systems

•  There is a need for adequate prototyping tools!




Prototyping. What do we need?


Prototyping Indoor Navigation Interfaces

•  Requirements:

–  Prototyping in early development

–  Ease change of the location (e.g. choose a new environment)

–  Evaluation of usability, efficiency and effectivity

–  Controlled conditions of localization conditions (e.g. location and orientation accuracy)

–  Evaluation of individual features in real-world scenarios



The Wizard of Oz Approach

Experimenter‘s Phone(Server)

Subject‘s Phone(Client)

Send image directly OR

send image ID of local database Local Image DB

Trigger UI events



User Interface Concept

•  Use the camera of the smartphone

•  Enable the usage of: –  Virtual Reality (VR) with panorama images (compare to

Google StreetView) taken in advance –  Augmented Reality (AR) with a live view of the environment

•  In both cases the user needs to hold the camera device towards interesting regions.

•  How to motivate the user to hold up the smartphone?



Augmented Reality •  Display the live camera-image

•  Draw information within the image

•  Hold the smartphone in eye-height

•  Live view of the environment

•  Ease match from to the real environment




Virtual Reality •  Preloaded panorama

images

•  Similar images to the real environment

•  Holding the smartphone to the ground

•  No orientation change needed

•  User has to match real and virtual reality


Mockup


•  Testing the augmentation content


What if the algorithm needs a feature-rich region?


•  Motivate the user to look at the phone to follow the instructions (in both AR and VR conditions!)


Theoretical Evaluation

•  Evaluation based on the initial requirements –  Use the prototype at early project stage (even before localization is

implemented) –  Testing AR and VR interface concepts on a real smartphone –  Inaccuracy possible due to manual transmission of images

–  Different motivation screens to point upon a feature-rich region

•  All initial requirements are met.



•  5 subjects evaluating the mockup system •  Predefined route through the building:

–  About 100m –  Dataset recorded with LadyBug

Camera system –  Corridors and a hall including doors

Practical Evaluation



Findings of the first evaluation

•  AR considered as nice and convenient •  AR Overlay was not always accurate, felt disturbing

(in case of orientation error)

•  VR had offset in location •  But user can deal better with inaccuracy, matching panoramas to the real

environment



Interface Design and Usability

•  Future research questions:

–  Which visualization deals better with varying conditions?

–  Automatic vs. manual adjustments by the user?

–  Preferences for proactive behavior?

–  Are VR and AR sufficient for navigation tasks?



Ongoing related work

•  Test the application within a large environment (a larger route with multiple turns)

•  Use dataset of a complete building recorded in advance (for VR)

•  Evaluate the user‘s experience with the application user-interface

•  Survey with 30 participants



Current Implementation


Subject’s Screen (top) Drawing of images in AR or VR mode, connected to the server device

Experimenter’s Screen (right) Send specific images to the client


Path trough the building


•  Including hallways, corridors and different turns

•  Evaluate the user’s experience in case of errors

•  Current research topic and related user study will be published soon


Summary

•  All initial requirements are met –  Testing of the prototype without having an actual localization –  Different views and augmentations are possible

•  A larger study with a longer way is currently evaluated with the same prototype

•  Full functional prototype framework available (for public on request)

•  Wizard-of-Oz approach can be replaced step by step



Map



Thank you for your attention! Questions?

? ? 21.09.2012 Luis Roalter 25

[email protected] www.vmi.ei.tum.de/team/luis-roalter.html


Paper Reference

•  Please find the associated paper at: https://vmi.lmt.ei.tum.de/publications/2012/mobivis_v02_preprint.pdf

•  Please cite this work as follows: •  Andreas Möller, Christian Kray, Luis Roalter, Stefan Diewald, Rorbert Huitl,

Matthias Kranz. 2012. Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation In: Workshop on Mobile Vision and HCI (MobiVis) on MobileHCI 2012, San Francisco, USA, September 2012



If you use BibTex, please use the following entry to cite this work:


@INPROCEEDINGS{Mobivis12moeller, author={Andreas M\"{o}ller and Christian Kray and Luis Roalter and Stefan Diewald and Robert Huitl and Matthias Kranz}, title={{Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation}}, booktitle={{Proceedings of the Workshop on Mobile Vision and HCI (MobiVis). Held in Conjunction with Mobile HCI}}, year={2012}, month=sep, location={San Francisco, USA}, abstract={Vision-based approaches are a promising method for indoor navigation, but prototyping and evaluating them poses several challenges. These include the effort of realizing the localization component, difficulties in simulating real-world behavior and the interaction between vision-based localization and the user interface. In this paper, we report on initial findings from the development of a tool to support this process. We identify key requirements for such a tool and use an example vision- based system to evaluate a first prototype of the tool.}, }

tool support for prototyping interfaces

Documents

motivationlocation information

localization vision

luis roalter10

luis roalter4

luis roalter13

luis roalter14

luis roalter12

luis roalter2