mobimed: comparing object identification techniques on smartphones

Technische Universität München Distributed Multimodal Information Processing Group

MobiMed: Comparing Object Identification Techniques

on Smartphones

Andreas Möller1, Stefan Diewald1, Luis Roalter1, Matthias Kranz2

1Technische Universität München, Germany 2Luleå University of Technology, Department of Computer Science,

Electrical and Space Engineering, Luleå, Sweden

October 15, 2012 NordiCHI, Copenhagen, Denmark


Outline

Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 2

Background and Motivation

Scenario and Prototype

User Study

Discussion and Conclusion



•  Idea of bridging the gap between the physical and the virtual world for easier interaction and additional functionality –  Connect physical objects with virtual representations by tags

(Want et al., 1999) –  Physical mobile interaction (Rukzio, 2006)

•  Investigation and comparison of different interaction techniques done earlier, BUT: –  meanwhile outdated technologies (e.g. IR) –  older comparisons based on (nowadays) limited hardware

(VGA cameras, small screens, slow mobile CPUs) –  new technologies have emerged (e.g. vision-based approaches) –  user knowledge and experience has changed

Suggesting a new comparison of (state-of-the-art) interaction techniques



Outline




User Study



Scenario for Physical Mobile Interaction

•  MobiMed: identifying medication packages with the smartphone

•  Target groups: active people pursuing a healthy lifestyle, elderly people

•  Physical mobile interaction to get information on drugs –  package insert –  side effects –  active ingredients –  cross-correlations




Investigated Interaction Types

Pointing (tag-less vision-based identification)

Touching (radio tags, e.g. NFC or RFID)

Scanning (visual tags, e.g. bar codes)

Text Input (e.g. name, ID, …)


Excursus: Pointing (Vision-based Recognition)

•  Image processing is used to detect visual features of an image

•  A query in feature space returns similar images from a reference database

•  Good choice of feature type allows very reliable results (e.g. MSER) –  High distinctiveness (e.g. by

using text-related features) –  Scale invariance (works at

different distances) –  Rotation invariance (works at

different angles) •  Enabled by rise in mobile CPU

performance (multi-core...)



Prototype

•  Implementation as Android application •  47,000 drugs in query database •  100,000 reference images



Outline




User Study



Research Questions

•  RQ1: What advantages and disadvantages of identification techniques, as presented in MobiMed, can be determined?

–  ...in terms of effectiveness? large-scale, online –  ...in terms of efficiency? lab

•  RQ2: Which method is preferred by users? –  ...a priori? large-scale, online –  ...after practical use? lab

•  RQ3: What potential do people see for MobiMed as a whole? –  ...a priori? large-scale, online –  ...after practical use? lab



Methodology

•  Online study –  Human Intelligence Task at Amazon mTurk –  149 participants

•  74 females, 75 males •  17-79 years (average: 31, standard deviation: 11)

–  Questionnaire survey

•  Lab study –  16 participants

•  6 females, 10 males •  22-69 years (average: 31, standard deviation: 12)

–  Experimental task + Questionnaire survey •  Identification of 10 packages

with each of four methods •  Within-subjects design, permuted order



Results: RQ1 (Individual Method Comparison)


Method Advantages Disadvantages

Scanning Quick, precise, high familiarity

Visual code + camera required, need to find and focus on code

Touching Hassle-free, fool-proof, quick

NFC augmentation and NFC-capable phone required, privacy skepticism

Pointing Intuitive to use, „most human form“ of interaction, works from any angle, works also with catalog/website images, no product tagging required

Computational demand, ambiguous results possible

Text Highest familiarity, accurate, search term flexibility

High amount of typing, misspelling, slow, difficult


Results: RQ1 (Efficiency)



Results: RQ2 (User Preferences)


Observations/interpretations: •  Touching was only #3 in online survey, but rated best in lab study •  Possible explanation: low familiarity (as soon as people used it, they liked it)

-3 = strongly disagree, +3 =strongly agree


Results: RQ3 (Utility of Tool in Scenario)

•  Information sources on drugs: –  Doctor/pharmacist (75%) –  Package insert (69%) –  Books/internet (56%)

•  Would you be interested in MobiMed as alternative source for drug information? 88%

•  Would you use a system such as MobiMed? 82%

•  Average amount of money subjects would spend: $8.40 (aged >25: $14.01)

•  Suggestions for additional features –  Price comparison –  Active ingredient analysis –  Self-diagnose –  Personalized medication

management



Results: RQ3 (Usability of Prototype)



Outline




User Study




•  Physical Mobile Interaction is popular and efficient –  Was preferred over conventional (text) search –  Was faster than text search

•  Touching and Scanning evaluated best –  Fastest and most popular physical mobile interaction methods –  Touching faster and more popular than scanning in lab study –  Scanning more popular in online survey (familiarity)

•  Vision-based Search (pointing) as future alternative? –  Natural; works for any object (no augmentation needed) –  Reliability/speed improvement needed, but almost as fast as scanning

•  Best method depends on intended scenario •  General demand for medical apps



Thank you for your attention! Questions?

? ? Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 19

[email protected] www.vmi.ei.tum.de/team/andreas-moeller.html


References

•  Slide 3: –  Rukzio, E. Physical mobile interactions: Mobile devices as pervasive mediators for interactions

with the real world. PhD thesis, 2006 –  Want, R., Fishkin, K., Gujar, A., and Harrison, B. Bridging physical and virtual worlds with

electronic tags. In Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, ACM (1999), 370–377.

•  Slide 10: https://www.mturk.com/mturk/welcome

•  All other images: Microsoft ClipArt 2012



Paper Reference

•  Please find the associated paper at: http://dx.doi.org/10.1145/2399016.2399022

•  Please cite this work as follows: •  Andreas Möller, Stefan Diewald, Luis Roalter, and Matthias Kranz. 2012.

MobiMed: comparing object identification techniques on smartphones. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12). ACM, New York, NY, USA, 31-40. DOI=10.1145/2399016.2399022 http://doi.acm.org/10.1145/2399016.2399022



If you use BibTex, please use the following entry to cite this work:


@inproceedings{Moller:2012:MCO:2399016.2399022, author = {M\"{o}ller, Andreas and Diewald, Stefan and Roalter, Luis and Kranz, Matthias}, title = {MobiMed: comparing object identification techniques on smartphones}, booktitle = {Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design}, series = {NordiCHI '12}, year = {2012}, isbn = {978-1-4503-1482-4}, location = {Copenhagen, Denmark}, pages = {31--40}, numpages = {10}, url = {http://doi.acm.org/10.1145/2399016.2399022}, doi = {10.1145/2399016.2399022}, acmid = {2399022}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {object identification, physical mobile interaction, pointing, scanning, touching}, }

mobimed: comparing object identification techniques on smartphones

Technology

visionbased identification

human form of interaction

motivation scenario

questionnaire surveylab

physical objects

practical use

standard deviation

years average