occlusion based interaction methods tangible augmented...
TRANSCRIPT
VR Lab HITLabNZPOSTECH Univ. of Canterbury
Occlusion based Interaction Methodsfor
Tangible Augmented Reality Environments
Gun A. Leeα Mark Billinghurstβ Gerard J. Kimα
α Virtual Reality Laboratory, Pohang University of Science and Technology
β Human Interface Technology Laboratory New Zealand,University of Canterbury
VRCAI 2004
VR LabHITLabNZ
2VRCAI 2004
Introduction
■ Tangible Augmented Reality[Kato 2001]
■ Each virtual object registered to physical object
■ Users manipulate virtual objects by manipulating corresponding physical objects
VR LabHITLabNZ
3VRCAI 2004
Introduction
■ Tangible AR Applications
▲MagicBook
ARGroove ►
▼ VOMAR
◄ Tiles
VR LabHITLabNZ
4VRCAI 2004
Introduction
■ Tangible Augmented Reality■ Easy natural 3D interactions
■ Full 6DOF 3D Direct manipulation
■ 1/2D interactions?■ Buttons?■ Sliders?■ Menus?■ Keypads?
VR LabHITLabNZ
5VRCAI 2004
Related Works
■ 2D Input methods in AR■ Screen stabilized fashion
■ Tinmith [Piekarski 2002]
■ Magic Ring [Dias 2003]
■ World stabilized fashion■ Windows on World■ ARGUI [Geiger 2003]
■ Hand gesture mouse [McDonald 2003]
■ Used indirect input methods (mouse, getsture, etc) rather than direct touch
VR LabHITLabNZ
6VRCAI 2004
Related Works
■ Large scale touch sensor■ SmartSkin [Rekimoto 2002]
■ Touch implies occlusion■ Commercial products using
occlusions as inputs■ Virtual Keyboard, Canesta■ Parapara Paradise, KONAMI
VR LabHITLabNZ
7VRCAI 2004
Occlusion based Interaction
■ 2D Interaction using pointers■ Pointer centered view
■ Tracking the pointers■ Cost depends on number of pointers + interaction
points■ Interaction point centered view
■ Sensing on interaction points■ Cost depends on number of interaction points
■ In TAR environments, interaction points (props) are trackedrather than pointers (hands).
VR LabHITLabNZ
8VRCAI 2004
Occlusion based Interaction
■ Occlusion Detection in TAR■ Tracking a set of visual markers with
predefined spatial relationship■ Provides robustness■ Invisible marker positions are inferred from
the visible ones
■ Occlusion = Invisible + Within the view■ Two methods for detecting occlusions
■ Boundary Marker Method■ Estimated Marker Projection Method
VR LabHITLabNZ
9VRCAI 2004
Occlusion based Interaction
■ Boundary Marker Method
VR LabHITLabNZ
10VRCAI 2004
Occlusion based Interaction
■ Estimated Marker Projection Method
VR LabHITLabNZ
11VRCAI 2004
Occlusion based Interaction
■ Boundary Marker Method■ Easy and straight forward, reliable
implementation■ Lesser computation
■ Estimated Marker Projection Method■ No need of boundary markers■ More computation required■ Estimation errors due to lens distortion
■ Over estimating the size of markers helps
VR LabHITLabNZ
12VRCAI 2004
Occlusion based Interaction
■ Interaction Design■ Time out constraints
■ Point & hold -> Explicit command■ Preventing accidental commands■ Repeating the command (common in
keyboards)
VR LabHITLabNZ
13VRCAI 2004
Occlusion based Interaction
■ Interaction Design (cont’d)
■ Sub-marker level measurement■ Two consecutive markers occluded
-> The pointer is in-between them■ Relatively lesser markers needed
VR LabHITLabNZ
14VRCAI 2004
Occlusion based Interaction
■ Interaction Design (cont’d)
■ Tip point marker detection■ 2D grid of markers■ Top-left one from the
occluded marker blob■ Other heuristics applicable
– e.g. select top-right one when the blob is skewed to the right
VR LabHITLabNZ
15VRCAI 2004
Implementation
■ Desktop■ Custom built PC with Windows XP■ Athlon 1.5GHz / 512MB■ NVIDIA GeForce4 MX
■ Mobile■ Apple Macintosh iBook with Mac OS X■ G3 900MHz / 640MB■ ATI Radeon 7500
VR LabHITLabNZ
16VRCAI 2004
Implementation
■ ARToolKit (2.65DS / 2.61Mac)■ Logitech Notebook PRO USB Camera
■ 320x240 @ 30Hz■ i-Visor HMD
■ Video image stretched to fit 800x600■ OpenGL drawn in full resolution
■ Performance: 19~30fps■ 5fps with 35 markers on mobile
configuration (low computation power)
VR LabHITLabNZ
17VRCAI 2004
Applications
■ Buttons
VR LabHITLabNZ
18VRCAI 2004
Applications
■ Sliders
VR LabHITLabNZ
19VRCAI 2004
Applications
■ Menu bars
VR LabHITLabNZ
20VRCAI 2004
Applications
■ Ball-pushing Game
VR LabHITLabNZ
21VRCAI 2004
Applications
■ Calculator
VR LabHITLabNZ
22VRCAI 2004
Applications
■ Board games
VR LabHITLabNZ
23VRCAI 2004
Discussion
■ Informal user study■ 6 subjects■ 7 scale questionnaire
■ Easy to learn: 5.6 (stddev=0.55)■ Easy to use: 5 (stddev=0.7)
■ Two 2D game applications demonstrated on public■ Over 100 peoples tried and gave
positive feedbacks
VR LabHITLabNZ
24VRCAI 2004
Discussion
■ The interaction appeared natural■ Using bare hands■ Direct manipulation (pointing)
■ Occlusion of users’ hands by virtual objects were reported unnatural ■ Partially solved by making the interface
semitransparent when occluded■ Easily applicable to mobile and/or
wearable systems
VR LabHITLabNZ
25VRCAI 2004
Conclusion & Future Works
■ Occlusion based Interaction for Tangible AR Environments■ Simple, easy to use, and natural 2D interaction■ Low development and computational cost
■ Future Works■ Try other type of visual markers or visual
tracking methods■ Add passive tactile feedbacks