3D Interaction Techniques for Virtual Environments

Doug A. Bowman

(C) 2005 Doug Bowman, Virginia Tech 2

Terminology

Interaction Technique (IT) – method for accomplishing a task

3D application – system that displays 3D information

3D interaction – performing user actions in three dimensions

(C) 2005 Doug Bowman, Virginia Tech 3

Didn’t we already cover input devices?

SystemSoftware

Use

r in

terf

ace

soft

war

e

User

Inputdevices

Outputdevices

ITs

(C) 2005 Doug Bowman, Virginia Tech 4

Video example: ISAAC

QuickTime™ and a MPEG-4 Video decompressor are needed to see this picture.

(C) 2005 Doug Bowman, Virginia Tech 5

Universal interaction tasks

Navigation Travel – motor component Wayfinding – cognitive component

Selection

Manipulation

System control

Symbolic input

(C) 2005 Doug Bowman, Virginia Tech 6

Selection & Manipulation

Selection: specifying one or more objects from a set

Manipulation: modifying object properties (position, orientation, scale, shape, color, texture, behavior, etc.)

(C) 2005 Doug Bowman, Virginia Tech 7

QuickTime™ and a MPEG-4 Video decompressor are needed to see this picture.

(C) 2005 Doug Bowman, Virginia Tech 8

Goals of selection

Indicate action on object

Query object

Make object active

Travel to object location

Set up manipulation

(C) 2005 Doug Bowman, Virginia Tech 9

Selection performance

Variables affecting user performanceObject distance from userObject sizeDensity of objects in areaOccluders

(C) 2005 Doug Bowman, Virginia Tech 10

Common selection techniques

Touching with virtual hand

Ray/cone casting

Occlusion / framing

Naming

Indirect selection

(C) 2005 Doug Bowman, Virginia Tech 11

Enhancements to basic techniques

Arm-extensionMapping“Reeling”

2D / 3D World in MiniatureSelect iconic objects

(C) 2005 Doug Bowman, Virginia Tech 12

th

thth

Go-Go implementation

Requires “torso position” t - tracked or inferred

Each frame: Get physical hand position h in world CS Calculate physical distance from torso dp = dist(h, t)

Calculate virtual hand distance dv = gogo(dp)

Normalize torso-hand vector

V. hand position v = t + dv *(h-t) (in world CS)

(C) 2005 Doug Bowman, Virginia Tech 13

QuickTime™ and a MPEG-4 Video decompressor are needed to see this picture.

(C) 2005 Doug Bowman, Virginia Tech 14

Selection classification

Selection

Feedback

Object indication

Indication to select

graphicaltactileaudio

object touchingpointingindirect selection

buttongesturevoice

(C) 2005 Doug Bowman, Virginia Tech 15

Evaluation: Selection Task

Ray-casting and image-plane generally more effective than Go-Go

Exception: selection of very small objects can be more difficult with pointing

Ray-casting and image-plane techniques result in the same performance (2DOF)

Image-plane technique less comfortable

(C) 2005 Doug Bowman, Virginia Tech 16

Implementation issues for selection techniques How to indicate selection event Object intersections Feedback

Graphical Aural Tactile

Virtual hand avatar List of selectable objects

(C) 2005 Doug Bowman, Virginia Tech 17

Implementation issues for selection techniquesHow to indicate selection event

Object intersections

Feedback Graphical Aural Tactile

Virtual hand avatar

List of selectable objects

(C) 2005 Doug Bowman, Virginia Tech 18

SIGGRAPH 2001 tutorial 자료

(C) 2005 Doug Bowman, Virginia Tech 19

Common selection techniques

Simple virtual hand

Ray-casting

Sticky finger (occlusion)

Go-go (arm-extension)

(C) 2005 Doug Bowman, Virginia Tech 20

Simple virtual hand technique

One-to-one mapping between physical and virtual hands

Object can be selected by “touching” or intersecting v. hand with object

(C) 2005 Doug Bowman, Virginia Tech 21

Ray-casting technique

“Laser pointer” attached to v. hand

First object intersected by ray may be selected

User only needs to control 2 DOFs

Empirically proven to perform well

(C) 2005 Doug Bowman, Virginia Tech 22

Ray-casting implementationNaïve: intersect ray with each

polygon Parametric equation Only consider intersections with t > 0

Better: transform vertices (or bounding box) to hand’s CS Drop new z coordinate of every vertex Ray intersects polygon iff (0,0) is in the

polygon Count the number of times the polygon

edges cross the positive x-axis

(C) 2005 Doug Bowman, Virginia Tech 23

(C) 2005 Doug Bowman, Virginia Tech 24

Occlusion techniqueImage-plane technique - truly

2D

Occlude/cover desired object with selector object (e.g. finger)

Nearest object along ray from eye through finger may be selected

(C) 2005 Doug Bowman, Virginia Tech 25

Occlusion implementationSpecial case of ray-

casting technique

Must consider position of eye/camera

Can use 2nd ray-casting algorithm; requires special object

ß

ß

(C) 2005 Doug Bowman, Virginia Tech 26

Go-Go techniqueArm-extension

techniqueLike simple v. hand,

touch objects to select them

Non-linear mapping between physical and virtual hand position

Local and distant regions

(C) 2005 Doug Bowman, Virginia Tech 27

Go-Go implementationRequires “torso position” t - tracked or inferred

Each frame: Get physical hand position h in world CS Calculate physical distance from torso dp = dist(h, t)

Calculate virtual hand distance dv = gogo(dp)

Normalize torso-hand vector

V. hand position v = t + dv*th (in world CS)

th

thth