3d interaction techniques for virtual environments

3D Interaction Techniques for Virtual Environments

Doug A. Bowman

Edited by C. Song

(C) 2005 Doug Bowman, Virginia Tech 2

Technique Classificationby metaphor

VE manipulation techniquesExocentric metaphor

Egocentric metaphor

World-In-Miniature

Virtual Hand metaphor

Virtual Pointer metaphor

Go-GoIndirect, stretch Go-Go

"Classical" virtual hand

Ray-castingApertureFlashlightImage plane

Scaled-world grab


Manipulation metaphors I

Ray casting little effort requiredExact positioning and orienting very difficult

(lever arm effect)


5.4.2 Interacting by Pointing Selection process

When the vector defined by the direction of pointing intersects a virtual object, the user can select it by triggering event that confirms the selection.

Examples of triggers are buttons and voice command A number of pointing techniques have been reported

How the pointing direction is defined By the shape of the selection volume

Pointing is a powerful selection technique Pointing, however, is generally a very poor positioning

(Manipulation) technique Rotation can be effectively accomplished only about one

axis. Expressive 6-DOF manipulation is therefore impossible


5.4.2 Interacting by Pointing

Simple ray-casting technique

Two-handed pointing

Flashlight / Aperture techniques

Image-plane techniques

Fishing-reel techniques


Ray Casting (I) The user points at objects with a virtual ray

that defines the direction of pointing. Attached to the virtual hand controlled by a 6-DOF

sensor in immersive environment. Attached to a 3D widget controlled by a mouse in a

desktop 3D environment. Pointing direction

p() = h + P, where P = the direction of user’s virtual hand, h= 3D position of the virtual hand

More than one object can be intersected by the line p(), and only one closest to the user should be selected.


Ray Casting (II) The shape of the ray

Short line segment; Figure 5.4 Infinitely long virtual ray; better visual feedback

Virtual ray casting is a very powerful selection technique Except when a very high precision of selection is

required; selecting small or faraway objects At close range, ray-casting is perhaps the most

simple and efficient selection technique.


Two-Handed Pointing Two handed technique: one hand specifies the origin

of the virtual ray, while the other hand specifies where the ray is pointing to

p() = hl + ( hr – hl ) , where hr and hl = 3D position of the right and left virtual hands respectively.

Disadvantage Both hands must be tracked.

Advantage Allow for richer and more effective pointing interaction Curve the virtual pointer by twisting the hand slightly. Fig. 5.5 참조 .


Flashlight and Aperture Techniques (I) Spotlight or flashlight technique

It replaces the virtual ray with a conic selection volume, with the apex of the cone at the input device; Fig. 5.6

Objects that fall within this selection cone can be selected. Easy selection of small objects even when they are

located far from the user Disambiguation

When more than one object falls into the spot light. Disadvantages:

When selection of small objects or tightly grouped objects


Flashlight and Aperture Techniques (II) Aperture Technique

Modification of the flashlight technique and improve it. Selection line P() p() = e + (h – e ) , e = virtual viewpoint

h = hand The user can interactively control the spread angle of the

selection volume simply by bringing the hand sensor closer or moving it father away; Figure 5.6

Simplify selection of virtual objects by using the orientation of the pointer around a central avis as an additional disambiguation metric. Figure 5.7b

Used easily in both desktop and immersive VEs.


Image-Plane Techniques The user selects and manipulates 3D objects by

touching and manipulating their 2D projections on a virtual image plane located in front of the user. Figure 5.8

Sticky-finger technique Figure 5.8

Head-crusher technique Another Image-plane tech. With a data glove devices With two fingers, his thumb and index fingers

Allows the user to modify the orientation of 3D objects, but their distance from the user can not be directly controlled. Mine’s scaled-world grab, Pierce’s Voodoo Doll technique


Fishing-Reel Technique

The ray direction is controlled by the spatial movements of user’s hand,

While distance is controlled by other means. Controlling the length of the virtual ray.A simple mechanical slider or a pair of

buttons added to the tracking device


Manipulation metaphors II

Simple virtual handNatural but limited


5.4.3 Direct Manipulation: Virtual Hand Techniques

3D cursor 3D model of human hand Semitransparent volumetric cursor

Selection The user intersects 3D cursor with the target of selection

Trigger technique : to pick it up Button press, voice command, hand gesture The object is attached to the virtual hand and can be easily

translated and rotated within V.E Release

The user release it with another trigger.


Virtual Hand Interaction Techniques

Simple (“Classical” ) virtual hand technique

Go-Go technique

Indirect Go-Go


Simple Virtual Hand (I) Direct mapping :

transfer functions or control-display gain functionpv = αpr , Rv = Rr (Eq. 5.4)

pr Rr are the position and orientation(3*3 matrix) of the user’s real hand.

pv , Rv are the corresponding position and orientation of the virtual handα is a scaling ratio to match the scales of the real

and virtual coordinate systems.

rv pp


Simple Virtual Hand (II)

Scaling rotationIt is useful to “scale” 3D device rotations similar to

the way we scale translations

Fundamental problemIn order to select objects located further away, the

user must employ a travel technique. Inconvenient and increase the complexity of the 3D UI.

rv pp


Go-Go Interaction TechniqueWhile the user’s real hand is close to the user (the distance to the hand is smaller than threshold D), the mapping is one to one, the movementof the virtual hand correspondto the real hand movements

As the user extends her handbeyond D, the mapping becomes nonlinear and the virtual arm “grows”, thus permitting the user to access and manipulate remote objects. C1 continuity is ensured.


Go-Go IT

Different mapping function can be used to achieve a different control-display gain between the real and virtual hands.

Advantages: Provide direct, seamless, 6-DOF object manipulation both

close to the user and at a distance. Disadvantages:

As the distance increases, the technique maps small movements of the user’s hand into large movements of the virtual hand, which complicates precise positioning at a distance.


5.4.5 Combining Techniques

Aggregation of techniques Mechanism for choosing the desired manipulation

technique from a limited set of possible options

Technique integration Combining techniques in which the interface switches

transparently between interaction techniques depending on the current task context.

Selection 과 manipulation 이 반복 적용되므로 , 각 mode 에서 최선의 방법들을 선택하여 적용한다 .


HOMER technique

Hand-Centered Object Manipulation Extending Ray-Casting

Select: ray-casting Manipulate: hand

Time


Manipulation metaphors III

HOMER (ray-casting + arm-extension)Easy selection & manipulationExpressive over range of distancesHard to move objects away from you


HOMER implementation Requires torso position t Upon selection, detach virtual hand from tracker,

move v. hand to object position in world CS, and attach object to v. hand (w/out moving object)

Get physical hand position h and distance

dh = dist(h, t)

Get object position o and distance

do = dist(o, t)


HOMER implementation (cont.)

Each frame:Copy hand tracker matrix to v. hand matrix (to

set orientation)Get physical hand position hcurr and distance:

dh-curr = dist(hcurr, t)V. hand distance

Normalize torso-hand vectorV. hand position vh = t + dvh*(thcurr)

dvh dh curr dodh

thcurr hcurr thcurr t


Manipulation metaphors IV

World-in-miniature (WIM)All manipulation in reachDoesn’t scale well, indirect

Scaled-world grabEasy, natural manipulationUser discomfort with use


5.4.4 World-in-miniature (WIM)

“Dollhouse” world held in user’s hand

Miniature objects can be manipulated directly

Moving miniature objects affects full-scale objects


World-in-Miniature (WIM) Provides the user with miniature handheld model of VE, which is

an exact copy of the VE at a small scale. Careful use of back face culling techniques. Only the “inside” of

the walls of the room model should be rendered. WIM combine navigation with manipulation. Although WIM works well for small and medium-sized

environments, using WIM in a very large environment would require an extreme scale factor, resulting in very small object copies in the WIM. This would make accurate selection and manipulation extremely difficult.

It has been successfully used in 3D interfaces for Augmented reality, in desktop 3D Uis. In fact, it can be considered a 3D generalization of the traditional overview maps that are often used in 3D games.


WIM implementation

Root

head hand room

table

Root

head hand room

tableWIM room(scaled)

table copy


Scale-world Grab Based on HOMER Selection : Image-plane selection tech. is used. Manipulation :

Scale down the entire VE around the user’s virtual viewpoint switching into manipulation mode

In HOMER, scaling the user’s hand motion The scaling coefficient s = Dv / Do where Dv is the distance

from the virtual viewpoint to the virtual hand, and Do is the distance from the virtual viewpoint to the selected object.

Well for operations at a distance but not effective when the user wants to pick up an object located within arm’s reach and move it farther away.


Voodoo Dolls To overcome the scaling approach such as HOMER, scaled-

World grab when the user needs to move local objects farther away.

Image-plane and WIM with a pair of pinch gloves Manipulate virtual objects indirectly using miniature handheld

copies of objects called “dolls”; Figure 5.13 Selecting the target object with an image-plane technique, which

creates the dolls representing the target objects and places them in the user’s hand.

The doll in her non-dominant hand : the corresponding virtual object does not move when the user moves this doll.

To start manipulation, the user simply passes the doll into dominant hand.


Voodoo Dolls Advantages:

Powerful IT allowing to perform some sophisticated tasks, such as the manipulation of moving, animated objects

Key idea is very universal and important insight Separating functionality depending on the dominant and

non-dominant hands

Disadvantages Increases H/W demand Direct application to a desktop might be difficult.


Technique Classification by metaphor

VE manipulation techniquesExocentric metaphor

Egocentric metaphor

World-In-Miniature

Virtual Hand metaphor

Virtual Pointer metaphor

Go-GoIndirect, stretch Go-Go

"Classical" virtual hand

Ray-castingApertureFlashlightImage plane

Scaled-world grab


Technique Classificationby components

Manipulation

Object Attachment

Object Position

Object Orientation

Feedback

attach to handattach to gazehand moves to objectobject moves to handuser/object scaling

no control1-to-N hand to object motionmaintain body-hand relationother hand mappingsindirect control

no control1-to-N hand to object rotationother hand mappingsindirect control

graphicalforce/tactileaudio


Evaluation: positioning task

Ray casting effective if the object is repositioned at constant distance

Scaling techniques (HOMER, scaled world grab) difficult in outward positioning of objects: e.g. pick an object located within reach and move it far away

If outward positioning is not needed then scaling techniques might be effective


Evaluation: orientation task

Setting precise orientation can be very difficult

Shape of objects is important

Orienting at-a-distance harder than positioning at-a-distance

Techniques should be hand-centered


Manipulation notes

No universally best technique

Constraints and reduced DOFs

Naturalism not always desirable

If VE is not based in the real, design it so manipulation is optimized


Manipulation enhancements

Constraints

2-handed manipulation

Haptic feedback

Multi-modal manipulation


Implementation issues for manipulation techniques

Integration with selection technique

Disable selection and selection feedback while manipulating

What happens upon release?


SIGGRAPH 2001 for Manipulation

참고할 것 .


Common manipulation techniques

Simple virtual hand

HOMER

Scaled-world grab

World-in-miniature


Simple virtual hand techniqueAttach object to virtual hand,

by making object a child of the hand (w/out moving object)

On release, reattach object to world (w/out moving object)

Also applies to Go-Go (and other arm-extension techniques) and ray-casting

Root

head hand building

Root

head hand

building


HOMER techniqueHand-CenteredObject ManipulationExtending Ray-Casting

Select: ray-casting

Manipulate: hand

Time

1.0 m

0.3 m

2.0 m

0.6 m

torso physicalhand

torso physicalhand


HOMER implementationRequires torso position t

Upon selection, detach virtual hand from tracker, move v. hand to object position in world CS, and attach object to v. hand (w/out moving object)

Get physical hand position h and distance

dh = dist(h, t)

Get object position o and distance do = dist(o, t)


HOMER implementation (cont.)Each frame:

Copy hand tracker matrix to v. hand matrix (to set orientation)

Get physical hand position hcurr and distance:

dh-curr = dist(hcurr, t)

V. hand distance

Normalize torso-hand vector V. hand position vh = t + dvh*(thcurr)

th

thth

curr

currcurr

h

ocurrhvh d

ddd


Scaled-world grab techniqueOften used w/ occlusion

At selection, scale user up (or world down) so that v. hand is actually touching selected object

User doesn’t notice a change in the image until he moves


Scaled-world grab implementationAt selection:

Get world CS distance from eye to hand deh

Get world CS distance from eye to object deo

Scale user (entire user subtree) uniformly by deo / deh

Ensure that eye remains in same position Attach selected object to v. hand (w/out moving object)

At release: Re-attach object to world (w/out moving object) Scale user uniformly by deh / deo

Ensure that eye remains in same position


World-in-miniature (WIM) technique

“Dollhouse” world held in user’s hand

Miniature objects can be manipulated directly

Moving miniature objects affects full-scale objects

Can also be used for navigation


WIM implementation

Root

head hand room

table

Root

head hand room

tableWIM room(scaled)

table copy


WIM implementation (cont.)

On selection: Determine which full-scale object

corresponds to the selected miniature object Attach miniature object to v. hand (w/out

moving object)Each frame:

Copy local position matrix of miniature object to corresponding full-scale object


Implementation issues for manipulation techniques

Integration with selection technique

Disable selection and selection feedback while manipulating

What happens upon release?

3d interaction techniques for virtual environments

Documents