3d user interface design for virtual environments

3D User Interface Design for Virtual Environments

Doug A. BowmanEdited by C. Song

(C) 2005 Doug Bowman, Virginia Tech 2

Background: Human-computer interaction HCI studies communication Users and computers communicate via the

interface 2D HCI issues:

Direct manipulation vs. commands Interface layout Interaction techniques Cognitive directness

3D

(C) 2005 Doug Bowman, Virginia Tech 3

Why 3D interaction?

3D / VE apps. should be useful ImmersionNatural skills Immediacy of visualization

But, current VE apps eitherare not complex interactively, orhave serious usability problems

(C) 2005 Doug Bowman, Virginia Tech 4

What makes 3D interaction difficult? Spatial input Lack of constraints Lack of standards Lack of tools

Lack of precision Fatigue Layout more

complex Perception

(C) 2005 Doug Bowman, Virginia Tech 5

Isn’t the 3D interface obvious?

Naturalism: make VE work exactly like real world

Magic: give user new abilitiesPerceptualPhysicalCognitive

(C) 2005 Doug Bowman, Virginia Tech 6

Goals of interaction design

Performance (efficiency, accuracy, productivity)

Usability (ease of use, ease of learning, user comfort)

Usefulness (users focus on tasks, interaction helps users meet system goals)

(C) 2005 Doug Bowman, Virginia Tech 7

Outline

System controlSymbolic input2D interaction in VEsConstraintsPassive haptic feedbackTwo-handed interaction

(C) 2005 Doug Bowman, Virginia Tech 8

8. System control

Issuing a command toChange the system modeChange the system state

Often composed of other tasksSometimes seen as a “catch-all” for 3D

interaction techniques other than travel, selection, & manipulation

(C) 2005 Doug Bowman, Virginia Tech 9

Classification of system control techniques

(C) 2005 Doug Bowman, Virginia Tech 10

Floating menus

Can occlude environment

Familiar interaction Using 3D selection for a

1D task Attach the menu

User’s headTracked physical surface (a tablet)

(C) 2005 Doug Bowman, Virginia Tech 11

1 DOF menu

Ring menu & selection baseket

Correct number of DOFs for the task

Can be put awayOnly one menu level at a

time

(C) 2005 Doug Bowman, Virginia Tech 12

TULIP menus

Goal: display all options but retain efficiency and comfort

Pinch Gloves: Three-up, Labels In Palm (TULIP) only three items available for

selection other items appear in sets of

three on the palm “more” item linked to next set

(C) 2005 Doug Bowman, Virginia Tech 13

Command and control cube (C3)

(C) 2005 Doug Bowman, Virginia Tech 14

Gestural commands

Can be “natural” Limited vocabulary Lack of affordances Fuzzy recognition issues Gesture as command - doesn’t mimic our use of

gestures in the real world Pen-based gestures can be powerful Perhaps more appropriate in multimodal interfaces

(C) 2005 Doug Bowman, Virginia Tech 15

System control design guidelines

Don’t disturb flow of actionUse consistent spatial reference Allow multimodal inputStructure available functionsPrevent mode errors by giving feedback

(C) 2005 Doug Bowman, Virginia Tech 16

9. Symbolic input

Communication of symbols (text, numbers, and other symbols/marks) to the system

Is this an important task for 3D UIs?Why is symbolic input in 3D UIs different

from symbolic input elsewhere?

(C) 2005 Doug Bowman, Virginia Tech 17

Symbolic input scenarios

Which technique(s) would you choose when… … architects will be making annotations to various

parts of a design? … engineers need to enter file names to save their

work? … an AR user needs to add hundreds of labels to

an environment?

(C) 2005 Doug Bowman, Virginia Tech 18

Symbolic input techniquesKeyboard-based

Miniature keyboards Low key-count keyboards Chord keyboards Pinch Keyboard Soft keyboards

Pen-based Pen-stroke gesture

recognition Digital Ink

(C) 2005 Doug Bowman, Virginia Tech 19

Symbolic input techniques 2

Gesture-based Sign language gestures Numeric gestures Instantaneous gestures

Speech-based Single-character speech recognition Whole-word speech recognition Unrecognized speech input

(C) 2005 Doug Bowman, Virginia Tech 20

Pinch keyboard video

(C) 2005 Doug Bowman, Virginia Tech 21

10. Usability Engineering

Methodology for constructing 3D UIsIterative process

Requirement gatheringDesign (Chapter 10)Prototype buildingEvaluation (Chapter 11)

(C) 2005 Doug Bowman, Virginia Tech 22

2D interaction in 3D VEs

Quite useful for appropriate tasks (match task and input DOFs)

Can integrate seamlessly with 3D If presence is important, the 2D interface should be embedded, not overlaid

Examples: Interaction on the projection surface or viewplane Using a PDA for VE input

(C) 2005 Doug Bowman, Virginia Tech 23

Constraints

Artificial limitations designed to help users interact more precisely or efficiently

Examples:Snap-to grid Intelligent objectsSingle DOF controls

(C) 2005 Doug Bowman, Virginia Tech 24

Passive haptic feedback

Props or “near-field” hapticsExamples:

Flight simulator controlsPirates’ steering wheel, cannonsElevator railing

Increase presence, improve interaction

(C) 2005 Doug Bowman, Virginia Tech 25

Two-handed interaction

Symmetric vs. AsymmetricDominant vs. non-dominant handGuiard’s principles

ND hand provides frame of referenceND hand used for coarse tasks, D hand for

fine-grained tasksManipulation initiated by ND hand

(C) 2005 Doug Bowman, Virginia Tech 26

Two-handed interaction examples

(C) 2005 Doug Bowman, Virginia Tech 27

Pen & tablet interaction

Involves 2D interaction, two-handed interaction, constraints, and props

(C) 2005 Doug Bowman, Virginia Tech 28

Pen & tablet interaction

Can put away Constrained surface

for input Combine 2D/3D

interaction

Handwriting input? Use any type of 2D

interface, not just menus

(C) 2005 Doug Bowman, Virginia Tech 29

QuickTime™ and a MPEG-4 Video decompressor are needed to see this picture.

Usability one of the most crucial issues facing VE applications

Implementation details critical to ensure usability

Ease of coding not equal to ease of useSimply adapting 2D interfaces is not

sufficient

Conclusions

(C) 2005 Doug Bowman, Virginia Tech 30

More work needed on…

System control performance Symbolic input and markup Usability evaluation methods Mapping interaction techniques to devices Integrating interaction techniques into

complete UIs

(C) 2005 Doug Bowman, Virginia Tech 31

Resources

3DUI groupMailing listAnnotated bibliographywww.mic.atr.co.jp/~poup/3dui.html

3DI research at VT research.cs.vt.edu/3di/

(C) 2005 Doug Bowman, Virginia Tech 32

Acknowledgments

Joe LaViola, Brown UniversityErnst Kruijff, GMDIvan Poupyrev, Sony CSL