interaction in virtual environments

43
Interaction in Interaction in Virtual Virtual Environments Environments Benjamin Lok Benjamin Lok This Lecture contains notes This Lecture contains notes created by created by Doug Bowman Doug Bowman Virginia Tech Virginia Tech Sherman & Craig, pp. 332-363 Sherman & Craig, pp. 332-363

Upload: liseli

Post on 09-Feb-2016

58 views

Category:

Documents


3 download

DESCRIPTION

Interaction in Virtual Environments. Benjamin Lok. This Lecture contains notes created by Doug Bowman Virginia Tech Sherman & Craig, pp. 332-363. 3D Interfaces. We live in a 3D world Natural interfaces are better Therefore 3D interfaces will be the ultimate What’s wrong with the above? - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Interaction in Virtual Environments

Interaction in Interaction in Virtual Virtual

EnvironmentsEnvironmentsBenjamin LokBenjamin Lok

This Lecture contains notes created This Lecture contains notes created by by Doug BowmanDoug BowmanVirginia TechVirginia TechSherman & Craig, pp. 332-363Sherman & Craig, pp. 332-363

Page 2: Interaction in Virtual Environments

3D 3D InterfacesInterfaces

We live in a 3D worldWe live in a 3D world Natural interfaces are betterNatural interfaces are better Therefore 3D interfaces will be the Therefore 3D interfaces will be the

ultimateultimate What’s wrong with the above?What’s wrong with the above?

Natural interfaces aren’t always Natural interfaces aren’t always better!better!

Making the interface simple (thus Making the interface simple (thus unnatural) often aids performanceunnatural) often aids performance

Constrains movementConstrains movement Limiting possible actionsLimiting possible actions

Depends on application and goal Depends on application and goal of the user interfaceof the user interface

Surgery simulationSurgery simulation Military simulation (general vs. Military simulation (general vs.

soldier training)soldier training) Architecture, education, product Architecture, education, product

designdesign Video gamesVideo games

Page 3: Interaction in Virtual Environments

3D Interfaces3D Interfaces What we really want are What we really want are enhancedenhanced interfaces interfaces

Give us powers we don’t Give us powers we don’t normally havenormally have Flying, x-ray vision, teleportation, Flying, x-ray vision, teleportation,

undo, etc.undo, etc. Be careful we don’t become Be careful we don’t become

overzealousoverzealous Air traffic control 3D displayAir traffic control 3D display Library interfaces using a books Library interfaces using a books

on shelves (what is it good for? on shelves (what is it good for? What is it poor for?)What is it poor for?)

Hurts performanceHurts performance Study results: Study results: 3D Bar charts 3D Bar charts

don’t help don’t help So what is helped by 3D?So what is helped by 3D?

Page 4: Interaction in Virtual Environments

Good 3DGood 3D Social interfaces + 3D can Social interfaces + 3D can

be very powerfulbe very powerful MMORPG (EveQuest)MMORPG (EveQuest) ActivedWorldsActivedWorlds The Sims OnlineThe Sims Online

ExperiencesExperiences Art gallaryArt gallary 3D Desktops (Mac’s latest)3D Desktops (Mac’s latest) Office metaphors did not Office metaphors did not

take off (BOB, Task Gallary)take off (BOB, Task Gallary) 3D Webbrowsing. Sure you 3D Webbrowsing. Sure you

can arrange 16 web pages can arrange 16 web pages spatially, but why?spatially, but why?

Compromises to provide Compromises to provide 3D interfaces might be 3D interfaces might be undermine usabilityundermine usability Think RTS gamesThink RTS games

Discussion: Is the interface Discussion: Is the interface holding back 3D?holding back 3D?

Page 5: Interaction in Virtual Environments

3D 3D InterfacesInterfaces

Use occlusion, shadows, perspective carefullyUse occlusion, shadows, perspective carefully Improves use of spatial memory (Ark ’98)Improves use of spatial memory (Ark ’98) Distracting and confusingDistracting and confusing

Minimize navigation stepsMinimize navigation steps Keep text readable (good contrast, 30 degree tilt Keep text readable (good contrast, 30 degree tilt

max)max) Simple user movement (why lock to a floor?) Simple user movement (why lock to a floor?)

Descent vs QuakeDescent vs Quake Prevent Errors (put in guides to help)Prevent Errors (put in guides to help) Simplify object movement (connecting two parts, Simplify object movement (connecting two parts,

for example, can be abstracted… most of the time)for example, can be abstracted… most of the time) Organize groups of items into alignments that Organize groups of items into alignments that

facilitate visual search and recall (allow user facilitate visual search and recall (allow user choice)choice)

Page 6: Interaction in Virtual Environments

3D Interface 3D Interface DevelopmentDevelopment

Developments that show promise:Developments that show promise: 3D sound3D sound Stereo display (Ware and Frank ’96)Stereo display (Ware and Frank ’96) Haptic feedback (mouse)Haptic feedback (mouse)

3D can help by:3D can help by: Provide overviews to see big pictureProvide overviews to see big picture Rapid teleportation (context shifts)Rapid teleportation (context shifts) Zooming (aid disabled)Zooming (aid disabled) Multiple coordinated views (3dsmax)Multiple coordinated views (3dsmax) 3D icons can represent abstract or recognizable 3D icons can represent abstract or recognizable

conceptsconcepts

Page 7: Interaction in Virtual Environments

TeleoperatioTeleoperationn Combines:Combines:

Direct ManipulationDirect Manipulation Process ControlProcess Control

Human operators control Human operators control physical processes in physical processes in complex environmentscomplex environments

Example applications: Example applications: Mars rover control, flying Mars rover control, flying airplanes (Predator), airplanes (Predator), manufacturing, medicine manufacturing, medicine (surgery)(surgery)

Supervisory control Supervisory control (Sheridan ’92)(Sheridan ’92) Different levels of human Different levels of human

control (automation)control (automation)

Page 8: Interaction in Virtual Environments

TeleoperatioTeleoperationn Direct Manipulation IssuesDirect Manipulation Issues

Adequate feedback (data Adequate feedback (data quality, latency (transmission quality, latency (transmission and operation delays), and operation delays), incomplete, interference)incomplete, interference)

PresencePresence Point and click or more natural Point and click or more natural

interaction vs. typinginteraction vs. typing Example project: Example project:

Nanomanipulator (show Nanomanipulator (show video)video)

Page 9: Interaction in Virtual Environments

VR InteractionVR Interaction Trying to simulate reality or an experienceTrying to simulate reality or an experience

Training, Learning, ExploringTraining, Learning, Exploring ExpensiveExpensive DangerousDangerous Logistically DifficultLogistically Difficult

Best interaction?Best interaction? Flight simulators (they can cost $10 mil, but that’s still Flight simulators (they can cost $10 mil, but that’s still

a good deal!)a good deal!) Why?Why?

Why do video game flight sims not cut it? (only $40!)Why do video game flight sims not cut it? (only $40!) Okay, we have monitors that show 3D worlds, Okay, we have monitors that show 3D worlds,

what else do we possibly need?what else do we possibly need?

Page 10: Interaction in Virtual Environments

Why 3D interaction?Why 3D interaction? 3D / VE apps. should be useful3D / VE apps. should be useful

ImmersionImmersion Natural skillsNatural skills Immediacy of visualizationImmediacy of visualization

But, current VE apps are But, current VE apps are Not complex interactivelyNot complex interactively Have serious usability problemsHave serious usability problems

Page 11: Interaction in Virtual Environments

What makes 3D What makes 3D interaction difficult?interaction difficult?

Spatial inputSpatial input Lack of constraintsLack of constraints Lack of standardsLack of standards Lack of toolsLack of tools

Lack of precisionLack of precision FatigueFatigue Layout more complexLayout more complex PerceptionPerception

You’ve seen this in creating Project #2!

Page 12: Interaction in Virtual Environments

Let’s look at this example Let’s look at this example (ISAAC)(ISAAC)

Page 13: Interaction in Virtual Environments

Universal interaction Universal interaction taskstasks

NavigationNavigation Travel – motor componentTravel – motor component Wayfinding – cognitive Wayfinding – cognitive

componentcomponent SelectionSelection ManipulationManipulation System controlSystem control

Page 14: Interaction in Virtual Environments

Goals of interaction Goals of interaction designdesign

Performance Performance (efficiency, (efficiency, accuracy, productivity)accuracy, productivity)

Usability Usability (ease of use, (ease of use, ease of learning, user comfort)ease of learning, user comfort)

Usefulness Usefulness (users focus (users focus on tasks, interaction helps on tasks, interaction helps users meet system goals)users meet system goals)

Page 15: Interaction in Virtual Environments

Philosophies of Philosophies of interaction designinteraction design

Artistic approachArtistic approach Intuition about users, Intuition about users,

taskstasks Heuristics, metaphorsHeuristics, metaphors AestheticsAesthetics AdaptationAdaptation

Scientific approachScientific approach Formal analysisFormal analysis Formal evaluationFormal evaluation Performance Performance

requirementsrequirements

Page 16: Interaction in Virtual Environments

TravelTravel Motor component of Motor component of

navigationnavigation Movement between 2 Movement between 2

locations, setting the locations, setting the position (and orientation) position (and orientation) of the user’s viewpointof the user’s viewpoint

Most basic and common Most basic and common VE interaction techniqueVE interaction technique

Used in almost any large-Used in almost any large-scale VEscale VE

Page 17: Interaction in Virtual Environments

Types of Travel TasksTypes of Travel Tasks ExplorationExploration

Travel which has no specific Travel which has no specific targettarget

Build knowledge of environmentBuild knowledge of environment SearchSearch

Naive: travel to find a target Naive: travel to find a target whose position is not knownwhose position is not known

Primed: travel to a target whose Primed: travel to a target whose position is knownposition is known

Build layout knowledgeBuild layout knowledge Move to task locationMove to task location

ManeuveringManeuvering Travel to position viewpoint for Travel to position viewpoint for

tasktask Short, precise movementsShort, precise movements

Page 18: Interaction in Virtual Environments

Steering metaphorSteering metaphor Continuous Continuous

specification of specification of direction of motiondirection of motion Gaze-directedGaze-directed PointingPointing Physical device Physical device

(steering wheel, flight (steering wheel, flight stick)stick)

Page 19: Interaction in Virtual Environments

Target-based metaphorTarget-based metaphor Discrete specification Discrete specification

of goalof goal Point at objectPoint at object Choose from listChoose from list Enter coordinatesEnter coordinates

E.g. Google EarthE.g. Google Earth

Page 20: Interaction in Virtual Environments

Route-planning Route-planning metaphormetaphor

One time specification One time specification of pathof path Place markers in worldPlace markers in world Move icon on mapMove icon on map

Manual manipulation of Manual manipulation of viewpointviewpoint ““Camera in hand”Camera in hand” Fixed object Fixed object

manipulationmanipulation

Page 21: Interaction in Virtual Environments

““Natural” travel Natural” travel metaphorsmetaphors

Walking techniquesWalking techniques TreadmillsTreadmills BicyclesBicycles Other physical motionOther physical motion

VMC / Magic carpetVMC / Magic carpet Disney’s river raft rideDisney’s river raft ride Simulation of flyingSimulation of flying

Page 22: Interaction in Virtual Environments

Technique classificationTechnique classification

Travel

Start to move

Stop moving

Indicate position

Indicate orientation

position

velocity

acceleration

Target specificationRoute specificationContinuousspecification

Page 23: Interaction in Virtual Environments

Alternate Technique Alternate Technique classificationclassification

Travel

Direction/TargetSelection

Velocity/Accel.Selection

Conditions of Input

gaze-directedpointingphysical props

gestureslow in, slow outphysical props

start/stop buttonsautomatic start/stopconstant movement

Ontologies by Bowman99 and Hand97

Page 24: Interaction in Virtual Environments

Evaluation resultsEvaluation results Steering techniques have Steering techniques have

similar performance on similar performance on absolute motion tasksabsolute motion tasks

Non-head-coupled steering Non-head-coupled steering better for relative motionbetter for relative motion

““Teleportation” can lead to Teleportation” can lead to significant disorientationsignificant disorientation

Environment complexity Environment complexity affects information gatheringaffects information gathering

Travel interaction technique Travel interaction technique and user’s strategies affect and user’s strategies affect spatial orientationspatial orientation

Page 25: Interaction in Virtual Environments

Evaluation results – 3 Evaluation results – 3 Manipulation-based Manipulation-based

techniques efficient for techniques efficient for relative motionrelative motion

Manipulation-based Manipulation-based techniques not requiring techniques not requiring an object efficient for an object efficient for search, but tiringsearch, but tiring

Steering techniques best Steering techniques best for naïve for naïve andand primed primed searchsearch

Map-based techniques Map-based techniques not effective in unfamiliar not effective in unfamiliar environments, or when environments, or when any precision is requiredany precision is required

Page 26: Interaction in Virtual Environments

MythsMyths

There is one optimal travel technique for VEs.There is one optimal travel technique for VEs. A “natural” technique will always be better A “natural” technique will always be better

than another technique.than another technique. Desktop 3D, workbench, and CAVE Desktop 3D, workbench, and CAVE

applications should use the same travel ITs applications should use the same travel ITs as HMD-based VEs.as HMD-based VEs.

Page 27: Interaction in Virtual Environments

Design guidelinesDesign guidelinesMake simple travel tasks simple (target-based Make simple travel tasks simple (target-based techniques for motion to an object, steering techniques for motion to an object, steering techniques for search).techniques for search).

Provide multiple travel techniques to support Provide multiple travel techniques to support different travel tasks in the same application.different travel tasks in the same application.

Page 28: Interaction in Virtual Environments

More design guidelinesMore design guidelines Use transitional motions if overall environment Use transitional motions if overall environment

context is important.context is important. Train users in sophisticated strategies to help Train users in sophisticated strategies to help

them acquire survey knowledge.them acquire survey knowledge. Consider integrated (“cross-task”) ITs if travel is Consider integrated (“cross-task”) ITs if travel is

used in the context of another task (e.g. used in the context of another task (e.g. manipulation).manipulation).

Page 29: Interaction in Virtual Environments

WayfindingWayfinding Cognitive process of Cognitive process of

defining a path through defining a path through an environment using and an environment using and acquiring spatial acquiring spatial knowledgeknowledge

6DOF makes wayfinding 6DOF makes wayfinding hardhard

Human beings have Human beings have different abilities to orient different abilities to orient themselves in an themselves in an environmentenvironment

Observing wayfinding as Observing wayfinding as a decision making a decision making processprocess

Page 30: Interaction in Virtual Environments

Wayfinding tasksWayfinding tasks General, explorative General, explorative

search search Search without Search without

targettarget NNaïaïveve search search

target position target position unkunknownnown

Primed searchPrimed searchtarget seen before target seen before

(known)(known) Specified trajectory Specified trajectory

movementmovementPredefined pathPredefined path

Page 31: Interaction in Virtual Environments

Cognitive MapCognitive MapDuring wayfinding, a During wayfinding, a person makes use of person makes use of three kinds of three kinds of knowledge to built up a knowledge to built up a cognitive map of the cognitive map of the environment:environment: Landmark knowledge Landmark knowledge Procedural knowledge Procedural knowledge Survey knowledgeSurvey knowledge

Page 32: Interaction in Virtual Environments

Reference framesReference frames Egocentric reference frame: Egocentric reference frame:

position, orientation, position, orientation, movement of object with movement of object with respect to position and respect to position and orientation of the:orientation of the: eyes eyes head head bodybody

Exocentric reference frame:Exocentric reference frame: position, orientation and position, orientation and movement are defined in movement are defined in coordinates external to bodycoordinates external to body object shapeobject shape object orientationobject orientation object motionobject motion

Page 33: Interaction in Virtual Environments

Travel technique effectsTravel technique effects Steering technique with Steering technique with

good strategy helps good strategy helps spatial orientationspatial orientation

A good travel technique A good travel technique will integrate aids to will integrate aids to wayfinding wayfinding

Jumping between points Jumping between points disturbs spatial disturbs spatial orientationorientation

Page 34: Interaction in Virtual Environments

Support of spatial Support of spatial knowledge acquisition knowledge acquisition

Allow a wide field of viewAllow a wide field of view Provide motion cues for Provide motion cues for

judging depth and judging depth and direction of movement direction of movement

Audio could enhance Audio could enhance visual spatial perceptionvisual spatial perception

Support sense of Support sense of presence: it could presence: it could strengthen the strengthen the construction of a construction of a cognicognitive maptive map

Page 35: Interaction in Virtual Environments

Support of spatial Support of spatial knowledge acquisitionknowledge acquisition

Design Design legiblelegible environments - allow the environments - allow the user to easily see the user to easily see the spatial organisation of an spatial organisation of an environment, enabling the environment, enabling the establishment of a establishment of a cognitive map. cognitive map.

Divide a large-scale Divide a large-scale environment into parts environment into parts with a distinct characterwith a distinct character

Page 36: Interaction in Virtual Environments

Support of spatial Support of spatial knowledge acquisitionknowledge acquisition

Create a simple spatial Create a simple spatial organisation in which the organisation in which the relations between the relations between the parts are clearparts are clear

Support the matching Support the matching process between the process between the egocentric and exocentric egocentric and exocentric frames of reference by frames of reference by (visual) cues, including (visual) cues, including directional cuesdirectional cues

Page 37: Interaction in Virtual Environments

Support of Spatial Support of Spatial KKnowledge Acquisitionnowledge Acquisition

Use real-world wayfinding Use real-world wayfinding principles to build up your principles to build up your environmentenvironment

Natural environment Natural environment principlesprinciples

Urban design principles Urban design principles [Lynch][Lynch]

Architectural design Architectural design principlesprinciples

Artificial cuesArtificial cues

Page 38: Interaction in Virtual Environments

Examples of wayfinding Examples of wayfinding aidsaids

Page 39: Interaction in Virtual Environments

Map usage guidelinesMap usage guidelines

Provide you are here markerProvide you are here marker Provide gridProvide grid Choose either north-up or Choose either north-up or

forward-up map, depending forward-up map, depending on taskon task

Example: World-in-MiniatureExample: World-in-Miniature

Page 40: Interaction in Virtual Environments

Manipulation TechniquesManipulation Techniques Wand-basedWand-based

Go-GoGo-Go Ray CastingRay Casting HOMER (Hand-HOMER (Hand-

centered Object centered Object Manipulation Manipulation Extending Ray-Extending Ray-casting)casting)

Page 41: Interaction in Virtual Environments

Two Handed and Two Handed and Body-Centered Body-Centered

InteractionInteraction What can you do with two hands?What can you do with two hands? What if you use your body as a What if you use your body as a

reference point?reference point? Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin

(1997). (1997). Moving Objects in Space: Exploiting Moving Objects in Space: Exploiting ProprioceptionProprioception in Virtual-Environment Interaction. in Virtual-Environment Interaction. Proceedings of Proceedings of SIGGRAPH 97, Los Angeles, CA. SIGGRAPH 97, Los Angeles, CA. (133K (133K pdfpdf version) version)

Page 42: Interaction in Virtual Environments

What can you do with What can you do with two hands?two hands?

What if you use your What if you use your body as a reference body as a reference point?point?

Mine, Mark, Frederick P. Brooks Jr., Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin (1997). and Carlo Sequin (1997). Moving Objects in Space: Exploiting Moving Objects in Space: Exploiting ProprioceptionProprioception in Virtual-Environment Interaction. in Virtual-Environment Interaction. Proceedings of SIGGRAPH 97, Los Proceedings of SIGGRAPH 97, Los Angeles, CA. Angeles, CA. (133K (133K pdfpdf version) version)

Two Handed and Two Handed and Body-Centered Body-Centered

InteractionInteraction

Page 43: Interaction in Virtual Environments

Other Manipulation Other Manipulation ApproachesApproaches

Tablet/hand-heldTablet/hand-held Pinch GlovePinch Glove