interaction in virtual environments
DESCRIPTION
Interaction in Virtual Environments. Benjamin Lok. This Lecture contains notes created by Doug Bowman Virginia Tech Sherman & Craig, pp. 332-363. 3D Interfaces. We live in a 3D world Natural interfaces are better Therefore 3D interfaces will be the ultimate What’s wrong with the above? - PowerPoint PPT PresentationTRANSCRIPT
Interaction in Interaction in Virtual Virtual
EnvironmentsEnvironmentsBenjamin LokBenjamin Lok
This Lecture contains notes created This Lecture contains notes created by by Doug BowmanDoug BowmanVirginia TechVirginia TechSherman & Craig, pp. 332-363Sherman & Craig, pp. 332-363
3D 3D InterfacesInterfaces
We live in a 3D worldWe live in a 3D world Natural interfaces are betterNatural interfaces are better Therefore 3D interfaces will be the Therefore 3D interfaces will be the
ultimateultimate What’s wrong with the above?What’s wrong with the above?
Natural interfaces aren’t always Natural interfaces aren’t always better!better!
Making the interface simple (thus Making the interface simple (thus unnatural) often aids performanceunnatural) often aids performance
Constrains movementConstrains movement Limiting possible actionsLimiting possible actions
Depends on application and goal Depends on application and goal of the user interfaceof the user interface
Surgery simulationSurgery simulation Military simulation (general vs. Military simulation (general vs.
soldier training)soldier training) Architecture, education, product Architecture, education, product
designdesign Video gamesVideo games
3D Interfaces3D Interfaces What we really want are What we really want are enhancedenhanced interfaces interfaces
Give us powers we don’t Give us powers we don’t normally havenormally have Flying, x-ray vision, teleportation, Flying, x-ray vision, teleportation,
undo, etc.undo, etc. Be careful we don’t become Be careful we don’t become
overzealousoverzealous Air traffic control 3D displayAir traffic control 3D display Library interfaces using a books Library interfaces using a books
on shelves (what is it good for? on shelves (what is it good for? What is it poor for?)What is it poor for?)
Hurts performanceHurts performance Study results: Study results: 3D Bar charts 3D Bar charts
don’t help don’t help So what is helped by 3D?So what is helped by 3D?
Good 3DGood 3D Social interfaces + 3D can Social interfaces + 3D can
be very powerfulbe very powerful MMORPG (EveQuest)MMORPG (EveQuest) ActivedWorldsActivedWorlds The Sims OnlineThe Sims Online
ExperiencesExperiences Art gallaryArt gallary 3D Desktops (Mac’s latest)3D Desktops (Mac’s latest) Office metaphors did not Office metaphors did not
take off (BOB, Task Gallary)take off (BOB, Task Gallary) 3D Webbrowsing. Sure you 3D Webbrowsing. Sure you
can arrange 16 web pages can arrange 16 web pages spatially, but why?spatially, but why?
Compromises to provide Compromises to provide 3D interfaces might be 3D interfaces might be undermine usabilityundermine usability Think RTS gamesThink RTS games
Discussion: Is the interface Discussion: Is the interface holding back 3D?holding back 3D?
3D 3D InterfacesInterfaces
Use occlusion, shadows, perspective carefullyUse occlusion, shadows, perspective carefully Improves use of spatial memory (Ark ’98)Improves use of spatial memory (Ark ’98) Distracting and confusingDistracting and confusing
Minimize navigation stepsMinimize navigation steps Keep text readable (good contrast, 30 degree tilt Keep text readable (good contrast, 30 degree tilt
max)max) Simple user movement (why lock to a floor?) Simple user movement (why lock to a floor?)
Descent vs QuakeDescent vs Quake Prevent Errors (put in guides to help)Prevent Errors (put in guides to help) Simplify object movement (connecting two parts, Simplify object movement (connecting two parts,
for example, can be abstracted… most of the time)for example, can be abstracted… most of the time) Organize groups of items into alignments that Organize groups of items into alignments that
facilitate visual search and recall (allow user facilitate visual search and recall (allow user choice)choice)
3D Interface 3D Interface DevelopmentDevelopment
Developments that show promise:Developments that show promise: 3D sound3D sound Stereo display (Ware and Frank ’96)Stereo display (Ware and Frank ’96) Haptic feedback (mouse)Haptic feedback (mouse)
3D can help by:3D can help by: Provide overviews to see big pictureProvide overviews to see big picture Rapid teleportation (context shifts)Rapid teleportation (context shifts) Zooming (aid disabled)Zooming (aid disabled) Multiple coordinated views (3dsmax)Multiple coordinated views (3dsmax) 3D icons can represent abstract or recognizable 3D icons can represent abstract or recognizable
conceptsconcepts
TeleoperatioTeleoperationn Combines:Combines:
Direct ManipulationDirect Manipulation Process ControlProcess Control
Human operators control Human operators control physical processes in physical processes in complex environmentscomplex environments
Example applications: Example applications: Mars rover control, flying Mars rover control, flying airplanes (Predator), airplanes (Predator), manufacturing, medicine manufacturing, medicine (surgery)(surgery)
Supervisory control Supervisory control (Sheridan ’92)(Sheridan ’92) Different levels of human Different levels of human
control (automation)control (automation)
TeleoperatioTeleoperationn Direct Manipulation IssuesDirect Manipulation Issues
Adequate feedback (data Adequate feedback (data quality, latency (transmission quality, latency (transmission and operation delays), and operation delays), incomplete, interference)incomplete, interference)
PresencePresence Point and click or more natural Point and click or more natural
interaction vs. typinginteraction vs. typing Example project: Example project:
Nanomanipulator (show Nanomanipulator (show video)video)
VR InteractionVR Interaction Trying to simulate reality or an experienceTrying to simulate reality or an experience
Training, Learning, ExploringTraining, Learning, Exploring ExpensiveExpensive DangerousDangerous Logistically DifficultLogistically Difficult
Best interaction?Best interaction? Flight simulators (they can cost $10 mil, but that’s still Flight simulators (they can cost $10 mil, but that’s still
a good deal!)a good deal!) Why?Why?
Why do video game flight sims not cut it? (only $40!)Why do video game flight sims not cut it? (only $40!) Okay, we have monitors that show 3D worlds, Okay, we have monitors that show 3D worlds,
what else do we possibly need?what else do we possibly need?
Why 3D interaction?Why 3D interaction? 3D / VE apps. should be useful3D / VE apps. should be useful
ImmersionImmersion Natural skillsNatural skills Immediacy of visualizationImmediacy of visualization
But, current VE apps are But, current VE apps are Not complex interactivelyNot complex interactively Have serious usability problemsHave serious usability problems
What makes 3D What makes 3D interaction difficult?interaction difficult?
Spatial inputSpatial input Lack of constraintsLack of constraints Lack of standardsLack of standards Lack of toolsLack of tools
Lack of precisionLack of precision FatigueFatigue Layout more complexLayout more complex PerceptionPerception
You’ve seen this in creating Project #2!
Let’s look at this example Let’s look at this example (ISAAC)(ISAAC)
Universal interaction Universal interaction taskstasks
NavigationNavigation Travel – motor componentTravel – motor component Wayfinding – cognitive Wayfinding – cognitive
componentcomponent SelectionSelection ManipulationManipulation System controlSystem control
Goals of interaction Goals of interaction designdesign
Performance Performance (efficiency, (efficiency, accuracy, productivity)accuracy, productivity)
Usability Usability (ease of use, (ease of use, ease of learning, user comfort)ease of learning, user comfort)
Usefulness Usefulness (users focus (users focus on tasks, interaction helps on tasks, interaction helps users meet system goals)users meet system goals)
Philosophies of Philosophies of interaction designinteraction design
Artistic approachArtistic approach Intuition about users, Intuition about users,
taskstasks Heuristics, metaphorsHeuristics, metaphors AestheticsAesthetics AdaptationAdaptation
Scientific approachScientific approach Formal analysisFormal analysis Formal evaluationFormal evaluation Performance Performance
requirementsrequirements
TravelTravel Motor component of Motor component of
navigationnavigation Movement between 2 Movement between 2
locations, setting the locations, setting the position (and orientation) position (and orientation) of the user’s viewpointof the user’s viewpoint
Most basic and common Most basic and common VE interaction techniqueVE interaction technique
Used in almost any large-Used in almost any large-scale VEscale VE
Types of Travel TasksTypes of Travel Tasks ExplorationExploration
Travel which has no specific Travel which has no specific targettarget
Build knowledge of environmentBuild knowledge of environment SearchSearch
Naive: travel to find a target Naive: travel to find a target whose position is not knownwhose position is not known
Primed: travel to a target whose Primed: travel to a target whose position is knownposition is known
Build layout knowledgeBuild layout knowledge Move to task locationMove to task location
ManeuveringManeuvering Travel to position viewpoint for Travel to position viewpoint for
tasktask Short, precise movementsShort, precise movements
Steering metaphorSteering metaphor Continuous Continuous
specification of specification of direction of motiondirection of motion Gaze-directedGaze-directed PointingPointing Physical device Physical device
(steering wheel, flight (steering wheel, flight stick)stick)
Target-based metaphorTarget-based metaphor Discrete specification Discrete specification
of goalof goal Point at objectPoint at object Choose from listChoose from list Enter coordinatesEnter coordinates
E.g. Google EarthE.g. Google Earth
Route-planning Route-planning metaphormetaphor
One time specification One time specification of pathof path Place markers in worldPlace markers in world Move icon on mapMove icon on map
Manual manipulation of Manual manipulation of viewpointviewpoint ““Camera in hand”Camera in hand” Fixed object Fixed object
manipulationmanipulation
““Natural” travel Natural” travel metaphorsmetaphors
Walking techniquesWalking techniques TreadmillsTreadmills BicyclesBicycles Other physical motionOther physical motion
VMC / Magic carpetVMC / Magic carpet Disney’s river raft rideDisney’s river raft ride Simulation of flyingSimulation of flying
Technique classificationTechnique classification
Travel
Start to move
Stop moving
Indicate position
Indicate orientation
position
velocity
acceleration
Target specificationRoute specificationContinuousspecification
Alternate Technique Alternate Technique classificationclassification
Travel
Direction/TargetSelection
Velocity/Accel.Selection
Conditions of Input
gaze-directedpointingphysical props
gestureslow in, slow outphysical props
start/stop buttonsautomatic start/stopconstant movement
Ontologies by Bowman99 and Hand97
Evaluation resultsEvaluation results Steering techniques have Steering techniques have
similar performance on similar performance on absolute motion tasksabsolute motion tasks
Non-head-coupled steering Non-head-coupled steering better for relative motionbetter for relative motion
““Teleportation” can lead to Teleportation” can lead to significant disorientationsignificant disorientation
Environment complexity Environment complexity affects information gatheringaffects information gathering
Travel interaction technique Travel interaction technique and user’s strategies affect and user’s strategies affect spatial orientationspatial orientation
Evaluation results – 3 Evaluation results – 3 Manipulation-based Manipulation-based
techniques efficient for techniques efficient for relative motionrelative motion
Manipulation-based Manipulation-based techniques not requiring techniques not requiring an object efficient for an object efficient for search, but tiringsearch, but tiring
Steering techniques best Steering techniques best for naïve for naïve andand primed primed searchsearch
Map-based techniques Map-based techniques not effective in unfamiliar not effective in unfamiliar environments, or when environments, or when any precision is requiredany precision is required
MythsMyths
There is one optimal travel technique for VEs.There is one optimal travel technique for VEs. A “natural” technique will always be better A “natural” technique will always be better
than another technique.than another technique. Desktop 3D, workbench, and CAVE Desktop 3D, workbench, and CAVE
applications should use the same travel ITs applications should use the same travel ITs as HMD-based VEs.as HMD-based VEs.
Design guidelinesDesign guidelinesMake simple travel tasks simple (target-based Make simple travel tasks simple (target-based techniques for motion to an object, steering techniques for motion to an object, steering techniques for search).techniques for search).
Provide multiple travel techniques to support Provide multiple travel techniques to support different travel tasks in the same application.different travel tasks in the same application.
More design guidelinesMore design guidelines Use transitional motions if overall environment Use transitional motions if overall environment
context is important.context is important. Train users in sophisticated strategies to help Train users in sophisticated strategies to help
them acquire survey knowledge.them acquire survey knowledge. Consider integrated (“cross-task”) ITs if travel is Consider integrated (“cross-task”) ITs if travel is
used in the context of another task (e.g. used in the context of another task (e.g. manipulation).manipulation).
WayfindingWayfinding Cognitive process of Cognitive process of
defining a path through defining a path through an environment using and an environment using and acquiring spatial acquiring spatial knowledgeknowledge
6DOF makes wayfinding 6DOF makes wayfinding hardhard
Human beings have Human beings have different abilities to orient different abilities to orient themselves in an themselves in an environmentenvironment
Observing wayfinding as Observing wayfinding as a decision making a decision making processprocess
Wayfinding tasksWayfinding tasks General, explorative General, explorative
search search Search without Search without
targettarget NNaïaïveve search search
target position target position unkunknownnown
Primed searchPrimed searchtarget seen before target seen before
(known)(known) Specified trajectory Specified trajectory
movementmovementPredefined pathPredefined path
Cognitive MapCognitive MapDuring wayfinding, a During wayfinding, a person makes use of person makes use of three kinds of three kinds of knowledge to built up a knowledge to built up a cognitive map of the cognitive map of the environment:environment: Landmark knowledge Landmark knowledge Procedural knowledge Procedural knowledge Survey knowledgeSurvey knowledge
Reference framesReference frames Egocentric reference frame: Egocentric reference frame:
position, orientation, position, orientation, movement of object with movement of object with respect to position and respect to position and orientation of the:orientation of the: eyes eyes head head bodybody
Exocentric reference frame:Exocentric reference frame: position, orientation and position, orientation and movement are defined in movement are defined in coordinates external to bodycoordinates external to body object shapeobject shape object orientationobject orientation object motionobject motion
Travel technique effectsTravel technique effects Steering technique with Steering technique with
good strategy helps good strategy helps spatial orientationspatial orientation
A good travel technique A good travel technique will integrate aids to will integrate aids to wayfinding wayfinding
Jumping between points Jumping between points disturbs spatial disturbs spatial orientationorientation
Support of spatial Support of spatial knowledge acquisition knowledge acquisition
Allow a wide field of viewAllow a wide field of view Provide motion cues for Provide motion cues for
judging depth and judging depth and direction of movement direction of movement
Audio could enhance Audio could enhance visual spatial perceptionvisual spatial perception
Support sense of Support sense of presence: it could presence: it could strengthen the strengthen the construction of a construction of a cognicognitive maptive map
Support of spatial Support of spatial knowledge acquisitionknowledge acquisition
Design Design legiblelegible environments - allow the environments - allow the user to easily see the user to easily see the spatial organisation of an spatial organisation of an environment, enabling the environment, enabling the establishment of a establishment of a cognitive map. cognitive map.
Divide a large-scale Divide a large-scale environment into parts environment into parts with a distinct characterwith a distinct character
Support of spatial Support of spatial knowledge acquisitionknowledge acquisition
Create a simple spatial Create a simple spatial organisation in which the organisation in which the relations between the relations between the parts are clearparts are clear
Support the matching Support the matching process between the process between the egocentric and exocentric egocentric and exocentric frames of reference by frames of reference by (visual) cues, including (visual) cues, including directional cuesdirectional cues
Support of Spatial Support of Spatial KKnowledge Acquisitionnowledge Acquisition
Use real-world wayfinding Use real-world wayfinding principles to build up your principles to build up your environmentenvironment
Natural environment Natural environment principlesprinciples
Urban design principles Urban design principles [Lynch][Lynch]
Architectural design Architectural design principlesprinciples
Artificial cuesArtificial cues
Examples of wayfinding Examples of wayfinding aidsaids
Map usage guidelinesMap usage guidelines
Provide you are here markerProvide you are here marker Provide gridProvide grid Choose either north-up or Choose either north-up or
forward-up map, depending forward-up map, depending on taskon task
Example: World-in-MiniatureExample: World-in-Miniature
Manipulation TechniquesManipulation Techniques Wand-basedWand-based
Go-GoGo-Go Ray CastingRay Casting HOMER (Hand-HOMER (Hand-
centered Object centered Object Manipulation Manipulation Extending Ray-Extending Ray-casting)casting)
Two Handed and Two Handed and Body-Centered Body-Centered
InteractionInteraction What can you do with two hands?What can you do with two hands? What if you use your body as a What if you use your body as a
reference point?reference point? Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin
(1997). (1997). Moving Objects in Space: Exploiting Moving Objects in Space: Exploiting ProprioceptionProprioception in Virtual-Environment Interaction. in Virtual-Environment Interaction. Proceedings of Proceedings of SIGGRAPH 97, Los Angeles, CA. SIGGRAPH 97, Los Angeles, CA. (133K (133K pdfpdf version) version)
What can you do with What can you do with two hands?two hands?
What if you use your What if you use your body as a reference body as a reference point?point?
Mine, Mark, Frederick P. Brooks Jr., Mine, Mark, Frederick P. Brooks Jr., and Carlo Sequin (1997). and Carlo Sequin (1997). Moving Objects in Space: Exploiting Moving Objects in Space: Exploiting ProprioceptionProprioception in Virtual-Environment Interaction. in Virtual-Environment Interaction. Proceedings of SIGGRAPH 97, Los Proceedings of SIGGRAPH 97, Los Angeles, CA. Angeles, CA. (133K (133K pdfpdf version) version)
Two Handed and Two Handed and Body-Centered Body-Centered
InteractionInteraction
Other Manipulation Other Manipulation ApproachesApproaches
Tablet/hand-heldTablet/hand-held Pinch GlovePinch Glove