interaction in virtual environments

Download Interaction in Virtual Environments

Post on 09-Feb-2016




3 download

Embed Size (px)


Interaction in Virtual Environments. Benjamin Lok. This Lecture contains notes created by Doug Bowman Virginia Tech Sherman & Craig, pp. 332-363. 3D Interfaces. We live in a 3D world Natural interfaces are better Therefore 3D interfaces will be the ultimate Whats wrong with the above? - PowerPoint PPT Presentation


  • Interaction in Virtual EnvironmentsBenjamin LokThis Lecture contains notes created by Doug BowmanVirginia TechSherman & Craig, pp. 332-363

  • 3D InterfacesWe live in a 3D worldNatural interfaces are betterTherefore 3D interfaces will be the ultimateWhats wrong with the above?Natural interfaces arent always better!Making the interface simple (thus unnatural) often aids performanceConstrains movementLimiting possible actionsDepends on application and goal of the user interfaceSurgery simulationMilitary simulation (general vs. soldier training)Architecture, education, product designVideo games

  • 3D InterfacesWhat we really want are enhanced interfacesGive us powers we dont normally haveFlying, x-ray vision, teleportation, undo, etc.Be careful we dont become overzealousAir traffic control 3D displayLibrary interfaces using a books on shelves (what is it good for? What is it poor for?)Hurts performanceStudy results: 3D Bar charts dont help So what is helped by 3D?

  • Good 3DSocial interfaces + 3D can be very powerfulMMORPG (EveQuest)ActivedWorldsThe Sims OnlineExperiencesArt gallary3D Desktops (Macs latest)Office metaphors did not take off (BOB, Task Gallary)3D Webbrowsing. Sure you can arrange 16 web pages spatially, but why?Compromises to provide 3D interfaces might be undermine usabilityThink RTS gamesDiscussion: Is the interface holding back 3D?

  • 3D InterfacesUse occlusion, shadows, perspective carefullyImproves use of spatial memory (Ark 98)Distracting and confusingMinimize navigation stepsKeep text readable (good contrast, 30 degree tilt max)Simple user movement (why lock to a floor?) Descent vs QuakePrevent Errors (put in guides to help)Simplify object movement (connecting two parts, for example, can be abstracted most of the time)Organize groups of items into alignments that facilitate visual search and recall (allow user choice)

  • 3D Interface DevelopmentDevelopments that show promise:3D soundStereo display (Ware and Frank 96)Haptic feedback (mouse)3D can help by:Provide overviews to see big pictureRapid teleportation (context shifts)Zooming (aid disabled)Multiple coordinated views (3dsmax)3D icons can represent abstract or recognizable concepts

  • TeleoperationCombines:Direct ManipulationProcess ControlHuman operators control physical processes in complex environmentsExample applications: Mars rover control, flying airplanes (Predator), manufacturing, medicine (surgery)Supervisory control (Sheridan 92)Different levels of human control (automation)

  • TeleoperationDirect Manipulation IssuesAdequate feedback (data quality, latency (transmission and operation delays), incomplete, interference)PresencePoint and click or more natural interaction vs. typingExample project: Nanomanipulator (show video)

  • VR InteractionTrying to simulate reality or an experienceTraining, Learning, ExploringExpensiveDangerousLogistically DifficultBest interaction?Flight simulators (they can cost $10 mil, but thats still a good deal!)Why?Why do video game flight sims not cut it? (only $40!)Okay, we have monitors that show 3D worlds, what else do we possibly need?

  • Why 3D interaction?3D / VE apps. should be usefulImmersionNatural skillsImmediacy of visualizationBut, current VE apps are Not complex interactivelyHave serious usability problems

  • What makes 3D interaction difficult?Spatial inputLack of constraintsLack of standardsLack of toolsLack of precisionFatigueLayout more complexPerceptionYouve seen this in creating Project #2!

  • Lets look at this example (ISAAC)

  • Universal interaction tasksNavigationTravel motor componentWayfinding cognitive componentSelectionManipulationSystem control

  • Goals of interaction designPerformance (efficiency, accuracy, productivity)Usability (ease of use, ease of learning, user comfort)Usefulness (users focus on tasks, interaction helps users meet system goals)

  • Philosophies of interaction designArtistic approachIntuition about users, tasksHeuristics, metaphorsAestheticsAdaptationScientific approachFormal analysisFormal evaluationPerformance requirements

  • TravelMotor component of navigationMovement between 2 locations, setting the position (and orientation) of the users viewpointMost basic and common VE interaction techniqueUsed in almost any large-scale VE

  • Types of Travel TasksExplorationTravel which has no specific targetBuild knowledge of environmentSearchNaive: travel to find a target whose position is not knownPrimed: travel to a target whose position is knownBuild layout knowledgeMove to task locationManeuveringTravel to position viewpoint for taskShort, precise movements

  • Steering metaphorContinuous specification of direction of motionGaze-directedPointingPhysical device (steering wheel, flight stick)

  • Target-based metaphorDiscrete specification of goalPoint at objectChoose from listEnter coordinatesE.g. Google Earth

  • Route-planning metaphorOne time specification of pathPlace markers in worldMove icon on mapManual manipulation of viewpointCamera in handFixed object manipulation

  • Natural travel metaphorsWalking techniquesTreadmillsBicyclesOther physical motionVMC / Magic carpetDisneys river raft rideSimulation of flying

  • Technique classificationTravelStart to moveStop movingIndicate positionIndicate orientationpositionvelocityaccelerationTarget specificationRoute specificationContinuousspecification

  • Alternate Technique classificationTravelDirection/TargetSelectionVelocity/Accel.SelectionConditions of Inputgaze-directedpointingphysical propsgestureslow in, slow outphysical propsstart/stop buttonsautomatic start/stopconstant movementOntologies by Bowman99 and Hand97

  • Evaluation resultsSteering techniques have similar performance on absolute motion tasksNon-head-coupled steering better for relative motionTeleportation can lead to significant disorientationEnvironment complexity affects information gatheringTravel interaction technique and users strategies affect spatial orientation

  • Evaluation results 3 Manipulation-based techniques efficient for relative motionManipulation-based techniques not requiring an object efficient for search, but tiringSteering techniques best for nave and primed searchMap-based techniques not effective in unfamiliar environments, or when any precision is required

  • MythsThere is one optimal travel technique for VEs.A natural technique will always be better than another technique.Desktop 3D, workbench, and CAVE applications should use the same travel ITs as HMD-based VEs.

  • Design guidelinesMake simple travel tasks simple (target-based techniques for motion to an object, steering techniques for search).Provide multiple travel techniques to support different travel tasks in the same application.

  • More design guidelinesUse transitional motions if overall environment context is important.Train users in sophisticated strategies to help them acquire survey knowledge.Consider integrated (cross-task) ITs if travel is used in the context of another task (e.g. manipulation).

  • WayfindingCognitive process of defining a path through an environment using and acquiring spatial knowledge6DOF makes wayfinding hardHuman beings have different abilities to orient themselves in an environmentObserving wayfinding as a decision making process

  • Wayfinding tasksGeneral, explorative search Search without targetNave search target position unknownPrimed search target seen before (known)Specified trajectory movement Predefined path

  • Cognitive MapDuring wayfinding, a person makes use of three kinds of knowledge to built up a cognitive map of the environment:Landmark knowledge Procedural knowledge Survey knowledge

  • Reference framesEgocentric reference frame: position, orientation, movement of object with respect to position and orientation of the:eyes head bodyExocentric reference frame: position, orientation and movement are defined in coordinates external to bodyobject shapeobject orientationobject motion

  • Travel technique effectsSteering technique with good strategy helps spatial orientationA good travel technique will integrate aids to wayfinding Jumping between points disturbs spatial orientation

  • Support of spatial knowledge acquisition Allow a wide field of viewProvide motion cues for judging depth and direction of movement Audio could enhance visual spatial perceptionSupport sense of presence: it could strengthen the construction of a cognitive map

  • Support of spatial knowledge acquisitionDesign legible environments - allow the user to easily see the spatial organisation of an environment, enabling the establishment of a cognitive map. Divide a large-scale environment into parts with a distinct character

  • Support of spatial knowledge acquisitionCreate a simple spatial organisation in which the relations between the parts are clearSupport the matching process between the egocentric and exocentric frames of reference by (visual) cues, including directional cues

  • Support of Spatial Knowledge AcquisitionUse real-world wayfinding principles to build up your environmentNatural environment principlesUrban design principles [Lynch]Architectural design principlesArtificial cues

  • Examples of wayfinding aids

  • Map usage guidelinesProvide you are here markerProvide gridChoose either north-up or forward-up map, depending on taskExample: World-in-Miniature

  • Manipulation TechniquesWand-basedGo-GoRay CastingHOMER (Hand-centered Object Manipulation Extending Ray-casting)

  • Two Handed and Body-Centered InteractionWhat can you do with two hands?What if