Chapter 6 – Direct Manipulation and Virtual Environments

6.1 Introduction

• Good interfaces produce positive feelings

• Desirable:– Visibility of objects

– Visibility of actions

– Rapid, reversible, incremental actions

– Direct manipulation of objects of interest

6.2 Examples of Direct Manipulation Interfaces

• Windows environment (Xerox, Apple, Microsoft)• Air Traffic Control • Automobile?• WYSIWYG Word Processors• Mapping and GIS• Modern computer games• CAD • Programming of industrial robots by moving robot by hand

(actions recorded)

5.3 Explaining Direct Manipulation

• Beneficial attributes: – Novices learn quickly

– Experts work rapidly

– Intermittent users can retain concepts

– Error messages are rarely needed

– Users see if their actions are furthering their goals

– Users experience less anxiety

– Users gain confidence and mastery - encourages exploration

Explaining Direct Manipulation

• Problems with direct manipulation – Spatial or visual representations can be too spread out

– Users must learn the graphical representations

– The visual representation may be misleading

– Typing commands with the keyboard may actually be faster

– Choosing the right objects, actions, metaphors is not easy

– May need greater system resources

– History and other tracing may be difficult to maintain

– Visual impaired users may have difficulty

Icons• An icon is an image, picture, or symbol representing a

concept • Icon-specific guidelines

• Represent the object or action in a familiar manner

• Limit the number of different icons

• Make icons stand out from the background

• Consider three-dimensional icons

• Ensure a selected icon is visible from unselected icons

• Ensure harmony and distinctiveness

• Design the movement animation

• Add detailed information

• Explore combinations of icons to create new objects or actions

Direct Manipulation Programming

• Robots Programmed by workers leading them through task once (e.g. painting motion)

• Excel macros programmed via doing the tasks by hand

• MS Access Query by Example

Programming in the User Interface

• Five challenges of programming in the user interface: – Sufficient computational generality

– Access to the appropriate data structures and operators

– Ease in programming and editing programs

– Simplicity in invocation and assignment of arguments

– Low risk

• Possible alternative to Agents

6.7 Remote Direct Manipulation • Examples:

– Telemedicine

– Robotic Space exploration

– Home automation

• Complicating factors in the architecture of remote environments: – Time delays

– Incomplete feedback

– Feedback from multiple sources

– Unanticipated interferences

6.8 Virtual Environments

• Virtual reality

• Augmented reality

• Situational awareness

Virtual Reality

• Examples– Architecture – actually walk into building

and look around (instead of more traditional direct manipulation looking at it from various directions and perspectives)

– Possible - medicine

– On the boundary – real flight simulator

Example: Firefighter Training • This system will simulate the progress of a fire in a single family

dwelling

• will respond to actions made by the user to rescue occupants and put the fire out.

• The user of the VE will be a Fire Company Officer being trained or evaluated for his/her skills at commanding a fire crew.

• In the VE, the user will speak commands that are translated by an operator into a predetermined animation sequence in the virtual environment.

• As the fire company officer issues commands, the virtual fire crew will go through animations reflecting these commands,

• fire burns in response to virtual crew actions.

Example: Meditation Chamber

• The goal of this research is design and build an immersive virtual environment that uses visual, audio, and tactile cues to create, guide, and maintain a patient's guided relaxation and meditation experience. 

• The use of meditation and guided imagery is well established as helpful in the treatment and prevention of a number of diseases

• The possibility of increasing the effectiveness and repeatability of this type of therapy

• This project is aimed at creating a working prototype of this system

Example: Virtual Geographic Information System

• VGIS (Virtual Geographic Information System) is a large, multifaceted project to allow navigation of and interaction with very large and high resolution, dynamically changing databases while retaining real-time display and interaction.

• The system allows users to navigate accurate geographies with sustained frame rates of 15-20 frames per second.

• The user can not only see these terrains from any viewing angle but also buildings, roads, high resolution imagery draped on the terrain, and other features    

Example: Virtual Reality Phobia Therapy

• Virtual Reality Exposure involves exposing the patient to a virtual environment containing the feared stimulus in place of taking the patient into a real environment or having the patient imagine the stimulus, which is what traditional exposure therapy usually involves.

Example: Helping Burn Patients Cope with Pain

• using immersive VR for pain control (in addition to pain medicine).

• Their first virtual world used was SpiderWorld. Spiderworld was originally designed to treat spider phobics, but has also proved quite distracting for burn patients.

• now developing several new virtual environments specifically designed for treating pain (e.g., especially attention-grabbing virtual environments).

• SnowWorld has been developed with support from the Paul Allen Foundation for Medical Research.

• Patients fly through an icy canyon with a river and frigid waterfall. Patients shoot snowballs at snowmen and igloos (with animated impacts).

• Since patients often report re-living their original burn experience during wound care, SnowWorld was designed to help put out the fire.

Example:Virtual Gorilla Exhibit

• being developed to explore techniques for using Virtual Reality to present information to users experientially that would otherwise be difficult for them to learn.

• Based upon actual data from the Zoo Atlanta gorilla exhibit,

• modeling an environment where the user can explore areas that are normally off limits to the casual visitor.

Maybe not quite virtual reality• "Macys.com gets rights to 3-D model software"

by Reuters, CNET News.com, September 2, 1999

• Software developer Broderbund, sell its new Cosmopolitan Fashion Makeover software exclusively through Macys.com.

• With the Cosmopolitan software women can create their own three-dimensional model based on their own body measurements and digitally ``try on'' brand name clothing.

• Users can also link via the makeover software directly to the Macys.com online shopping site, where they can buy the clothing online.

Virtual Reality Headgear

(1999)

Alternative to Headgear

Exploring using the CAVE

Virtual Environments

• Successful virtual environments depend on the smooth integration of: – Visual Display

– Head position sensing

– Hand-position sensing

– Force feedback

– Sound input and output

– Other sensations

– Cooperative and competitive virtual reality

1. Virtual Retinal Display (VRD)2. Shared Space3. Learning in Virtual Environments4. PAIN MAN5. Virtual Motion Controller6. Interactive VRD7. Virtual Pilot8. Greenspace9. Virtual Chess10. Starship11. New Media11. Expert Surgical Assistant13. Tactile Augmentation13. Geoscientific Visualization13. Motion Sickness16. FLIGHT17. Blocksmith18. SS Working Group19. Parkinson's Project20. SS Working Group21. Driving Simulator22. Two-Handed User Interface22. Motion Sickness24. Wearable Interfaces25. Virtual Classroom26. Situation Awareness27. CEDeS Lab27. Endoscopic Surgery Simulator27. Virtual Playground27. Virtual Reality Toolkit

31. Engineering Study of an Endoscope Design31. Virtual Mirrors31. Starship31. Human-Computer Symbiote31. Virtual Chess36. 4d mouse36. collaborative mixed reality36. VR Interaction Techniques39. Collaboration through Wearables39. Phobia Desensitization41. VRD 41. architecture and virtual reality 41. Multimodal Interfaces41. situation awareness41. Medical Robotic Interfaces46. VRD Emulator46. laparoscopic surgical simulator48. Flicker Sensitivity48. knowledge base project 48. /48. self-motion perception52. Design for a Low Vision Aid52. PRISM54. LIMIT54. Interface Sickness54. design for a low vision aid using a scanned laser display

57. visual-inertial nulling: cross-over asymmetry57. design for a low vision aid57. Measures for Presence60. functional effects of refractive surgery on driving performance

U Washington Virtual Reality Projects

Progress on Visual Display• The Virtual Retinal Display (VRD) team has been focused

on developing improvements to the current prototype systems and on creating the parts needed for future prototypes. The VRD, based on the concept of scanning an image directly on the retina of the viewer's eye, was invented at the HIT Lab in 1991. The

development program began in November 1993 with the goal of producing a full color, wide field-of-view, high resolution, high brightness, low cost virtual display.

• http://www.hitl.washington.edu/research/vrd/project.html

Progress on Movement Sensing• For some applications, a hands-free, body-operated

walking interface is ideal;

• the UW HIT Lab has been developing prototypes of "sufficient-motion" interfaces, which allow the user to interact by using a subset of the real-world kinesthetic inputs.

• Though the ranges of motion are less than full, these inputs are sufficient to convince the user that he or she is moving in the virtual world.

• Development of these interface devices is called the Virtual Motion Controller (VMC) Project.

Virtual Motion Controller

• The HIT Lab's VMC working prototype measures body position over the working surface with an arrangement of four weight sensors

• The curved working surface provides important feedback to the user about his or her physical location, and therefore body locomotion input to the device.

Progress on Cooperative Augmented Reality

• The Shared Space interface demonstrates how augmented reality, the overlaying of virtual objects on the real world, can radically enhance face-to-face and remote collaboration.

• For remote collaboration, system allows life-sized live virtual video images of remote user to be overlaid on the local real environment, supporting spatial cues and removing the need to be physically present at a desktop machine to conference.

• computer vision techniques are used to precisely register virtual images with physical objects, extending the currently popular "Tangible Interface" metaphor.

• work in the context of a collaborative card-game application that allows face-to-face and remote users to collaboratively interact with each other and virtual animations.

• http://www.hitl.washington.edu/research/shared_space/

Science Fiction?

• Most of this stuff won’t be in an office near you next year …

• But the future comes quickly in computer science !

End Chapter 6