ve input devices - virginia techcourses.cs.vt.edu/cs5754/lectures/input.pdfve input devices doug...

1

VE Input Devices

Doug BowmanVirginia Tech

2

(C) 2006 Doug Bowman, Virginia Tech 2

Goals and Motivation

Provide practical introduction to the inputdevices used in VEs

Examine common and state of the art inputdevices look for general trendsspark creativity

Advantages and disadvantagesDiscuss how different input devices affect

interface design

In this lecture we will discuss the various input and output devices that are used in3D user interfaces and virtual environment applications.

3


Input devices

Hardware that allows the user tocommunicate with the system

Input device vs. interaction techniqueSingle device can implement many ITs

4


Human-computer interface

SystemSoftware

Use

r int

erfa

ce s

oftw

are

User

Inputdevices

Outputdevices

ITs

5


Human-VE interface

Tracking system

Env. modelSimulation loop:-render-check for events-respond to events-iterate simulation-get new tracker data

Display(s)

Input device(s)

6


Input device characteristics

Degrees of Freedom (DOFs) & DOFcomposition (integral vs. separable)

Type of electronics: Digital vs. analogRange of reported values:

discrete/continuous/hybridData type of reported values: Boolean vs.

integer vs. floating point

7


More input device characteristics

User action required: active/passive/hybridMethod of providing information: “push” vs.

“pull” Intended use: locator, valuator, choice, …Frame of reference: relative vs. absoluteProperties sensed: position, motion, force,

…

8


Practical classification system

Desktop devicesTracking devices3D miceSpecial-purpose devicesDirect human input

9


Desktop devices: keyboards

Chord keyboards1

Arm-mountedkeyboards2

“Soft” keyboards(logical devices)

10


Desktop devices: 6-DOF devices

6 DOFs withouttracking

Often isometricExs: SpaceBall,

SpaceMouse,SpaceOrb

11


Tracking devices: position trackers

Measure position and/or orientation of asensor

Degrees of freedom (DOFs)Most VEs track the head

motion parallaxnatural viewing

12


Other uses for trackers

Track hands, feet, etc.“whole body” interactionmotion capture application

Correspondence betweenphysical/virtual objectsProps5,6

spatial input devices

13


Tracking physical objects (props)

14


Electromagnetic trackers

Exs: Polhemus Fastrak,Ascension Flock ofBirds

Most common (?) Transmitter Receiver(s) Noisy Affected by metal

15


Optical/vision-based trackers

Exs: Vicon, HiBall,ARToolkit

Advantages accurate can capture a large volume allow for untethered tracking

Disadvantages image processing

techniques occlusion problem

16


Inertial trackers

Exs: Intersense IS-300,Intertrax2

Less noise, lag Drift problem Only 3 DOFs

(orientation)

17


Hybrid tracking

Ex: IS-600 / 900 inertial (orient.) acoustic (pos.) additional complexity,

cost

18


Tracking devices: eye tracking

Eye tracking systems provide applications with knowledge of the user’s gazedirection. This information opens the door to a number of interesting interactiontechniques such as eye directed selection and manipulation. The figure on the leftshows the Eyegaze system, a non-intrusive approach which uses an infra-red sourcethat reflects off of the pupil, developed by LC Technologies. The figure on the rightshows iView, a head-mounted eye tracking device developed by SensoMotoricInstruments.

References:www.eyegaze.comwww.smi.de

19


Tracking devices: bend-sensinggloves

CyberGlove7, 5DT Reports hand

posture Gesture:

single postureseries of posturesposture(s) + location or

motion

20


Tracking devices: pinch gloves

Conductive cloth atfingertips

Any gesture of 2 to 10fingers, pluscombinations ofgestures

> 115,000 gestures

21


Case study: Pinch Gloves

Pinch gloves are designed to be acombination device (add a positiontracker)

Very little has been done with PinchGloves in VEs - usually 1 or 2 gestures for:Object selectionTool selectionTravel

22


Characteristics of Pinch Gloves

Relatively low costVery lightUser’s hand becomes the deviceUser’s hand posture can changeAllow two-handed interactionHuge number of possible gestures

23


Characteristics of Pinch Gloves II

Much more reliable than data glovesSupport eyes-off inputCan diminish “Heisenberg effect”Support context-sensitive gesture

interpretation

24


Pinch Gloves in SmartScene13

Lots of two-handedgesturesScale worldRotate worldTravel by “grabbing

the air”Menu selection

25


Pinch Gloves for menus

TULIP system14

ND hand selects menu, Dhand selects item withinmenu

Limited to comfortablegestures

Visual feedback on virtualhands

rapMenu

26


Pinch Gloves for text input

Pinch Keyboard14

Emulate QWERTY Pinch finger to thumb

to type letter underthat finger

Move/rotate hands tochange active letters

Visual feedback

27


3D mice Ring Mouse Fly Mouse Wand Cubic Mouse Dragonfly …

The Ring Mouse (top picture) is a small device worn on the user’s finger which usesultrasonic tracking. It also has two buttons for generating discrete events. The mainadvantages of this device is that it is wireless and inexpensive. The Fly Mouse is a3D mouse that also uses ultrasonic tracking. This device has five buttons instead oftwo and also can be used as a microphone.The Cubic Mouse (shown in the figure on the right) is an input device developed atGMD that allows users to intuitively specify three-dimensional coordinates ingraphics applications. The device consists of a box with three perpendicular rodspassing through the center and buttons for additional input.

28


Special-purpose devices: usingconductive clothVirtual toolbelt

Used to select virtual toolsGood use of proprioceptive cues

Interaction slippers3

Step on displayed optionsClick heels to “go home”

29


Special-purpose devices: PaintingTable4

The Painting Table is another example of a special-purpose input device that is usedin the CavePainting application, a system for painting 3D scenes in a virtualenvironment. The device uses a set of conductive cloth contacts as well astraditional buttons and digital sliders. Users can dip the paint brush prop into thecolored cups to change brush strokes. The bucket is used to throw paint around thevirtual canvas.

References:Keefe, D., Acevedo, D., Moscovich, T., Laidlaw, D., and LaViola, J.“CavePainting: A Fully Immersive 3D Artistic Medium and InteractiveExperience”, Proceedings of the 2001 Symposium on Interactive 3D Graphics, 85-93, 2001.

30


Special-purpose devices:ShapeTape11

ShapeTape is a continuous bend and twist sensitive strip which encourages two-handed manipulation. A BAT is attached and the tool (shown in the figure on theright) is used for creating and editing curves and surfaces along with cameralcontrol and command access. ShapeTape senses bend and twist with two fiberoptic sensors at 6cm intervals.

References:Balakrishnan, Ravin, George Fitzmaurice, Gordon Kurtenbach, and Karan Singh.“Exploring Interactive Curve and Surface Manipulation Using a Bend and TwistSensitive Input Strip” Proceedings of the 1999 Symposium on Interactive 3DGraphics, 111-118, 1999.

31


Human input: speech

Frees handsAllows multimodal inputNo special hardwareSpecialized software Issues: recognition, ambient noise,

training, false positives, …

32


Human input: Bioelectric Control

A recent development at NASA Ames Research Center is a bioelectric input devicewhich reads muscle nerve signals emanating from the forearm. These nerve signalsare captured by a dry electrode array on the arm. The nerve signals are analyzedusing pattern recognition software and then routed through a computer to issuerelevant interface commands. The figure on the left shows a user entering numberson a virtual numeric keypad while the figure on the right shows a user controlling avirtual 757 aircraft.

References:Jorgensen, Charles, Kevin Wheeler, and Slawomir Stepniewski. Bioelectric Controlof a 757 Class High Fidelity Aircraft Simulation,http://ic.arc.nasa.gov/publications/index.html, 1999.

33


Human input: Body Sensing Devices

The MIT Media Lab’s affective computing group has developed a PrototypePhysiological Sensing System which includes a Galvanic Skin Response sensor, aBlood Volume Pulse sensor, a Respiration sensor, and an Electromyogram. Byusing this prototype, interface developers can monitor a user’s emotional state todynamically modify an application’s interface to better fit the user’s needs.

References:http://www.media.mit.edu/affect/

34


More human input

Breathing device -OSMOSE

Brain-body actuatedcontrolmuscle movementsthoughts!

35


Locomotion devices

TreadmillsStationary cyclesVMC / magic carpetWalking/flying simulations (use

trackers)

36


UNIPORT

First Locomotion DeviceFor U.S. Army (1994)

Proof-of-conceptdemonstration

Developed in six weeks Difficult to change

direction of travel Small motions such as

side-stepping areimpossible

37


Treadport

Developed in 1995 Based on a standard

treadmill with the user beingmonitored and constrainedby mechanical attachment tothe user’s waist

User actually walks or jogsinstead of pedaling

Physical movement isconstrained to one direction

38


Individual Soldier Mobility Simulator(Biport)

Most sophisticated locomotiondevice

Designed for the conduct oflocomotion studies

Hydraulic-based locomotiondriven w/ force sensors at thefeet

Safeguards limitedresponsiveness

Too awkward to operate

39


Omni-Directional Treadmill15,16

Most recently developedlocomotion device for U.S. Army

Revolutionary device that enablesbipedal locomotion in anydirection of travel

Consists of two perpendiculartreadmills

Two fundamental types ofmovementUser initiated movementSystem initiated movement

40


Torus treadmill

41


ODT video

42


Virtual Motion Controller17

Weight sensors in platformsense user’s position overplatform

Step in direction to movethat direction

Step further to go faster

43


Walking in place18,19

Analyze tracker information fromhead, body, feet

Neural network (Slater)GAITER project (Templeman)Shown to be better than purely

virtual movement, but worse thanreal walking20

44


Classification of locomotiondevices/techniques

Virtual turning Real turning

Virtualmotion

Desktop VEsVehicle simulators

CAVE wand

Most HMD systemsWalking in place

VMC

Realmotion

Stationary cyclesTreadport

Biport

Wide-area trackingUNIPORT

ODT

45


Input and output with a single device

Classic example - touch screenLCD tablets or PDAs with pen-based inputPhantom haptic deviceFEELEX haptic device21

46


PDA as ideal VE device?22

Offers both input and outputHas on-board memoryWireless communicationPortable, light, robustAllows text / number inputCan be tracked to allow spatial input

47


Conclusions

When choosing a device, consider:CostGeneralityDOFsErgonomics / human factorsTypical scenarios of useOutput devicesInteraction techniques

48


Acknowledgments

Joe LaViola, Brown University, for slidesand discussions

Ron Spencer, presentation on locomotiondevices used by the Army

49


References [1] Matias, E., MacKenzie, I., & Buxton, W. (1993). Half-QWERTY: A One-handed Keyboard Facilitating Skill

Transfer from QWERTY. Proceedings of ACM INTERCHI, 88-94. [2] Thomas, B., Tyerman, S., & Grimmer, K. (1998). Evaluation of Text Input Mechanisms for Wearable

Computers. Virtual Reality: Research, Development, and Applications, 3, 187-199. [3] LaViola, J., Acevedo, D., Keefe, D., & Zeleznik, R. (2001). Hands-Free Multi-Scale Navigation in Virtual

Environments. Proceedings of ACM Symposium on Interactive 3D Graphics, Research Triangle Park, NorthCarolina, 9-15.

[4] Keefe, D., Feliz, D., Moscovich, T., Laidlaw, D., & LaViola, J. (2001). CavePainting: A Fully Immersive 3DArtistic Medium and Interactive Experience. Proceedings of ACM Symposium on Interactive 3D Graphics,Research Triangle Park, North Carolina, 85-93.

[5] Bowman, D., Wineman, J., Hodges, L., & Allison, D. (1998). Designing Animal Habitats Within an ImmersiveVE. IEEE Computer Graphics & Applications, 18(5), 9-13.

[6] Hinckley, K., Pausch, R., Goble, J., & Kassell, N. (1994). Passive Real-World Interface Props for NeurosurgicalVisualization. Proceedings of CHI: Human Factors in Computing Systems, 452-458.

[7] Kessler, G., Hodges, L., & Walker, N. (1995). Evaluation of the CyberGlove(TM) as a Whole Hand InputDevice. ACM Transactions on Computer-Human Interaction, 2(4), 263-283.

[8] LaViola, J., & Zeleznik, R. (1999). Flex and Pinch: A Case Study of Whole-Hand Input Design for VirtualEnvironment Interaction. Proceedings of the International Conference on Computer Graphics and Imaging, 221-225.

[9] Ware, C., & Jessome, D. (1988). Using the Bat: a Six-Dimensional Mouse for Object Placement. IEEEComputer Graphics and Applications, 8(6), 65-70.

[10] Zeleznik, R. C., Herndon, K. P., Robbins, D. C., Huang, N., Meyer, T., Parker, N., & Hughes, J. F. (1993). AnInteractive 3D Toolkit for Constructing 3D Widgets. Proceedings of ACM SIGGRAPH, Anaheim, CA, USA, 81-84.

50


References (2) [11] Balakrishnan, R., Fitzmaurice, G., Kurtenbach, G., & Singh, K. (1999). Exploring Interactive Curve and

Surface Manipulation Using a Bend and Twist Sensitive Input Strip. Proceedings of the ACM Symposium onInteractive 3D Graphics, 111-118.

[12] Froehlich, B., & Plate, J. (2000). The Cubic Mouse: A New Device for Three-Dimensional Input. Proceedingsof ACM CHI.

[13] Mapes, D., & Moshell, J. (1995). A Two-Handed Interface for Object Manipulation in Virtual Environments.Presence: Teleoperators and Virtual Environments, 4(4), 403-416.

[14] Bowman, D., Wingrave, C., Campbell, J., & Ly, V. (2001). Using Pinch Gloves for both Natural and AbstractInteraction Techniques in Virtual Environments. Proceedings of HCI International, New Orleans, Louisiana.

[15] Darken, R., Cockayne, W., & Carmein, D. (1997). The Omni-directional Treadmill: A Locomotion Device forVirtual Worlds. Proceedings of ACM Symposium on User Interface Software and Technology, 213-221.

[16] Iwata, H. (1999). Walking About Virtual Environments on an Infinite Floor. Proceedings of IEEE VirtualReality, Houston, Texas, 286-293.

[17] Wells, M., Peterson, B., & Aten, J. (1996). The Virtual Motion Controller: A Sufficient-Motion WalkingSimulator. Proceedings of IEEE Virtual Reality Annual International Symposium, 1-8.

[18] Slater, M., Usoh, M., & Steed, A. (1995). Taking Steps: The Influence of a Walking Technique on Presence inVirtual Reality. ACM Transactions on Computer-Human Interaction, 2(3), 201-219.

[19] Slater, M., Steed, A., & Usoh, M. (1995). The Virtual Treadmill: A Naturalistic Metaphor for Navigation inImmersive Virtual Environments, Virtual Environments '95: Selected Papers of the Eurographics Workshops (pp.135-148). New York: SpringerWien.

[20] Usoh, M., Arthur, K., Whitton, M., Bastos, R., Steed, A., Slater, M., & Brooks, F. (1999). Walking > Walking-in-Place > Flying, in Virtual Environments. Proceedings of ACM SIGGRAPH, 359-364.

51


References (3) [21] Iwata, H., Yano, H., Nakaizumi, F., & Kawamura, R. (2001). Project FEELEX: adding haptic surface to

graphics. Proceedings of ACM SIGGRAPH, Los Angeles, 469-476. [22] Watsen, K., Darken, R., & Capps, M. (1999). A Handheld Computer as an Interaction Device to a Virtual

Environment. Proceedings of the Third Immersive Projection Technology Workshop. [23] Zhai, S. (1998). User Performance in Relation to 3D Input Device Design. Computer Graphics, 32(4), 50-54.

ve input devices - virginia techcourses.cs.vt.edu/cs5754/lectures/input.pdfve input devices doug...

Documents