input devices, interaction techniques, and interaction tasks
DESCRIPTION
Input Devices, Interaction Techniques, and Interaction Tasks. Chapter 8. High-quality user interfaces are in many ways the last frontier in providing computing to a wide variety of users. The quality of user interface often determines whether users enjoy or despise a system - PowerPoint PPT PresentationTRANSCRIPT
Input Devices, Interaction Techniques,
and Interaction TasksChapter 8Chapter 8
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
2
High-quality user interfaces are in many ways High-quality user interfaces are in many ways the last frontier in providing computing to a the last frontier in providing computing to a wide variety of users.wide variety of users.
The quality of user interface often determines The quality of user interface often determines whether users enjoy or despise a systemwhether users enjoy or despise a system
We focus in this chapter on input devices, We focus in this chapter on input devices, interaction technologies, and interaction tasks.interaction technologies, and interaction tasks. These are the basic building blocks from which user These are the basic building blocks from which user
interfaces are constructed.interfaces are constructed.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
3
1. Interaction Hardware
Here we introduce some interaction devices not Here we introduce some interaction devices not covered in Section 4.5, elaborate on how they covered in Section 4.5, elaborate on how they work, and discuss the advantages and work, and discuss the advantages and disadvantages of various devices.disadvantages of various devices. advantages and disadvantages can be discussed on advantages and disadvantages can be discussed on
three levels: three levels: • device -- hardware characteristicsdevice -- hardware characteristics
• task -- we might compare interaction techniques uding task -- we might compare interaction techniques uding different devices for the same taskdifferent devices for the same task
• dialogue -- we consider not just individual interaction tasks dialogue -- we consider not just individual interaction tasks but also sequences of tasks.but also sequences of tasks.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
4
1.1 Locator Devices
It is useful to classify locator devices It is useful to classify locator devices according to three independent according to three independent characteristics:characteristics: absolute or relativeabsolute or relative direct or indirectdirect or indirect discrete or continuousdiscrete or continuous
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
5
Absolute devicesAbsolute devices have a frame of reference or origin, and report have a frame of reference or origin, and report
positions with respect to that originpositions with respect to that origin examples include: data tablets or touch panelexamples include: data tablets or touch panel
Relative DevicesRelative Devices have no absolute origin and report only changes have no absolute origin and report only changes
from their former positionfrom their former position Examples include: mice, trackballs, and velocity-Examples include: mice, trackballs, and velocity-
control joystickscontrol joysticks
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
6
Direct DeviceDirect Device the user points directly at the screen with a finger or the user points directly at the screen with a finger or
surrogate finger.surrogate finger. Examples include a touch screenExamples include a touch screen
Indirect DeviceIndirect Device the user moves a cursor on the screen using a device the user moves a cursor on the screen using a device
not on the screen.not on the screen. Examples include: a tablet, mouse, or joystickExamples include: a tablet, mouse, or joystick
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
7
Continuous DeviceContinuous Device one in which smooth hand motion can create a one in which smooth hand motion can create a
smooth cursos motion.smooth cursos motion. Examples include: tablets, joysticks, and miceExamples include: tablets, joysticks, and mice These typically allow more natural, easier, and These typically allow more natural, easier, and
faster cursor movement than do discrete devices.faster cursor movement than do discrete devices.
Discrete DeviceDiscrete Device Examples include: cursor control keys.Examples include: cursor control keys.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
8
Speed of cursor positioning with a continuous Speed of cursor positioning with a continuous device is affected by the device is affected by the control-to-display control-to-display ratioratio, commonly called the C/D ratio., commonly called the C/D ratio. It is the ratio between hand movement (the control) It is the ratio between hand movement (the control)
and cursor movement (the display)and cursor movement (the display) A large ratio is good for accuracy, but makes rapid A large ratio is good for accuracy, but makes rapid
movements tedious.movements tedious. A small ratio is good for speed, but not for accuracy.A small ratio is good for speed, but not for accuracy. Fortunately, for a relative positioning device the Fortunately, for a relative positioning device the
ratio does not have to be constant, but can be ratio does not have to be constant, but can be changed adaptively.changed adaptively.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
9
1.2 Keyboard Devices
The well known QWERTY keyboard has The well known QWERTY keyboard has been with us for many years.been with us for many years.
It is ironic that this keyboard was originally It is ironic that this keyboard was originally designed to designed to slow downslow down typists, so that the typists, so that the typewriter hammers would not be so likely typewriter hammers would not be so likely to jam.to jam.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
10
1.3 Valuator Devices
Some valuators are bounded (like the Some valuators are bounded (like the volume control on a radio).volume control on a radio).
A continuous-turn potentiometer can be A continuous-turn potentiometer can be turned an unbounded number of times in turned an unbounded number of times in either direction.either direction. given an initial value, it can return absolute given an initial value, it can return absolute
values, otherwise it returns a relative value.values, otherwise it returns a relative value.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
11
1.4 Choice Devices
Function keys are common choice devices.Function keys are common choice devices. Their placement affects their usability:Their placement affects their usability:
keys on the CRT are harder to use than keys on keys on the CRT are harder to use than keys on the keyboard, or in a nearby separate unit.the keyboard, or in a nearby separate unit.
A foot switch can be used in applications in A foot switch can be used in applications in which the user’s hands are engaged yet a single which the user’s hands are engaged yet a single switch closure must be frequently made.switch closure must be frequently made.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
12
1.5 Other Devices
Here we discuss some of the less common, Here we discuss some of the less common, and in some ways experimental, 2D and in some ways experimental, 2D interaction devices.interaction devices. voice recognizersvoice recognizers pressure sensitive data tablets.pressure sensitive data tablets.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
13
1.6 3D Interaction Devices
Some 2D interaction devices are readily Some 2D interaction devices are readily extended to 3D.extended to 3D. joysticks can have a shaft that twists for a third joysticks can have a shaft that twists for a third
dimensiondimension trackballs can be made to sense rotation about trackballs can be made to sense rotation about
the vertical axis in addition to that about the the vertical axis in addition to that about the two horizontal axes.two horizontal axes.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
14
A number of devices can record 3D hand A number of devices can record 3D hand movementsmovements The data glove records hand position and orientation The data glove records hand position and orientation
as well as finger movement.as well as finger movement.
FIGURE 8.2FIGURE 8.2
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
15
Several other technologies can be used to Several other technologies can be used to record 3D positions.record 3D positions. In one, using optical sensors, LEDs are mounted on In one, using optical sensors, LEDs are mounted on
the user the user • (either at a single point, such as the finger tip, or all over (either at a single point, such as the finger tip, or all over
the body, to measure body movement)the body, to measure body movement)
• Light sensors are mounted high in the corners of a Light sensors are mounted high in the corners of a semidarkened room in which the user works, and each semidarkened room in which the user works, and each LED is intensified in turn.LED is intensified in turn.
• The sensors can determine the plane in which the LED The sensors can determine the plane in which the LED lieslies
In another, a television camera records hand In another, a television camera records hand movementsmovements
• image processing techniques are used to find the position image processing techniques are used to find the position of the fingers and hand. of the fingers and hand.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
16
2. Basic Interaction Tasks (BITs)
With a basic interaction task, the user of an With a basic interaction task, the user of an interactive system enters a unit of information interactive system enters a unit of information that is meaningful in the context of the that is meaningful in the context of the application.application.
BITs are indivisible. We discuss composite BITs are indivisible. We discuss composite interaction tasks in the next section.interaction tasks in the next section.
Examples include: positioning, selecting, Examples include: positioning, selecting, entering text, and entering numeric quantities.entering text, and entering numeric quantities.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
17
2.1 The Position Interaction Task
The positioning task involves specifying a (x,y) The positioning task involves specifying a (x,y) or an (x,y,z) position to the application or an (x,y,z) position to the application program.program. The customary interaction techniques for carrying The customary interaction techniques for carrying
out this task involve either moving a screen cursor out this task involve either moving a screen cursor to the desired location and then pushing a button, or to the desired location and then pushing a button, or typing the desired position’s coordinates on either a typing the desired position’s coordinates on either a real or simulated keyboard.real or simulated keyboard.
The positioning device can either be direct or The positioning device can either be direct or indirect, continuous or discrete, absolute or relative.indirect, continuous or discrete, absolute or relative.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
18
There are two types of positioning tasks, spatial There are two types of positioning tasks, spatial and linguistic.and linguistic.
In a spatial positioning task, the user knows where In a spatial positioning task, the user knows where the intended position is, in spatial relation to nearby the intended position is, in spatial relation to nearby elements (as in drawing a line between two elements (as in drawing a line between two rectangles, or centering an object between two rectangles, or centering an object between two others).others).
In a linguistic positioning task, the user knows the In a linguistic positioning task, the user knows the numeric values of the (x,y) coordinates of the numeric values of the (x,y) coordinates of the positionposition
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
19
2.2 The Select Interaction Task -- Variable-Sized Set of Choices
The selection task is that of choosing an element The selection task is that of choosing an element from a choice set.from a choice set. Typical sets are commands, attribute values, object Typical sets are commands, attribute values, object
classes, object instances, …classes, object instances, … Some interaction techniques can be used to select from Some interaction techniques can be used to select from
any of these four types of choice sets; others are less any of these four types of choice sets; others are less general.general.
• For example, pointing at a visual representation of a set element For example, pointing at a visual representation of a set element can serve to select it, no matter what the set typecan serve to select it, no matter what the set type
• On the other hand, it is difficult to use function keys, since the On the other hand, it is difficult to use function keys, since the size of the choice set is variable, and often is large. (larger than size of the choice set is variable, and often is large. (larger than the number of function keys.)the number of function keys.)
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
20
We use the terms We use the terms (relatively) fixed-size choice (relatively) fixed-size choice setset and and varying-sized choice setvarying-sized choice set The first characterizes command, attribute, and The first characterizes command, attribute, and
object-class choice setsobject-class choice sets• The term relatively is used because the size of the set can The term relatively is used because the size of the set can
change, but it will not change frequently, or very much.change, but it will not change frequently, or very much.
Varying-sized choice sets, on the other hand, can Varying-sized choice sets, on the other hand, can become quite large, and can change frequently.become quite large, and can change frequently.
In this section, we discuss techniques that are In this section, we discuss techniques that are particularly well suited to potentially large particularly well suited to potentially large varying-sized choice sets. In the next section varying-sized choice sets. In the next section we cover techniques for fixed-size choice sets.we cover techniques for fixed-size choice sets.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
21
Selecting Objects by NamingSelecting Objects by Naming The user can type the choice’s name.The user can type the choice’s name.
But they may not know, or have no reason to know But they may not know, or have no reason to know the objects name.the objects name.
Nevertheless this technique is useful in several Nevertheless this technique is useful in several situations.situations.
• If you know the name, it may be easier & faster to use it.If you know the name, it may be easier & faster to use it.
If naming by typing is necessary, feedback can If naming by typing is necessary, feedback can help expedite the process.help expedite the process. A useful form of feedback is autocompletionA useful form of feedback is autocompletion
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
22
Selecting Objects by Pointing Selecting Objects by Pointing Any of the pointing techniques mentioned in Any of the pointing techniques mentioned in
the intro to this section can be used to select an the intro to this section can be used to select an object.object.
If an object has multiple levels of hierarchy (as If an object has multiple levels of hierarchy (as does a robot), you may want multiple select does a robot), you may want multiple select commands (select_robot, select_arm,…)commands (select_robot, select_arm,…)
If the number of hierarchies is large and If the number of hierarchies is large and unknown, at least two commands are needed:unknown, at least two commands are needed: up_hierarchyup_hierarchy down_hierarchydown_hierarchy
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
23
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
24
2.3 The Select Interaction Task -- Relatively Fixed-Size Choice Set
Menu selection is one of the richest Menu selection is one of the richest techniques for selecting from a relatively techniques for selecting from a relatively fixed-size choice set.fixed-size choice set.
In this section we will discuss several key In this section we will discuss several key factors in menu designfactors in menu design
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
25
Single-level vs. hierarchical designSingle-level vs. hierarchical design The set is too large to display all at once, what The set is too large to display all at once, what
do you do?do you do? Scroll bar on a menuScroll bar on a menu
hierarchical menu.hierarchical menu.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
26
Menu placementMenu placement Menus can static and permanently visible, or Menus can static and permanently visible, or
can appear dynamically on request. (tear-off, can appear dynamically on request. (tear-off, appearing, pop-up, pull-down, and pull-out)appearing, pop-up, pull-down, and pull-out)
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
27
Current selectionCurrent selection If a system has the concept of a currently If a system has the concept of a currently
selected element of a choice set, a radio button selected element of a choice set, a radio button selection technique is one way to implement selection technique is one way to implement this.this.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
28
Size and shape of menu itemsSize and shape of menu items Pointing accuracy and speed are affected by the Pointing accuracy and speed are affected by the
size of each individual item.size of each individual item. Large items are faster to selectLarge items are faster to select Smaller items take less space, and permit more Smaller items take less space, and permit more
menu items to be displayed in the fixed area, but menu items to be displayed in the fixed area, but they induce more errors during selection.they induce more errors during selection.
Thus you have a tension between the two in Thus you have a tension between the two in your designyour design
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
29
Pattern RecognitionPattern Recognition In selection techniques involving pattern In selection techniques involving pattern
recognition, the user makes a sequence of recognition, the user makes a sequence of movements with a continuous-positioning movements with a continuous-positioning device, such as a tablet or mouse.device, such as a tablet or mouse. Proofreader’s marks indicating delete, capitalize, Proofreader’s marks indicating delete, capitalize,
move, and so on are attractive candidates for this move, and so on are attractive candidates for this approachapproach
Advances in these type of algorithms have led Advances in these type of algorithms have led to the explosion in handheld PDAs (palm,…)to the explosion in handheld PDAs (palm,…)
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
30
Function keysFunction keys Elements of the choice set can be associated Elements of the choice set can be associated
with function keys with function keys We can think of every key as a function keyWe can think of every key as a function key Unfortunately there never seem to be enough keys Unfortunately there never seem to be enough keys
to go around.to go around.
The keys can be used in a hierarchical-selection The keys can be used in a hierarchical-selection fashion, or their meanings can be altered using fashion, or their meanings can be altered using chords of keys.chords of keys.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
31
2.4 The Text Interaction Task
The text-string input task entails entering a The text-string input task entails entering a character string to which the application does character string to which the application does not ascribe any special meaning.not ascribe any special meaning. Typing a command name is not a text-entry task.Typing a command name is not a text-entry task. Typing legends for a graph and typing text into a Typing legends for a graph and typing text into a
word processor are.word processor are. Clearly the most common text-input technique is Clearly the most common text-input technique is
the use of the QWERTY keyboard.the use of the QWERTY keyboard.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
32
2.5 The Quantify Interaction Task
The quantify interaction task involves The quantify interaction task involves specifying a numeric value between some specifying a numeric value between some minimum and maximum value.minimum and maximum value.
Typical interaction techniques are Typical interaction techniques are typing the valuetyping the value setting a dial to the valuesetting a dial to the value using an up-down counter to select the valueusing an up-down counter to select the value
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
33
One means of entering values is a One means of entering values is a potentiometer (pot). potentiometer (pot). Then you have to decide on a rotary or linear one.Then you have to decide on a rotary or linear one.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
34
2.6 3D Interaction Tasks
Two of the four interaction tasks described Two of the four interaction tasks described previously for 2D applications become more previously for 2D applications become more complicated in 3D: position and selectcomplicated in 3D: position and select
The first part of this section deals with The first part of this section deals with positioning and selection, which are closely positioning and selection, which are closely related.related.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
35
This figure shows a common way to position in This figure shows a common way to position in 3D. 3D.
By providing three projections of an object, it is By providing three projections of an object, it is sometimes easier to manipulate it.sometimes easier to manipulate it. It is also common to provide an oblique/perspective It is also common to provide an oblique/perspective
view of the scene.view of the scene.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
36
As with locating and selecting, the issues in 3D As with locating and selecting, the issues in 3D rotation are rotation are understanding depth relationshipsunderstanding depth relationships mapping 2D interaction devices into 3Dmapping 2D interaction devices into 3D ensuring stimulus-response compatibility.ensuring stimulus-response compatibility.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
37
An easily implemented 3D rotation technique An easily implemented 3D rotation technique provides slider dials or gauges that control provides slider dials or gauges that control rotation about the three axes.rotation about the three axes.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
38
It is often necessary to combine 3D interaction It is often necessary to combine 3D interaction tasks.tasks.
Thus, rotation requires Thus, rotation requires a select task for the object to be rotateda select task for the object to be rotated a position task for the center of the rotationa position task for the center of the rotation and an orient task for the actual rotationand an orient task for the actual rotation
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
39
3. Composite Interaction Tasks
CITs are built on top of Basic Interaction Tasks CITs are built on top of Basic Interaction Tasks described in the previous section, and are described in the previous section, and are actually a combination of BITsactually a combination of BITs
There are basically three major forms of CITsThere are basically three major forms of CITs dialogue boxes -- used to specify multiple units of dialogue boxes -- used to specify multiple units of
informationinformation construction -- used to create objects requiring two construction -- used to create objects requiring two
or more positionsor more positions manipulation -- used to reshape existing objectsmanipulation -- used to reshape existing objects
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
40
3.1 Dialogue Boxes
We often need to select multiple elements of a We often need to select multiple elements of a selection setselection set For example: text attributes (italic, bold, underline,For example: text attributes (italic, bold, underline,
…)…) Menu approaches are useful for a single one, but Menu approaches are useful for a single one, but
disappear when a selection is madedisappear when a selection is made
This problem can be overcome with dialogue This problem can be overcome with dialogue boxesboxes the box is typically dismissed explicitly with a the box is typically dismissed explicitly with a
command.command.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
41
3.2 Construction Techniques
One way to construct a line is to have the user One way to construct a line is to have the user indicate one endpoint and then the other.indicate one endpoint and then the other. Then the line is drawn.Then the line is drawn.
A far superior technique is rubberbanding A far superior technique is rubberbanding (discussed in Chapter 2)(discussed in Chapter 2) when the user pushes a button, the beginning point is when the user pushes a button, the beginning point is
selected.selected. As the cursor moves, so does the endpoint of the line. As the cursor moves, so does the endpoint of the line. When the button is released, the endpoint is frozen.When the button is released, the endpoint is frozen.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
42
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
43
An entire genre of interaction techniques is An entire genre of interaction techniques is derived from rubberband line drawing.derived from rubberband line drawing. Rubber-rectangle Rubber-rectangle
• Anchor one corner and the opposite corner is dynamically Anchor one corner and the opposite corner is dynamically lined to the cursor until the button is released.lined to the cursor until the button is released.
Rubber-circle Rubber-circle • The center is at the initial positionThe center is at the initial position
• The circle passes through the current cursor position.The circle passes through the current cursor position.
All these techniques have a common user-All these techniques have a common user-interaction sequence of button-down, move locator interaction sequence of button-down, move locator and see feedback, button-up.and see feedback, button-up.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
44
3.3 Dynamic Manipulation
It is not sufficient to create lines, rectangles, It is not sufficient to create lines, rectangles, and so on. In many situations, the user and so on. In many situations, the user must be able to modify previously created must be able to modify previously created geometric entities.geometric entities.
Dragging, rotating, and scaling are Dragging, rotating, and scaling are important techniques that affect an entire important techniques that affect an entire object and the scene it is in.object and the scene it is in.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
45
Click and DragClick and Drag dragging moves a selected symbol from one dragging moves a selected symbol from one
position to another under control of a cursor.position to another under control of a cursor.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
46
HandlesHandles The concept of handles is useful to provide The concept of handles is useful to provide
scaling of an objectscaling of an object
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
47
Changing components of an object.Changing components of an object. Vertices could be named, and the user could Vertices could be named, and the user could
enter the new (x,y) coordinate.enter the new (x,y) coordinate. But the same point-and-drag strategy used to But the same point-and-drag strategy used to
move an entire object is more attractive.move an entire object is more attractive.
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
48
4. Interaction-Technique Toolkits.
The look and feel of a user-computer interface is The look and feel of a user-computer interface is determined largely by the collection of determined largely by the collection of interaction techniques provided for it.interaction techniques provided for it. Designing and implementing a good set of Designing and implementing a good set of
interaction techniques is time consuminginteraction techniques is time consuming Using interaction-technique toolkits helps ensure a Using interaction-technique toolkits helps ensure a
consistent look and feel among application programs, consistent look and feel among application programs, and is clearly a sound software-engineering practice.and is clearly a sound software-engineering practice.
Examples include: the Macintosh toolkit, Motif, Examples include: the Macintosh toolkit, Motif, OPEN LOOK, … OPEN LOOK, …
Chapter 8 -- Input Devices, Interaction Techniques, and Interaction Tasks
49
Summary
We have presented some of the msot important We have presented some of the msot important concepts of user interfaces:concepts of user interfaces: input devicesinput devices interaction techniquesinteraction techniques interaction tasksinteraction tasks
There are many more aspects of user interface There are many more aspects of user interface techniques and design, however, that we have not techniques and design, however, that we have not discussed.discussed. Foley, vanDam, et.al (the big white book) has a Foley, vanDam, et.al (the big white book) has a
thorough treatment of those topics. thorough treatment of those topics.