the growth of cognitive modeling in human-computer interaction since goms presented by: daniel...
Post on 19-Dec-2015
216 views
TRANSCRIPT
The Growth of Cognitive modeling The Growth of Cognitive modeling in Human-Computer Interaction in Human-Computer Interaction
Since GOMSSince GOMS
Presented by:Presented by:Daniel Loewus-DeitchDaniel Loewus-Deitch
Douglas GrimesDouglas Grimes
Purpose of Article (1)Purpose of Article (1)
Evaluate the current status of our Evaluate the current status of our methods for modeling HCI cognition methods for modeling HCI cognition and analyzing task performance.and analyzing task performance.
Review the evolution of GOMS and Review the evolution of GOMS and MHP and how subsequent research MHP and how subsequent research has supported and extended this has supported and extended this framework. framework.
Purpose of Article (2)Purpose of Article (2)
Examine 3 new directions for this Examine 3 new directions for this framework:framework: Study of learning and transfer.Study of learning and transfer. Study of errors.Study of errors. Analysis of parallel processes.Analysis of parallel processes.
Some Central IssuesSome Central Issues
How people transfer between skilled How people transfer between skilled performance and problem solving.performance and problem solving.
Designing consistent user interfaces.Designing consistent user interfaces. How people produce and manage How people produce and manage
errors.errors. How we interpret visual displays.How we interpret visual displays. When processes are parallel and When processes are parallel and
when they are serial.when they are serial.
Usefulness and Goals of Cognitive Usefulness and Goals of Cognitive ModelingModeling
Main goal:Main goal: Predicting how users will interact with Predicting how users will interact with
proposed designs.proposed designs.
Usefulness and Goals of Cognitive Usefulness and Goals of Cognitive ModelingModeling
Useful in:Useful in: Initially constraining the design space.Initially constraining the design space. Answering specific design decisions, Answering specific design decisions,
including various tradeoffs.including various tradeoffs. Estimating the total time for task Estimating the total time for task
performance.performance. Estimate training time and guide Estimate training time and guide
training documentation.training documentation. Discovering the most resource-intensive Discovering the most resource-intensive
and error-prone stages of an activity.and error-prone stages of an activity.
GOMS and MHPGOMS and MHP
Card et al. proposed their framework Card et al. proposed their framework to help system designers to help system designers Gather detailed knowledge about the Gather detailed knowledge about the
processes of perception to action.processes of perception to action. Generate predictions about human Generate predictions about human
behavior and task performance in “real, behavior and task performance in “real, naturalistic settings.”naturalistic settings.”
GOMS and MHPGOMS and MHP
Two key components:Two key components: MHP (Model Human Processor) – a MHP (Model Human Processor) – a
general characterization of the human general characterization of the human as an information-processing system, as an information-processing system, including aincluding a
System architectureSystem architecture Library of quantitative parameters that Library of quantitative parameters that
breaks down task performance into specific breaks down task performance into specific components.components.
GOMS and MHPGOMS and MHP
Two key components:Two key components: GOMS (Goals, Operators, Methods, and GOMS (Goals, Operators, Methods, and
Selecction rules) – A family of models Selecction rules) – A family of models that describes what the user needs to that describes what the user needs to know in order to perform computer-know in order to perform computer-based tasks.based tasks.
GOMS and MHPGOMS and MHP
Steps of theoretical process:Steps of theoretical process: User perceives activity on screen.User perceives activity on screen. Evaluates whether it is what is expected.Evaluates whether it is what is expected. Sets up an intention of the next step (goals).Sets up an intention of the next step (goals). Retrieves way to enact this intent on the Retrieves way to enact this intent on the
system.system. Executes appropriate motor movements.Executes appropriate motor movements. Repeat process.Repeat process.
Critical Steps of User ActivitiesCritical Steps of User Activities
AssumptionsAssumptions
Routine cognitive skills can be Routine cognitive skills can be described as a serial sequence of described as a serial sequence of cognitive operations and motor cognitive operations and motor activities.activities.
Each time parameter is independent Each time parameter is independent of context (the same in any task).of context (the same in any task).
Empirical data derived from use of Empirical data derived from use of text editors, graphic systems, etc. is text editors, graphic systems, etc. is generalizable.generalizable.
StrengthsStrengths
Can accurately predict the time it takes a Can accurately predict the time it takes a skilledskilled user to execute a task based on a set user to execute a task based on a set of composite actions.of composite actions.
Certain component parameters have been Certain component parameters have been found to be very consistent across various found to be very consistent across various tasks and stable across repeated tasks and stable across repeated experiments.experiments. KeystrokeKeystroke PointingPointing Moving handsMoving hands Retrieving a chunk of info. from LTM.Retrieving a chunk of info. from LTM.
StrengthsStrengths
Allows comparisons of different Allows comparisons of different design alternatives.design alternatives.
LimitationsLimitations
Generalizable to new domains?Generalizable to new domains? Applied only to skilled users.Applied only to skilled users. No account of learning or delayed recall.No account of learning or delayed recall. No account of errors (predicts perfect No account of errors (predicts perfect
performance).performance). Cognitive processes treated in same Cognitive processes treated in same
manner as perceptual and motor manner as perceptual and motor activities.activities.
No consideration of parallel processes.No consideration of parallel processes.
LimitationsLimitations
No consideration of mental workload.No consideration of mental workload. Does not determine functionality.Does not determine functionality. Does not address user fatigue.Does not address user fatigue. No account for individual differences.No account for individual differences. Does not predict user satisfaction or Does not predict user satisfaction or
acceptance.acceptance. Does not consider CSCW issues.Does not consider CSCW issues.
Advances in Modeling Specific Advances in Modeling Specific Serial Components (1)Serial Components (1)
Direct tests of two assumptions:Direct tests of two assumptions: Serial processingSerial processing Consistency of time parameters across Consistency of time parameters across
taskstasks Helped define and expand the MHP.Helped define and expand the MHP. Determined by studies involving Determined by studies involving
entering editor commands with entering editor commands with keyboard commands and entering keyboard commands and entering formulas in spreadsheets.formulas in spreadsheets.
Advances in Modeling Specific Advances in Modeling Specific Serial Components (2)Serial Components (2)
Parameters are grouped into 3 Parameters are grouped into 3 classes:classes: Motor movementMotor movement PerceptionPerception Memory and cognitionMemory and cognition
Parameters can be mapped onto the Parameters can be mapped onto the critical steps of the user activity critical steps of the user activity process (previous figure).process (previous figure).
Motor MovementsMotor Movements
KeyingKeying 230 msec230 msec Can vary with skill level of typist, Can vary with skill level of typist,
frequency of particular key use, and frequency of particular key use, and predictability of text being typed.predictability of text being typed.
Moving a mouseMoving a mouse 1100 msec per selection1100 msec per selection Varies with distance of movement and Varies with distance of movement and
size of target (Fitt’s Law).size of target (Fitt’s Law). Always a constant time of 1 s to began Always a constant time of 1 s to began
moving.moving.
Motor MovementsMotor Movements
Hand MovementsHand Movements Move from keyboard to pointing device.Move from keyboard to pointing device. Large-muscle movement.Large-muscle movement. Variation among different pointing Variation among different pointing
devices.devices.
PerceptionPerception
Perceptual processor is estimated to be 100 Perceptual processor is estimated to be 100 msec.msec.
Saccade is estimated at 230 msec.Saccade is estimated at 230 msec. ““Scanning” parameter, identified by Olson and Scanning” parameter, identified by Olson and
Nielson, found in experiment to be 2300 msec.Nielson, found in experiment to be 2300 msec. User scans spreadsheet screen, looking for cell User scans spreadsheet screen, looking for cell
addresses.addresses. Composite of scanning, storing, and retrieving.Composite of scanning, storing, and retrieving. Roughly supports previously identified time Roughly supports previously identified time
parameter estimate when broken down into parameter estimate when broken down into components.components.
Memory and Cognitive ProcessesMemory and Cognitive Processes
Memory retrievalMemory retrieval M (“mental”) = time to retrieve the next M (“mental”) = time to retrieve the next
unit of information from LTM.unit of information from LTM. Estimated at 1350 msec.Estimated at 1350 msec. On repeated trials, retrieval times drop On repeated trials, retrieval times drop
50% and remain flat, but keying times 50% and remain flat, but keying times remain the same.remain the same.
Memory and Cognitive ProcessesMemory and Cognitive Processes
Executing steps in a taskExecuting steps in a task GOMS provides an explicit GOMS provides an explicit
representation of the mental steps representation of the mental steps involved.involved.
Kieras and Polson programmed the Kieras and Polson programmed the procedures in production system procedures in production system formalism.formalism.
Memory and Cognitive ProcessesMemory and Cognitive Processes
Choosing among methodsChoosing among methods MHP assumes that more choices for a MHP assumes that more choices for a
response lead to longer response times.response lead to longer response times. Research suggests that a choice Research suggests that a choice
between methods is a complex cognitive between methods is a complex cognitive task, requiring a number of cognitive task, requiring a number of cognitive steps.steps.
Summary of cognitive engineering Summary of cognitive engineering parameters derived:parameters derived:
Enter a keystrokeEnter a keystroke 230 msec230 msec
Point with a mousePoint with a mouse 1500 msec1500 msec
Move hands to mouseMove hands to mouse 360 msec360 msec
PerceivePerceive 100 msec100 msec
Make a saccadeMake a saccade 230 msec230 msec
Retrieve from Retrieve from memorymemory
1200 msec1200 msec
Execute a mental Execute a mental stepstep
70 msec70 msec
Choose among Choose among methodsmethods
1250 msec1250 msec
Example of Applied Use of GOMSExample of Applied Use of GOMS
Walker et al. demonstrated how Walker et al. demonstrated how GOMS can help a designer narrow GOMS can help a designer narrow down his search.down his search.
Goal was to shorten menu-selection Goal was to shorten menu-selection time with nested menus.time with nested menus.
Example of Applied Use of GOMSExample of Applied Use of GOMS
3 adjustments were made and tested 3 adjustments were made and tested using GOMS techniques:using GOMS techniques: Pop up submenus on right rather than Pop up submenus on right rather than
bottom (shorten total distance user bottom (shorten total distance user much move cursor to make a selection).much move cursor to make a selection).
Target size grows as the distance from Target size grows as the distance from the cursor increases (Fittsized menus).the cursor increases (Fittsized menus).
Add a virtual border around the pop-up Add a virtual border around the pop-up menu (increase target size).menu (increase target size).
Accuracy in Predicting Composite Accuracy in Predicting Composite PerformancePerformance
Young and MacLean measured the times Young and MacLean measured the times of people entering a block of values in a of people entering a block of values in a spreadsheet using 2 different methods:spreadsheet using 2 different methods: Mouse methodMouse method Menu methodMenu method
Showed that using these parameters can Showed that using these parameters can provide sufficient accuracy for this level provide sufficient accuracy for this level of analysis (14% error range in this of analysis (14% error range in this case).case).
Extensions of Basic FrameworkExtensions of Basic Framework
Task-Action Grammar (TAG)Task-Action Grammar (TAG) Knowledge a user must have in order to Knowledge a user must have in order to
translate from goals to actions in a translate from goals to actions in a particular system.particular system.
Predicts learning with the relationship Predicts learning with the relationship between system features and natural between system features and natural world associations.world associations.
Consists of commands, features of goal, Consists of commands, features of goal, dictionary of tasks, and rules that dictionary of tasks, and rules that translate goals into actions.translate goals into actions.
Extensions of Basic FrameworkExtensions of Basic Framework
Production systemsProduction systems Represent GOMS structure and aspects Represent GOMS structure and aspects
of MHP.of MHP. Makes underlying knowledge much Makes underlying knowledge much
more explicit.more explicit.
Learning and TransferLearning and Transfer
Reaction to GOMS’ narrow focus on Reaction to GOMS’ narrow focus on skilled performance.skilled performance.
Time to learn has been advanced byTime to learn has been advanced by Cognitive Complexity TheoryCognitive Complexity Theory Payne and Green’s research on grammar Payne and Green’s research on grammar
rules.rules. Transfer of training has been advanced Transfer of training has been advanced
byby Kieras and Polson’s production system Kieras and Polson’s production system
modelsmodels
Kieras and Polson’s Kieras and Polson’s Cognitive Cognitive Complexity Theory Complexity Theory (1)(1)
Made advances by focusing onMade advances by focusing on Time to learn new procedures.Time to learn new procedures. Transfer of training between procedures.Transfer of training between procedures.
Used specialized language called Used specialized language called NGOMSL to facilitate the NGOMSL to facilitate the programming of production system programming of production system representations.representations.
Kieras and Polson’s Kieras and Polson’s Cognitive Cognitive Complexity Theory Complexity Theory (2)(2)
First determined number of steps in First determined number of steps in procedure and then assessed the procedure and then assessed the time it takes a person to learn the time it takes a person to learn the procedure.procedure.
Can’t generalize quantified learning Can’t generalize quantified learning times to naturalistic settings.times to naturalistic settings.
Payne and GreenPayne and Green
Number of rules determines ease of Number of rules determines ease of learning.learning. More often a rule can be used, the more More often a rule can be used, the more
consistent a system is.consistent a system is. More critical is whether features of More critical is whether features of
rules follow real-world rules already rules follow real-world rules already familiar to the user.familiar to the user.
Kieras and Polson’s Production Kieras and Polson’s Production System ModelsSystem Models
Makes explicit exactly what it is that a Makes explicit exactly what it is that a person has to learn to use a new system.person has to learn to use a new system.
Productions = units of learning.Productions = units of learning. Number of productions the two systems Number of productions the two systems
share is a good prediction of transfer.share is a good prediction of transfer. Makes consistency of design across Makes consistency of design across
systems explicit and quantifiable.systems explicit and quantifiable. TAG has some similar potential.TAG has some similar potential.
(Time to Learn Scatter Plot – (Time to Learn Scatter Plot – Poulson)Poulson)
Analysis of ErrorsAnalysis of Errors
Hypothesis: When working memory Hypothesis: When working memory is overloaded, errors will increases.is overloaded, errors will increases.
Test: Calculate Profit = Revenues – Test: Calculate Profit = Revenues – Costs, using Lotus 1-2-3 & IFPSCosts, using Lotus 1-2-3 & IFPS
Result: IFPS Result: IFPS burdens working burdens working memory memory fewer errors fewer errors
Similar studies on SQL queries.Similar studies on SQL queries.
Parallel ProcessesParallel Processes
Earlier cognitive studies assumed Earlier cognitive studies assumed serial processing.serial processing.
Skilled motor and perceptual action is Skilled motor and perceptual action is highly parallel:highly parallel: Expert typist, musician, athlete, driver, …Expert typist, musician, athlete, driver, …
Serial models overestimate time for Serial models overestimate time for parallel tasksparallel tasks
Skilled action is more parallel; novice Skilled action is more parallel; novice action more serial.action more serial.
Critical Path Analysis: Critical Path Analysis: Parallel + SequentialParallel + Sequential
Theory: 3 parallel processors: Theory: 3 parallel processors: perceptual, cognitive, and motorperceptual, cognitive, and motor
Studies of skilled typing and skilled Studies of skilled typing and skilled menu use. menu use. (Author’s interpretation, not (Author’s interpretation, not speaker’s):speaker’s): Expert (~33 wpm) limited by cognitive Expert (~33 wpm) limited by cognitive
processor (e.g., reading speed for typist)processor (e.g., reading speed for typist) Novice (~6 wpm) limited by motor Novice (~6 wpm) limited by motor
speed.speed.
Q’s on Critical Path AnalysisQ’s on Critical Path Analysis If cognitive processor is the bottleneck If cognitive processor is the bottleneck
for expert typist, why can expert typists for expert typist, why can expert typists read much faster than they type? read much faster than they type? Do we Do we “chunk” letters together in preconscious processing, “chunk” letters together in preconscious processing, then “unchunk” them to type?then “unchunk” them to type?
Where do we draw the line between Where do we draw the line between perception and cognition? perception and cognition? Example shows Example shows reading is one word at a time with reading is one word at a time with constant timeconstant time per per word, but recognizing words is a cognitive process, word, but recognizing words is a cognitive process, whether preconscious or not. Also ocular-motor skill.whether preconscious or not. Also ocular-motor skill.
Doesn’t address looping/cascading.Doesn’t address looping/cascading. In spite of Q’s In spite of Q’s more accurate more accurate
predictions for some tasks than purely predictions for some tasks than purely serial models.serial models.
What cognitive modeling (serial & What cognitive modeling (serial & parallel) has not revealed parallel) has not revealed
How we move from skilled to unskilled How we move from skilled to unskilled tasks smoothly throughout the day.tasks smoothly throughout the day.
Learning: Learning: “Athough we know that consistency in a “Athough we know that consistency in a system may make that system easier to learn … and system may make that system easier to learn … and easier to operate…, we do not know much more than easier to operate…, we do not know much more than that.”that.”
Errors: More than overloading working Errors: More than overloading working memorymemory
Individual differences in HCIIndividual differences in HCI Fatigue and other stress factors: machines Fatigue and other stress factors: machines
don’t tire or get stressed out; people do.don’t tire or get stressed out; people do.
From GOMS to GamesFrom GOMS to Games Prior studies based on static visual Prior studies based on static visual
display of text and numbers.display of text and numbers. Bonnie John & Alonso Vera extended Bonnie John & Alonso Vera extended
GOMS model to more interactive GOMS model to more interactive HCI’s --video games, call handling, & HCI’s --video games, call handling, & browsing.browsing.
Subject: 9 year-old expertSubject: 9 year-old expert Result – 2 levels: Result – 2 levels:
GOMS predicts functional interaction wellGOMS predicts functional interaction well Keystroke accuracy ~ 50%Keystroke accuracy ~ 50%
Program (HI-SOAR) learns ~ humansProgram (HI-SOAR) learns ~ humans
ConclusionConclusion
GOMS and its successors are useful for GOMS and its successors are useful for many repetitive HCI tasks – bank tellers, many repetitive HCI tasks – bank tellers, ATM’s, airline reservations, ATM’s, airline reservations, bookkeeping… Cognitive modeling bookkeeping… Cognitive modeling saves $ + time saves $ + time less need to build and test prototypes on less need to build and test prototypes on
users when models can predict results.users when models can predict results. User testing still needed for reality check User testing still needed for reality check
– “situated cognition” = understanding – “situated cognition” = understanding tasks in their human and environmental tasks in their human and environmental context.context.
(Start of extra slides)(Start of extra slides)
Learning Computer TasksLearning Computer Tasks
Limitation of GOMS: Restricted to Limitation of GOMS: Restricted to skilled performance. Didn’t address skilled performance. Didn’t address learning.learning.
Cognitive Complexity Theory Cognitive Complexity Theory (Kieras & Polson)(Kieras & Polson)
2 aspects of their learning model:2 aspects of their learning model: Time to learn a new system or taskTime to learn a new system or task Transferring knowledge between tasksTransferring knowledge between tasks
Time to LearnTime to Learn
Kieras developed a high-level Kieras developed a high-level programming language to describe programming language to describe learning simple computer tasks, like learning simple computer tasks, like data entry or writing an SQL query.data entry or writing an SQL query.
Keiras & Polson found learning each Keiras & Polson found learning each step of a task takes ~ 30 s. Other step of a task takes ~ 30 s. Other studies found shorter times.studies found shorter times.
Current best guess ~ 25 s./step for Current best guess ~ 25 s./step for learning.learning.
Learning & ComplexityLearning & Complexity
Number of rules to learn is not as Number of rules to learn is not as important how well the rules follow important how well the rules follow real-world features the learner real-world features the learner already knows.already knows.
Implication: Metaphors work, as long Implication: Metaphors work, as long as they are consistent.as they are consistent.
Learning Computer Tasks (1): Learning Computer Tasks (1): TheoryTheory
K & P’s production rules make K & P’s production rules make explicit exactly what a person has to explicit exactly what a person has to learn to master a new system.learn to master a new system.
Hypothesis: If # of productions Hypothesis: If # of productions (~steps) is a measure of how long it (~steps) is a measure of how long it takes to learn, then 2 systems with takes to learn, then 2 systems with same # of productions should take same # of productions should take same time to learn.same time to learn.
Learning Computer Tasks (2): Learning Computer Tasks (2): StudiesStudies
Example: Compare time it takes to Example: Compare time it takes to learn:learn: How to copy a floppy How to copy a floppy How to print a documentHow to print a document
Result: Learning times are consistent Result: Learning times are consistent across systems.across systems.
Value: When designing systems, favor Value: When designing systems, favor alternatives with simpler production rulesalternatives with simpler production rules
(easier to learn!)(easier to learn!)
Philosophical Critique of GOMS Philosophical Critique of GOMS
Does it reduce human to machine?Does it reduce human to machine? Answer: Not intended to address Answer: Not intended to address
philosophy. Only intended to aid in philosophy. Only intended to aid in system design.system design.
Bottom line: Predictions average Bottom line: Predictions average within 14% of observed values for a within 14% of observed values for a narrow range of serial tasks.narrow range of serial tasks.
Practical Critiques of GOMS Practical Critiques of GOMS
Restricted to skilled performanceRestricted to skilled performance Doesn’t predict learningDoesn’t predict learning Doesn’t apply to parallel tasksDoesn’t apply to parallel tasks
Extensions to Basic GOMS FrameworkExtensions to Basic GOMS Framework
Purpose: Less restrictive framework Purpose: Less restrictive framework for understanding HCI.for understanding HCI.
3 areas of analysis since GOMS:3 areas of analysis since GOMS:1.1. Learning Learning
2.2. ErrorsErrors
3.3. Parallel processesParallel processes
Cognitive Engineering Cognitive Engineering GrammarsGrammars
Purpose: Define cognitive tasks in Purpose: Define cognitive tasks in terms of countable components for terms of countable components for production rules.production rules.
Example: Task-Action Grammar (TAG)Example: Task-Action Grammar (TAG) Commands, e.g. “Left-click,” “press Commands, e.g. “Left-click,” “press
<enter>”<enter>” Features of goal, e.g. direction of cursorFeatures of goal, e.g. direction of cursor Tasks, e.g., “Add 3 numbers.”Tasks, e.g., “Add 3 numbers.” Rules to translate goals into action.Rules to translate goals into action.