stopping rule use during information search in design tasks glenn j. browne texas tech university...
TRANSCRIPT
Stopping Rule Use During Stopping Rule Use During Information Search in Information Search in
Design TasksDesign Tasks
Glenn J. Browne
Texas Tech University
Mitzi G. Pitts
University of Memphis
OutlineOutline
Introduction: Aims of the StudyBackgroundSetting and MethodResultsConclusion
Introduction: Aims of the Introduction: Aims of the StudyStudy
Improving requirements determination for systems development
Understanding how analysts decide when to stop gathering information
Understanding role of experience in that decision
Background:Background:The Decision-Making ProcessThe Decision-Making Process
Simon’s Model– Intelligence– Design– Choice
Information SearchInformation Search
Why we might expect information search in different stages of the decision-making process to differ.
“Stopping Rules”
Information AcquisitionInformation Acquisition
Problems in acquisition– Underacquiring– Overacquiring
HeuristicsHeuristics
Defined – rules of thumb for taking actions in various situations
Examples– “80-20 Rule”– “Feed a cold, starve a fever”
Heuristics for Assessing Heuristics for Assessing LikelihoodLikelihood
Normative – Relative FrequencyDescriptive
– E.g., availability, representativeness, anchoring and adjustment
Heuristics for ChoiceHeuristics for Choice
Normative– E.g., expected value of information, expected
value of additional information, expected loss from terminating information acquisition.
Descriptive– E.g., Dominance, Conjunctive, Disjunctive,
“The Minimalist,” “Take the Best.”
Heuristics for Intelligence Heuristics for Intelligence Gathering and DesignGathering and Design
?
Heuristics for Intelligence Gathering Heuristics for Intelligence Gathering and Design:and Design:Some IdeasSome Ideas
Nickles, Curley, and Benson (1995)– Difference Threshold– Magnitude Threshold– Mental List– Representational Stability
Difference Threshold Stopping Rule
Magnitude Threshold Stopping Rule
Mental List Stopping Rule
Representational Stability Stopping RuleRepresentational Stability Stopping Rule
Impact of Analyst ExperienceImpact of Analyst Experience
On information gatheredOn stopping rules used
The ContextThe Context
Requirements gathering for information systems development
Application for grocery shopping on world wide web
54 practicing systems analysts in the Baltimore-Washington metro area
Participants asked to gather requirements until they felt they had enough information to draw diagrams representing requirements and proceed with system design.
Measuring Information Measuring Information RequirementsRequirements
Requirements TaxonomyTotal requirements (Quantity)BreadthDepth
HypothesesHypotheses
H1a: The use of some stopping rules will result in different quantities of requirements than the use of others.
H1b: The use of some stopping rules will result in different breadth of requirements than the use of others.
H1c: The use of some stopping rules will result in different depth of requirements than the use of others.
Hypotheses (cont.)Hypotheses (cont.) H2a: A greater number of experienced analysts will
use the mental list rule than will use the representational stability rule.
H2b: A greater number of experienced analysts will use the mental list rule than will use the difference threshold rule.
H2c: A greater number of experienced analysts will use the magnitude threshold rule than will use the representational stability rule.
H2d: A greater number of experienced analysts will use the magnitude threshold rule than will use the difference threshold rule.
Hypotheses (cont.)Hypotheses (cont.)
H3a: There will be no relationship between the experience of the analyst and the quantity of requirements elicited.
H3b: There will be no relationship between the experience of the analyst and the quality of requirements elicited.
Data AnalysisData Analysis
Verbal protocols and questionnairesCodingInterrater reliabilityStopping rule identification
ResultsResults
Stopping Rule Use– Difference Threshold – 22– Representational Stability – 13– Mental List - 10– Magnitude Threshold – 9
Results (cont.)Results (cont.)
Requirements Elicited by Stopping Rule– Quantity – F(3,50) = 2.72; p = .05– Breadth - F(3,50) = 1.72; p = .17
– Depth - 2(3) = 8.98; p = .03
Results (cont.)Results (cont.)
Impact of Experience on Stopping Rule Use– Mental List = 14.30 years– Magnitude Threshold = 14.06 years– Difference Threshold = 11.11 years– Representational Stability = 7.65 years
Results (cont.)Results (cont.)
Impact of Experience on Stopping Rule Use– Mental List rule users were more experienced than
users of the Representational Stability rule (t(21) = 2.27; p = .019), supporting Hypothesis 2a.
– Users of the Magnitude Threshold rule were also more experienced than users of the Representational Stability rule (t(20) = 2.00; p = .03), supporting Hypothesis 2c.
– Other two hypotheses were not supported.
Results (cont.)Results (cont.)
Impact of Experience on Requirements Elicited– Analysts’ years of experience were unrelated to
the total number of requirements elicited (Pearson’s r2 = .08; p = .59), supporting H3a.
– Breadth of requirements (r2 = .15; p = .27) and depth of requirements (r2 = .02; p = .91) were also unrelated to analysts’ years of experience, supporting H3b.
ConclusionConclusion
Identification of stopping rules during information search
Impacts of analyst experienceImpact on information systems
development process