linear models of judgment
DESCRIPTION
Linear Models of Judgment. Judgment vs. choice Multiattribute model of judgment Actuarial model of the environment Experts and computers Bootstrapping models. Judgment vs. Choice. Judgment = assign a score or category e.g., How much would you pay for a one-week trip to Aspen? - PowerPoint PPT PresentationTRANSCRIPT
Linear Models of Judgment
Judgment vs. choice Multiattribute model of judgment Actuarial model of the
environment Experts and computers Bootstrapping models
M ileage P rice
R epairs
C os t
C rash Test A B S
A irbags
S afety
Looks H andling
M arque Features
Fun
E xternal C abin
T runk
S ize
D esirability
Judgment vs. Choice
Judgment = assign a score or categorye.g., How much would you pay for a one-week trip to Aspen?How much do you like Bill Clinton?What will the price of Intel be in 6 months?
Choice = pick from a set of alternativese.g., which car? investment? job?
Multiattribute Choice Model Choice = select most desirable Desirability is judged from attributes Attributes can be FACTS (e.g., price),
COMPOSITES (e.g., safety), or subjective VALUES (e.g., prestige)
There is often a hierarchical structure to attributes and judgments
How to make tradeoffs, e.g., weight & add
Desirability of a Car
M ileage P rice
R epairs
C os t
C rash T est A B S
A irbags
S afety
Looks H andling
M arque F eatures
F un
E xternal C abin
T runk
S ize
D esirability
Lens Model
Judgment (Ys) is an attempt to represent or predict the environment from cues
There is a criterion (Ye) that allows us to estimate correctness of the judgment
YsYe
c
u
e
s
Models of Decision Makers
Slovic’s study of two stockbrokers:A: near term prospects, P/E, earnings qtly trend
B: earnings yearly trend, P/E, profit margin trend
Ratings of business schools:USNWR: reputation with academics, reputation with CEOs, selectivity, placement
BusWeek: recruiters rate analytics, teaming, global; graduates rate teaching, curriculum,
placement
Actuarial Environmental Models Occupation
clergy 46executive 62professional 62student 46teacher 46unemployed 33no answer 47
Job Tenure< .5 years 31.5 - 5.5 245.5 - 8.5 268.5 - 15.5 31> 15.5 years 39
Capon, J. Marketing, 1982, 46, 82-91.
Major Retailer’s Credit Scoring Table
Older economists make more extreme forecasts
Comparisons Using Models
How consistent are individuals? How consensual are experts? How accurate are judges? (Ye vs. Ys)
What are judges doing? (Ysm)
What predicts the criterion? (Yem) How good are our models? Do judges understand the environment?
(Yem vs. Ysm)
Graduate Admissions Example Ys = judgment of admissions committee
(1 to 5 scale) Ye = faculty ratings of performance
Ysm = prediction model of judgments
= -4.17 +.0032*GRE +1.02*GPA +.0791*QI Yem= actuarial model of performance
= -.71 +.0006*GRE +.76*GPA +.2518*QI
Admission and Job Interviews
Harvard Business School stopped conducting interviews– Are interviews accurate?– Are interviews overweighted?– What is their proper role?
HBS no longer uses the GMAT HBS criteria: academics and
character
Advantages of Models
Makes strategy explicit Can see how experts vary Train new judges Learn about environments Enhance or replace experts Can use the model when expert
gone
Judges vs. Environment
Which should be more accurate, expert judges or actuarial models?
Judges have their experience, ability to use cues in complex ways
Actuarial models are simple, typically linear in form, consistent
Judges vs. Actuarial Model
Task Judge Actuarial Model
Credit scoring .80 .95
Stock analysis .23 .80
Personnel .35 .57
Cancer survival -.01 .35
Graduate GPA .33 .69
Why Don’t Experts Do Better? They have the wrong rules They don’t use their rules
- distractions- fatigue, boredom- “exceptions”- unable to make tradeoffs
Bootstrapping Models
If intuitive decision makers have good rules but fail to use them consistently, can we separate signal from noise?
Consensus of judges (see groups later) Model of a judge (bootstrapping)
Judgment = Linear + Nonlinear + Noise What wins: Judge vs. linear model?
A Tale of Three Models
Task JudgeBootstrapModel
ActuarialModel
Credit scoring .80 .85 .95
Stock analysis .23 .29 .80
Personnel .35 .46 .57
Cancer survival -.01 .13 .35
Graduate GPA .33 .50 .69
Some Typical Results
Some tasks are much harder than others Actuarial models almost always win Bootstrapping works! Linear models correlate with any
monotonic function, work well when there is noise, positively correlated cues, work with random or unit weights
To improve on linear models, you need lots of data
Experts and Models
What do experts do best? What do computers do best? How can they be combined? Should we give the model to the
expert or give the expert to the model?
Batterymarch Example
Stock portfolio company Manage $12 Billion with 37
employees Experts identify variables, suggest
rules, design tests, deal with clients Computer keeps databases, runs
tests of rules, buys and sells stocks 10-12 rules identify attractive stocks
Working With the Political Lens: Separating Facts and Values Selecting a bullet for Denver Police
- police want to immobilize suspects- community concerned about injuries- experts testify on each side
What kinds of information are needed? How should this decision be made?
A Frame for Conflict Resolution
Facts Values Desirabilityweight Injury potentialspeedshape Stopping poweretc.
Threat tobystanders
Denver Bullet Resolution Experts combine facts into
judgments on each value Constituencies compromise on how
to weight the values into overall worth
Stopping Power
InjuryPotential
. . . . . . . . . . . . . . . . . . . . P. . . . . . . . . . . . . . . . . . . . . . C. . . . . . . . .. . . . . . . . . . .
proposed by Police
proposed by Community
Role of Technical Experts
Executive whose daughter had a hip deformity
One doctor said, “Wait” A second said, “Brace for 6 months” The third said, “Operate” How would you make this decision?