testing transitivity with a true and error model
DESCRIPTION
Testing Transitivity with a True and Error Model. Michael H. Birnbaum California State University, Fullerton. Testing Algebraic Models with Error-Filled Data. Models assume or imply formal properties such as transitivity: If A > B and B > C then A > C - PowerPoint PPT PresentationTRANSCRIPT
Testing Transitivity with a True and Error Model
Michael H. BirnbaumCalifornia State University,
Fullerton
Testing Algebraic Models with Error-Filled Data
• Models assume or imply formal properties such as transitivity:
If A > B and B > C then A > C• But such properties may not hold if
data contain “error.”• And different people might have
different “true” preferences.
Error Model Assumptions
• Each choice in an experiment has a true choice probability, p, and an error rate, e.
• The error rate is estimated from (and is the “reason” given for) inconsistency of response to the same choice by same person over repetitions
One Choice, Two Repetitions
A B
A
B€
pe2
+ ( 1 − p )( 1 − e )2
p ( 1 − e ) e + ( 1 − p )( 1 − e ) e
p ( 1 − e ) e + ( 1 − p )( 1 − e ) e
€
p ( 1 − e )2
+ ( 1 − p ) e2
Solution for e
• The proportion of preference reversals between repetitions allows an estimate of e.
• Both off-diagonal entries should be equal, and are equal to:
( 1 − e ) e
Ex: Stochastic Dominance
: 05 tickets to win $12
05 tickets to win $14
90 tickets to win $96
B: 10 tickets to win $12
05 tickets to win $90
85 tickets to win $96
122 Undergrads: 59% repeated viols (BB) 28% Preference Reversals (AB or BA) Estimates: e = 0.19; p = 0.85170 Experts: 35% show 2 violations (BB) 31% Reversals (AB or BA) Estimates: e = 0.196; p = 0.50
Testing Higher Properties• Extending this model to properties
relating 2, 3, or 4 choices:• Allow a different error rate on
each choice.• Estimate true probability for each
choice pattern. Different people can have different “true” patterns, which need not be transitive.
New Studies of Transitivity
• Work currently under way testing transitivity under same conditions as used in tests of other decision properties.
• Participants view choices via the WWW, click button beside the gamble they would prefer to play.
Some Recipes being Tested
• Tversky’s (1969) 5 gambles.• LS: Preds of Priority Heuristic• Starmer’s recipe• Additive Difference Model (regret;
majority rule)• Birnbaum, Patton, & Lott (1999) recipe.• Recipes based on Bleichrodt & Schmidt
context-dependent utility models.
Replications of Tversky (1969) with Roman
Gutierez• First two studies used Tversky’s 5
gambles, but formatted with tickets instead of pie charts.
• Two studies with n = 417 and n = 327 with small or large prizes ($4.50 or $450)
• No pre-selection of participants.• Participants served in other risky DM
studies, prior to testing (~1 hr).
Three of Tversky’s (1969) Gambles
• A = ($5.00, 0.29; $0, 0.79)• C = ($4.50, 0.38; $0, 0.62)• E = ($4.00, 0.46; $0, 0.54)Priority Heurisitc Predicts:A preferred to C; C preferred to E, and E preferred to A. Intransitive.
Tversky (1969) reported viol of WST
Response Combinations
Notation (A, B) (B, C) (C, A)
000 A B C *
001 A B A
010 A C C
011 A C A
100 B B C
101 B B A
110 B C C
111 B C A *
Weak Stochastic Transitivity
€
P ( A f B ) = P ( 000 ) + P ( 001 ) + P ( 010 ) + P ( 011 )
P ( B f C ) = P ( 000 ) + P ( 001 ) + P ( 100 ) + P ( 101 )
P ( C f A ) = P ( 000 ) + P ( 010 ) + P ( 100 ) + P ( 110 )
WST Can be Violated even when Everyone is Perfectly
Transitive
€
P ( 001 ) = P ( 010 ) = P ( 100 ) =1
3
€
P ( A f B ) = P ( B f C ) = P ( C f A ) =2
3
Triangle Inequality has similar problems:
• It is possible that everyone is transitive but WST is violated.
• It is possible that people are systematically intransitive and WST is satisfied.
• Possible that everyone is intransitive and triangle inequality is satisfied.
Model for Transitivity
€
P ( 000 ) = p000
( 1 − e1
)( 1 − e2
)( 1 − e3
) + p001
( 1 − e1
)( 1 − e2
) e3
+
+ p010
( 1 − e1
) e2
( 1 − e3
) + p011
( 1 − e1
) e2e
3+
+ p100
e1
( 1 − e2
)( 1 − e3
) + p101
e1
( 1 − e2
) e3
+
+ p110
e1e
2( 1 − e
3) + p
111e
1e
2e
3
A similar expression is written for the other seven probabilities. These can in turn be expanded to predict the probabilities of showing each pattern repeatedly; i.e., up to six errors.
Expand and Simplify• There are 8 X 8 data patterns in an
experiment with 2 repetitions.• However, most of these have very small
frequencies.• Examine probabilities of each of 8
repeated patterns.• Frequencies of showing each of 8
patterns in one replicate OR the other, but NOT both. Mutually exclusive, exhaustive partition.
Tests of WSTPercentage Choosing Column >pr Row Gamble
Row A B C D E
A 73 77 80 85
B 30 68 79 79
C 16 29 74 78
D 11 16 24 63
E 13 17 15 33
Patterns for A, C, EPattern Rep. 1 Rep 2 Both
000 14 28 5
001 18 25 15
010 23 38 1
011 12 5 3
100 24 33 7
101 5 6 1
110 301 256 220
111 19 25 5
Sum 416 416 257
Pattern Both Rep 1 or 2Not both
Est p Pred Both
Pred 1 or 2 Not both
000 5 16 .03 8.1 8.6
001 15 6.5 .07 15.3 6.5
010 1 29.5 .00 4.7 37.2
011 3 5.5 .01 2.8 5.9
100 7 21.5 .03 7.8 26.0
101 1 4.5 .00 0.9 5.5
110 220 58.5 .85 196.6 67.6
111 5 17 .02 4.6 17.9
Sum 257 159 1 240.9 175.1
Comments• Results are surprisingly transitive, unlike
Tversky’s data (est. 95% transitive).• Of those 115 who were perfectly reliable,
93 perfectly consistent with EV (p), 8 with opposite ($), and only 1 intransitive.
• Differences: no pre-test, selection;• Probability represented by # of tickets
(100 per urn), rather than by pies.• Participants have practice with variety of
gambles, & choices;• Tested via Computer.
Pie Chart Format
Pies: with or without Numerical probabilities
• 321 participants randomly assigned to same study; except probabilities displayed as pies (spinner), either with numerical probabilities displayed or without.
• Of 105 who were perfectly reliable, 84 were perfectly consistent with EV (prob), 13 with the opposite order ($); 1 consistent with LS.
Lower Standards
• Look at AB,BC,CD,DE choices and EA choices only:
• 10 of 321 participants showed this pattern; all in the pies-only condition. Although the rate is low (6% of 160), association with condition is clear!
• By still lower standard used by Tversky: 75% agreement with above pattern: 37 people, 27 in pies-only condition.
Tests of Lexicographic Semi-order and Additive Difference• LS implies no integration of
contrasts (additive difference model allow integration)
• Both LS and additive difference models imply no interactions between probability and consequences.
Test of Interaction
R S Pies & p
Pies & No p
($7.25, .95; $1.25, .05)
($4.25, .95; $3.25, .05)
16 22
($7.25, .05; $1.25, .95)
($4.25, .05; $3.25, .95)
84 77
Among the 37 Leniently classified as Intransitive
• Are those 37 who are 75% consistent with the LS in the 5 choices also approx. consistent with LS in tests of Interaction?
• No. 26 of these have all four choices in the pattern of interaction predicted by TAX and other utility models.
Summary• Priority Heuristic model’s predicted
violations of transitivity are rare and rarely repeated when probability and prize information presented numerically.
• Violations of transitivity are still rare but more frequent when prob information presented only graphically.
• Evidence of Dimension Interaction violates PH and additive Difference models.
Conclusions
• Violations of transitivity are probably not due to intransitive strategy (LS or additive difference model), but rather to a configural assimilation of the probability values, which are then used in a numerical utility model.
• We are still unable to produce the higher rates of intransitivity reported by Tversky and others.
Transitivity Test: ADLTrial Choice % Response Pattern
RepsEst. parameters
00 01 10 11 p e
8 50 to win $10050 to win $20
50 to win $6050 to win $27
18 190 23 25 23 0.10
0.10
3 50 to win $6050 to win $27
50 to win $4550 to win $34
29 140 44 39 37 0.17
0.20
21 50 to win $4550 to win $34
50 to win $10050 to win $20
74 35 33 20 172
0.85
0.12
Results-ADLpattern Rep 1 Rep 2 Both
000 LPH 21 13 1
001 TAX 134 147 106
010 20 18 8
011 38 37 10
100 15 9 0
101 14 10 0
110 12 15 7
111 6 11 1
Sum 260 260 133