This Pump Sucks: Testing Transitivity with

Individual Data

Michael H. Birnbaum and Jeffrey P. Bahra

California State University, Fullerton

Transitivity of Preference

• If A > B and B > C then A > C.• Satisfy it or become a money

pump.• But transitivity may not hold if

data contain “error.”• And different people might have

different “true” preferences.

Tversky (1969)

• Tversky (1969) reported that selected subjects showed a pattern of intransitive data consistent with a lexicographic semi-order.

• Tversky tested Weak Stochastic Transitivity: If P(A>B) > 1/2 and P(B>C) > 1/2 then P(A>C) > 1/2.

Issues

• Iverson & Falmagne (1985) argued that Tversky’s statistical analysis was incorrect of WST.

• Tversky went on to publish transitive theories of preference (e.g., CPT).

Renewed Interest in Intransitive Preference

• New analytical methods for analysis of transitivity (Iverson, Myung, & Karabatsos; Regenwetter & Stober, et al); Error models (Sopher & Gigliotti, ‘93; Birnbaum, ‘04; others).

• Priority Heuristic (Brandstaetter, et al., 2006); stochastic difference model (González-Vallejo,

2002; similarity judgments, Leland, 1994; majority rule, Zhang, Hsee, Xiao, 2006). Renewed interest in Fishburn, as well as in Regret Theory.

Lexicographic Semi-order• G = (x, p; y, 1 - p). F = (x’, q; y’, 1 -

q).

• If y - y’ ≥ L choose G (L = $10)

• If y’ - y ≥ L choose F

• If p - q ≥ P choose G (P = 0.1)

• If q - p ≥ P choose F

• If x > x’ choose G; if x’ > x choose F;• Otherwise, choose randomly.

Priority Heuristic• “Aspiration level” is 10% of largest

prize, rounded to nearest prominent number.

• Compare gambles by lowest consequences. If difference exceeds the aspiration level, choose by lowest consequence.

• If not, compare probabilities; choose by probability if difference ≥ 0.1

• Compare largest consequences; choose by largest consequences.

New Studies of Transitivity

• Work currently under way testing transitivity using same procedures as used in other decision research.

• Participants view choices via the WWW, click button beside the gamble they would prefer to play.

• Today’s talk: Single-S data.

Studies with Roman Gutierez

• Four studies used Tversky’s 5 gambles, formatted with tickets or with pie charts.

• Studies with n = 417 and n = 327 with small or large prizes ($4.50 or $450)

• No pre-selection of participants.• Participants served in other risky DM

studies, prior to testing (~1 hr).

Three of Tversky’s (1969) Gambles

• A = ($5.00, 0.29; $0, 0.79)• C = ($4.50, 0.38; $0, 0.62)• E = ($4.00, 0.46; $0, 0.54)Priority Heurisitc Predicts:A preferred to C; C preferred to E, and E preferred to A.

Findings• Results were surprisingly transitive,

unlike Tversky’s data (est. 95% transitive).

• Of those 115 who were perfectly reliable, 93 perfectly consistent with EV (p), 8 with opposite ($), and only 1 intransitive.

• Differences: no pre-test; Probability represented by # of tickets (100 per urn), rather than by pies; Participants have practice with variety of gambles, & choices;Tested via Computer.

Pie Chart Format

Pies: with or without Numerical probabilities

• 321 participants randomly assigned conditions with probabilities displayed as pies (spinner), either with numerical probabilities displayed or without.

• Of 105 who were perfectly reliable, 84 were perfectly consistent with EV (prob), 13 with the opposite order ($); 1 consistent with LS.

Findings• Priority Heuristic predicted violations of

transitivity were rare and rarely repeated when probability and prize information presented numerically.

• Violations of transitivity are still rare but more frequent when probability information presented only graphically.

• Evidence of Dimension Interaction violates PH and additive Difference models.

Response to Birnbaum-Gutierrez

• Perhaps the intransitivity only develops in longer studies. Tversky used 20 replications of each choice.

• Perhaps consequences of Tversky’s gambles diminished since 1969 due to inflation. Perhaps transitivity occurs because those prizes are too small.

Birnbaum & Bahra• Collected up to 40 choices/pair per

person. (20 reps). 2 Sessions, 1.5 hrs, 1 week apart.

• Cash prizes up to $100. • 51 participants, of whom 10 to win

the prize of one of their chosen gambles.

• 3 5 x 5 Designs to test transitivity vs. Priority heuristic predictions

Notation-Two-branch Gambles

• G = (x, p; y, 1 - p); x > y ≥ 0• L = Lower Consequence• P = Probability to win higher prize• H = Higher consequence

LH Design

• A = ($84, .50; $24)• B = ($88, .50; $20)• C = ($92, .50; $16)• D = ($96, .50; $12)• E = ($100, .50; $8)

LP Design

• A = ($100, .50; $24)• B = ($100, .54; $20)• C = ($100, .58; $16)• D = ($100, .62; $12)• E = ($100, .66; $8)

PH Design

• A = ($100, .50; $0)• B = ($96, .54; $0)• C = ($92, .58; $0)• D = ($88, .62; $0)• E = ($84, .66; $0)

Priority Heuristic Predictions

• LH Design: E > D > C > B > A, but A > E

• LP Design: A ~ B ~ C ~ D ~ E, but A > E

• PH Design: A > B > C > D > E but E > A

One Rep = 2 choices/pair

Second GambleFirst A B C D E

A 2 2 2 2B 1 2 2 2C 1 1 2 2D 1 1 1 2E 1 1 1 1

Analysis

• Each replication of each design has 20 choices; hence 1,048,576 possible data patterns (220) per rep.

• There are 1024 possible consistent patterns (Rij = 2 iff Rji = 1, all i, j).

• There are 120 (5!) possible transitive patterns.

Within-Rep Consistency

• Count the number of consistent choices in a replicate of 20 choices (10 x 2).

• If a person always chose the same button, consistency = 0.

• If a person was perfectly consistent, consistency = 10.

• Randomly choosing between 1 and 2 produces expected consistency of 5.

Intransitive and Consistent

LH Second GambleFirst A B C D E

A 2 2 1 1B 1 2 2 1C 1 1 2 2D 2 1 1 2E 2 2 1 1

Within-Replicate Consistency

• The average rate of agreement was 8.63 (86% self-agreement).

• 46.4% of all replicates were scored 10; an additional 19.9% were scored 9.

LH Design: Overall Proportions Choosing Second Gamble

Second GambleFirst A B C D E

A 0.41 0.38 0.34 0.27B 0.58 0.40 0.36 0.30C 0.61 0.59 0.44 0.32D 0.64 0.61 0.55 0.33E 0.70 0.69 0.66 0.66

LP Design: Overall Proportions Choosing Second Gamble

Second GambleFirst A B C D E

A 0.44 0.43 0.42 0.36B 0.54 0.42 0.42 0.38C 0.54 0.55 0.45 0.40D 0.56 0.56 0.53 0.41E 0.60 0.59 0.57 0.56

PH Design: Overall Proportions Choosing Second Gamble

PH Second GambleFirst A B C D E

A 0.61 0.64 0.64 0.64B 0.37 0.61 0.63 0.65C 0.34 0.37 0.64 0.64D 0.34 0.35 0.33 0.63E 0.34 0.33 0.35 0.34

Majority Data WST

• LH Design A>B>C>D>E• LP Design A>B>C>D>E• PH Design E>D>C>B>A• Patterns consistent with special

TAX with “prior” parameters.• But this analysis hides individual

diffs

Individual Data

• Choice proportions calculated for each individual in each design.

• These were further broken down within each person by replication.

S# 8328 C = 9.6 Rep = 20

LH Second GambleA B C D E

A 0.02 0.02 0.00 0.02B 0.02 0.00 0.02C 0.02 0.00D 0.02E

S# 8328 C = 9.8 Rep = 20

LP Second GambleA B C D E

A 0.05 0.02 0.00 0.00B 0.00 0.00 0.00C 0.05 0.02D 0.00E

S# 8328 C = 9.9 Rep = 20

PH Second GambleA B C D E

A 1.00 1.00 1.00 0.98B 1.00 1.00 1.00C 0.95 1.00D 0.95E

S# 6176 C = 9.8 Rep = 20; started with this pattern, then switched to perfectly consistent

with the opposite pattern for 4 replicates at the end of the first day; back to this pattern for 10

reps on day 2.

PH Second GambleA B C D E

A 0.28 0.20 0.23 0.20B 0.25 0.20 0.20C 0.20 0.20D 0.20E

S# 684 C = 8.1 Rep = 14; an intransitive pattern opposite that predicted by priority heuristic.

LP Second GambleA B C D E

A 0.07 0.57 0.71 0.50B 0.18 0.54 0.68C 0.14 0.57D 0.14E

S# 7663 C = 6.3 Rep = 10; an intransitive pattern consistent with priority heuristic, P = 0.05. Few reps and low self-consistency in this

case.

PH Second GambleA B C D E

A 0.15 0.75 0.75 0.90B 0.45 0.55 0.65C 0.30 0.55D 0.30E

Data Summary

• For n = 51, there are 153 matrices. Of these, 90% were perfectly consistent with WST: P(A,B) ≥ 1/2 & P(B,C) ≥ 1/2 then P(A,C) ≥ 1/2.

• 29 people had all three arrays fitting WST; no one had all three arrays with intransitive patterns.

Summary of WST Individuals

29 People with 3 Perfectly WST Patterns

Within-Person Changes in Preference Pattern

• Criterion: Person must show perfect consistency (10 out of 10) to one pattern in one replication, and perfect consistency to another pattern on another replication.

• 15 Such cases were found (10%). There may be other cases where the data are less consistent.

Delta = 1; Preference for A or E in LH, LP, and PH Designs, respectively

0

0.5

1

1.5

2

2.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

Gamma

PH

LH

LP

AAA (1)

AAE (14)

AEE (3)

EAA (4)EEE (3)

EEA (4)

Summary

• Recent studies fail to confirm systematic violations of transitivity predicted by priority heuristic. Adds to growing case against this descriptive model.

• Individual data are mostly transitive.• Next Q: From individual data, can we

predict, for example, from these data to other kinds of choices by same person, e. g., tests of SD?