a dynamic stochastic framework for dual process models€¦ · overview motivation and background...

A dynamic stochastic framework for dual process models

Adele Diederich

Jacobs University Bremen

Purdue Winer Memorial Lectures

November 9 – 12, 2018

Adele Diederich (JUB) Dual process November 9 – 12, 2018 1 / 65

Overview

Motivation and Background

Basic assumptions of sequential sampling models

Dynamic Dual Process model framework

Predictions

Variations

Model account for data

Further directions


De Martino, Kumaran, Seymour, & Dolan (2006), Science


Framing effect

Framing effect: cognitive bias, in which people react to a particularchoice in different ways depending on how it is presented; e.g. as a loss oras a gain.

Preference reversal

Shift in preference

(cf. externality, description-invariance)


Risky choice framing

Choice between two options

Lotteries

Option A is typically riskless

Option B is risky

Situation 1 Outcomes are framed as gains (positive frame)

Situation 2 Outcomes are framed as losses (negative frame)


Gain frame

Keep


Loss frame

Lose


.

Kahneman & Tversky (1979), Prospect Theory (PT), reflectioneffect


De Martino et al. (2006)


De Martino et al. (2006)

Increased activation in the amygdala was associated with subjects’tendency to be risk-averse in the Gain frame and risk-seeking in theLoss frame, supporting the hypothesis that the framing effect isdriven by an affect heuristic underwritten by an emotional system.

When subjects’ choices ran counter to their general behavioraltendency, there was enhanced activity in the anterior cingulate cortex(ACC). This suggests an opponency between two neural systems,with ACC activation consistent with the detection of conflict betweenpredominantly ”analytic” response tendencies and a more”emotional” amygdala-based system.


Loewenstein, O’Donoghue, & Bhatia (2015), Decision

”Evolutionarily older brain regions, such as the limbic system, whichincludes areas such as the amygdala and the hypothalamus, evolvedto promote survival and reproduction, incorporate affectivemechanisms (MacLean, 1990). In contrast, the seemingly uniquehuman ability to choose deliberately, by focusing on broader goals,relies on the prefrontal cortex (Damasio, 1994; Lhermitte, 1986;Miller & Cohen, 2001), the region of the brain that expanded mostdramatically in the course of human evolution (Manuck et al., 2003).Indeed, these results have led to dual-system frameworks for theneuroscience of decision making. ”


Loewenstein et al. (2015)

Framework for intertemporal choice, risky decisions, and social preferences

Affective system

Deliberate system


Dual process models

System 1 Intuitive (fast, emotional, parallel, holistic, . . . )

System 2 Deliberate (slow, rational, serial, . . .)

Applied to cognitive processes including reasoning and judgments

Problems for most approaches:

Verbal – allows no quantitative predictions

Unclear about processing

Reverse inference


New modeling approach

Multi-stage decision model aka multiattribute attention switching(MAAS) model

Belongs to the class of sequential sampling models

Predictions of choice response times and choice frequencies

Linked to ”fact” in biology (natural construct): evidenceaccumulation and threshold


Sequential sampling approach

The information built-up or evidence or preference is incrementedaccording to a diffusion process:

dX (t) = µ(X (t), t)dt + σ(X (t), t)dW (t)

The effective drift rate µ(x , t) describes the instantaneous rate ofexpected increment change at time t and state x = X (t).

The diffusion coefficient σ(x , t) in front of the instantaneousincrements dW (t) of a standard Wiener process W (t) relates to thevariance of the increments.


Sequential sampling approach – Basic assumptions

0

Criterion for choosing Sure

Criterion for choosing Gamble

time (t) P(t

)

Preference sampledcontinuously over time

Random fluctuation inaccumulating preferencestrength

P(t) stochastic process

Each trajectory representsthe accumulation process forone trial


Basic assumptions – Starting point

0



time (t) P(t

)

Initial state of preferencestrength P(0)

P(0) = 0 : neutral

P(0) > 0 : favoring Gamble

P(0) < 0 : favoring Sure

Fixed position → initialstate z

Random location → initialdistribution Z

A priori bias


Basic assumptions – Increments

0



time (t) P(t

)

Increments of preferencesampled at any moment intime dP(t)

dP(t) > 0 : favoringGamble at t

dP(t) < 0 : favoring Sure att

Continuous update ofpreference


Basic assumptions – Decision criterion

0



time (t) P(t

)

Process stops and responseis initiated when a criterionis reached

Instructions or strategiesaffects the criterion

Function of time constraints


Basic assumptions – Thresholds: Stopping times

0



time (t) P(t

)

Optional stopping tIme

P(t) = θG > 0 – choose G

P(t) = θS < 0 – choose S

Internally controlled decisionthreshold

0 t

Acc

umul

atio

n P

roce

ss X

(t)

Fixed Stopping Time

Alternative A

Alternative B

Fixed stopping time

Externally controlleddecision threshold


Variable decision boundaries

Diederich & Oswald (2016) Multi-stage sequential sampling modelswith finite or infinite time horizon and variable boundaries. Journal ofMathematical Psychology


Basic assumptions – Drift rate

0



time (t)P(t

)

Quality of evidencedetermines drift rate (meandrift, drift coefficient)

The better the evidence thelarger µ

The better the alternativescan be discriminated thehigher the the drift rate.


Multi-stage decision model: Basic assumptions

For each system a different accumulation process takes place.

Attention switches from one system to the other system and the twosystems are processed serially

Alternative: The two systems are processes in parallel.


Preference Process

Preference strength is updated from one moment, t, to the next, (t + h)by an reflecting the momentary comparison of consequences produced byimagining the choice of either option G or S with

P(t + h) = P(t) + Vi (t + h),

V (t) : input valence

V (t) = V G (t)− V S(t)

V G (t) : momentary valence for the gamble

V S(t) : momentary valence for the sure option


Gain and Loss frame

E (S) = E (G )Gain frame

P(t

)

0

θS

θG

t

t1

System 1 System 2

Loss frame

P(t

)

0

θS

θG

t

t1

System 1 System 2

Attention switches from system 1 to system 2 at time t1

Switching time may be deterministic or random (according todistribution)

Solid lines indicate the drift rates


Predictions

Prediction 1: The size of the framing effect is a function of the timethe DM operates in System 1.

Prediction 2: The size of the framing effect is a function of the time(limit) the DM has for making a choice.


Prediction 1: Attention times

The size of the framing effect is a function of the time the DM operates inSystem 1.

0

θS

θG

t

t1

System 1 System 2

P(t

)

0

θS

θG

t

t1

System 1 System 2

P(t

)

Gain frameSystem 1: drift rate < 0→ SureSystem 2: drift rate = 0→ indifferent between Gamble and Sure


Prediction 2: Deadlines

The size of the framing effect is a function of the time (limit) the DM hasfor making a choice.

0

θS

θG

t

t1

System 1 System 2

P(t

)

P(t

)

0

θS

θG

t

t1

System 1 System 2

Gain frameSystem 1: drift rate < 0→ SureSystem 2: drift rate = 0→ indifferent between Gamble and Sure


In the following

WienerdV (t) = δidt + σidW (t)

with

drift rate δi , i = 1, 2, one for each system

diffusion coefficient σ2i = 1


Modeling the drift rates – Example 1

System 1Preferences in System 1 are constructed according to prospecttheory

System 2Preferences in System 2 are constructed according to expectedutility theory


System 1: Prospect Theory

The value V of a simple prospect that pays x (here the startingamount) with probability p (and nothing otherwise) is given by:

V(x , p) = w(p)v(x)

with probability weighting function

w(p) =pγ

(pγ + (1− p)γ)1/γ

and value function

v(x) =

{xα if x ≥ 0

−λ|x |β if x < 0


Weighting and value functions

Risk aversion in the positivedomainRisk seeking in the negativedomain


System 2 : Expected Utility Theory

For simplicity, we assume a risk-neutral DM.

The utility equals the outcome, i.e u(x) = x .

The expected utility equals the expected value (EV)

EU(x , p) = EV (p, x) = p · u(x) = p · x


Mean difference in valence (drift rate)

System 1 – gain frame

δ1g = VG − VSg

System 1 – loss frameδ1g = VG − VSl

System 2δ2 = EV (G )− EV (S)


For the predictions

Amount given: 25, 50, 75, 100Probability of keeping: 0.2, 0.4, 0.6, 0.8


Predictions – Example 1

Parameters (PT from Tversky & Kahneman, 1992)

System 1 System 2

α = .88 no parametersβ = .88 to be estimatedλ = 2.25 δ2 = 0γ = .61

boundary: θattention time: t, E (T1)


Predictions 1: Attention times

The size of the framing effect is a function of the time the DM operates inSystem 1. θ = 15; t1 = 0, 100, 500, ∞

Amount given25 50 75100 25 50 75100 25 50 75100 25 50 75100

Pr(

Gam

ble

)

0

0.1

0.3

0.5

0.7

0.9

1

Pr(keep)0.2 0.4 0.6 0.8

Loss frame

Gain frame


Predictions 1: Attention times

θ = 15; t1 = 0, 100, 500, ∞

Amount given25 50 75100 25 50 75100 25 50 75100 25 50 75100

Pr(

Ga

mb

le)

0

0.1

0.3

0.5

0.7

0.9

1

Pr(keep)0.2 0.4 0.6 0.8

0

50

100

150

200

250

300

Mean R

T G

am

ble

0.2 0.4 0.6 0.8

Pr(keep)

25 50 75100 25 50 75100 25 50 75100 25 50 75100

Amount given

0

50

100

150

200

250

300

Mean R

T S

ure


Predictions 2: Time limits

The size of the framing effect is a function of the time (limit) the DM hasfor making a choice. t1 = 100; θ = 10, 15, 20

25 50 75100 25 50 75100 25 50 75100 25 50 75100

Amount given

0

0.1

0.3

0.5

0.7

0.9

1

Pr(

Ga

mb

le)

0.2 0.4 0.6 0.8

Pr(keep)

Loss frame

Gain frame


Predictions 2: Time limits

t1 = 100; θ = 10, 15, 20

25 50 75100 25 50 75100 25 50 75100 25 50 75100

Amount given

0

0.1

0.3

0.5

0.7

0.9

1

Pr(

Gam

ble

)

0.2 0.4 0.6 0.8

Pr(keep)

0

100

200

300

400

500

Mean R

T G

am

ble

0.2 0.4 0.6 0.8

Pr(keep)

25 50 75100 25 50 75100 25 50 75100 25 50 75100

Amount given

0

100

200

300

400

500

Mean R

T S

ure



System 1Preferences in System 1 follow PT.

System 2Preferences in System 2 are a weighted average of PT and EU.


System 2: Weighted average of PT and EU

δ∗2 = w · (VG − VS) + (1− w) · (EV (G )− EV (S))

= w · δ1 + (1− w) · δ2.

Qualitative predictions remain as before.



System 1Preferences in System 1 are modeled according to a Motivationalfunction weighted by Willpower strength and Cognitive demand(MWC). (Loewenstein et al., 2015)

System 2Preferences in System 2 are modeled according to EU.


System 1: MWC

Motivational function M(x , a); a captures the intensity of affectivemotivations

Function h(W , σ) reflects the willpower strength W and cognitivedemands σ.


MWC

M(x , a) =∑

w(pi )v(xi , a)

w(p) is a probability-weighting function

w(p) = c + bp with w(0) = 1,w(1) = 1, and 0 < c < 1− b

v(x , a) is a value function that incorporates loss aversion

v(x , a) =

{a u(x) if x ≥ 0

aλ u(x) if x < 0

h(W , σ) is not specified but meant to be decreasing in W andincreasing in σ.

V(x) =∑

u(xi ) + h(W , σ) ·∑

w(pi )v(xi , a)


Note

WMC: static-deterministic model

For predicting choice probabilities and choice responses time

→ dynamic-stochastic framework


System 1 and System 2

MWC model assumes that both processes operate simultaneously.

Therefore, System 1 and System 2 merge into a single drift rate andthe two stages basically collapse into one single stochastic process.

With VG and VS indicating the subjective value of the gamble andthe sure option, respectively, the mean difference in valences (driftrates) in a gain and loss frame become

δg = VG − VSg

δl = VG − VSl ,


Predictions

100

Amount given

75

50

p = 0.2

25 .1 .5

h(W,σ)

1 1.5

0.9

0.7

0.5

0.3

0.1

Pr(

Ga

mb

le)

100

Amount given

75

50

p = 0.4

25 .1 .5

h(W,σ)

1 1.5

0.9

0.7

0.5

0.3

0.1

Pr(

Ga

mb

le)

100

Amount given

75

50

p = 0.6

25 .1 .5

h(W,σ)

1 1.5

0.9

0.7

0.5

0.3

0.1

Pr(

Ga

mb

le)

100

Amount given

75

50

p = 0.8

25 .1 .5

h(W,σ)

1 1.5

0.9

0.7

0.5

0.3

0.1

Pr(

Ga

mb

le)

100

Amount given

75

50

p = 0.2

25 .1 .5

h(W,σ)

1 1.5

0.1

0.3

0.5

0.7

0.9

Pr(

Ga

mb

le)

100

Amount given

75

50

p = 0.4

25 .1 .5

h(W,σ)

1 1.5

0.1

0.3

0.5

0.7

0.9P

r(G

am

ble

)

100

Amount given

75

50

p = 0.6

25 .1 .5

h(W,σ)

1 1.5

0.1

0.3

0.5

0.7

0.9

Pr(

Ga

mb

le)

100

Amount given

75

50

p = 0.8

25 .1 .5

h(W,σ)

1 1.5

0.9

0.7

0.5

0.3

0.1

Pr(

Ga

mb

le)

100

Amount given

75

50

p = 0.2

25 .1 .5

h(W,σ)

1 1.5

300

200

100

0

Me

an

RT

Ga

mb

le

100

Amount given

75

50

p = 0.4

25 .1 .5

h(W,σ)

1 1.5

300

200

100

0

Me

an

RT

Ga

mb

le

100

Amount given

75

50

p = 0.6

25 .1 .5

h(W,σ)

1 1.5

300

200

100

0

Me

an

RT

Ga

mb

le100

Amount given

75

50

p = 0.8

25 .1 .5

h(W,σ)

1 1.5

300

200

100

0

Me

an

RT

Ga

mb

le

100

Amount given

75

50

p = 0.2

25 .1 .5

h(W,σ)

1 1.5

0

100

200

300

Me

an

RT

Ga

mb

le

100

Amount given

75

50

p = 0.4

25 .1 .5

h(W,σ)

1 1.5

300

200

100

0

Me

an

RT

Ga

mb

le

100

Amount given

75

50

p = 0.6

25 .1 .5

h(W,σ)

1 1.5

300

200

100

0

Me

an

RT

Ga

mb

le

100

Amount given

75

50

p = 0.8

25 .1 .5

h(W,σ)

1 1.5

300

200

100

0

Me

an

RT

Ga

mb

le


Quantitative Model Evaluation

De Martino et al. (2006), Science

Guo, Trueblood, Diederich (2017), Psychological Science


De Martino et al. 2006 behavioral data

Gain frame Loss framepro

b(G

am

ble

)

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Prob(keep)0.2 0.4 0.6 0.8

pro

b(G

am

ble

)

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Prob(keep)0.2 0.4 0.6 0.8

pro

b(G

am

ble

)

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Initial amount25 50 75 100

pro

b(G

am

ble

)

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Initial amount25 50 75 100


Experiment

Guo et al. 2017

2 × (time limits: no, 1 sec) × 2 (frames: gain, loss)

72 gambles per condition, collapsed to 9 ”gambles” per condition

8 catch trials per condition

195 participants


Models tested

Number ofModel parameters RMSEA

PTk 8 22,030PT with additional scaling factor 9 2,106Dual with PT and EU 10 932Dual with PT and weighted PT and EU 11 931MWCk 10 20,786MWC with additional scaling factor 11 3,177MWC2stages 12 2,578


Model acccounts: Probabilities

Gainframe

PTDualMWCMWC2

Lossframe

no TP TPPr(keep)

0.27 0.42 0.57

Pr(

Gam

ble

)

0.25

0.5

0.75

Pr(keep) 0.27 0.42 0.57

Amount given32 56 79 32 56 79 32 56 79

Pr(

Gam

ble

)

0.25

0.5

0.75

Amount given32 56 79 32 56 79 32 56 79


Model acccounts: RT – no TP

Gainframe

PTDualMWCMWC2

Lossframe

0.27 0.42 0.57

1.5

2

2.5

Pr(keep)

mean R

T G

am

ble

[sec]

Pr(keep) 0.27 0.42 0.57

me

an

RT

Su

re [

se

c]

1.5

2

2.5

Amount given32 56 79 32 56 79 32 56 79

me

an

RT

Ga

mb

le [

se

c]

1.5

2

2.5

Amount given32 56 79 32 56 79 32 56 79

me

an

RT

Su

re [

se

c]

1.5

2

2.5


Model acccounts: RT – TP

Gainframe

PTDualMWCMWC2

Lossframe

Pr(keep) 0.27 0.42 0.57

me

an

RT

Ga

mb

le [

se

c]

0.4

0.5

0.6

Pr(keep) 0.27 0.42 0.57

me

an

RT

Su

re [

se

c]

0.4

0.5

0.6

Amount given32 56 79 32 56 79 32 56 79

me

an

RT

Ga

mb

le [

se

c]

0.4

0.5

0.6

Amount given32 56 79 32 56 79 32 56 79

me

an

RT

Su

re [

se

c]

0.4

0.5

0.6


More details

Diederich, A., Trueblood, J. (2018) A Dynamic Dual Process Modelof Risky Decision-making, Psychological Review, 125, 2, 270 – 292,doi.org/10.1037/rev0000087


Further directions

Risk attitudes

Gainframe

Lossframe

Risk averse Risk seeking

0

θS

θG

t

t1

System 1 System 2

P(t

)

0

θS

θG

t

t1

System 1 System 2

P(t

)0

θS

θG

t

t1

System 1 System 2

P(t

)

0

θS

θG

t

t1

System 1 System 2

P(t

)


Further directions

Baron & Gurcay (2017) for moral judgmentsRT = b0 + b1AD + b2U + b3AD · U


Further directions

0P(t

)

t

t1

t2

t3

1 > 0

2 < 0

1 > 0

2 > 0 Multi-stage decision model

Time schedule: deterministicor random

Order schedule:deterministic or random

Diederich & Oswald (2014) Frontiers in Human Neuroscience


Further directions/Remark

Quantum Decision Theory (QDT), Favre et al. 2016 PlosOne

No claim that neurological processes are quantum in nature

Observables A = {A1,A2} and B = {B1,B2}Aj gambles; Bj confidence; (j = 1, 2)

Two states: decision-maker; prospect



Decision-maker state |ψ〉 ∈ HAB:

|ψ〉 = α11|A1B1〉+ α12|A1B2〉+ α21|A2B1〉+ α22|A2B2〉

with αmn ≡ αmn(t) ∈ CSuperposition state reflects indecision until choice is made

Time evolution related to endogenous (breathing, digestions, feeling,thought) and exogenous (interaction with surrounding) factors



Prospect state |τj〉 ∈ HAB:

|τj〉 = |Aj〉 ⊗ {γjj1|B1〉+ γjj2|B2〉}

= γjj1|AjB1 + γjj2|AjB2〉,

γjkl = 0, ∀j , k 6= j , l ∈ {1, 2}, γjkl ∈ C, j , k, l ∈ {1, 2}State reflects DMs indefinite mixed feelings about the setup andcontext choosing either lottery



Probability measure for choosing lottery Lj

p(Lj) :=|〈ψ|τj〉|2

|〈ψ|τ1〉|2 + |〈ψ|τ2〉|2

Compare to Luce choice rule


Concluding remarks

Two-stage processes outperforms single processes.

Race (parallel processing): makes the notion of a faster System 1redundant (always the winner)

Mixture of two systems: makes the notion of two interactingprocesses operating together redundant.

QDT may be incorporated in framework.


Thank you

Supported by


a dynamic stochastic framework for dual process models€¦ · overview motivation and background...

Documents