neuroeconomics { where science and economics meet

Neuroeconomics – where science and economicsmeet

Agnieszka Tymula

2017 European Economic Science Association Meeting

24,649 papers as of 21/08/2017

Outline

1. How does brain encode value?

2. Examples of neuroscience informing economics

neuroscience → economics

I divisive normalizationI choice set effectsI Prospect Theory-like behaviors

I gray matter volume

3. Neuroeconomics - useful resources

Outline

How does brain encode value?

I Cortex consists of neurons

I Neurons produce electrochemical impulses called actionpotentials

I Neurons convey information by the rate at which they produceaction potentials - firing rate (up to 200 Hz)

I Activity of neurons can be precisely recorded using implantedelectrodes and approximated using fMRI

Subjective value brain map

Bartra, McGuire & Kable, NeuroImage, 2013

Four very important facts about neurons:

1. Firing rate is biophysically restricted by a maximum rate ofaction potentials

2. We have limited number of neurons in the brain

3. Brain activity is stochastic

4. Neurons are expensive - brain takes 3% of body mass and20% of total calorie consumption

jec�

reward (x)

jec�

reward (x)xmin xmax

jec�

reward (x)xmin xmax

jec�

reward (x)xmin xmax

jec�

reward (x)xmin xmax

jec�

reward (x)xmin xmax

jec�

reward (x)xmin xmax

x1 x2x x

jec�

reward (x)xmin xmax

x1 x2x x

jec�

reward (x)xmin xmax

Neuroeconomic models ofdecision-making

Three dominant models of choice in neuroeconomics:

1. Divisive normalizationI Glimcher, Heeger

2. Stochastic network modelsI Rustichini, Padoa-Schioppa

3. Drift-diffusion modelI Rangel, Shadlen

I Divisive normalization originates from enormous amount ofresearch on sensory systems

I Perception of sensory stimulus depends not only on itsobjective features but also on:

I other currently presented stimuli (spatial normalization /choice-set effects)

I previously experienced stimuli (temporal normalization / riskattitudes)

Spatial normalization

Ri (x1, x2, ..., xn) = rmaxxαi

σα+∑

j 6=i Xαj

+ εCarandini & Heeger, Nature Reviews, 2012

This general form occurs from retina to cortex and mediatescontrast adaptation, surround suppression, visual attention,multisensory integration, ...

σα+∑

j 6=i Xαj

σα+∑

j 6=i Xαj

σα+∑

j 6=i Xαj

σα+∑

j 6=i Xαj

σα+∑

j 6=i Xαj

Spatial normalization in valuation

Normalization outperforms other models in predicting neuronalfiring rates (Louie et al., 2011)

It leads to specific choice and brain activity patterns - now testedin monkeys and humans (Louie et al., 2013, 2015, Webb et al.,2012, 2014)

u1u2u3

u3 u4u5

It is optimal maximization tool under costly value representation,equivalent to reducing Shannon entropy (Steverson et al. 2016)

References: Kenway Louie at NYU

Temporal normalization

Stimulus history has strong effect in sensory processing

contrast level

Ohzawa, Scalar & Freeman, Journal of neurophysiology, 1985

R(x) = rmaxxα

xα+t−1∑i=1

γ(1−γ)t−ixi

LoFaro et al., Letters in Biomathematics, 2014

Louie et al., Journal of Neuroscience, 2014

contrast level

Ohzawa, Scalar & Freeman, Journal of neurophysiology, 1985

R(x) = rmaxxα

xα+t−1∑i=1

γ(1−γ)t−ixi

LoFaro et al., Letters in Biomathematics, 2014

Louie et al., Journal of Neuroscience, 2014

Temporal normalization of value

Hypothesis: After high (low) adapt block, bids in test block will belower (higher). This effect will gradually wear off.

Khaw, Glimcher, Louie, under revision

Hypothesis: After high (low) adapt block, bids in test block will belower (higher). This effect will gradually wear off.

Expected Subjective Value Theory: A Representation ofDecision in Time under Risk and Certainty

by Paul Glimcher and Agnieszka Tymula

Theoretically explore the implications of divisive normalizationalgorithm for:

I risk attitudes,

I loss aversion, and

I probability weighting

The subjective value (V ) of a prize x ∈ IR+ at time t is given by:

V (x) =xα

xα + Mαt

α - predisposition (free parameter)Mt - reference point / reward expectation

Mt =t−1∑i=1

γ(1− γ)t−ixi

γ ∈ (0, 1) sets the forgetting rate

The expected subjective value of a lottery (x , p) with outcomesx1, x2, ..., xn occurring with corresponding probabilitiesp1, p2, ..., pn:

EV (x , p) =n∑

piV (xi )

The subjective value (V ) of a prize x ∈ IR+ at time t is given by:

V (x) =xα

xα + Mαt

α - predisposition (free parameter)Mt - reference point / reward expectation

Mt =t−1∑i=1

γ(1− γ)t−ixi

γ ∈ (0, 1) sets the forgetting rate

The expected subjective value of a lottery (x , p) with outcomesx1, x2, ..., xn occurring with corresponding probabilitiesp1, p2, ..., pn:

EV (x , p) =n∑

piV (xi )

Examples of subjective value functions: predisposition

α=1, M=5

α=2, M=5

α=5, M=5

α=10, M=5

V ∈ [0, 1]

Examples of subjective value functions: reference point

3 5 7x

α=5, M=5

α=5, M=3

α=5, M=7

For all Mt ,V (Mt) = 12

Risk taking

Arrow-Pratt index of relative risk aversion (RRA):

RRAt = −xV ′′

V ′=

(1 + α)xα − (α− 1)Mαt

xα + Mαt

Proposition 1 Individual is risk averse if

x > α

√α− 1

α + 1Mt

I predisposition (α), in addition to the reference (Mt),determines risk attitude

Arrow-Pratt index of relative risk aversion (RRA):

RRAt = −xV ′′

V ′=

(1 + α)xα − (α− 1)Mαt

xα + Mαt

Proposition 1 Individual is risk averse if

x > α

√α− 1

α + 1Mt

I predisposition (α), in addition to the reference (Mt),determines risk attitude

Predisposition and risk attitude

x > α

√α− 1

α + 1Mt

α ≤ 1

α ∈ (1,∞) α =∞

risk averse for x > 0

x > α

√α−1α+1Mt risk averse for x > Mt

0 2 4 6 8 10x

Concave u�lity for

(Bernoulli)

0 2 4 6 8 10x

Prospect Theory

x > α

√α− 1

α + 1Mt

α ≤ 1

α ∈ (1,∞)

α =∞risk averse for x > 0

x > α

√α−1α+1Mt

risk averse for x > Mt

0 2 4 6 8 10x

(Bernoulli)

0 2 4 6 8 10x

Prospect Theory

x > α

√α− 1

α + 1Mt

α ≤ 1 α ∈ (1,∞) α =∞risk averse for x > 0 x > α

√α−1α+1Mt risk averse for x > Mt

0 2 4 6 8 10x

(Bernoulli)

0 2 4 6 8 10x

Prospect Theory

Reference point and risk attitudeProposition 4 For any given gamble, individual is more risktolerant the larger his reference point is.

Proof: ∂RRAt

∂Mt< 0

3 5 7x

α=5, M=5

α=5, M=3

α=5, M=7

A kind of “wealth effect”, but not exactly

Reference point and risk attitude

Risk attitudes depend not only on the total rewards earned butalso their timing and value

total t=1 t=2 t= 3

increasing income 30 5 10 15 more risk tolerantdecreasing income 30 15 10 5 more risk averse

Proposition 5 Keeping the total past income and predispositionconstant, increasing income streams lead to more risk tolerancethan decreasing income streams.

Reference point and risk attitude

Risk attitudes depend not only on the total rewards earned butalso their timing and value

total t=1 t=2 t= 3

increasing income 30 5 10 15 more risk tolerantdecreasing income 30 15 10 5 more risk averse

Proposition 5 Keeping the total past income and predispositionconstant, increasing income streams lead to more risk tolerancethan decreasing income streams.

Reflection effects

People on aggregate tend to be risk averse in gains and riskseeking in losses (Kahneman and Tversky, 1979)

But not on individual level (Baucells and Villasis, 2010; Cohen etal., 1987; Schoemaker, 1990; Tymula et al., 2013)!

Risk attitude − losses

F. data from Tymula et al. (PNAS, 2013)

seeking averse

Tymula et al., PNAS, 2013

correlation = 0.254, p = 0.003; 79/128 in PT region

Reflection effects

People on aggregate tend to be risk averse in gains and riskseeking in losses (Kahneman and Tversky, 1979)

But not on individual level (Baucells and Villasis, 2010; Cohen etal., 1987; Schoemaker, 1990; Tymula et al., 2013)!

F. data from Tymula et al. (PNAS, 2013)

Risk attitude − losses

seeking averse

Tymula et al., PNAS, 2013

correlation = 0.254, p = 0.003; 79/128 in PT region

Reflection effects: model simulation

−10 −5 0 5 10

RRA − losses

averseseeking

averse

seeking

α = 5M = 10

xl = 5 lossxg = 15 gain

−10 −5 0 5 10

RRA − losses

−10 −5 0 5 10

RRA − losses

A. alpha~U(0,10), M~U(0,20) B. alpha~N(5,2), M~N(10,2)

−10 −5 0 5 10

RRA − losses

A. alpha~U(0,10), M~U(0,20)

high M and

su!ciently high α

low α and/or

correlation = 0.287, p < 0.001 - opposite of reflection effect

Loss aversion

Common approach to loss aversion

The most commonly used utility specification:

U(x) =

{u(x) if x ≥ 0

λu(x) if x < 0

where λ > 1 - loss aversion

But the kink in the utility function is not necessary to produce lossaverse behavior (e.g. Blavatsky and Pogrebna, 2009; Ert and Erev,2013)

xxk kM

Common approach to loss aversion

The most commonly used utility specification:

U(x) =

{u(x) if x ≥ 0

λu(x) if x < 0

where λ > 1 - loss aversion

But the kink in the utility function is not necessary to produce lossaverse behavior (e.g. Blavatsky and Pogrebna, 2009; Ert and Erev,2013)

xxk kM

ESVT and loss aversion

We use traditional definition of loss aversion:

λt =V (Mt)− V (Mt − k)

V (Mt + k)− V (Mt)

k ∈ IR+ - distance from the reference point Mt

λt =(Mα

t − (Mt − k)α)(Mαt + (Mt + k)α)

((Mt − k)α + Mαt )((Mt + k)α −Mα

Individual is loss averse if λt > 1

Proposition 7 An individual with preferences represented by valuefunction V (x) is always loss averse, though not necessarily to asignificant degree.

V (Mt + k)− V (Mt)

λt =(Mα

t − (Mt − k)α)(Mαt + (Mt + k)α)

V (Mt + k)− V (Mt)

λt =(Mα

t − (Mt − k)α)(Mαt + (Mt + k)α)

0 2 4 6 8 10k

Proposition 8 For any stake, an individual with lowerpredisposition is more loss averse. Loss aversion first increases andthen decreases with stake size. Loss aversion peaks at lower stakesfor individuals with higher predisposition.

0 2 4 6 8 10k

Proposition 8 For any stake, an individual with lowerpredisposition is more loss averse. Loss aversion first increases andthen decreases with stake size. Loss aversion peaks at lower stakesfor individuals with higher predisposition.

Endowment effect: literature

People endowed with a mug demand more money to sell it thanthose who do not have a mug are willing to pay for it

Traditional explanation: loss aversion (Thaler, 1980). Impliesendowment effect should always occur

This explanation has been the subject of a heated debate(Engelmann and Hollard, 2010; Isoni et al., 2011; List, 2002 &2003; Plott and Zeiler, 2005, 2007 & 2011). Reverse endowmenteffect has been documented in Lerner et al. (2004), Lin et al.(2006), Martinez et al. (2011)

Sense of ownership emerges as the key determinant of theendowment effect (review by Ericson & Fuster (2014))

Endowment effect in ESVTNotation:

before endowment after endowment

ownership x0 x1reference M0 M1

Proposition 9 The endowment effect occurs if(Mα

0 −Mα1 )(Mα

0 Mα1 − xα0 x

α1 ) > 0.

Endowment effect does not occur if:

(Mα0 −Mα

1 )(Mα0 M

α1 − xα0 x

α1 ) = 0

I there is no change in reference point after endowment:M0 = M1

I when the reference point is equal to the current ownershiplevel: M0 = x0 and M1 = x1

0 −Mα1 )(Mα

0 Mα1 − xα0 x

α1 ) > 0.

(Mα0 −Mα

1 )(Mα0 M

α1 − xα0 x

α1 ) = 0

0 −Mα1 )(Mα

0 Mα1 − xα0 x

α1 ) > 0.

(Mα0 −Mα

1 )(Mα0 M

α1 − xα0 x

α1 ) = 0

0 −Mα1 )(Mα

0 Mα1 − xα0 x

α1 ) > 0.

(Mα0 −Mα

1 )(Mα0 M

α1 − xα0 x

α1 ) = 0

0 −Mα1 )(Mα

0 Mα1 − xα0 x

α1 ) > 0.

(Mα0 −Mα

1 )(Mα0 M

α1 − xα0 x

α1 ) = 0

No endowment effect in ESVT

There is no endowment effect if M0 = x0 and M1 = x1

1V(mugs)

Endowment effect in ESVT

Assume M0 = x0. Endowment effect occurs if M1 < x1

1V(mugs)

Reverse endowment effect in ESVT

Let M0 < M1. RRE if M0 > x0 and M1 > x1 (e.g. after a loss)

1V(mugs)

Diminishing WTP and WTA: divisive versus difference

divisive model - ESVT

1V(mugs)

2 3 4 5mugs

difference-based model

U(mugs)

Diminishing WTP and WTA: divisive versus difference

divisive model - ESVT

1V(mugs)

2 3 4 5mugs

difference-based model

2 3 4U(mugs)

Probability weighting

Probability weighting - theory versus practice

Theory

Practice

Tobler et al., Journal of Neuroscience (2008)

There is a lot of unexplained heterogeneity in the estimatedprobability weighting functions on the individual level

Theory Practice

Probability weighting - neuro evidence

Essentially all neuroeconomic studies agree that neurons arecapable of encoding probability without any distortions

I Abler et al. 2006; Berns et al. 2008; Preuschoff, Bossaerts,and Quartz 2006; Tobler et al. 2008

There seem to be brain regions that distort probabilities, but either

I distortions have no influence on choice (Tobler et al. 2008)

I cannot reject the null of linear coding in the brain overall(Berns et al. 2008)

I estimates may be biased by the functional assumptions (Hsuet al. 2008)

Probability weighting - neuro evidence

Essentially all neuroeconomic studies agree that neurons arecapable of encoding probability without any distortions

I Abler et al. 2006; Berns et al. 2008; Preuschoff, Bossaerts,and Quartz 2006; Tobler et al. 2008

There seem to be brain regions that distort probabilities, but either

I distortions have no influence on choice (Tobler et al. 2008)

I cannot reject the null of linear coding in the brain overall(Berns et al. 2008)

I estimates may be biased by the functional assumptions (Hsuet al. 2008)

Probability weighting an artefact of normalization?

1. We simulated choices for 9 individuals (3 reference points x 3predispositions) with ESVT-preferences in the Gonzalez andWu (1999) lottery task

EV (x , p) =n∑

piV (xi )

w(p) = p

2. We fitted these choices with

U(x) = x r

w(p) =δpγ

δpγ + (1− p)γ

1. We simulated choices for 9 individuals (3 reference points x 3predispositions) with ESVT-preferences in the Gonzalez andWu (1999) lottery task

EV (x , p) =n∑

piV (xi )

w(p) = p

2. We fitted these choices with

U(x) = x r

w(p) =δpγ

δpγ + (1− p)γ

low reference point

medium reference point high reference point

α=1α=2α=5

Summary of ESVT

I Our model, based on established neuronal computationdivisive normalization, captures many of the observedbehavioral phenomena with only two parameters

I It can capture:I reflection effects in risk attitudesI loss aversionI probability weightingI choice set effects

I Additionally, it captures heterogeneity in all of the above dueto individual differences in reference points and predispositions

I Lack of discontinuity in the value function allows for easyestimation of the reference point

Summary of ESVT

Neuroeconomic variables - graymatter volume

Study set up

I n=61 young adults (half exploration / half replication)

Tymula∗, Gilaie-Dotan∗,Cooper, Kable, Glimcher & Levy. J of Neuroscience, 2014

Gray matter volume estimation

I 10 minute structural scans

I Siemens Allegra 3T head-only scanner

Study 1 Result

Gray matter volume and risk attitudes in ageing

I Gray matter volume decreases with age

Salat et al., 2012

I Risk aversion increases with age (von Gaudecker et al. 2011,Dohmen et al., 2011, Tymula et al., 2013, ...)

Study 2

I n=52 (30 female)I 18-88 years old, mean: 54.7, s.d.:22.1

Grubb, Tymula, Gilaie-Dotan, Glimcher & Levy, Nature Communications, 2016

Study 2 Result

Grubb, Tymula, Gilaie-Dotan, Glimcher & Levy, Nature Communications, 2016

Rationality and gray matter volume

I 39 subjects (14 male)I 65-92 years old (mean: 72.46)I no signs of dementia (tested on mini-mental)

Chung, Tymula, Glimcher, r&r

Rationality measurement

I Harbaugh, Krause, Berry, AER 2002

good A

u(a)>u(d) choice

u(d)>u(b) monotonicity

u(b)>u(c) choice

u(c)>u(a) monotonicity

good A

u(a)>u(d) choice

u(b)>u(c) choice

good A

u(a)>u(d) choice

u(b)>u(c) choice

good A

I Rationality is positively correlated with grey matter volumes inleft anterior prefrontal cortex (a47r)

Chung, Tymula, Glimcher, r&r

Why is this interesting?

I TheoryI Constrains possible neural mechanisms underlying risk

attitudes and rationality

I Provides biological evidence for incorporating neural capacityin models of choice

I Simon QJE 1955, Rayo & Becker JPE 2007, Netzer AER2009, Woodford AER 2012, Robson & Whitehead, 2016

I Opportunity to unite numerous previously documentedassociations between host of brain-changing variables and riskpreferences and rationality into one coherent theory

I EmpiricalI New datasets - simple measurement of risk attitudes in

existing brain scans

I PolicyI New interventions

attitudes and rationalityI Provides biological evidence for incorporating neural capacity

in models of choiceI Simon QJE 1955, Rayo & Becker JPE 2007, Netzer AER

2009, Woodford AER 2012, Robson & Whitehead, 2016

Perspective

I Specific population of participants - we need to study differentsamples

I Dementia sample at U of Sydney

I Findings do not imply a fixed deterministic relation betweengenetics and risk attitude

I Environment, own behavior may shape brain structure

I We cannot infer causality from this data

“The task is to replace the global rationality of economic man witha kind of rational behavior that is compatible with the access toinformation and the computational capacities that are actually

possessed by organisms.”

Herbert Simon, 1955, p. 241

Neuroeconomics - usefulresources

Neuroeconomics - useful resources

Join the Society for Neuroeconomics - https://neuroeconomics.org

and Society for Neuroeconomics Annual Meetings (since 2005)

Neuroeconomics - useful resources

Do a postdoc - https://neuroeconomics.org/career/

Join us for the summer school - next edition in 2019

Collaborate (to learn and avoid embarrassment)

Be respectful and ask questions

Thank you!

Coauthors:

Paul Glimcher, Ifat Levy, Hui-Kuan Chung, Joe Kable, Sharon Gilaie-Dotan,

Michael Grubb, Natalie Cooper

Funding bodies:

How is reference point, Mt , computed in the brain?

I Dopaminergic reward prediction error system computes arecency-weighted average of past rewards

I Typically, it is modeled using discrete time, where at each stept, the brain computes future reward expectation, Pt , as:

Pt = Pt−1 + γ(xt − Pt−1)

= (1− γ)tP0 +t∑

γ(1− γ)t−ixi

I Assuming that initial expectation P0 = 0 (Sutton and Barto,1998):

Mt = Pt−1 =t−1∑i=1

γ(1− γ)t−ixi

Back to model

neuroeconomics { where science and economics meet

Documents