The fussy brain:

What makes one option more attractive than another?

Steve Fleming and Louise Whiteley

Value

Some decisions are about information gathering, where what matters is being accurate. Many everyday decisions are about what is valuable to us now, and in the future…

OR ?

Predicting the future

What is a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Bentham and probability

Jeremy Bentham believed that using “felicific calculus” it was possible to work out the best action to take

“Nature has placed mankind under the guidance of two sovereign masters; pain and pleasure. It is for them alone to point out what we ought to do, as well as to determine what we shall do”

Jeremy Bentham, 1748-1832

Darling’s investment – predicting the future

The value of the share can rise or fall…

Down5%

Up 10%p = 0.2

p = 0.8

Darling’s investment – predicting the future

Down5%

Up 10%p = 0.2

p = 0.8

Expected value of share = weight each outcome by its probability, then add them all up

Darling’s investment – predicting the future

Down5%

Up 10%p = 0.2

p = 0.8

Expected value of share = weight each outcome by its probability, then add them all up

EV(share) = outcomes p(outcome) x r(outcome)

= (0.2 x 10) + (0.8 x -5) = -2

Darling’s investment – discounting the future

Down20%

Up 25%p = 0.2

p = 0.8Time

Value

Down20%

Up 15%p = 0.2

p = 0.8Time

Value

Share 1

Share 2

In six months…

In six weeks…

OR

Darling’s investment – discounting the future

EV(share) = outcomes p(outcome) x r(outcome)

EV(share) = outcomes λ x p(outcome) x r(outcome)

Time

Value

Measuring impulsivity…

Time

Value

How are these values learnt?

What is a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

+ Learning

Investigating value in the brain

1.Find neurons that signal our preferences

2.Work out how these neurons learn from experience to predict future values

3.See how these neurons are affected by probability

4.See how these neurons are affected by when you get the reward

1. Find neurons that signal our preferences

OFC – Orbitofrontal cortex

MT

LIP

OFC

Neurons representing value of choice…

vs.

We want to know if OFC neurons can keep track of different preferences

V(pineapple) V(orange)

orangepineapple

during instruction

just before reward

Preferences in the OFC

• Different groups of neurons within OFC are associated with different types of reward (e.g. orange vs. pineapple)

• OFC neurons also know how much reward is on offer - e.g. six apples vs. one piece of cake

Padoa-Schioppa & Assad (2006)

2. Work out how neurons predict future values

• Learn from the past!

New value = prediction + new information

= difference between prediction and what happened…

So:

New value = prediction + α(outcome – prediction)

How does the brain predict future values?

(outcome – prediction)

Schultz et al. (1997) Science

Reward unpredicted, reward occurs

Reward predicted, reward occurs

Reward predicted, reward absent

Changing our predictions with new information

Time

Read label

It’s corked

Taste…

New information in the brain…

Seymour et al. (2004)

Basal ganglia

3. See how these neurons are affected by probability

p = 0.8 p = 0.2

LIP

Platt & Glimcher (1997)

How does the brain respond to probability?

Knutson et al. (2005)

OFC Basal ganglia

EV = outcomes p(outcome) x r(outcome)

How does the brain respond to probability?

Would you like a) £900 now or b) £1000 in one month’s time?

4. See how these neurons are affected by when you get a reward

Short- and long-term gain in the brain

Kable & Glimcher (2007)

OFC

Brain data help us refine our theory

OFC

Kable & Glimcher (2007)

• Two theories: –a) brain region knows about “absolute” value, communicates it to somewhere else which knows about how far away it is in time–b) discounting the future is inherent to our value system

What is involved in making a decision?

What happens when things get more complicated…?

DecisionShort- vsLong-term

gain

Context

Risk

Information gathering

Prior Beliefs

Value

Many decision systems in parallel

We’ve been focusing on how the brain learns values from experience, building up habits that can be used again

Many decision systems in parallel

Sometimes, we can’t learn habits, and need to look ahead in a more sophisticated way…

Complicated or one-off decisions…

Many decision systems in parallel

And sometimes we don’t need to bother - we have innate values attached to things like food and shelter

Bentham again…

“the game of push pin is of equal value with poetry”

vs. J.S. Mill…

“it is better to be … Socrates dissatisfied than a fool satisfied”

Complicated value

Many decision systems in parallel

In the next talk we hear more about these three systems, about how the brain chooses which system to use, and how

this can lead us astray…

Any questions...?

What is a decision?

DecisionShort- vs

Long-termgain

Context

Risk

Information gathering

Prior Beliefs

Value

Neurons representing value of choice…

Padoa-Schioppa & Assad (2006)