Neural Correlates of Variations in Event Processing during Learning in

Basolateral Amygdala

Matthew R. Roesch*, Donna J. Calu, Guillem R. Esber, and Geoffrey Schoenbaum

* Department of Psychology and Program in Neuroscience and Cognitive Science, University of Maryland College Park

(2010)(2010)

Background…

• To optimize reward, animals must learn to associate cues with rewards and recognize the difference between the reward expected and that which actually occurs to guide their behaior

• The prediction error• 2 categories of learning models:

Category 1: “Signed error” Models

• If a reward is larger than expected(+), the association between the cue and reward will be strengthened, whereas if the reward is smaller than expected(-), the association will be weakened.

• …predict that the sign of the prediction error (i.e., whether the reward is bigger or smaller than expected) will be encoded in neural activity.

Category 1

• This correlate has been shown in midbrain dopamine neurons (Rescorla and Wagner, 1972; Sutton and Barto, 1998; Mirenowicz and Schultz, 1994; Montague et al., 1996; Schultz et al., 1997; Hollerman and Schultz, 1998; Waelti et al., 2001; Bayer and Glimcher, 2005; Pan et al., 2005; Bayer et al., 2007; D'Ardenne et al., 2008; Matsumoto and Hikosaka,

2009) . • Firing in these neurons increases in the face of

unexpected reward (+) and is suppressed when reward is unexpectedly omitted (-).

• Evidence also from other brain areas (Hong and Hikosaka, 2008; Matsumoto and Hikosaka, 2009) .

Category 2: “Unsigned error’ models

• Prediction errors tell an animal that it must learn more about the cue–reward association and therefore serve to drive attention.

• A cue should be more thoroughly processed (and learned about) when it is a poor predictor of reward. When the cue becomes a more reliable predictor, processing (and learning) should decline (Pearce–Hall model (1980, 1982)).

Category 2

• These models predict that neural activity encoding prediction errors will be similar regardless of the sign of the error(+/-).

• …lack of evidence for neural correlates of unsigned prediction errors—e.g., increased firing when reward is either better or worse than expected.

What did this paper do?• Basolateral Amygdalar (ABL) Neurons Encode

Unsigned Prediction Errors. • This neural signal increased immediately after

a change in reward, and stronger firing was

evident whether the value of the reward increased or decreased.

How did they do it• Recording single unit activity in a behavioral

task in which rewards were unexpectedly delivered or omitted.

• Basic paradigm is a choice task

Reward well Odor Port

Reward well

3 different odor cues: one signaled reward on the right (forced-choice), a second for left (forced-choice), and a third for either well (free-choice).

• Trials and Blocks

Each Block consists of at least 60 trials; In between blocks, rewarding value shifted (i.e. value of the port for rats changed)

-

-+

+

+

<

<>

>

Results• Performance and recording sites

• 70 reward-responsive ABL neurons recorded ;• 58/70 exhibited differential firing base on

timing (short/long delay) or size of the reward(large/small) after learning, signed coding theory; outcome-selective

• They also exhibited changes in reward-related firing between the beginning and end trials of each block, regardless reward upshift or downshift, unsigned coding theory

• 2 factor ANOVA analysis in each neuron across learning (early vs late) and shift type (upshift

vs downshift)• 10 of the 58 neurons (17%) fired significantly

more early in a block(after a change in

reward), than later(after learning).

• Indices [(early – late)/(early + late)], representing the difference in firing to reward delivery (within 1 s) during trials 3–10 (early) and during the last 10 trials (late) after shifts .

Main contribution of this paper

• The activity in the outcome-selective ABL neurons was higher at the start of a new training block, whether reward was better or worse than expected, and declined as the rats

learned to predict the value of reward. • This pattern of firing is generally consistent

with the notion of an “unsigned error” models such as that of Pearce and Hall (1980)

Another distinctive feature • Their firing did not

immediately increase at the start of a new block, in response to a change in reward, but rather appeared to gather momentum and peak a few trials into the block (3rd trial).

• Given the remarkable fit provided by the amended Pearce–Hall model (1982) and the role attributed to unsigned errors within this theoretical context, it seems natural to speculate that this ABL signal may be related to variations in event processing (title).

• Especially in view of the striking similarity between changes in the ABL signal and changes in the rats' latency to approach the odor port at the start of each trial.

Increase in speed of orienting to the odor port

Trial by trial analysis

The close relationship between the ABL signal and a behavioral measure “speed of orienting”

Explanation• Faster odor-port approach latencies may reflect

error-driven increases in the processing of trial events (e.g., cues and/or reward), because rats accelerate the reception of those events when shifted contingencies need to be worked out.

• In this sense, approaching the odor port faster

can be looked upon as similar to conditioned orienting. Conditioned orienting responses, also known as investigatory reflexes, means to recover from habituation when learned

contingencies are shifted.

• To further investigate the relationship between the ABL signal and odor-port

approach latency, ABL was inactivated in some rats during performance of the recording task.

• Inactivation of ABL disrupted the change in orienting. ns

• Inactivation of ABL also retarded learning in

response to changes in reward.

• Inactivation of ABL with DNQX

Block 1 well 1> well2, rats prefer well1

• Block 2, well 1< well 2, rats continue to approach well 1

Choice performance in vehicle versus NBQX sessions, plotted according to whether the well values in a particular trial block were similar to or opposite from those learned at the end of the prior session.

Conclusions

• Basolateral Amygdalar Neurons Encode Unsigned Prediction Errors ;

• This neural signal was correlated with faster orienting to predictive cues after changes in reward, and abolition of it disrupted this change in orienting and retarded learning in response to changes in reward.

• These results suggest that basolateral amygdala serves a critical function in attention for learning.