dorsal striatum responses to reward and punishment...

Cognitive Affective amp Behavioral Neuroscience2003 3 (1) 27-38

Rewards play an important role in motivated behaviorThe hedonic properties of a potential reward can lead toapproach behavior and a sense of pleasure after con-sumption serving to reinforce such behavior (Schultz2000) Thus it is important to consider how the humanbrain perceives different properties of a salient stimulussuch as valence and magnitude The striatum is one ofseveral regions implicated by previous research in theprocessing of reward-related information Activity in thestriatum has an important role in detecting the presenceof an affective stimulus (Apicella Ljungberg Scarnatiamp Schultz 1991 Hikosaka Sakamoto amp Usui 1989Hollerman Tremblay amp Schultz 2000) its predictability(Berns McClure Pagnoni amp Montague 2001 PagnoniZink Montague amp Berns 2002 Schultz Tremblay ampHollerman 1998) and the valencemdashreward or punish-mentmdashassociated with such a stimulus (Breiter AharonKahneman Dale amp Shizgal 2001 Delgado NystromFissell Noll amp Fiez 2000) The striatum thereforeseems to be in a position to influence and guide behaviorby coding the affective properties of a stimulus To exert

such a function the striatum should also have the capac-ity to rank feedback according to magnitude or preferenceAccordingly when presented with feedback the striatumshould parametrically order its response in accordancewith the valence and magnitude of the feedback

As was previously discussed a variety of data stronglysupports the involvement of both the dorsal and the ven-tral striatum in a circuit responsible for detecting a reward-related stimuli and further detecting its valence Less isknown however about how the striatum responds to otherstimulus properties such as magnitude The literaturehas shown that magnitude manipulations affect activityin other areas implicated in a potential reward circuitrymdashmostly the prefrontal cortex and the amygdala For ex-ample a correlation exists between the speed at which ratsrun down a runway to retrieve a reward and the magni-tude of such a reward (Crespi 1942) After amygdala le-sions rats become insensitive to reductions in the rewardamount (Salinas Packard amp McGaugh 1993) In mon-keys enhancement of activity was observed in the dor-solateral prefrontal cortex during the memory phase of avisual-memory task for trials associated with a large ver-sus a small reward (Leon amp Shadlen 1999) The orbitalfrontal cortex an area heavily implicated in emotion(Bechara Damasio amp Damasio 2000 Grant Con-toreggi amp London 2000 Rolls 1999 2000) is also af-fected by changes in magnitude In humans for exam-ple the orbital frontal cortex was active in a paradigm inwhich participants chose between small likely rewardsand large unlikely rewards (Rogers et al 1999) and in areversal learning paradigm in which different magni-

27 Copyright 2003 Psychonomic Society Inc

This work was supported by NIH Grant ROIDA14103 NSF GrantLIS 9720350 and an NSF Graduate Research Fellowship The authorsthank Leigh Nystrom and Kate Fissell for technical assistance andCameron Carter Anthony Grace Carl Olson and Peter Strick for in-formative discussion and constructive criticism The authors also thankClay Holroyd and an anonymous reviewer for their helpful commentson an earlier draft of the manuscript Correspondence concerning thisarticle should be addressed to M R Delgado Department of Psychol-ogy New York University Meyer Hall Room 873 6 Washington PlaceNew York NY 10003 (e-mail mdelgadonyuedu)

Dorsal striatum responses to reward andpunishment Effects of valence and magnitude

manipulations

M R DELGADO H M LOCKE V A STENGER and J A FIEZUniversity of Pittsburgh Pittsburgh Pennsylvania

and Center for the Neural Basis of Cognition Pittsburgh Pennsylvania

The goal of this research was to further our understanding of how the striatum responds to the de-livery of affective feedback Previously we had found that the striatum showed a pattern of sustainedactivation after presentation of a monetary reward in contrast to a decrease in the hemodynamic responseafter a punishment In this study we tested whether the activity of the striatum could be modulated byparametric variations in the amount of financial reward or punishment We used an event-related fMRIdesign in which participants received large or small monetary rewards or punishments after perfor-mance in a gambling task A parametric ordering of conditions was observed in the dorsal striatum ac-cording to both magnitude and valence In addition an early response to the presentation of feedbackwas observed and replicated in a second experiment with increased temporal resolution This study fur-ther implicates the dorsal striatum as an integral component of a reward circuitry responsible for thecontrol of motivated behavior serving to code for such feedback properties as valence and magnitude

28 DELGADO LOCKE STENGER AND FIEZ

tudes of reinforcement were given (OrsquoDoherty Kringel-bach Rolls Hornak amp Andrews 2001)

One area that receives projections from both the frontalcortex (Haber Kunishio Mizobuchi amp Lynd-Balta1995 Middleton amp Strick 2000) and the amygdala(Groenewegen Wright Beijer amp Voorn 1999) is thestriatum where magnitude effects and even investiga-tions of magnitude manipulations have been less fre-quently reported Recently Hassani Cromwell andSchultz (2001) showed that monkey striatal neurons firemore vigorously for preferred rewards suggesting thatsome ranking based on preference may occur in the stria-tum In a clever paradigm Breiter et al (2001) foundhigher ventral striatum activity associated with higheroutcomes in a ldquowheel of fortunerdquo like task A more directmeasurement of the effects of magnitude in the striatumwas done by Knutson Adams Fong and Hommer (2001)who scanned participants while they anticipated rewardsand punishments that varied in amounts The studyshowed that ventral striatum activity was associated withanticipation of larger rewards whereas the dorsal stria-tum was activated while both rewards and punishmentsof larger magnitude were anticipated Although in thisstudy magnitude changes in the striatum were looked atthe primary focus was on the effects of magnitude in theanticipatory phase Thus more research is needed tofully understand how the striatum responds to the actualdelivery of rewards and punishments of different magni-tudes

In a previous study we developed a gambling task(card-guessing paradigm) in which participants wereasked to guess the value of a card (Delgado Nystromet al 2000) A correct guess yielded a monetary rewardwhereas an incorrect guess led to a monetary punish-ment Using an event-related design we observed dif-ferential responses to reward and punishment feedbackin the striatum where the hemodynamic response for re-wards was significantly higher than that for punishmentsin the 6- to 9-sec time window after feedback presenta-tion In this paper we will further examine the previ-ously observed dissociation of valence in the striatum bystudying how the response is affected by the magnitudeof the outcome In our first experiment we used a mod-ified version of the card-guessing paradigm to measurestriatal activity following the delivery of monetary re-wards and punishments that varied parametrically Theoutcomes were either a large ($400) or small ($040)monetary reward or a large ($200) or small ($020)monetary loss The primary goal was to replicate the dif-ferences in activation according to valence and to go be-yond prior studies by varying the magnitude of the out-comemdashthus providing further evidence that the striatuminfluences or guides motivated behavior by processingthe motivational properties of a stimulus

In our previous study (Delgado Nystrom et al 2000)we also observed a differential response to punishmentsand rewards that evolved within 3 sec of feedback pre-sentation The rapidity of this response caused us to

question its reliability Although it is likely that the pre-sentation of a surprising and affective stimulus (eg abear appearing from nowhere) causes fast immediatecognitive and physiological responses our understand-ing of the properties of the hemodynamic response lim-its any possible interpretation Hence a secondary goalof this research was to determine whether an early re-sponse (of less than 3 sec) to a feedback could be repli-cated and further characterized temporally This was ac-complished by examining the activation detected within3 sec of feedback presentation in the first experimentand by running a second experiment with increased tem-poral sampling

EXPERIMENT 1

MethodParticipants Twenty right-handed volunteers participated in

this study (11 females 9 males) The participants were mostly grad-uate and undergraduate students drawn from the University of Pitts-burgh (average age = 229 years SD = 326) Two participants wereremoved from all analysis because of excessive motion during theirscan sessions The participants were asked to fill out a brief ques-tionnaire to ensure that they had prior experience with gambling butwere not abusive or excessive in such behavior (ie have youplayed cards for money not at all less than once a week or once aweek or more) The questionnaire was based on the South OaksGambling Screen (Lesieur amp Blume 1987) Information about anyfamily history of gambling was not acquired All the participantsgave informed consent according to the policies of the InstitutionalReview Board at the University of Pittsburgh

Cognitive task The paradigm involved a series of 144 inter-leaved trials divided into nine runs of 16 trials each Each triallasted 15 sec and began with the presentation of a visually displayedcard projected onto a screen The card had an unknown value rang-ing from 1 to 9 and the participant was instructed to make a guessabout the value of the card A question mark appeared in the centerof the card indicating that the participant had 25 sec to guesswhether the card value was higher or lower than the number 5 Theparticipant pressed the left or the right button of a response unit toindicate his or her selection After the choice-making period anumber appeared in the center of the card for 500 msec followedby an arrow that was also displayed for another 500 msec The ap-pearance of a green arrow pointing upward indicated that the par-ticipant had correctly guessed the card value A large green arrowcorresponded to a large reward of $400 A smaller green arrow in-dicated a small reward of $040 (Figure 1A) The appearance of ared arrow pointing downward indicated that the participant had in-correctly guessed the card value leading to a penalty of $200 if thearrow was large and a $020 penalty if the arrow was small Unlikeour previous design (Delgado Nystrom et al 2000) there was noneutral condition and the number 5 never appeared as the value ofa card Therefore there were four types of trials (large and small re-ward and large and small punishment) The same gain-to-loss ratioas that from our previous design was kept (21) in accordance withclassic decision-making literature suggesting that the impact ofnegative outcomes such as losses is larger than that of positive out-comes such as gains (Kahneman amp Tversky 1979 Tversky amp Kah-neman 1981) For trials in which a response was not made on timethe feedback was a pound sign () and such trials were excludedfrom all analyses After the 35-sec period between presentation ofthe response cue (question mark) and the rewardpunishment out-come there was an 115-sec delay before the onset of the next trialAn experimental session therefore consisted of 144 trials of 15 sec

VALENCE AND MAGNITUDE EFFECTS IN THE STRIATUM 29

each (Figure 1B) Stimulus presentation and behavioral data acqui-sition were controlled by a Macintosh computer with PsyScopesoftware (Macwhinney Cohen amp Provost 1997)

Unbeknownst to the participant the outcome of each trial waspredetermined to be of a specific valence and magnitude Card val-ues were selected only after the participant had indicated his or herguess on each trial Each participant performed 36 trials of eachcondition The nine runs were broken down into three groupsGroup A had more reward trials during the beginning of the task

whereas Group B had more punishment trials and Group C wasevenly matched This allowed for counterbalancing of runs acrossparticipants to help minimize effects of scanner drift The partici-pants were told they could keep the final sum of monetary out-comes at the end of the experiment although they were not giveninformation regarding cumulative earnings throughout the session

Data acquisition and analysis A conventional 15-T GE Signawhole-body scanner and standard RF coil were used to obtain 20contiguous slices (375 3 375 3 38 mm voxels) parallel to the

Figure 1 (A) Description of task and events in card-guessing paradigm large reward small re-ward large punishment and small punishment The sequence of events is depicted in the timelinewhere cue (presentation of a question mark) is the first event followed by the choice-making periodduring which the participants were asked to guess whether the value of the presented card washigher or lower than 5 After a choice was made the value of the card was revealed (outcome) andthis was followed by reward or punishment feedback that varied according to magnitude (B) Tem-poral and scanning sequence of events in one trial A single trial consisted of five scans of 3 sec each(total 15 sec) Analysis was performed during the postoutcome period (T2ndashT5)

Cue Choice-Period Outcome Feedback

HIGH or LOW

Trial Events

REWARDTRIAL

ScanningSequence

TEMPORALSEQUENCE

SMALL REWARD$040

LARGE REWARD$400

SMALL PUNISHMENTndash$020

LARGE PUNISHMENTndash$200

A)

B)

Card Outcome Card

Scan 1 Scan 2 Scan 3 Scan 4 Scan 5 Scan 1

0 3 6 9 12 15

Seconds


AC-PC line Structural images were acquired in the same locationsas the functional images using a standard T1-weighted pulse se-quence Functional images were acquired using a two-interleavedspiral pulse sequence (TR = 1500 msec TE = 34 msec FOV =24 cm flip angle = 70ordm Noll Cohen Meyer amp Schneider 1995)This T2-weighted pulse sequence allowed 20 slices to be acquiredevery 3 sec Images were reconstructed and corrected for motionwith AIR (Woods Cherry amp Mazziotta 1992) adjusted for scan-ner drift between runs with an additive baseline correction appliedto each voxel-wise time course independently and were detrendedwith a simple linear regression to adjust for drift within runs Struc-tural images of each participant were coregistered to a common ref-erence brain (Woods Mazziotta amp Cherry 1993) Both statisticalmaps created in analysis and the reference brain were transformedto standard Talairach stereotaxic space (Talairach amp Tournoux1988) using AFNI software (Cox 1996) Functional images werethen globally mean-normalized to minimize differences in image in-tensity within a session and between subjects and were smoothedusing a three-dimensional Gaussian filter (4-mm FWHM) to ac-count for between-subjects anatomic differences Peak activity ofeach region of interest was reported using Talairach coordinates(Talairach amp Tournoux 1988)

A repeated measures three-way analysis of variance (ANOVA)was performed on the entire set of coregistered data with partici-pants as a random factor Within-subjects factors included valence(reward or punishment) magnitude (large or small) and time (thepostoutcome period four sequential 3-sec scans in a trial of 15 secreferred to as T2ndashT5) The first scan (T1 period) represented thechoice-making period and was not included in the analysis Compar-isons of interest included interactions of each factor (magnitude andvalence) with time (T2ndashT5) because these should consistently cap-ture differences in the time course of the blood oxygen level depen-dent (BOLD) response associated with each trial type Special focuswas given to the three-way interaction between magnitude valenceand time [F(351) = 481 p lt 005] Our previous work (DelgadoNystrom et al 2000) supports the assumption that changes in ourselected variables (eg valence) can be better characterized by ex-amining activity over the duration of a single trial (eg time) Re-gions of interest (ROIs) consisting of five or more contiguous voxelswere selected as a precaution against Type 1 errors (Forman et al1995) Inferences were therefore made on regions defined by strengthof effect ( p lt 005) and size (5 or more voxels) To confirm and ex-tend the ANOVA findings post hoc one-tailed t tests were performedat specific time points using event-related time series data for eachROI to provide fMRI mean intensity value for each condition forTime Periods T1ndashT5 The focus of these analyses was Time Point T4which occurred in the 6 to 9 sec time window after the presentationof a reward This was the time period in which the biggest differ-entiation between reward and punishment had been observed in theprevious study as well as the period in which the difference shouldbe most pronounced due to the hemodynamic lag which usually re-sults in a peak of activity 4ndash6 sec after presentation of an event(Kwong et al 1992) A secondary analysis motivated by prior re-sults was performed to further investigate any response at TimePoint T2 (during the initial 3 sec when the reward was presented)

ResultsI Interaction of time magnitude and valence The

main comparison of interest was the three-way interactionbetween magnitude and valence over time (Table 1) Areasidentified by this analysis [F(351) = 481 p lt 005] werethe left angular gyrus which decreased in activity theleft lingual gyrus which initially showed a decrease inactivation followed by an increase and the left dorsalstriatum where the activity was localized to the caudatenucleus

The activation in the left caudate nucleus was charac-terized by the previously observed pattern of sustainedactivation for a reward event as opposed to a decaybelow baseline for punishment events (Delgado Nys-trom et al 2000) The critical difference between re-ward and punishment was observed 6 sec after the pre-sentation of the feedback when a typical hemodynamicresponse would be expected A differential response be-tween valences was observed when the outcome magni-tude was small (Figure 2A) and when it was large (Fig-ure 2B) Thus irrespective of magnitude the left dorsalstriatum differentiated between reward and punishment

In addition the magnitude of an outcome as indicatedby the three-way interaction did influence the responseof the caudate nucleus (Figure 3) The highest activationwas associated with the large reward trials followed bythe small reward trials The lowest activation was asso-ciated with large punishment trials whereas small pun-ishment trials were slightly higher One-tailed pairedt tests were performed post hoc to determine whether thefour conditions (large and small reward and large andsmall punishment) significantly differed during the 6- to9-sec time window after the outcome (Time Point T4)Large reward was higher than small reward [t(17) = 225p lt 05] Similarly small punishment was higher thanlarge punishment [t(17) = 207 p lt 05] Small rewardwas significantly higher than small punishment [t(17) =291 p lt 01] and large reward was higher than largepunishment [t(17) = 604 p lt 0001] Thus a paramet-ric ordering according to magnitude of the outcome wasobserved in the left caudate nucleus

The inverse pattern was observed during the initial3 sec after the feedback was presented (Time Point T2)Large punishment produced the highest activation and itwas significantly higher than large reward [t(17) = 442p lt 001] Although small punishment was of a higherorder than small reward the two conditions were not dif-ferentiated at Time Point T2 [t(17) = 02 p = 58] Theparametric order of this early response therefore wasalmost the complete inverse of the later response that oc-curred at the 6- to 9-sec time window although the dif-ferences between the conditions were less robust

Activity in the other two ROIs identified by this con-trast was marked by decreases in activation from the onsetof the trial The left angular gyrus showed a decrease fromthe onset of the trial with large reward events showing asharper decrease then all other events at later time pointsbefore its eventual return to baseline at the end of the trialThe left lingual gyrus showed an initial decrease in activ-

Table 1Interaction of Magnitude Valence and Time

Brodmann Talairach Coordinates AverageRegion of Activation Areas x y z F Ratio

Left angular gyrus 39 238 262 35 571Left caudate nucleus 211 212 7 594Left lingual gyrus 18 23 256 3 531

Notemdashp lt 005


ity followed by a sharp increase after Time Point T2 Thepattern of activation following the increase was highestfor large reward and next highest for large punishmentfollowed by small reward and small punishment trials (atTime Point T4) This ROI appeared to be coding the sizeof the visual stimulus (the large arrows) followed by ei-

ther color or orientation (green or red or pointing up ordown) The lingual gyrus may extract details from the pre-sented visual stimulus and indeed it has been linked withidentification of such stimuli as landscapes and buildings(Takahashi amp Kawamura 2002) However this suggestiondeserves future research since at the onset of the trial

Figure 2 Time series for left caudate nucleus showing that irrespective of magni-tude the dorsal striatum differentiates between reward and punishment There wasan increase in activation at the onset of the trial that was sustained when a reward wasreceived and that decayed when a punishment was received during both the smallmagnitude comparison (A) and the large magnitude comparison (B) Standard errorbars were calculated on a per participant basis across both time and conditions ThefMRI mean intensity value is displayed on the y-axis whereas the temporal frame ofa trial 15 sec per trial is displayed in the x-axis (time in seconds) The green arrowrepresents the onset of the feedback arrow (reward or punishment) according to thetemporal frame of the trial (3 sec after the onset of the trial) The scan progression(T1ndashT5) is also displayed above the time scale (gray boxes) and each data point in thegraph represents one scan acquisition or time point

A)

B)

Small Magnitude of OutcomeDifferences between reward and punishment

Large Magnitude of OutcomeDifferences between reward and punishment

3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity

Val

ueTYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD


there was a decrease in activation and the literature ofBOLD fMRI has yet to fully provide an understanding ofwhat such decreases from the onset of trials signify

II Interaction of time and magnitude Regions thatvaried with magnitude over time are listed in Table 2[F(351) = 481 p lt 005] Activation was observed in thecuneus and the lingual gyrus where the large magnitudetrials had a higher signal than did the small trials A sim-ilar pattern was observed in the parahippocampal gyrusThe activation showed an increase when feedback was re-vealed and large reward events had the highest signal Theoverall pattern observed in these regions suggests that theactivity was driven mostly by the size of the arrows al-though an effect of valence (such as an influence of re-ward in the parahippocampal gyrus) cannot be discounted

III Interaction of time and valence A contrast ofthe brain regions activated over time that differed ac-cording to valence yielded the ROIs shown in Table 3

[F(351) = 481 p lt 005] The largest effects were ob-served in the bilateral dorsal striatum and the thalamicnucleus a replication of our previous experiment wherereward significantly differed from punishment (Del-gado Nystrom et al 2000) Other brain regions acti-vated by this contrast that showed an increase in activityat the onset of the trial included the right inferior parietalcortex (BA 40) and the ventromedial frontal gyrus(BA 10) The inferior parietal ROI showed a higher re-sponse for punishment trials at later time points a pat-tern opposite to the one observed in the ventromedialfrontal ROI where reward trials had a higher responsealthough a noisier time series was recorded in this ROIbecause of its proximity to the edge of the brain

A set of other ROIs showed patterns of decreasing ac-tivation relative to baseline at the onset of the trial In themiddle frontal gyri (bilateral BA 6) and the right supe-rior frontal gyrus (BA 9) ROIs activity decreased sharply

Figure 3 Activation of the left caudate nucleus (dorsal striatum) identified by a three-way interaction of magnitude valence andtime During the 6- to 9-sec time window after the delivery of a feedback (Time Point T4) a parametric ordering was observed sincethe largest activation was large reward followed by small reward then small punishment and finally large punishment The inverseordering was observed during the initial 3 sec after the feedback was presented (Time Point T2) during which an early response tolarge punishment was higher than that to large reward

Table 2Interactions of Magnitude and Time


Right middle frontal gyrus 8 235 235 45 533Right precuneus 7 210 267 38 579Right posterior cingulate gyrus 31 2 7 244 38 568Right inferior parietal gyrus 40 242 237 37 545Right middle frontal gyrus 9 226 219 36 550Left cuneus 18 22 276 16 647Right occipital gyrus 19 229 285 15 597Right putamenglobus pallidus 227 27 7 516Left putamenglobus pallidus 223 2 1 2 529Left parahippocampal gyrus 30 211 240 22 654Right parahippocampal gyrus 3019 216 241 24 676Right lingual gyrus 19 216 252 24 653

Notemdashp lt 005

Left Caudatex y z = (ndash11 12 7)

3104

3102

3100

3098

3096

0 3 6 9 12Time (seconds)

Mea

n In

tens

ity

Val

ue

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

LARGE PUNISHMENT

LARGE REWARD


at the onset of the trial showing a larger decrease for re-ward events than for punishment events at later timepoints A similar pattern was observed in both right andleft cingulate gyrus ROIs where the reward events de-creased more than the punishment events at later timepoints before returning to baseline at the end of the trial

Other regions showed an initial decrease in activationfollowed by a rise above baseline at Time Point T2 In theleft and right middle frontal gyrus ROIs (BA 946) the riseabove baseline was highest for punishment trials whereasin the cuneus the rise was highest for reward trials

In summary this contrast identified regions that ex-hibited both increasing and decreasing patterns of acti-vation across time On the basis of our prior work wewere most interested in ROIs that showed an initial in-crease of activity These ROIs included a cortico-striatalloop comprising the bilateral striatum the thalamus andthe right ventromedial frontal gyrus all of which showeda higher response for reward than for punishment trialsThe right inferior parietal was also activated showing theopposite pattern (punishment higher than reward)

The left-striatum ROI included two peaks of activityone in the dorsal and one in the ventral portions of thestriatum (at or near the nucleus accumbens) Furtheranalyses were performed to investigate hemodynamic re-sponses in the ventral striatum Although a significantthree-way interaction between time valence and mag-nitude was not observed in the ventral striatum this re-gion did show an interaction between time and valenceThe locus of activity in the left ventral striatum showeda pattern of response similar to that observed in the leftdorsal striatum where reward was significantly higherthan punishment at T3 [t(17) = 373 p lt 01] and a trendwas observed at T4 [t(17) = 150 p lt 08] Interestinglyno significant differences between conditions were ob-served at T2 in this ventral striatum focus [t(17) = 016p = 44] similar to the null effect observed in our previ-ous study (Delgado Nystrom et al 2000)

Thus although dorsal and ventral striatum responsesappear to be similar during the 6- to 9-sec time windowafter reward delivery the largest effects are observed inthe dorsal striatum Furthermore an early response seemsto be observed only in the dorsal striatum To further un-derstand the validity of the differential responses observedduring the initial 3 sec after feedback presentation weconducted Experiment 2 which was a replication of thepresent experiment with increased temporal resolution

EXPERIMENT 2

MethodTwelve different paid volunteers participated in Experiment 2 (6

males 6 females average age = 2458 years SD = 38) Two par-ticipants were removed from final analysis due to excessive motionand artifact-induced noise in their data sets The cognitive task wasidentical to the first experiment The only difference was in theimaging parameters where functional images were acquired usinga one-shot spiral pulse sequence (TR = 1500 msec TE = 34 msecFOV = 24 cm flip angle = 70ordm Noll et al 1995) This allowed forgreater temporal resolution (15- vs 3-sec scans) and the acquisi-tion of 10 rather than 5 time points Because of a smaller samplesize and a more restricted focus of the investigation we looked atROIs defined by the interaction of condition and time where va-lence was collapsed across magnitude to increase the number of tri-als for each condition (Table 4)

ResultsA repeated measures two-way ANOVA was per-

formed [F(763) = 439 p lt 0005] and the left caudate

Table 3Interactions of Valence and Time


Right middle frontal gyrus 6 34 218 40 627Right superior frontal gyrus 9 32 39 38 616Left middle frontal gyrus 6 31 213 38 573Right inferior parietal gyrus 40 57 234 33 644Right cingulate gyrus 24 20 21 27 668Left cingulate gyrus 24 212 23 27 606Right middle frontal gyrus 946 51 38 26 545Left middle frontal gyrus 946 238 38 24 682Left cuneus 18 21 295 19 610Right thalamus 5 224 12 1033Left caudate nucleus 28 11 7 1155Right caudate nucleus 15 18 7 679Right ventromedial frontal gyrus 10 4 60 22 646Left ventral striatum 24 12 25 660Right ventral striatum 15 11 25 555

Notemdashp lt 005

Table 4Experiment 2 Interaction of Valence and Time


Right cuneus 18 2 2 276 18 561Right thalamus 2 6 224 11 532Left caudate nucleus 211 211 5 542

Notemdashp lt 0005


the thalamus and the cuneus showed an interaction ofvalence (reward vs punishment) and time (T3ndashT10 eachtime point reflecting 15 sec) Focusing on the striatal ac-tivation one can see a result similar to that shown in thevalence 3 time interaction of Experiment 1 where re-ward trials were significantly higher than punishmentevents (Figure 4) Two-tailed paired t tests of all striataltime points confirmed a difference for reward versuspunishment trials during the 6- to 9-sec time windowafter delivery of a feedback Time Points T7 [t(9) = 472p lt 01] and T8 [t(9) = 253 p lt 05 corresponding toT4 in Experiment 1] In previous work (Delgado Nys-trom et al 2000) and in Experiment 1 punishmentevents had a higher signal than did reward outcomes dur-ing the initial 3 sec when the reward was presented (TimePoint T2 corresponding to T3 and T4 in Experiment 2)The faster temporal acquisition allowed us to observethat in Experiment 2 the higher punishment responsewas not significant at T3 [0ndash15 sec after the feedbackt(9) = 193 p = 09] but was significant at T4 [15ndash3 secafter feedback t(9) = 283 p lt 05] This response wasexclusive to the caudate nucleus since thalamic activa-tion did not show significant differences between rewardand punishment trials during Time Points T3 [t(9) =015 p = 89] and T4 [t(9) = 032 p = 76] althoughthere was an effect of valence in the thalamus during the6- to 9-sec time window after delivery of feedback [T7t(9) = 294 p lt 05 T8 t(9) = 523 p lt 05] Exploratoryanalysis revealed a small ventral striatum focus[F(763) = 327 p lt 005 4 voxels] that did not show adifference during the 3 sec after presentation of a feed-back [T3 t(9) = 153 p = 16 T4 t(9) = 133 p = 22]but did show a higher response for reward trials at T7[t(9) = 35 p lt 05]

Thus the results of Experiments 1 and 2 suggest thatthe dorsal striatum is activated after presentation of anaffective stimulus and that such activation is sensitive tothe valence and magnitude of a stimulus The reward re-sponse is more sustained and significantly higher thanpunishment during the 6- to 9-sec time window after thepresentation of a reward The punishment responseshows an early peak roughly 15ndash3 sec after the presen-tation of a punishment feedback which decreases belowbaseline until the onset of the next trial The early re-sponse appears to be limited to the dorsal striatum Sig-nificant differences in the 15- to 3-sec time windowwere not found in the thalamus or ventral striatum in ei-ther Experiment 1 or Experiment 2 although in both ex-periments these regions were sensitive to valence in thelater 6- to 9-sec time window

GENERAL DISCUSSION

The goal of these experiments was to further our un-derstanding of how the human striatum responds to the de-livery of an affective feedback Using a modified versionof a gambling paradigm in which the delivery of rewardsand punishments varied according to magnitude wefound that the dorsal striatum differentiated between thevalence of an event (reward and punishment) irrespectiveof the magnitude (large or small) Furthermore duringthe 6- to 9-sec time window after the delivery of an out-come the dorsal striatum activation was parametricallyarranged according to magnitude where large rewardyielded the highest signal and large punishment the lowestsignal An early response to the presentation of a feedbackwas also observed and replicated in a second follow-upexperiment in which the temporal sampling was increased

Figure 4 Activation of the left caudate nucleus in Experiment 2 where the paradigm from Experiment 1 was replicated with greatertemporal resolution (15-sec scans) Identified by an interaction of valence and time the left caudate nucleus showed differential re-sponses to reward and punishment during the 6- to 9-sec time window after delivery of feedback (T7 and T8) A higher response topunishment was observed during the 15- to 3-sec time window after feedback presentation (T4) suggesting that an early responsewas evoked by the feedback presentation

Experiment 2 mdash Left Caudatex y z = (ndash11 11 5)

25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD


Converging support for our findings come from priorstudies of reward processing in animals and humans Forexample single-cell studies have shown striatal re-sponses to both the anticipation of a reward (ApicellaScarnati Ljungberg amp Schultz 1992 Hikosaka et al1989 Schultz Apicella Scarnati amp Ljungberg 1992)and the reception of a rewarding event such as a drop ofliquid (Aosaki et al 1994 Apicella et al 1991 Hikosakaet al 1989 Shidara Aigner amp Richmond 1998) Spe-cific recordings in the caudate nucleus have shown boththat neurons in this area are modulated by the expecta-tion of a reward (Hollerman Tremblay amp Schultz 1998Kawagoe Takikawa amp Hikosaka 1998) and that theyare sensitive to the reception of a stimulus of positive va-lence (Aosaki et al 1994 Apicella et al 1991 Hiko-saka et al 1989 Shidara et al 1998) The striatum hasalso been implicated in reward processing in differenttypes of imaging paradigms (Berns et al 2001 Breiteret al 2001 Breiter amp Rosen 1999 Delgado Nystromet al 2000 Elliott Friston amp Dolan 2000 KnutsonAdams et al 2001 Knutson Westdorp Kaiser amp Hom-mer 2000 Koepp et al 1998 Pagnoni et al 2002) Ac-tivation of the caudate nucleus specifically has been re-ported in paradigms in which anticipation of a monetaryreward (Breiter et al 2001 Knutson et al 2000) and re-sponses to the delivery of monetary rewards and punish-ments (Breiter et al 2001 Delgado Nystrom et al2000 Elliott et al 2000) have been measured as well asin paradigms in which responses to positive and negativenonmonetary feedback have been measured (Elliott Sa-hakian Michael Paykel amp Dolan 1998)

Although there is an extensive literature on how thestriatum responds to reward paradigms in which thequestion of how these responses are modulated by mag-nitude has been examined have been less commonChanges in reward magnitude have been shown to influ-ence neuronal activity in the prefrontal cortex (Leon ampShadlen 1999 OrsquoDoherty et al 2001 Rogers et al1999) and the amygdala (Salinas amp White 1998) Withinthe striatum recent recordings in the monkey have sug-gested that neurons in that region may encode more thanjust valence since activity varies according to the type ofliquid reinforcement expected during a spatial delayed-response task (Hassani et al 2001) These findings havebeen supported by recent neuroimaging studies (Breiteret al 2001 Delgado Sypher Stenger amp Fiez 2000Knutson Adams et al 2001 OrsquoDoherty et al 2001Rogers et al 1999) For example Breiter et al (2001)showed ventral striatum activity after the spinning of awheel where of three possible outcomes the largest($1000 reward) yielded the highest activity whereas thelowest ($000 reward) was less active Knutson Adamset al (2001) used a delay task in which participants an-ticipated a reward or a punishment of different magni-tudes The study showed that during the anticipationphase of the task the medial caudate responded to in-creases in both rewards and punishments

The present study concentrated on the consummatoryperiod and is unique in showing the effects of valence

and magnitude in the dorsal striatum after delivery of areward-related stimulus Not only does the caudate nu-cleus differentiate between valence (rewards and pun-ishments) but it also does so with respect to magnitude(large and small) The parametric ordering of the out-come is in accordance with how preferable or valuable aspecific outcome is to an individual Thus a possible in-terpretation of the role of the caudate in the coding of re-wards of different magnitudes is that during the antici-pation phase it elicits approach behavior on the basis ofmagnitude (Knutson Fong Adams Varner amp Hommer2001) During the outcome phase a more consummatoryand reinforcing role is performed (Robbins amp Everitt1992 1996) where the caudate detects and dissociatesbetween not only the valences but also the magnitudesof a stimulus (Delgado Sypher et al 2000)

Similar to our previous design (Delgado Nystromet al 2000) and in accordance with classic decision-making theory (Kahneman amp Tversky 1979 Tversky ampKahneman 1981) the ratio of gain to loss was 21 sincethe literature suggests that the impact of negative out-comes is larger than the impact of positive outcomes Itis unfair to say however that a reward outcome recruitsthe striatum more than does a punishment outcome Ourfindings suggest instead that both the dorsal striatumand the ventral striatum respond to the presentation ofmonetary rewards and monetary punishments showingdifferential responses to both events This result has beensupported by another neuroimaging study that also em-ployed a disproportionate ratio of gains and losses andshowed responses in the striatum to both monetary re-wards and punishments (Breiter et al 2001)

The differential responses to valence and magnitudein the striatum were observed during the 6- to 9-sec timewindow after the presentation of feedback The BOLDhemodynamic response can be elicited by a brief periodof neuronal activity (Aguirre Zarahn amp DrsquoEsposito1998 Buckner 1998 Buckner amp Logan 2001) Earlystudies in the motor and sensory realms showed signalchanges in response to finger movements that lasted aslittle as 05 sec (Bandettini 1999) The BOLD hemody-namic response also has a typical shape (Aguirre et al1998 Buckner amp Logan 2001) characterized by a delayof 2ndash6 sec between neuronal activity and the onset of thehemodynamic response (Kwong et al 1992) as well asa response that may last anywhere between 10ndash12 sec(Blamire et al 1992) Thus the point at which rewardand punishment should differ and furthermore orderparametrically in this paradigm should be roughly6ndash9 sec after the reward presentation

Although we found differences at the expected period inboth Experiments 1 and 2 we found significant differencesduring the initial 3 sec after the outcome was revealedEven though higher temporal sampling in Experiment 2allowed us to observe that the response was actually oc-curring 15ndash3 sec after feedback presentation this is stillearly relative to the typical hemodynamic response Anunexpected and interesting result this response is puz-zling nevertheless and merits further research


This early response has now been replicated in threedesigns however and it is conceivable that it reflects abrief almost immediate hemodynamic response in thedorsal striatum Although further research will be nec-essary to fully understand such a response three possi-ble explanations can be provided First most investiga-tions of the hemodynamic responses have focusedprimarily on motor and sensory regions Although somecognitive paradigms have been used more recently theyfocused mostly on cortical responses (Buckner et al1998 Cohen et al 1997 Courtney Ungerleider Keilamp Haxby 1997) and not much is known about the he-modynamic properties of subcortical structures Perhapsthe use of an ANOVA to analyze the data allowed us thesensitivity to pick up on these more rapid changes sinceunlike a more traditionally used general linear model anANOVA does not assume a shape for the hemodynamicresponse Second it is possible that the observed early re-sponse reflects an autonomic response rather than a cogni-tive process The presentation of a conditioned stimulus(such as a tone that was paired with a shock) elicits au-tonomic (changes in heart rate) and behavioral (freezing)responses (LeDoux 2000) The immediate defensive re-sponses displayed by rats in fear-conditioning paradigmssuggest that the early response to feedback might be re-lated to attention-like mechanisms necessary for instan-taneous fight-or-flight reactions The participant might ini-tially have an autonomic reaction to the large punishmentarrow for example which might show a different couplingto blood flow than do responses to the more cognitiveevaluation of the event A third possibility is that the earlyresponse reflects modulation of an already ongoing re-sponse At the onset of the trial there is a rise in activityas participants are presented with a question mark and areprompted to make a fast guess Since they find out theactual value of the card 25 sec into the trial the knowl-edge of being incorrect compounded by the almost im-mediate presentation of a large feedback arrow denotinga monetary loss of $200 may reflect a rapid modulationof an already rising hemodynamic response leading toan early differentiation between conditions

It is worth noting that many neuroimaging studies ofreward processes have focused on ventral striatal activa-tion (Aharon et al 2001 Berns et al 2001 Breiter et al2001 Elliott et al 2000 Knutson Adams et al 2001Knutson Fong et al 2001 Knutson et al 2000 Koeppet al 1998 Pagnoni et al 2002) because work in ani-mals indicates that the nucleus accumbens (part of theventral striatum) is integral to the brainrsquos reward systemand that it is linked to addictive behavior (Di Chiaraet al 1999 Everitt et al 1999 Koob 1999 Koob ampNestler 1997) A growing literature however also sug-gests that the dorsal striatum is involved in motivated be-haviors ranging from lesion and microdialysis studies inrats (Ito Dalley Robbins amp Everitt 2002 Robbins ampEveritt 1992) to single-cell recordings in nonhumanprimates (Kawagoe et al 1998 Lauwereyns Takikawaet al 2002 Lauwereyns Watanabe Coe amp Hikosaka

2002) and even to dopamine measurements in humans(Volkow et al 2002)

Regarding the ventral striatum we found that activityin this region showed a main effect of time and an inter-action of time and valence in both the present (Experi-ments 1 and 2) and previous studies (Delgado Nystromet al 2000) but a three-way interaction (magnitude va-lence and time) was found exclusively in the dorsalstriatum Another study has also not found magnitude in-fluences in the ventral striatum (Brown amp Bowman1995) In a cued task where a light indicated how manypellets of food a rat was about to receive decreases inreaction time were often observed After ventral striatallesions performance in the task or reaction time was notaffected In contrast a recent neuroimaging experimenthas suggested that the ventral striatum responds to theanticipation of increasing rewards (Knutson Adamset al 2001)

In our neuroimaging paradigm activation may be morerobust in the dorsal striatum because it may be more con-cerned with the consummatory period where rat lesionstudies suggest a larger role for the dorsal rather thanthe ventral striatum (Robbins amp Everitt 1992 1996) An-other potential difference between dorsal striatum andventral striatum activation was the observed early responseto the presentation of feedback which was significantonly in the dorsal striatum in the present and previousstudies (Delgado Nystrom et al 2000) perhaps due todifferential circuitry that includes separate inputs intoeach region

In summary the activation of the striatum in a gam-bling paradigm in which valence and magnitude are ma-nipulated is concurrent with animal and other neu-roimaging experiments This experiment furtherimplicates the dorsal striatum as an integral componentof a reward circuitry responsible for the control of moti-vated behavior where the striatum decodes the valenceof a feedback and ranks it on the basis of preference ormagnitude

REFERENCES

Aguirre G K Zarahn E amp DrsquoEsposito M (1998) The variabil-ity of human BOLD hemodynamic responses NeuroImage 8 360-369

Aharon I Etcoff N Ariely D Chabris C F OrsquoConnor E ampBreiter H C (2001) Beautiful faces have variable reward valuefMRI and behavioral evidence Neuron 32 537-551

Aosaki T Tsubokawa H Ishida A Watanabe K Graybiel A Mamp Kimura M (1994) Responses of tonically active neurons in theprimatersquos striatum undergo systematic changes during behavioralsensorimotor conditioning Journal of Neuroscience 14 3969-3984

Apicella P Ljungberg T Scarnati E amp Schultz W (1991)Responses to reward in monkey dorsal and ventral striatum Experi-mental Brain Research 85 491-500

Apicella P Scarnati E Ljungberg T amp Schultz W (1992)Neuronal activity in monkey striatum related to the expectation ofpredictable environmental events Journal of Neurophysiology 68945-960

Bandettini P A (1999) The temporal resolution of functional MRIIn C Moonen amp P A Bandettini (Eds) Functional MRI (pp 205-220) New York Springer-Verlag


Bechara A Damasio H amp Damasio A (2000) Emotion decisionmaking and the orbitofrontal cortex Cerebral Cortex 10 295-307

Berns G S McClure S M Pagnoni G amp Montague P R(2001) Predictability modulates human brain response to rewardJournal of Neuroscience 21 2793-2798

Blamire A M Ogawa S Ugurbil K Rothman D McCarthyGEllermann J M Hyder F Rattner Z amp Shulman R G(1992) Dynamic mapping of the human visual cortex by high-speedmagnetic resonance imaging Proceedings of the National Academyof Sciences 89 11069-11073

Breiter H C Aharon I Kahneman D Dale A amp Shizgal P(2001) Functional imaging of neural responses to expectancy andexperience of monetary gains and losses Neuron 30 619-639

Breiter H C amp Rosen B R (1999) Functional magnetic resonanceimaging of brain reward circuitry in the human In J F McGinty(Ed) Advancing from the ventral striatum to the extended amygdalaImplications for neuropsychiatry and drug abuse In honor ofLennart Heimer (Annals of the New York Academy of SciencesVol 877 pp 523-547) New York New York Academy of Sciences

Brown V J amp Bowman E M (1995) Discriminative cues indicat-ing reward magnitude continue to determine reaction time of rats fol-lowing lesions of the nucleus accumbens European Journal of Neu-roscience 7 2479-2485

Buckner R L (1998) Event-related fMRI and the hemodynamic re-sponse Human Brain Mapping 6 373-377

Buckner R L Goodman J Burock M Rotte M Koutstaal WSchacter D Rosen B amp Dale A M (1998) Functional-anatomiccorrelates of object priming in humans revealed by rapid presentationevent-related fMRI Neuron 20 285-296

Buckner R L amp Logan J M (2001) Functional neuroimagingmethods PET and fMRI In R Cabeza amp A Kingstone (Eds) Hand-book of functional neuroimaging of cognition (pp 27-48) Cam-bridge MA MIT Press

Cohen J D Perlstein W M Braver T S Nystrom L E NollD C Jonides J amp Smith E E (1997) Temporal dynamics ofbrain activation during a working memory task Nature 386 604-608

Courtney S M Ungerleider L G Keil K amp Haxby J V(1997) Transient and sustained activity in a distributed neural systemfor human working memory Nature 386 608-611

Cox R W (1996) AFNI Software for analysis and visualization offunctional magnetic resonance neuroimages Computers amp Biomed-ical Research 29 162-173

Crespi L P (1942) Quantitative variation of reinforcement and levelof performance American Journal of Psychology 55 467-517

Delgado M R Nystrom L E Fissell C Noll D C amp Fiez J A(2000) Tracking the hemodynamic responses to reward and punish-ment in the striatum Journal of Neurophysiology 84 3072-3077

Delgado M [R] Sypher H Stenger V amp Fiez J (2000) Dorsalstriatum responses to reward and punishment Effects of valence andmagnitude manipulations Society for Neuroscience Abstracts 261073

Di Chiara G Tanda G Bassareo V Pontieri F Acquas EFenu S Cadoni C amp Carboni E (1999) Drug addiction as adisorder of associative learning Role of nucleus accumbens shellextended amygdala dopamine In J F McGinty (Ed) Advancingfrom the ventral striatum to the extended amygdala Implications forneuropsychiatry and drug abuse In honor of Lennart Heimer (An-nals of the New York Academy of Sciences Vol 877 pp 461-485)New York New York Academy of Sciences

Elliott R Friston K J amp Dolan R J (2000) Dissociable neuralresponses in human reward systems Journal of Neuroscience 206159-6165

Elliott R Sahakian B J Michael A Paykel E S amp DolanR J (1998) Abnormal neural response to feedback on planning andguessing tasks in patients with unipolar depression PsychologicalMedicine 28 559-571

Everitt B J Parkinson J A Olmstead M C Arroyo M Rob-ledo P amp Robbins T W (1999) Associative processes in addic-tion and reward The role of amygdala-ventral striatal subsystems InJ F McGinty (Ed) Advancing from the ventral striatum to the ex-

tended amygdala Implications for neuropsychiatry and drug abuseIn honor of Lennart Heimer (Annals of the New York Academy ofSciences Vol 877 pp 412-438) New York New York Academy ofSciences

Forman S D Cohen J D Fitzgerald M Eddy W F MintunM A amp Noll D C (1995) Improved assessment of significant ac-tivation in functional magnetic resonance imaging (fMRI) Use of acluster-size threshold Magnetic Resonance Medicine 33 636-647

Grant S Contoreggi C amp London E D (2000) Drug abusersshow impaired performance in a laboratory test of decision makingNeuropsychologia 38 1180-1187

Groenewegen H J Wright C I Beijer A V amp Voorn P (1999)Convergence and segregation of ventral striatal inputs and outputs InJ F McGinty (Ed) Advancing from the ventral striatum to the ex-tended amygdala Implications for neuropsychiatry and drug abuseIn honor of Lennart Heimer (Annals of the New York Academy ofSciences Vol 877 pp 49-63) New York New York Academy ofSciences

Haber S N Kunishio K Mizobuchi M amp Lynd-Balta E (1995)The orbital and medial prefrontal circuit through the primate basalganglia Journal of Neuroscience 15 4851-4867

Hassani O K Cromwell H C amp Schultz W (2001) Influenceof expectation of different rewards on behavior-related neuronal ac-tivity in the striatum Journal of Neurophysiology 85 2477-2489

Hikosaka O Sakamoto M amp Usui S (1989) Functional proper-ties of monkey caudate neurons III Activities related to expectationof target and reward Journal of Neurophysiology 61 814-832

Hollerman J R Tremblay L amp Schultz W (1998) Influence ofreward expectation on behavior-related neuronal activity in primatestriatum Journal of Neurophysiology 80 947-963

Hollerman J R Tremblay L amp Schultz W (2000) Involvementof basal ganglia and orbitofrontal cortex in goal-directed behaviorProgress in Brain Research 126 193-215

Ito R Dalley J W Robbins T W amp Everitt B J (2002) Do-pamine release in the dorsal striatum during cocaine-seeking behav-ior under the control of a drug-associated cue Journal of Neuro-science 22 6247-6253

Kahneman D amp Tversky A (1979) Prospect theory An analysis ofdecision under risk Econometrica 47 263-291

Kawagoe R Takikawa Y amp Hikosaka O (1998) Expectation ofreward modulates cognitive signals in the basal ganglia Nature Neu-roscience 1 411-416

Knutson B Adams C M Fong G W amp Hommer D (2001) An-ticipation of increasing monetary reward selectively recruits nucleusaccumbens Journal of Neuroscience 21 RC159

Knutson B Fong G W Adams C M Varner J L amp Hommer D(2001) Dissociation of reward anticipation and outcome with event-related fMRI NeuroReport 12 3683-3687

Knutson B Westdorp A Kaiser E amp Hommer D (2000) FMRIvisualization of brain activity during a monetary incentive delay taskNeuroImage 12 20-27

Koepp M J Gunn R N Lawrence A D Cunningham V JDagher A Jones T Brooks D J Bench C J amp Grasby P M(1998) Evidence for striatal dopamine release during a video gameNature 393 266-268

Koob G F (1999) The role of the striatopallidal and extended amyg-dala systems in drug addiction In J F McGinty (Ed) Advancingfrom the ventral striatum to the extended amygdala Implications forneuropsychiatry and drug abuse In honor of Lennart Heimer (An-nals of the New York Academy of Sciences Vol 877 pp 445-460)New York New York Academy of Sciences

Koob G F amp Nestler E J (1997) The neurobiology of drug addictionJournal of Neuropsychiatry amp Clinical Neurosciences 9 482-497

Kwong K K Belliveau J W Chesler D A Goldberg I EWeisskoff R M Poncelet B P Kennedy D N Hoppel B ECohen M S Turner R Cheng H Brady T J amp Rosen B R(1992) Dynamic magnetic resonance imaging of human brain activ-ity during primary sensory stimulation Proceedings of the NationalAcademy of Sciences 89 5675-5679

Lauwereyns J Takikawa Y Kawagoe R Kobayashi S Koi-zumi M Coe B Sakagami M amp Hikosaka O (2002) Feature-


based anticipation of cues that predict reward in monkey caudate nu-cleus Neuron 33 463-473

Lauwereyns J Watanabe K Coe B amp Hikosaka O (2002) Aneural correlate of response bias in monkey caudate nucleus Nature418 413-417

LeDoux J E (2000) Emotion circuits in the brain Annual Review ofNeuroscience 23 155-184

Leon M I amp Shadlen M N (1999) Effect of expected reward mag-nitude on the response of neurons in the dorsolateral prefrontal cor-tex of the macaque Neuron 24 415-425

Lesieur H R amp Blume S B (1987) The South Oaks GamblingScreen (SOGS) A new instrument for the identification of patho-logical gamblers American Journal of Psychiatry 144 1184-1188

Macwhinney B Cohen J amp Provost J (1997) The PsyScopeexperiment-building system Spatial Vision 11 99-101

Middleton F A amp Strick P L (2000) Basal ganglia output andcognition Evidence from anatomical behavioral and clinical stud-ies Brain amp Cognition 42 183-200

Noll D C Cohen J D Meyer C H amp Schneider W (1995)Spiral K-space MR imaging of cortical activation Journal of Mag-netic Resonance Imaging 5 49-56

OrsquoDoherty J Kringelbach M L Rolls E T Hornak J amp An-drews C (2001) Abstract reward and punishment representations inthe human orbitofrontal cortex Nature Neuroscience 4 95-102

Pagnoni G Zink C F Montague P R amp Berns G S (2002)Activity in human ventral striatum locked to errors of reward predic-tion Nature Neuroscience 5 97-98

Robbins T W amp Everitt B J (1992) Functions of dopamine in thedorsal and ventral striatum Seminars in the Neurosciences 4 119-127

Robbins T W amp Everitt B J (1996) Neurobehavioural mecha-nisms of reward and motivation Current Opinion in Neurobiology 6228-236

Rogers R D Owen A M Middleton H C Williams E JPickard J D Sahakian B J amp Robbins T W (1999) Choosingbetween small likely rewards and large unlikely rewards activatesinferior and orbital prefrontal cortex Journal of Neuroscience 199029-9038

Rolls E T (1999) The brain and emotion Oxford Oxford UniversityPress

Rolls E T (2000) The orbitofrontal cortex and reward Cerebral Cor-tex 10 284-294

Salinas J A Packard M G amp McGaugh J L (1993) Amygdalamodulates memory for changes in reward magnitude Reversiblepost-training inactivation with lidocaine attenuates the response to areduction in reward Behavioural Brain Research 59 153-159

Salinas J A amp White N M (1998) Contributions of the hip-pocampus amygdala and dorsal striatum to the response elicited byreward reduction Behavioral Neurosciences 112 812-826

Schultz W (2000) Multiple reward signals in the brain Nature Re-views Neurosciences 1 199-207

Schultz W Apicella P Scarnati E amp Ljungberg T (1992)Neuronal activity in monkey ventral striatum related to the expecta-tion of reward Journal of Neuroscience 12 4595-4610

Schultz W Tremblay L amp Hollerman J R (1998) Reward pre-diction in primate basal ganglia and frontal cortex Neuropharma-cology 37 421-429

Shidara M Aigner T G amp Richmond B J (1998) Neuronal sig-nals in the monkey ventral striatum related to progress through a pre-dictable series of trials Journal of Neuroscience 18 2613-2625

Takahashi N amp Kawamura M (2002) Pure topographical disori-entation The anatomical basis of landmark agnosia Cortex 38 717-725

Talairach J amp Tournoux P (1988) Co-planar stereotaxic atlas ofthe human brain An approach to medical cerebral imaging NewYork Thieme Medical Publishers

Tversky A amp Kahneman D (1981) The framing of decisions andthe psychology of choice Science 211 453-458

Volkow N D Wang G J Fowler J S Logan J Jayne MFranceschi D Wong C Gatley S J Gifford A N Ding Y Samp Pappas N (2002) ldquoNonhedonicrdquo food motivation in humans in-volves dopamine in the dorsal striatum and methylphenidate ampli-fies this effect Synapse 44 175-180

Woods R P Cherry S R amp Mazziotta J C (1992) Rapid auto-mated algorithm for aligning and reslicing PET images Journal ofComputer Assisted Tomography 16 620-633

Woods R P Mazziotta J C amp Cherry S R (1993) MRI-PETregistration with automated algorithm Journal of Computer AssistedTomography 17 536-546

(Manuscript received August 5 2002revision accepted for publication February 19 2003)


tudes of reinforcement were given (OrsquoDoherty Kringel-bach Rolls Hornak amp Andrews 2001)

One area that receives projections from both the frontalcortex (Haber Kunishio Mizobuchi amp Lynd-Balta1995 Middleton amp Strick 2000) and the amygdala(Groenewegen Wright Beijer amp Voorn 1999) is thestriatum where magnitude effects and even investiga-tions of magnitude manipulations have been less fre-quently reported Recently Hassani Cromwell andSchultz (2001) showed that monkey striatal neurons firemore vigorously for preferred rewards suggesting thatsome ranking based on preference may occur in the stria-tum In a clever paradigm Breiter et al (2001) foundhigher ventral striatum activity associated with higheroutcomes in a ldquowheel of fortunerdquo like task A more directmeasurement of the effects of magnitude in the striatumwas done by Knutson Adams Fong and Hommer (2001)who scanned participants while they anticipated rewardsand punishments that varied in amounts The studyshowed that ventral striatum activity was associated withanticipation of larger rewards whereas the dorsal stria-tum was activated while both rewards and punishmentsof larger magnitude were anticipated Although in thisstudy magnitude changes in the striatum were looked atthe primary focus was on the effects of magnitude in theanticipatory phase Thus more research is needed tofully understand how the striatum responds to the actualdelivery of rewards and punishments of different magni-tudes

In a previous study we developed a gambling task(card-guessing paradigm) in which participants wereasked to guess the value of a card (Delgado Nystromet al 2000) A correct guess yielded a monetary rewardwhereas an incorrect guess led to a monetary punish-ment Using an event-related design we observed dif-ferential responses to reward and punishment feedbackin the striatum where the hemodynamic response for re-wards was significantly higher than that for punishmentsin the 6- to 9-sec time window after feedback presenta-tion In this paper we will further examine the previ-ously observed dissociation of valence in the striatum bystudying how the response is affected by the magnitudeof the outcome In our first experiment we used a mod-ified version of the card-guessing paradigm to measurestriatal activity following the delivery of monetary re-wards and punishments that varied parametrically Theoutcomes were either a large ($400) or small ($040)monetary reward or a large ($200) or small ($020)monetary loss The primary goal was to replicate the dif-ferences in activation according to valence and to go be-yond prior studies by varying the magnitude of the out-comemdashthus providing further evidence that the striatuminfluences or guides motivated behavior by processingthe motivational properties of a stimulus

In our previous study (Delgado Nystrom et al 2000)we also observed a differential response to punishmentsand rewards that evolved within 3 sec of feedback pre-sentation The rapidity of this response caused us to

question its reliability Although it is likely that the pre-sentation of a surprising and affective stimulus (eg abear appearing from nowhere) causes fast immediatecognitive and physiological responses our understand-ing of the properties of the hemodynamic response lim-its any possible interpretation Hence a secondary goalof this research was to determine whether an early re-sponse (of less than 3 sec) to a feedback could be repli-cated and further characterized temporally This was ac-complished by examining the activation detected within3 sec of feedback presentation in the first experimentand by running a second experiment with increased tem-poral sampling

EXPERIMENT 1

MethodParticipants Twenty right-handed volunteers participated in

this study (11 females 9 males) The participants were mostly grad-uate and undergraduate students drawn from the University of Pitts-burgh (average age = 229 years SD = 326) Two participants wereremoved from all analysis because of excessive motion during theirscan sessions The participants were asked to fill out a brief ques-tionnaire to ensure that they had prior experience with gambling butwere not abusive or excessive in such behavior (ie have youplayed cards for money not at all less than once a week or once aweek or more) The questionnaire was based on the South OaksGambling Screen (Lesieur amp Blume 1987) Information about anyfamily history of gambling was not acquired All the participantsgave informed consent according to the policies of the InstitutionalReview Board at the University of Pittsburgh

Cognitive task The paradigm involved a series of 144 inter-leaved trials divided into nine runs of 16 trials each Each triallasted 15 sec and began with the presentation of a visually displayedcard projected onto a screen The card had an unknown value rang-ing from 1 to 9 and the participant was instructed to make a guessabout the value of the card A question mark appeared in the centerof the card indicating that the participant had 25 sec to guesswhether the card value was higher or lower than the number 5 Theparticipant pressed the left or the right button of a response unit toindicate his or her selection After the choice-making period anumber appeared in the center of the card for 500 msec followedby an arrow that was also displayed for another 500 msec The ap-pearance of a green arrow pointing upward indicated that the par-ticipant had correctly guessed the card value A large green arrowcorresponded to a large reward of $400 A smaller green arrow in-dicated a small reward of $040 (Figure 1A) The appearance of ared arrow pointing downward indicated that the participant had in-correctly guessed the card value leading to a penalty of $200 if thearrow was large and a $020 penalty if the arrow was small Unlikeour previous design (Delgado Nystrom et al 2000) there was noneutral condition and the number 5 never appeared as the value ofa card Therefore there were four types of trials (large and small re-ward and large and small punishment) The same gain-to-loss ratioas that from our previous design was kept (21) in accordance withclassic decision-making literature suggesting that the impact ofnegative outcomes such as losses is larger than that of positive out-comes such as gains (Kahneman amp Tversky 1979 Tversky amp Kah-neman 1981) For trials in which a response was not made on timethe feedback was a pound sign () and such trials were excludedfrom all analyses After the 35-sec period between presentation ofthe response cue (question mark) and the rewardpunishment out-come there was an 115-sec delay before the onset of the next trialAn experimental session therefore consisted of 144 trials of 15 sec








HIGH or LOW

Trial Events

REWARDTRIAL

ScanningSequence

TEMPORALSEQUENCE

SMALL REWARD$040

LARGE REWARD$400



A)

B)

Card Outcome Card


0 3 6 9 12 15

Seconds













Notemdashp lt 005





A)

B)



3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity

Val

ueTYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD











Notemdashp lt 005


3104

3102

3100

3098

3096


Mea

n In

tens

ity

Val

ue

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

LARGE PUNISHMENT

LARGE REWARD







EXPERIMENT 2








Notemdashp lt 005




Notemdashp lt 0005




GENERAL DISCUSSION




25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD
















REFERENCES



















































































HIGH or LOW

Trial Events

REWARDTRIAL

ScanningSequence

TEMPORALSEQUENCE

SMALL REWARD$040

LARGE REWARD$400



A)

B)

Card Outcome Card


0 3 6 9 12 15

Seconds













Notemdashp lt 005





A)

B)



3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity

Val

ueTYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD











Notemdashp lt 005


3104

3102

3100

3098

3096


Mea

n In

tens

ity

Val

ue

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

LARGE PUNISHMENT

LARGE REWARD







EXPERIMENT 2








Notemdashp lt 005




Notemdashp lt 0005




GENERAL DISCUSSION




25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD
















REFERENCES
























































































Notemdashp lt 005





A)

B)



3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity

Val

ueTYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD











Notemdashp lt 005


3104

3102

3100

3098

3096


Mea

n In

tens

ity

Val

ue

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

LARGE PUNISHMENT

LARGE REWARD







EXPERIMENT 2








Notemdashp lt 005




Notemdashp lt 0005




GENERAL DISCUSSION




25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD
















REFERENCES
















































































A)

B)



3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

3104

3102

3100

3098

30960 3 6 9 12

Time (seconds)

Mea

n In

tens

ity

Val

ueTYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD











Notemdashp lt 005


3104

3102

3100

3098

3096


Mea

n In

tens

ity

Val

ue

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

LARGE PUNISHMENT

LARGE REWARD







EXPERIMENT 2








Notemdashp lt 005




Notemdashp lt 0005




GENERAL DISCUSSION




25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD
















REFERENCES






















































































Notemdashp lt 005


3104

3102

3100

3098

3096


Mea

n In

tens

ity

Val

ue

TYPE OF TRIAL

SMALL PUNISHMENT

SMALL REWARD

LARGE PUNISHMENT

LARGE REWARD







EXPERIMENT 2








Notemdashp lt 005




Notemdashp lt 0005




GENERAL DISCUSSION




25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD
















REFERENCES


















































































EXPERIMENT 2








Notemdashp lt 005




Notemdashp lt 0005




GENERAL DISCUSSION




25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD
















REFERENCES















































































GENERAL DISCUSSION




25075

2505

25025

2500

24975

2495

0 3 6 9 12

Time (seconds)

Mea

n In

tens

ity V

alue

TYPE OF TRIAL

PUNISHMENT

REWARD
















REFERENCES



























































































REFERENCES












































































dorsal striatum responses to reward and punishment...

Documents