A

RJSa

tb

c

a

ARRAA

KMfSR

1

irpd(&BHtmnlpa

aT

0d

Neuropsychologia 48 (2010) 2912–2921

Contents lists available at ScienceDirect

Neuropsychologia

journa l homepage: www.e lsev ier .com/ locate /neuropsychologia

virtual reality-based FMRI study of reward-based spatial learning

achel Marsha,∗, Xuejun Haoa, Dongrong Xua, Zhishun Wanga, Yunsuo Duana,un Liua, Alayar Kangarlua, Diana Martinezc, Felix Garciaa, Gregory Z. Taua,han Yua, Mark G. Packardb, Bradley S. Petersona

The MRI Unit and Division of Child & Adolescent Psychiatry in the Department of Psychiatry, the New York State Psychiatric Institute andhe College of Physicians and Surgeons, Columbia University, New York, NY, United StatesThe Department of Psychology, Texas A&M University, College Station, TX, United StatesThe Department of Psychiatry, the New York State Psychiatric Institute and the College of Physicians and Surgeons, Columbia University, New York, NY, United States

r t i c l e i n f o

rticle history:eceived 7 December 2009eceived in revised form 26 May 2010ccepted 28 May 2010vailable online 4 June 2010

eywords:

a b s t r a c t

Although temporo-parietal cortices mediate spatial navigation in animals and humans, the neural corre-lates of reward-based spatial learning are less well known. Twenty-five healthy adults performed a virtualreality fMRI task that required learning to use extra-maze cues to navigate an 8-arm radial maze and findhidden rewards. Searching the maze in the spatial learning condition compared to the control conditionswas associated with activation of temporo-parietal regions, albeit not including the hippocampus. Thereceipt of rewards was associated with activation of the hippocampus in a control condition when using

emory systemsMRIpatial learningeward

the extra-maze cues for navigation was rendered impossible by randomizing the spatial location of cues.Our novel experimental design allowed us to assess the differential contributions of the hippocampus andother temporo-parietal areas to searching and reward processing during reward-based spatial learning.This translational research will permit parallel studies in animals and humans to establish the functionalsimilarity of learning systems across species; cellular and molecular studies in animals may then informthe effects of manipulations on these systems in humans, and fMRI studies in humans may inform the

nce o

interpretation and releva

. Introduction

Previous fMRI studies of spatial navigation within virtual real-ty (VR) environments have consistently identified a network ofegions – including the hippocampus, parahippocampus, retros-lenial cortex, and posterior parietal cortex – that are engageduring virtual navigation relative to non-navigational control tasksAguirre, Detre, Alsop, & D’Esposito, 1996; Hartley, Maguire, Spiers,

Burgess, 2003; Hassabis et al., 2009; Iaria, Fox, Chen, Petrides, &arton, 2008; Spiers & Maguire, 2006b; Wolbers & Buchel, 2005).owever, many of these control tasks differed from the naviga-

ion tasks on salient features, such as the form of visual stimuli,otor demands, task instructions, or cognitive effort required for

avigation, thereby precluding full isolation of the neural corre-ates of spatial learning in standard functional imaging subtractionaradigms. In addition, none of the prior fMRI studies have directlyssessed the role of reward in spatial learning.

∗ Corresponding author at: Columbia University and the New York State Psychi-tric Institute, 1051 Riverside Drive, Unit 74, New York, NY 10032, United States.el.: +1 212 543 5384; fax: +1 212 543 0522.

E-mail address: MarshR@childpsych.columbia.edu (R. Marsh).

028-3932/$ – see front matter. Published by Elsevier Ltd.oi:10.1016/j.neuropsychologia.2010.05.033

f findings in animals.Published by Elsevier Ltd.

Our VR task for spatial learning is directly analogous to the stan-dard “win-shift” radial-arm maze paradigm that has been usedto study spatial learning and memory in animals. This paradigminvolves navigation of a maze with 8 identically appearing armsextending outward from a central platform that has extra-mazeobjects visible from within the maze. Rodents obtain food rewardsby visiting each arm of the maze once, with re-entries into mazearms previously visited scored as errors. Because performance onthis task requires rodents to remember those arms that have beenpreviously visited, it is regarded as a prototypical test of spatialmemory or cognitive mapping. Extensive evidence from animalstudies employing lesions (Olton & Samuelson, 1976; Packard,Hirsh, & White, 1989) and pharmacological stimulation (Packard &White, 1991) have demonstrated that the hippocampus contributesto performance on the win-shift radial maze task.

Analogous fMRI studies in humans can complement theseradial-arm maze studies of spatial learning in animals. For exam-ple, fMRI studies can more readily and simultaneously assess the

differential involvement of brain regions other than the hippocam-pus in spatial learning. In addition, fMRI studies can disentangle theroles of reward and learning in reward-based spatial learning, andthey more readily can study of de novo learning, without requiringprior training as in animal studies.

R. Marsh et al. / Neuropsychologia 48 (2010) 2912–2921 2913

F oting‘

hMtmtapaiaotdcm&MtrIcoicvswt

2

2

ssdAhwwt

2

tgcbepuae

o

ig. 1. (a) A view of the virtual reality environment, (b) the U.S. dollar signs dentrail-following’ control condition C.

We report an fMRI study of reward-based spatial learning inealthy adults performing a novel VR task. Participants used anRI-compatible joystick to navigate through an 8-arm radial maze

hat was surrounded by a naturalistic landscape consisting ofountains, trees, and flowers that had to be used for spatial naviga-

ion if participants were to avoid re-entries into previously visitedrms. Hidden at the end of each arm of the maze was a reward thatarticipants could earn and keep only if they did not re-enter anrm already visited. To isolate the neural correlates of spatial learn-ng, we included a rigorously defined control condition that sharedll salient features with the active learning task except for the usef the extra-maze cues for navigation. Our a priori hypothesis washat participants would engage medial temporal and parietal areasuring spatial learning in the active compared with the controlondition. Based on prior findings of reward-based learning in ani-als and humans (Adcock, Thangavel, Whitfield-Gabrieli, Knutson,Gabrieli, 2006; Ito, Robbins, Pennartz, & Everitt, 2008; Vanni-ercier, Mauguiere, Isnard, & Dreher, 2009), we also hypothesized

hat activation of mesolimbic areas would be associated with theeceipt of rewards in the active compared to the control condition.n addition, we explored whether mesolimbic activation was asso-iated with the anticipation of rewards, and whether it dependedn the actual receipt of rewards or on the involvement of learn-ng processes. Finally, to ensure that any confusion or additionalognitive effort in the control condition was not producing acti-ation in the active condition, we compared brain activity duringpatial learning and the receipt of rewards in the active conditionith brain activity in a ‘trail-following’ condition that eliminated

he attempted use of the extra-maze cues for navigation.

. Methods

.1. Participants

Participants were recruited through flyers posted in the local community toerve as healthy control participants for a neuroimaging study of the neural basis ofubstance dependence. The Structured Clinical Interview for DSM-IV-TR Axis I disor-ers (First, Spitzer, Gibbon, & Williams, 2002) was administered to all participants.ny who met DSM-IV criteria for a current Axis I disorder, or who had a lifetimeistory of substance abuse disorder, neurological illness, past seizures, head traumaith loss of consciousness, mental retardation, or pervasive developmental disorderere excluded from participating. The Institutional Review Board of NYSPI approved

his study, and all participants gave informed consent prior to participating.

.2. Spatial learning paradigm

Creation of the virtual environments was accomplished through use of softwarehat was built upon C++ and OpenGL, an API (Application Programming Interface) ofraphics. The virtual environments were composed of an 8-arm radial maze with aentral starting location and a low outer perimeter wall. The maze was surroundedy a naturalistic landscape (e.g., mountains, trees and flowers) that constituted thextra-maze cues that could be used for spatial navigation (Fig. 1a). Prior to scanning,

articipants underwent a 10-min training session on a desktop computer to practicesing a joystick to move and navigate freely around a virtual maze that was similar inppearance to the experimental environment used for scanning, but with differentxtra-maze cues.

Stimuli during scanning were presented through non-magnetic goggles (Res-nance Technology, Inc.). Participants used an MRI-compatible joystick (Current

rewards that were hidden at the ends of the maze arms, and (c) a view of the

Designs Inc.) to navigate the maze. Before entering the scanner, participants wereinformed that they would find themselves in the center of a virtual maze with 8 iden-tical runways extending outwards, and that hidden rewards (U.S. $, Fig. 1b) wouldbe available at the very end of the runways. They were instructed to try to find therewards and were told that they could earn and keep as many as they found, butthat they would loose money if they revisited arms they had already visited. Theywere told that they would complete several sessions of the task. They were not toldthat the conditions of the sessions would differ from one another, and thereforethey believed that they would be performing the same task multiple times. Partic-ipants were not given actual monetary rewards for their performance on the task,but rather paid a set amount for their participation in the study.

The first session (condition A) required spatial learning. To find and retain all8 rewards, participants had to learn to use the extra-maze cues to avoid revisitingpreviously visited arms. Each trial began at the center platform. After reaching theend of an arm, participants reappeared automatically in the middle of the centerplatform to initiate a new trial, with the initial viewing perspective randomly ori-ented across the entire 360◦ of the circular maze, which compelled participants touse extra-maze cues to orient themselves for subsequent navigation. The random-ized orientation at the start of each trial also prevented use of S-R strategies, suchas “chaining” rules (e.g., exploring arms positioned successively to the left or rightof the last arm entered, regardless of extra-maze cues), when performing the task.

After obtaining all 8 rewards in condition A, participants saw a black screen withwhite written text indicating that a new session was about to begin. In this first con-trol condition (B), the same extra-maze cues as used in condition A were randomizedamong locations after each trial: the mountains that surrounded the maze were cutinto several pieces that were then shuffled and re-stitched into whole mountains,while the individual trees and flowers were themselves shuffled among locations.This randomization of the cues together with the randomized directional orientationof participants with respect to the various arms of the maze at the start of each trialdestroyed any possibility of using the spatial layout of the cues (spatial learning) toperform the task. As in condition A, the randomized orientation at the start of eachtrial also prevented the use of S-R or other procedural learning strategies. To controlfor the frequency of reward and punishment with the spatial learning task, however,participants were rewarded at the same frequency as in the previous spatial learningcondition, though now without regard to any characteristics of their actual perfor-mance. Participants were not told that the cues would be randomized in this sessionor that anything else would differ from the previous session. This control conditionthus shared all salient features with the spatial learning task (A), including all thelower-order stimulus features, such as color, texture, hue, luminance, the form andmovement of visual stimuli, as well as many higher-order task features, such asthe motor demands, task instructions, cognitive effort in searching for rewards, andreward and punishment experiences. Condition B was pre-programmed to termi-nate according to the number of trials that a given participant needed to obtain all8 rewards in condition A. For instance, if a participant required 10 trials to find all8 rewards in condition A (i.e., 8 correct and 2 error trials), they would be given 2unbaited trails randomly in condition B. After completion of condition B, anotherblack screen appeared that instructed participants to “follow the arrows” duringthe next session (condition C).

Condition C was a second control condition to contrast with condition A. Con-dition C was added to ensure that the use of the cues or the additional confusion orcognitive effort in condition B was not producing any observed activation in condi-tion A (when compared to B). Condition C was a “trail-following” condition in whichthe extra-maze cues were randomized among locations in the same manner as incontrol condition B, but a red arrow at eye level indicated the path to follow (thearm to be traversed, Fig. 1c). Because participants were told simply to follow thearrow on each trial, they were presumably not trying to use the extra-maze cuesfor navigation, and in fact they were unable to use those cues because of the ran-domization of the extra-maze cues upon the initiation of each trial. Because the

allocation of rewards was randomized, however, the arrow did not always point tothe correct arm (where a reward was hidden). This control condition shared manysalient features with both the spatial learning task (A) and the control condition B,including the form and movement of visual stimuli, motor demands, and rewardand punishment experiences. It did not require as much cognitive effort in search-ing for rewards as did condition B, the task instruction (“follow the arrow”) differed

2 chologia 48 (2010) 2912–2921

fBl

semoiCcspevmsar

2

2

IdpmwdTTFcbc

2

sedws2&stet8

2

rrd8cvciCioeoprmmWreht(

events during spatial learning (in condition A) allowed us to disentangle componentsof the learning process.

Group composite activation maps were generated by calculating t statistics ateach pixel of the image using a random effects analysis, covarying for age and sex. Wereport voxels that were identified on these maps using a P-value threshold <0.025

Table 1Contrast images used to assess neural activity associated with and without spatiallearning during each event.

Event Spatial learning Attempted learning

Searching A vs. Ba B vs. CAnticipation A vs. B B vs. CReceipt of reward A vs. Bb B vs. CNo receipt of reward A vs. Bb B vs. CNo reward vs. reward Ac Bd

In exploratory analyses, the A vs. C contrast was used to identify neural activityassociated with the same processes as the A vs. B contrast.

a These contrasts were used to identify neural activity associated with spatiallearning.

b These contrasts were used to identify neural activity associated with reward

914 R. Marsh et al. / Neuropsy

rom conditions A and B, and the task generated less confusion than did conditionwhen trying to use and remember the extra-maze cues for navigation when the

ocations of those cues were randomly interspersed.Each participant underwent two scanning runs, each run containing one pre-

entation of conditions A, B, C presented in that order, for a total of 2 runs ofach condition. Because the use of A to establish the rate of reward and punish-ent in conditions B and C required presenting A first for all participants, the

rder of conditions was not counterbalanced across participants. In addition, test-ng our a priori hypotheses required comparing conditions A and B. Condition

was added only to ensure that any confusion or additional cognitive effort inondition B was not producing activation when contrasted with condition A. Thepatial learning paradigm contained 48 rewarded navigation events (8 rewardser each of 3 conditions × 2 runs). The number of unrewarded events varied forach participant. The 3D coordinates of each participant’s movements through theirtual environment were recorded with a temporal resolution of 50 ms. Perfor-ance on the task was scored automatically using the same criteria as in animal

tudies (e.g., the arms entered and order of entry, overall latency to visit baitedrms, total number of errors, number of choices before the first error were allecorded).

.3. Image acquisition and preprocessing

.3.1. AcquisitionHead positioning in the magnet was standardized using the canthometal line.

mages were acquired on a GE Signa 3 T LX scanner (Milwaukee, WI) and a stan-ard quadrature GE head coil. A T1-weighted sagittal localizing scan was used toosition the axial functional images parallel to anterior commissure–posterior com-issure (AC–PC) line. In all participants, a 3D spoiled gradient recall (SPGR) imageas acquired for coregistration with the axial functional images and with the stan-ard Talairach coordinate system. The functional images were obtained using a2*-sensitive gradient-recalled, single-shot, echo-planar pulse sequence having aR = 2800 ms, TE = 25 ms, 90◦ flip angle, single excitation per image, 24 cm × 24 cmOV, a 64 × 64 matrix, 43 slices 3 mm thick, no gap, covering the entire brain. Weollected a maximum of 322 EPI volumes for each of two run, with the actual num-er of volumes determined by the performance of each participant during the activeonditions of each run.

.3.2. PreprocessingOur preprocessing procedures were run in batch mode and implemented using

ubroutines in SPM2. Functional images were first corrected for timing differ-nces between slices using a windowed Fourier interpolation to minimize theirependence on the reference slice (Jazzard, Matthews, & Smith, 2002). Imagesere then motion-corrected and realigned to the middle image of the middle

canning run. Images were discarded if the estimates for peak motion exceeded-mm displacement or 28◦ of rotation (Friston, Williams, Howard, Frackowiak,

Turner, 1996). The corrected images were normalized and coregistered to atandard MNI template by warping each participant’s SPGR image to the MNIemplate avg152T1 (resolution = 2 mm × 2 mm × 2 mm per voxel) and then warpingach functional image to the SPGR image for that person. Images were then spa-ially smoothed using a Gaussian-kernel filter with a full width at half maximum ofmm.

.4. Behavioral analysis

We tested whether participants would improve in spatial learning over scanuns using a mixed models analysis (PROC MIXED) with repeated measures overuns performed in SAS version 8.0 (SAS Institute Inc, Cary, NC). Participants whoemonstrate spatial learning on the VR paradigm should be faster at obtaining allrewards (i.e., they should have shorter average latency scores) during the second

ompared with the first active, spatial learning condition (e.g., condition A in run 2s. condition A in run 1). With latency defined as the time to complete each learningondition (A) for each individual, latency was entered as the dependent variablen a linear mixed model. Run (1 or 2) was entered as the within-subjects factor.ovariates included age, sex, and minority status. The hypothesized improvement

n spatial learning was tested by assessing statistical significance of the main effectf run, representing the change in latency from run 1 to run 2. Participants whongaged in spatial learning should have required fewer trials to obtain all 8 rewardsver the 2 runs. The rate of learning (8/the number of trials) was calculated for eacherson and entered as the dependent variable in another linear mixed model withun as the within-subjects factor and age, sex, and race as covariates. The improve-ent in the rate of spatial learning was assessed by testing the significance of theain effect of run in this model, representing the change in rate from run 1 to run 2.

e also considered for inclusion in both of these models all interactions of latency or

ate of learning with age and run. Terms that were not statistically significant wereliminated via backward stepwise regression, with the constraint that the modelsad to be hierarchically well-formulated at each step (i.e., all possible lower-ordererms had to be included in the model, regardless of their statistical significance)Morrell, Pearson, & Brant, 1997).

Fig. 2. Study design: Shown here is a schematic diagram of the event-related design,including the 4 events that were modeled as regressors. These events were definedbased on the participants’ behavioral performance, consistent with the performanceof rats in the original radial-arm maze studies.

2.5. Image analysis

First-level parametric analyses were conducted individually for each participantusing GLM in a modified version of SPM2 run in batch mode. The GLM included thefollowing 4 regressors, corresponding to specific events that occurred in each condi-tion and in both scan runs: (1) ‘searching’ began at the start of a trial and lasted until10% of the length of an arm had been traversed, (2) ‘anticipation of reward’ beganafter the first 10% of an arm was traversed and extended until reaching its baitedarea, (3) ‘receipt of reward’ was defined as comprising the images at the end of thetrial when a reward was present, and (4) ‘no receipt of reward’ was the time whenthe end of an arm was reached but no reward was received (Fig. 2). All regressorswere convolved with a canonical hemodynamic response function (HRF), with thedurations of the regressors for each participant modeled according to the durationsof these events during performance in condition A. Low frequency noise from scan-ner drift was removed using a high-pass filter based on discrete cosine transform(DCT).

Twelve contrast images were created for each person by applying a t contrastvector to the parameters (beta j) that were estimated for voxel j. Thus, beta j = b1j,b2j, b3j, b4j, b5j, . . ., bnj, where n = 12 because 4 regressors corresponded to 4 events(searching, anticipation, receipt of reward, and no receipt of reward) for each of 3conditions (A, B, and C). Table 1 lists the contrast images that were used to identifyneural activity associated with spatial learning and with attempted spatial learningduring each of the 4 events. Below we describe the specific information providedby each of these contrasts and how each contrast was used to test our a priori andexploratory hypotheses. Because spatial learning was rendered impossible by therandomization of extra-maze cues in condition B, our comparison of neural activityassociated with events in condition B with neural activity associated with the same

experience during spatial learning.c These contrasts were used to identify neural activity associated with the effects

of reward when spatial learning was possible.d These contrasts were used to identify neural activity associated with the effects

of reward when learning was attempted but rendered impossible (e.g., reward dis-entangled from learning).

R. Marsh et al. / Neuropsychologia 48 (2010) 2912–2921 2915

Table 2Activations associated with spatial learning during searching.

Activated regions Location

Side BA # of voxels Peak location Significance testing Contrast

x y z t statistic P

SearchingSuperior temporal gyrus L 22 156 −44 −21 1 4.21 <0.001 A > B

R 699 65 −26 16 5.15 <0.001 A > BL 274 −55 −27 11 5.46 <0.001 A > CR 340 63 −25 12 4.60 <0.001 A > C

Lateral parietal area R 40 84 50 −36 48 4.45 0.01 A > BR 15 40 −50 47 3.37 0.01 A > C

Parahippocampal gyrus L 36 146 −34 −30 −15 3.45 <0.001 A > BR 157 38 −26 −19 3.93 <0.001 A > BL 191 −28 −22 −21 3.75 <0.001 A > CR 216 36 −24 −24 3.47 <0.001 A > C

Precuneus/posterior cingulate cortex L/R 7/31 613 8 −69 20 4.95 <0.001 A > BL/R 1002 6 −69 26 7.49 <0.001 A > C

Posterior cingulate/retrosplenial cortex L 30/31 707 −18 −64 5 7.17 <0.001 A > C

Primary motor cortex R 4 170 46 −12 37 3.47 <0.001 A > C

Mid-cingulate cortex L/R 24 105 6 −16 36 2.33 <0.01 A > C

tlftap

2

banc

•

•

2

nrsemeo

•

•

•

Caudate nucleusHead LBody L

ogether with the requirement that the activation occurred in a spatial cluster of ateast 20 adjacent pixels, a conjoint requirement which, based on an approximationormula (Friston, Worsley, Frackowiak, Mazziotta, & Evans, 1994), yields a conserva-ive effective P-value <0.000005. The combined application of a statistical thresholdnd cluster filter reduces substantially the false-positive identification of activatedixels at any given threshold (Forman et al., 1995).

.6. A priori hypothesis testing

We tested whether participants engaged medial temporal areas during reward-ased spatial learning on the task. Analyses of covariance (ANCOVAs) with centeredge and sex as covariates were performed on the following contrasts to identifyeural activity during searching and reward experiences in the active condition Aompared to control condition B:

Neural activity during searching in spatial learning (A vs. B): Identified neural activityassociated with spatial learning (i.e., using the extra-maze cues to navigate themaze), regardless of whether a trial was rewarded or not.Neural activity during the receipt or non-receipt of reward when learning (A vs. B):Isolated the neural correlates of reward experiences during spatial learning bycomparing neural activity associated with the receipt or non-receipt of rewardat the end of trials in condition A with neural activity associated with the samereward experience at the end of trials in condition B.

.7. Exploratory analyses

To ensure that any confusion or additional cognitive effort in condition B wasot producing activation in condition A, neural activity during searching and theeceipt of reward in condition A was also compared to neural activity during theame events in control condition C (A vs. C). We also tested (1) whether participantsngaged medial temporal areas when attempting to learn, (2) whether they engagedesolimbic areas during the anticipation of reward, and (3) whether mesolimbic

ngagement depended on reward and condition. Thus, ANCOVAs were performedn the following contrasts:

Searching when attempts at spatial learning were rendered impossible (B vs. C): Iden-tified neural activity associated with attempting to use the extra-maze cues forspatial learning when learning was experimentally rendered impossible by therandomization of cues in condition B.Reward anticipation (A vs. B): Identified neural activity associated with rewardanticipation during spatial learning. This contrast was appropriate because learn-

ing, and therefore reward anticipation, was only possible in condition A. Rewardswere not based on performance in condition B since cue randomization madelearning impossible and reward frequency was based only on performance incondition A.Reward-specific activation (no reward vs. reward) in condition A: Identified theeffects of reward on neural activity when spatial learning was possible (i.e., com-

−14 21 −1 2.49 0.01 A < C−16 14 10 2.72 0.006 A < C

paring activity during no receipt of reward at the end of trials with the receipt ofreward and the end of trials in condition A).

• Reward-specific activation (no reward vs. reward) in condition B: Identified theeffects of reward on neural activity when spatial learning was attempted but ren-dered impossible (i.e., comparing activity during no reward at the end of trialswith reward and the end of trials in condition B).

Finally, to ensure that our findings reported for the primary contrast (A vs. B)were not driven by the fixed order of conditions across participants, paired t-testswere used to compare neural activity associated with searching and with rewardexperiences at the end of trials during the first and second presentation of theseconditions.

3. Results

3.1. Participants

One person moved excessively during scanning and was there-fore excluded from further analysis. The participants who remainedfor statistical analyses included 25 healthy adults, 21 men and 4women, 21–44 (mean = 32.5 ± 7.6) years of age; 10 were Caucasian,10 African American, 4 Hispanic, and 1 was Asian American. Theywere from households of primarily medium socioeconomic status(SES = 33.7 ± 14, possible range 0–64) (Hollingshead, 1975). All butone of the participants were right-handed (Oldfield, 1971).

3.2. Behavioral performance

Age, sex, and minority status did not account for significant aproportion of variance in spatial learning in either model (Ps > 0.1)and were therefore eliminated from the models. A significant maineffect of Run in the first model (t(23) = 8.16, P < 0.01) indicated thatparticipants required less time to complete run 2 (8011 ± 3797 ms)compared to run 1 (11059 ± 6911). A significant main effect of Runin the second model (t(23) = 9.62, P < 0.01) indicated that partici-pants required fewer trials (attempts) to obtain all 8 rewards in run

2 (mean rate = 0.64 ± 0.3) compared to run 1 (0.48 ± 0.2). Thus, spa-tial learning, defined by a decline in latency and an increased rateof reward, improved over the two scanning runs. Across both scanruns, however, an average of 45% of the trials were not rewarded(range: 0–80%), prompting our assessment of neural activity

2916 R. Marsh et al. / Neuropsycholog

Fig. 3. Average brain activity during searching: These are t-maps for FMRI signal dur-ing searching in condition A compared to control conditions B (a) and C (b). Increasesin signal during condition A are shown in red and increases during the control condi-tions are shown in blue. Searching in condition A relative to condition B, regardlessof whether the trial was rewarded or not, activated medial temporal and parietalareas involved in spatial learning. (c) A t-map for FMRI signal during searching incondition B compared to C. Increases in signal during B are in red and increasesdPr

a(

3

SwptgP(m

lt

uring C are in Blue. STG, superior temporal gyrus; LIP, lateral inferoparietal area;CC, posterior cingulate cortex; PHi, parahippocampus; Cd, caudate nucleus; RSC,etrosplenial cortex.

ssociated with both the receipt and the non-receipt of rewardbelow).

.3. A priori hypothesis testing

Neural activity during searching in spatial learning (A vs. B):earching in condition A relative to condition B, regardless ofhether the trial was rewarded or not, activated temporal andarietal cortices (Fig. 3a, and Table 2), including bilateral superioremporal gyrus (STG; P < 0.001, BA 22), bilateral parahippocampalyrus (P < 0.001, BA 36), and right lateral inferoparietal cortex (LIP,= 0.01, BA 40). Searching also activated the posterior cingulate

PCC, BA31), precuneus (P < 0.001, BA 7/31, Fig. 3a), and primaryotor cortices (PMC, P < 0.001, BA 4, Fig. 3a).Neural activity during the receipt and non-receipt of reward when

earning (A vs. B): The receipt of reward in condition A vs. B deac-ivated the left hippocampus, which was equivalent to activating

ia 48 (2010) 2912–2921

this region more during the receipt of reward in condition B,when learning was impossible, than in condition A, when learningoccurred (P = 0.007, Fig. 4a, and Table 3). However, a small por-tion of the right hippocampus was activated in association withthe failure to receive reward at the end of trials in condition B vs.A (P = 0.008, Fig. 4d, and Table 3). The relatively greater activationof the hippocampus in condition B relative to condition A suggeststhat at the end of trials, the confusion, effort, or difficulty of condi-tion B relative to condition A could have activated the hippocampus,independent of the receipt of reward. Alternatively, the effects ofreward and non-reward on hippocampus activation at the end oftrials in condition B could be lateralized, with unanticipated rewardactivating the left hippocampus and the failure to receive a reward(unrewarded trials) activating the right hippocampus.

3.4. Exploratory analyses

Neural activity during searching in spatial learning (A vs. C): Sim-ilar to findings for the contrast of condition A with B, searchingin condition A relative to condition C also activated temporal andparietal cortices (Fig. 3b, and Table 2), including bilateral superiortemporal gyrus (STG; P < 0.001, BA 22), bilateral parahippocampalgyrus (P < 0.001, BA 36), and right lateral inferoparietal cortex (LIP,P = 0.01, BA 40). This contrast also revealed greater activation of theposterior cingulate and restrosplenial cortices (RSC) (P < 0.001, BA30/31, Fig. 3b), and greater activation of the mid-cingulate cortex(MCC, P < 0.01, BA 31, Fig. 3b).

Neural activity during searching when attempts at spatial learningwere rendered impossible (B vs. C): Searching in condition B relativeto C activated the PMC, MCC, and PCC bordering on the precuneusand RSC. In addition, searching deactivated the body and head ofthe left caudate nucleus (Ps < 0.01) and left nucleus accumbens(Ps < 0.01) or, equivalently, activated these regions in condition Crelative to conditions A (Fig. 3b) and B (Fig. 3c).

Neural activity during the receipt of reward when spatial learn-ing was possible (A vs. C): Activity in the left caudate nucleus andputamen was greater at the end of the trial in condition C than incondition A during both reward (Ps = 0.01, Fig. 4b) and no reward(Ps = 0.001, Fig. 4e).

Reward-specific activation (no reward vs. reward): Maps of neuralactivity during no reward compared to reward within conditions Aand B revealed greater activity in the left hippocampus and rightamygdala when not rewarded in condition B (Ps ≤ 0.003, Fig. 5b),but not in condition A (Fig. 5a). Thus, hippocampus activation wasgreater when not receiving reward than when receiving rewardin condition B, when learning was rendered impossible. In addi-tion, an interaction of condition (A or B) with reward (rewarded orunrewarded trials) was detected in the left hippocampus (P < 0.01,Fig. 5c), confirming statistically that activation of this region wasindeed specific to condition B when participants were not rewardedand learning was impossible. Finally, a main effect of reward wasdetected in premotor cortices bilaterally, indicating greater acti-vation of these regions during non-rewarded trials compared torewarded ones in conditions A (Fig. 3a) and B (Fig. 3b)—i.e., regard-less of whether learning was possible and whether rewards wereanticipated.

Reward anticipation (A vs. B): The anticipation of reward (begin-ning after the first 10% of an arm was traversed and extending untilparticipants reached its baited area) was not associated with activ-ity in mesolimbic regions or in any other brain areas in the activelearning (A) or control (B) conditions (maps not shown).

Order effects: A comparison of contrast maps (A vs. B) of neuralactivity during searching and during the receipt and non-receipt ofreward in run 1 with neural activity during these events in run 2suggested that order and practice did not contribute to our findingsof temporo-parietal activation during searching in spatial learning

R. Marsh et al. / Neuropsychologia 48 (2010) 2912–2921 2917

Fig. 4. Average brain activity during the receipt of reward and no reward: These are t-maps for fMRI signal during the receipt of reward in condition A compared to controlc mpara a) ande ing ths audat

ao

4

lcgrpodao

4

mdoi2

TA

onditions B (a) and C (b), and for fMRI signal during no reward in condition A core shown in blue. The hippocampus was activated during the receipt of reward (xperience itself during spatial learning. Also shown are t-maps for fMRI signal durignal during B are in red and increases during C are in blue. Hi, hippocampus; Cd, c

nd of hippocampal activation during the receipt and non-receiptf reward in condition B (maps not shown).

. Discussion

We used a novel VR paradigm to investigate the neural corre-ates of spatial learning in healthy individuals. In the active learningondition (A), participants had to use extra-maze cues to navi-ate the maze and find hidden rewards. In control condition B,andomization of the elements of the extra-maze scene preventedarticipants from using the extra-maze cues to learn the locationf rewards. As expected, participants demonstrated spatial learninguring the active conditions by taking less time and making fewerttempts to find all 8 rewards on the second compared to the firstf two runs.

.1. Neural activity during the search phase of spatial learning

Also as predicted, imaging results revealed that searching the

aze during spatial learning in condition A relative to control con-

ition B, when learning was impossible, activated large expansesf temporal and parietal cortices that participate in spatial nav-gation (Aguirre et al., 1996; Ekstrom et al., 2003; Hartley et al.,003; Hassabis et al., 2009; Iaria et al., 2008; Spiers & Maguire,

able 3ctivations associated with spatial learning during the receipt of reward, and no reward.

Activated regions Location

Side # of voxels in cluster P

x

RewardHippocampus L 34 −3Caudate nucleus L 25 −Lenticular nucleus (or putamen) L 64 −2

No rewardHippocampus R 43 3Putamen L 4900 −2

R 490 2Caudate nucleus L −Thalamus L/R

No reward vs. reward in BHippocampus L 2022 −2Amygdala R 2

ed to conditions B (d) and C (e). Increases in signal during the control conditionsno receipt of reward (d) in condition B and therefore associated with the reward

e receipt of reward (c) and no reward (f) in condition B compared to C. Increases ine nucleus; Put, putamen.

2006b; Wolbers & Buchel, 2005), including bilateral STG (BA 22),bilateral parahippocampal gyrus (BA 36), and right LIP (BA 40), aswell as the PCC and RSC. Exploratory analyses revealed the samepatterns of temporo-parietal activation during searching in condi-tion A relative to control condition C, when an arrow indicating thepath to follow minimized effort and confusion associated with theattempted use of extra-maze cues to navigate and find rewards.The similarity in findings across these contrasts (A vs. B and A vs.C) indicates that participants were likely engaging the same strate-gies across the task conditions. Although we did not counterbalancethe order of conditions across participants, these similar patterns oftemporo-parietal activation during searching, along with our find-ings of non-significant order effects, suggest that order and practicedid not contribute to these findings. Finally, we found that the PCCand RSC activated more in condition B relative to condition C, indi-cating that these regions activated when attempting to use spatialcues to navigate, even when success was rendered impossible incondition B by the randomization of extra-maze cues.

Our findings suggest that the parahippocampal cortex may

play a greater role than the hippocampus in processing spatialinformation, in contrast to findings from previous fMRI stud-ies that have implicated the hippocampus in spatial navigation(Gron, Wunderlich, Spitzer, Tomczak, & Riepe, 2000; Hartley etal., 2003; Iaria, Chen, Guariglia, Ptito, & Petrides, 2007). The tasks

eak location Significance testing Contrast

y z t statistic P

2 −16 −13 2.62 0.007 A < B8 14 9 2.11 0.01 A < C0 4 5 2.36 0.01 A < C

2 −14 −18 2.59 0.008 A < B2 4 2 3.99 <0.001 A < C8 −5 9 3.02 0.001 A < C4 16 −1 3.16 0.001 A < C0 0 7 3.43 0.001 A < C

0 −12 −16 4.08 <0.001 Reward ↑4 −1 −13 3.01 0.003 Reward ↑

2918 R. Marsh et al. / Neuropsycholog

Fig. 5. Average brain activity during no reward compared to reward: These are t-mapsfor fMRI signal during no reward compared to reward in conditions A (a) and B(b). Increases in signal associated with no reward are shown in red and increasesdwrH

usi(tbowtfcrppnnararusp

sfip(twrsrrapc

with the parahippocampus and hippocampus (Blatt, Pandya, &

uring the receipt of reward are shown in blue. The hippocampus and amygdalaere activated during no reward at the end of trials compared to the receipt of

eward at the end of trials in condition B, when learning was rendered impossible.i, hippocampus; Amy, Amygdala; PMC, primary motor cortex.

sed in these prior studies differed considerably from ours, in thatome included separate learning and retrieval conditions, and allncluded less rigorously specified control conditions. One studyIaria et al., 2007), for example, required participants to learnhe spatial relationships between landmarks in a VR environmentefore the experiment and then compared activity during retrievalf their learned, mental representation of the maze environmenthen navigating to those landmarks with activity during a con-

rol condition that was similar to our condition C (i.e., participantsollowed arrows along a path). The landmarks used in the controlondition, however, differed from those used during learning andetrieval. This comparison produced activation of the anterior hip-ocampus during learning and of the posterior hippocampus andarahippocampus during retrieval. In contrast, our participants didot undergo separate learning and retrieval procedures, they wereot instructed to learn the spatial relationships between the cues,nd they experienced identical elements of the extra-maze envi-onment during the active and control tasks. They were also likelyttending to the larger extra-maze environment and to the spatialelationships among cues, rather than to the details of the partic-lar cues and landmarks that were uncontrolled in the previoustudy. When searching the maze, our participants activated thearahippocampus rather than the hippocampus.

Our finding that the parahippocampus activates during theearch phase of spatial learning is consistent with prior fMRIndings suggesting that the parahippocampal cortex respondsreferentially to scenes or a spatial layout, but not to single objectsEpstein, Higgins, Jablonski, & Feiler, 2007), that it processes con-extual associations (Aminoff, Gronau, & Bar, 2007), and is activatedhen participants discriminate between two virtual reality envi-

onments (Hassabis et al., 2009). Consistent with our findings,ingle-unit recording studies have reported that neurons in theodent homolog of the parahippocampus strongly encode spatial

elationships among visual stimuli (Burwell & Hafeman, 2003). Inddition, lesion studies have demonstrated a role for the parahip-ocampal cortex in binding objects to a particular environmentalontext rather than in memory for particular objects themselves

ia 48 (2010) 2912–2921

(Norman & Eacott, 2005). Finally, our finding of parahippocampalactivation during spatial learning is consistent with findings thatlesions to parahippocampal areas, including the entorhinal cortex,impair spatial learning in rodents on the Morris water maze task(Kopniczky et al., 2006).

Our finding that spatial navigation in humans engages theparahippocampus rather than the hippocampus may at first seemto differ from findings of the radial-arm maze studies in rodents thatimplicate the hippocampus in spatial learning (Becker, Walker, &Olton, 1980; Packard et al., 1989). These animal studies showed thatlesions of the hippocampus impair the ability of rodents to navigatea radial-arm maze accurately and find hidden rewards. In addi-tion, the hippocampus is known to participate in path integration,the updating of one’s position in the environment based on self-motion cues (Etienne & Jeffery, 2004), self-motion cues determinethe firing properties of hippocampal place cells (Wiener, Paul, &Eichenbaum, 1989), and hippocampal-dependent spatial learningby rodents within a radial-arm maze has been reported to requiremovement (White, 2004). Thus, movement is likely required toengage the hippocampus during navigation. The ‘searching’ phasein our task included very brief periods of passive scanning withoutmovement (i.e., participants only spent 0.003% of the total searchtime motionless in the center of the maze before traversing anarm), possibly contributing to our failure to detect hippocampusactivation during that portion of the task.

The hippocampus likely works in concert with anatomicallyconnected regions, including parahippocampal areas and retros-plenial cortices, to create a map of the environment (Sharp, 1999).Single-cell recordings indicate that the retrosplenial cortex con-tains head-direction cells that fire when an animal’s head points ina specific direction in an open field (Cho & Sharp, 2001). Moreover,the regions that most commonly activate in human imaging studiesof spatial navigation, including virtual reality-based fMRI stud-ies, are the parahippocampal and retrosplenial cortices (Ekstromet al., 2003; Hassabis et al., 2009; Maguire et al., 1998; Spiers &Maguire, 2006a; Wolbers, Weiller, & Buchel, 2004). Patients withdamage to the RSC can recognize landmarks, but they have troublefinding their way around familiar environments (Ino et al., 2007),suggesting that the RSC plays an important role in processing thedirectional information between landmarks. These findings fromlesion studies are consistent with our imaging findings that theRSC activated during the searching phase of the task, particularlywhen contrasted with activity during condition C, when partic-ipants were simply following an arrow and not using internallygenerated processing of directional information.

Participants also engaged the LIP area during the searchingphase of the task. The LIP, a subdivision of the inferior parietallobe, is involved in visual attention (Corbetta & Shulman, 2002).Unilateral lesions of the LIP produce hemispatial neglect, a condi-tion that prevents responding to stimuli in the contralateral visualspace (Driver & Vuilleumier, 2001) and interferes with the mainte-nance of attention directed toward spatial information (Malhotra,Coulthard, & Husain, 2009). In addition, electrophysiological stud-ies in animals suggest that neurons in the LIP integrate spatialinformation with information concerning self-motion, includingshifting attention to salient or task-relevant objects (Balan &Gottlieb, 2006), representing the location of cues and associatedmotor responses (Oristaglio, Schneider, Balan, & Gottlieb, 2006),and encoding navigational epochs in a maze (Nitz, 2006). The lat-eral parietal cortex is anatomically connected to the retrosplenialcortex (Kobayashi & Amaral, 2003) and reciprocally connected

Rosene, 2003; Clower, West, Lynch, & Strick, 2001). Together,these considerations suggest that the LIP plays a crucial role insupporting the sensory and attentional requirements of spatialnavigation.

cholog

cvt2iepdnsc

4

aleccwolbrlcil

pwaatelrornpsrfh

goTrtuatGsbDrndnwb

R. Marsh et al. / Neuropsy

Participants also activated the STG during searching in the activeompared to the control conditions. The STG supports higher-orderisual processing (Kandel, Schwartz, & Jessell, 1991), includinghe perception of biological motion (Servos, Osu, Santi, & Kawato,002). In addition, fMRI studies have shown that the STG is involved

n imagined locomotion (Jahn et al., 2004), spatial orienting, andxploration (Himmelbach, Erb, & Karnath, 2006). Finally, partici-ants in our study activated mid-cingulate and premotor corticesuring the searching phase of the task in conditions A and B, butot in condition C, likely reflecting the greater requirements to con-truct a motor plan for navigation in conditions A and B than inondition C (Nakayama, Yamagata, Tanji, & Hoshi, 2008).

.2. Neural activity associated with reward

Our VR paradigm allowed us to assess brain activity associ-ted with reward when spatial learning was possible in the activeearning condition (A) and when it was attempted but renderedxperimentally impossible by the randomization of extra-mazeues in condition B (Table 1). By comparing brain activity asso-iated reward and no reward at the end of trials in condition Aith brain activity during the same reward experiences at the end

f trials in condition B, we were able to identify the neural corre-ates of reward experiences during spatial learning. By comparingrain activity during the receipt of reward at the end of trials vs. noeceipt of reward at the end of trials in condition A (when spatialearning was possible) with brain activity associated with the sameontrast in condition B (when spatial learning was attempted butmpossible), we were able to disentangle the effects of reward onearning from the effects of reward itself, in the absence of learning.

Activation of the left hippocampus was greater when partici-ants were rewarded at the end of trials in control condition B,hen learning was impossible, relative to condition A (Fig. 4), but

ctivation was greater when not rewarded than when rewardedt the end of trials in condition B (Fig. 5). These findings suggesthat hippocampus activation was associated with the confusion,ffort, or difficulty of condition B and not associated with spatialearning which was rendered impossible in this condition by theandomization of the extra-maze cues and the participants’ newrientation at the start of each trial. Finally, exploratory analysesevealed that activity of the caudate nucleus when rewarded andot rewarded at trial completion was greater in condition C com-ared with conditions A and B, likely indicating the formation of atimulus–response association between the arrow and an approachesponse, and consistent with extensive evidence indicating a roleor the dorsal striatum in S-R habit learning in lower animals andumans (Packard & Knowlton, 2002).

Activation of the left hippocampus and the right amygdala wasreater when not rewarded compared to when rewarded at the endf trials specifically within condition B, but not condition A (Fig. 5).he hippocampus and amygdala are components of the mesolimbiceward processing system (Heimer & Van Hoesen, 2006). Althoughhe roles of these regions in reward processing are not wellnderstood, accumulating evidence from single-unit recordings innimals suggests that the hippocampus is functionally connectedo the ventral tegmental area (VTA) (Lisman & Grace, 2005; Thierry,ioanni, Degenetais, & Glowinski, 2000), which projects dopamineignals most strongly to the ventral striatum (nucleus accum-ens) but also to the hippocampus and amygdala (Wise, 2004).opamine is intimately associated with reward processing and

eward-based learning (Schultz, 2007; Wise, 2004). Findings from

europhysiological studies suggest that reward unpredictabilityetermines the phasic responses of ventral striatal dopaminergiceurons to rewarding stimuli (Schultz, 2007). These neurons firehen rewards occur without being predicted, but their firing drops

elow baseline when predicted rewards are omitted. Activation of

ia 48 (2010) 2912–2921 2919

the left hippocampus, but not the amygdala, was also associatedwith the receipt of reward at the end of trials in condition B com-pared to condition A (Fig. 4a). Reward receipt in condition B wasunexpected because of the randomization of the extra-maze cues.Although lesion, neurophysiological, and fMRI studies typicallyimplicate the nucleus accumbens in processing reward predictionerrors (Cardinal, Pennicott, Sugathapala, Robbins, & Everitt, 2001;Delgado, 2007; Schoenbaum & Setlow, 2003), the nucleus accum-bens did not activate with the hippocampus during the unexpectedreceipt of rewards in condition B or deactivate (i.e., drop belowbaseline) when participants were not rewarded at the end of tri-als in condition B. Thus, the receipt and non-receipt of rewardsin condition B likely activated the hippocampus via a system ormechanism that is independent of the ventral striatum (Bunzeck& Duzel, 2006; Lisman & Grace, 2005; Wittmann, Bunzeck, Dolan,& Duzel, 2007) and independent of learning in general. Indeed,novel events that are devoid of behavioral relevance, such as thereceipt of rewards in condition B, are known to activate the hip-pocampus but not the substantia nigra or, presumably, the ventralstriatum (Stoppel et al., 2009). Moreover, the hippocampus con-tains neurons that respond to reward delivery (Watanabe & Niki,1985), and human electrophysiological studies demonstrate thatthe hippocampus encodes and computes an uncertainty signal(Vanni-Mercier et al., 2009), consistent with our findings and withthe possibility that hippocampal encoding of uncertainty and unex-pected rewards may be independent of dopaminergic firing and themore classical system that subserves prediction error.

Alternatively, greater hippocampal engagement during thereceipt of rewards in condition B compared to condition A mayhave reflected encoding of the novel (random) configuration of thecues in condition B, consistent with the role of the hippocampusin detecting associative novelty—when familiar stimuli are newlyarranged (Nyberg, 2005). The hippocampus was perhaps acting as amatch–mismatch detector in condition B, generating signals (e.g.,activating) when predictions derived from previous experiences(e.g., the arrangement of the cues in condition A) were violatedby current inputs (e.g., the random configuration of the cues incondition B) (Kumaran & Maguire, 2007). However, hippocam-pal activation was only observed during the reward experiencesat the end of trials in condition B, rather than when participantswere searching the maze. Thus, we believe that at the end of tri-als, the confusion, effort, or difficulty of condition B relative tocondition A activated the hippocampus, and that this activationwas not associated with learning, which was rendered experi-mentally impossible by the randomization of extra-maze cues incondition B.

5. Conclusions

Our findings have important implications for understanding theneural correlates of reward-based spatial learning. Our inclusionof two different control conditions and our isolation of differentevents that occurred during maze navigation allowed for the pre-cise study and disentangling of the neural mechanisms that supportspatial navigation and reward processing. Participants engagedmesial temporal, parietal, and retrosplenial areas when searchingthe maze, consistent with findings from prior studies of spatial nav-igation. Unlike tasks used in other studies, however, our task didnot require participants to process the details of the spatial cuesbut rather required them to process and remember the spatial lay-

out of the extra-maze environment. Thus, participants activated theparahippocampus rather than the hippocampus during the search-ing phase of the task. Unlike prior studies, we assessed the neuralcorrelates of reward receipt during spatial learning and found thatparticipants engaged the hippocampus when learning was ren-

2 cholog

dpmhcOtslptsttat

A

Kt(PoIf

R

A

A

A

B

B

B

B

B

C

C

C

C

D

D

E

E

E

F

920 R. Marsh et al. / Neuropsy

ered impossible in condition B. Although we did not offer thearticipants actual monetary rewards, which may have decreasedotivation and therefore performance on the task, previous studies

ave shown that virtual and abstract rewards also engage rewardircuits (O’Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001).ur novel design also allowed us to assess the differential contribu-

ions of the hippocampus and other temporo-parietal areas to theearching and reward processing aspects of reward-based spatialearning. Finally, this translational research paradigm will facilitatearallel studies in animals and humans to establish the processeshat are common and unique to each of the two species. Animaltudies using the task will then permit cellular and molecular inves-igations of learning processes that are not possible in humans, buthat may nevertheless inform experimental manipulations, suchs the development and testing of novel medications, during use ofhe identical task in humans.

cknowledgements

This work was supported in part by NIMH grants K02-74677 and01-MH077652, a NIH/NIBIB grant 1R03EB008235, a grant from

he National Alliance for Research on Schizophrenia and DepressionNARSAD), by funding from the Sackler Institute for Developmentalsychobiology, Columbia University and from the Opening Projectf Shanghai Key Laboratory of Functional Magnetic Resonancemaging, East China Normal University. We thank Tiago V. Maiaor his valuable comments.

eferences

dcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. (2006).Reward-motivated learning: Mesolimbic activation precedes memory forma-tion. Neuron, 50(3), 507–517.

guirre, G. K., Detre, J. A., Alsop, D. C., & D’Esposito, M. (1996). The parahippocampussubserves topographical learning in man. Cerebral Cortex, 6(6), 823–829.

minoff, E., Gronau, N., & Bar, M. (2007). The parahippocampal cortex mediatesspatial and nonspatial associations. Cerebral Cortex, 17(7), 1493–1503.

alan, P. F., & Gottlieb, J. (2006). Integration of exogenous input into a dynamicsalience map revealed by perturbing attention. Journal of Neuroscience, 26(36),9239–9249.

ecker, J. T., Walker, J. A., & Olton, D. S. (1980). Neuroanatomical bases of spatialmemory. Brain Research, 200(2), 307–320.

latt, G. J., Pandya, D. N., & Rosene, D. L. (2003). Parcellation of cortical afferentsto three distinct sectors in the parahippocampal gyrus of the rhesus monkey:An anatomical and neurophysiological study. Journal of Comparative Neurology,466(2), 161–179.

unzeck, N., & Duzel, E. (2006). Absolute coding of stimulus novelty in the humansubstantia nigra/VTA. Neuron, 51(3), 369–379.

urwell, R. D., & Hafeman, D. M. (2003). Positional firing properties of postrhinalcortex neurons. Neuroscience, 119(2), 577–588.

ardinal, R. N., Pennicott, D. R., Sugathapala, C. L., Robbins, T. W., & Everitt, B. J.(2001). Impulsive choice induced in rats by lesions of the nucleus accumbenscore. Science, 292(5526), 2499–2501.

ho, J., & Sharp, P. E. (2001). Head direction, place, and movement correlates for cellsin the rat retrosplenial cortex. Behavioral Neuroscience, 115(1), 3–25.

lower, D. M., West, R. A., Lynch, J. C., & Strick, P. L. (2001). The inferior parietallobule is the target of output from the superior colliculus, hippocampus, andcerebellum. Journal of Neuroscience, 21(16), 6283–6291.

orbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-drivenattention in the brain. Nature Reviews Neuroscience, 3(3), 201–215.

elgado, M. R. (2007). Reward-related responses in the human striatum. Annals ofthe New York Academy of Science, 1104, 70–88.

river, J., & Vuilleumier, P. (2001). Perceptual awareness and its loss in unilateralneglect and extinction. Cognition, 79(1–2), 39–88.

kstrom, A. D., Kahana, M. J., Caplan, J. B., Fields, T. A., Isham, E. A., Newman, E. L.,et al. (2003). Cellular networks underlying human spatial navigation. Nature,425(6954), 184–188.

pstein, R. A., Higgins, J. S., Jablonski, K., & Feiler, A. M. (2007). Visual scene process-ing in familiar and unfamiliar environments. Journal of Neurophysiology, 97(5),3670–3683.

tienne, A. S., & Jeffery, K. J. (2004). Path integration in mammals. Hippocampus,14(2), 180–192.

irst, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (2002). Structured clini-cal interview for DSM-IV-TR Axis I disorders, research version, non-patient edition(SCID-I/NP). New York: Biometrics Research, New York State Psychiatric Insti-tute.

ia 48 (2010) 2912–2921

Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C.(1995). Improved assessment of significant activation in functional magneticresonance imaging (fMRI): Use of a cluster-size threshold. Magnetic ResonanceMedicine, 33, 636–647.

Friston, K. J., Williams, S., Howard, R., Frackowiak, R. S., & Turner, R. (1996).Movement-related effects in fMRI time-series. Magnetic Resonance Medicine, 35,346–355.

Friston, K. J., Worsley, K. J., Frackowiak, R. S. J., Mazziotta, J. C., & Evans, A. C. (1994).Assessing the significance of focal activations using their spatial extent. HumanBrain Mapping, 1, 214–220.

Gron, G., Wunderlich, A. P., Spitzer, M., Tomczak, R., & Riepe, M. W. (2000). Brain acti-vation during human navigation: Gender-different neural networks as substrateof performance. Nature Neuroscience, 3(4), 404–408.

Hartley, T., Maguire, E. A., Spiers, H. J., & Burgess, N. (2003). The well-worn route andthe path less traveled: Distinct neural bases of route following and wayfindingin humans. Neuron, 37(5), 877–888.

Hassabis, D., Chu, C., Rees, G., Weiskopf, N., Molyneux, P. D., & Maguire, E. A. (2009).Decoding neuronal ensembles in the human hippocampus. Current Biology,19(7), 546–554.

Heimer, L., & Van Hoesen, G. W. (2006). The limbic lobe and its output channels:Implications for emotional functions and adaptive behavior. Neuroscience andBiobehavioral Reviews, 30(2), 126–147.

Himmelbach, M., Erb, M., & Karnath, H. O. (2006). Exploring the visual world: Theneural substrate of spatial orienting. Neuroimage, 32(4), 1747–1759.

Hollingshead, A. B. (1975). Four-factor index of social status. New Haven, CT: YaleUniversity Press.

Iaria, G., Chen, J. K., Guariglia, C., Ptito, A., & Petrides, M. (2007). Retrosplenial andhippocampal brain regions in human navigation: Complementary functionalcontributions to the formation and use of cognitive maps. European Journal ofNeuroscience, 25(3), 890–899.

Iaria, G., Fox, C. J., Chen, J. K., Petrides, M., & Barton, J. J. (2008). Detection ofunexpected events during spatial navigation in humans: Bottom–up atten-tional system and neural mechanisms. European Journal of Neuroscience, 27(4),1017–1025.

Ino, T., Doi, T., Hirose, S., Kimura, T., Ito, J., & Fukuyama, H. (2007). Directional dis-orientation following left retrosplenial hemorrhage: A case report with fMRIstudies. Cortex, 43(2), 248–254.

Ito, R., Robbins, T. W., Pennartz, C. M., & Everitt, B. J. (2008). Functional interactionbetween the hippocampus and nucleus accumbens shell is necessary for theacquisition of appetitive spatial context conditioning. Journal of Neuroscience,28(27), 6950–6959.

Jahn, K., Deutschlander, A., Stephan, T., Strupp, M., Wiesmann, M., & Brandt, T. (2004).Brain activation patterns during imagined stance and locomotion in functionalmagnetic resonance imaging. Neuroimage, 22(4), 1722–1731.

Jazzard, P., Matthews, P. M., & Smith, S. M. (2002). Functional MRI—An introductionto methods. New York: Oxford University.

Kandel, E. R., Schwartz, J. H., & Jessell, T. M. (Eds.). (1991). Principles of neural science.East Norwalk, CT: Appleton & Lange.

Kobayashi, Y., & Amaral, D. G. (2003). Macaque monkey retrosplenial cortex. II.Cortical afferents. Journal of Comparative Neurology, 466(1), 48–79.

Kopniczky, Z., Dochnal, R., Macsai, M., Pal, A., Kiss, G., Mihaly, A., et al. (2006). Alter-ations of behavior and spatial learning after unilateral entorhinal ablation ofrats. Life Science, 78(23), 2683–2688.

Kumaran, D., & Maguire, E. A. (2007). Match mismatch processes underlie humanhippocampal responses to associative novelty. Journal of Neuroscience, 27(32),8517–8524.

Lisman, J. E., & Grace, A. A. (2005). The hippocampal-VTA loop: Controlling the entryof information into long-term memory. Neuron, 46(5), 703–713.

Maguire, E. A., Burgess, N., Donnett, J. G., Frackowiak, R. S., Frith, C. D., & O’Keefe, J.(1998). Knowing where and getting there: A human navigation network. Science,280(5365), 921–924.

Malhotra, P., Coulthard, E. J., & Husain, M. (2009). Role of right posterior parietal cortexin maintaining attention to spatial locations over time. Brain.

Morrell, C. H., Pearson, J. D., & Brant, L. J. (1997). Linear transformations of linearmixed-effects models. American Statistician, 51, 338–343.

Nakayama, Y., Yamagata, T., Tanji, J., & Hoshi, E. (2008). Transformation of a virtualaction plan into a motor plan in the premotor cortex. Journal of Neuroscience,28(41), 10287–10297.

Nitz, D. A. (2006). Tracking route progression in the posterior parietal cortex. Neuron,49(5), 747–756.

Norman, G., & Eacott, M. J. (2005). Dissociable effects of lesions to the perirhinalcortex and the postrhinal cortex on memory for context and objects in rats.Behavioral Neuroscience, 119(2), 557–566.

Nyberg, L. (2005). Any novelty in hippocampal formation and memory? CurrentOpinion in Neurology, 18(4), 424–428.

O’Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001). Abstractreward and punishment representations in the human orbitofrontal cortex.Nature Neuroscience, 4(1), 95–102.

Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburghinventory. Neuropsychologia, 9, 97–113.

Olton, D. S., & Samuelson, R. J. (1976). Remembrance of places past: Spatial memoryin rats. Journal of Experimental Psychology, 2, 97–115.

Oristaglio, J., Schneider, D. M., Balan, P. F., & Gottlieb, J. (2006). Integration of visu-ospatial and effector information during symbolically cued limb movementsin monkey lateral intraparietal area. Journal of Neuroscience, 26(32), 8310–8319.

cholog

P

P

P

S

S

S

S

S

S

S

R. Marsh et al. / Neuropsy

ackard, M. G., Hirsh, R., & White, N. M. (1989). Differential effects of fornix andcaudate nucleus lesions on two radial maze tasks: Evidence for multiple memorysystems. Journal of Neuroscience, 9(5), 1465–1472.

ackard, M. G., & Knowlton, B. J. (2002). Learning and memory functions of the BasalGanglia. Annual Reviews of Neuroscience, 25, 563–593.

ackard, M. G., & White, N. M. (1991). Dissociation of hippocampus and caudatenucleus memory systems by posttraining intracerebral injection of dopamineagonists. Behavioral Neuroscience, 105, 295–306.

choenbaum, G., & Setlow, B. (2003). Lesions of nucleus accumbens disrupt learningabout aversive outcomes. Journal of Neuroscience, 23(30), 9833–9841.

chultz, W. (2007). Behavioral dopamine signals. Trends in Neuroscience, 30(5),203–210.

ervos, P., Osu, R., Santi, A., & Kawato, M. (2002). The neural substrates of biologicalmotion perception: An fMRI study. Cerebral Cortex, 12(7), 772–782.

harp, P. E. (1999). Complimentary roles for hippocampal versus subicu-lar/entorhinal place cells in coding place, context, and events. Hippocampus, 9(4),432–443.

piers, H. J., & Maguire, E. A. (2006a). Spontaneous mentalizing during an interactive

real world task: An fMRI study. Neuropsychologia, 44(10), 1674–1682.

piers, H. J., & Maguire, E. A. (2006b). Thoughts, behaviour, and brain dynamicsduring navigation in the real world. Neuroimage, 31(4), 1826–1840.

toppel, C. M., Boehler, C. N., Strumpf, H., Heinze, H. J., Hopf, J. M., Duzel, E., et al.(2009). Neural correlates of exemplar novelty processing under different spatialattention conditions. Human Brain Mapping, 30(11), 3759–3771.

ia 48 (2010) 2912–2921 2921

Thierry, A. M., Gioanni, Y., Degenetais, E., & Glowinski, J. (2000). Hippocampo-prefrontal cortex pathway: Anatomical and electrophysiological characteristics.Hippocampus, 10(4), 411–419.

Vanni-Mercier, G., Mauguiere, F., Isnard, J., & Dreher, J. C. (2009). The hippocampuscodes the uncertainty of cue-outcome associations: An intracranial electrophys-iological study in humans. Journal of Neuroscience, 29(16), 5287–5294.

Watanabe, T., & Niki, H. (1985). Hippocampal unit activity and delayed response inthe monkey. Brain Research, 325(1–2), 241–254.

White, N. M. (2004). The role of stimulus ambiguity and movement in spatialnavigation: A multiple memory systems analysis of location discrimination.Neurobiology of Learning and Memory, 82(3), 216–229.

Wiener, S. I., Paul, C. A., & Eichenbaum, H. (1989). Spatial and behavioral correlatesof hippocampal neuronal activity. Journal of Neuroscience, 9(8), 2737–2763.

Wise, R. A. (2004). Dopamine, learning and motivation. Nature Reviews Neuroscience,5(6), 483–494.

Wittmann, B. C., Bunzeck, N., Dolan, R. J., & Duzel, E. (2007). Anticipation of nov-elty recruits reward system and hippocampus while promoting recollection.Neuroimage, 38(1), 194–202.

Wolbers, T., & Buchel, C. (2005). Dissociable retrosplenial and hippocampal contribu-tions to successful formation of survey representations. Journal of Neuroscience,25(13), 3333–3340.

Wolbers, T., Weiller, C., & Buchel, C. (2004). Neural foundations of emerging routeknowledge in complex spatial environments. Brain Research. Cognitive BrainResearch, 21(3), 401–411.