DECISION NEUROSCIENCE

This page intentionally left blank

DECISIONNEUROSCIENCE

AN INTEGRATIVE PERSPECTIVE

Edited by

JEAN-CLAUDE DREHER

LEON TREMBLAY

Institute of Cognitive Science (CNRS), Lyon, France

AMSTERDAM • BOSTON • HEIDELBERG • LONDON

NEW YORK • OXFORD • PARIS • SAN DIEGO

SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO

Academic Press is an imprint of Elsevier

Academic Press is an imprint of Elsevier125 London Wall, London EC2Y 5AS, United Kingdom525 B Street, Suite 1800, San Diego, CA 92101-4495, United States50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United StatesThe Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

Copyright © 2017 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical,including photocopying, recording, or any information storage and retrieval system, without permission in writing fromthe publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and ourarrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can befound at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than asmay be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden ourunderstanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers may always rely on their own experience and knowledge in evaluating and using anyinformation, methods, compounds, or experiments described herein. In using such information or methods they should bemindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for anyinjury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use oroperation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication DataA catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN: 978-0-12-805308-9

For information on all Academic Press publicationsvisit our website at https://www.elsevier.com/

Publisher: Mara ConnerAcquisition Editor: April FarrEditorial Project Manager: Timothy BennettProduction Project Manager: Edward TaylorDesigner: Matthew Limbert

Typeset by TNQ Books and Journals

Contents

List of Contributors xiPreface xiii

IANIMAL STUDIES ON REWARDS,PUNISHMENTS, AND DECISION-

MAKING

1. Anatomy and Connectivity of theReward CircuitS.N. HABER

Introduction 3

Prefrontal Cortex 4

The Ventral Striatum 5

Ventral Pallidum 10

The Midbrain Dopamine Neurons 11

Completing the CorticaleBasal Ganglia RewardCircuit 14

References 15

2. Electrophysiological Correlates of RewardProcessing in Dopamine NeuronsW. SCHULTZ

Introduction 21

Basics of Dopamine Reward-Prediction ErrorCoding 21

Subjective Value and Formal Economic UtilityCoding 25

Dopamine and Decision-Making 27

Acknowledgments 29

References 29

3. Appetitive and Aversive Systems in theAmygdalaS. BERNARDI AND D. SALZMAN

Introduction 33

Conclusion 41

References 42

4. Ventral Striatopallidal PathwaysInvolved in Appetitive and AversiveMotivational ProcessesY. SAGA AND L. TREMBLAY

Introduction 47

The Corticobasal Ganglia Functional Circuits: TheRelation Between the Cortex and the BasalGanglia 49

The Direct and Indirect Pathways: From Inhibitionof Competitive Movement to AversiveBehaviors 49

Single-Unit Recording in Awake Animals toInvestigate the Neural Bases 50

Tasks to Investigate Appetitive Approach Behaviorand Positive Motivation 51

Ventral Striatum and Ventral PallidumAre Also Involved in Aversive Behaviors:First Evidence of Local Inhibitory Dysfunction 51

The Ventral Striatum and the Ventral PallidumEncode Aversive Future Events for Preparationof Avoidance and for Controlling Anxiety Level 53

Abnormal Aversive Processing Could BiasDecision-Making Toward PathologicalBehaviors 55

Conclusion 55

Acknowledgments 55

References 55

5. Reward and Decision Encodingin Basal Ganglia: Insights FromOptogenetics and Viral Tracing Studiesin RodentsJ. TIAN, N. UCHIDA AND N. ESHEL

Dopamine 59

Striatum 64

Conclusions 67

References 67

v

6. The Learning and Motivational ProcessesControlling Goal-Directed Action and TheirNeural BasesL.A. BRADFIELD AND B.W. BALLEINE

Learning “What Leads to What”: The Neural Basesof ActioneOutcome Learning 72

How Rewards Are Formed: Neural Bases ofIncentive Learning 74

Deciding What to Do: The Neural Bases ofOutcome Retrieval and Choice 76

How These Circuits Interact: Convergence on theDorsomedial Striatum 77

Conclusion 79

Acknowledgments 79

References 79

7. Impulsivity, Risky Choice, and ImpulseControl Disorders: Animal ModelsT.W. ROBBINS AND J.W. DALLEY

Delayed and Probabilistic Discounting: “ImpulsiveChoice” in Decision-Making 82

Premature Responding in the 5CSRTT 83

Stop-Signal Reaction Time 83

Neural Basis of Impulsivity 84

Neural Substrates of Waiting Impulsivity 84

Neural Substrates of Probability Discounting ofReward: Risky Choice 89

Motor Impulsivity: Stop-Signal Inhibition 90

Conclusion 90

Acknowledgments 91

References 91

8. Prefrontal Cortex in Decision-Making:The PerceptioneAction CycleJ.M. FUSTER

The PerceptioneAction Cycle 96

Inputs to the PerceptioneAction Cycle inDecision-Making 97

Prediction and Preparation Toward Decision 99

Execution of Decision 101

Feedback From Decision: Closure of thePerceptioneAction Cycle 102

References 103

IIHUMAN STUDIES ON MOTIVATION,PERCEPTUAL, AND VALUE-BASED

DECISION-MAKING

9. Reward, Value, and SalienceT. KAHNT AND P.N. TOBLER

Introduction 109

Value 110

Salience 113

Conclusions 118

References 118

10. Computational Principles of ValueCoding in the BrainK. LOUIE AND P.W. GLIMCHER

Introduction 121

Value and Choice Behavior 121

Value Coding in Decision Circuits 123

Context Dependence in Brain and Behavior 125

Neural Computations Underlying ValueRepresentation 127

Temporal Dynamics and CircuitMechanisms 130

Conclusions 133

References 133

11. Spatiotemporal Characteristics andModulators of Perceptual Decision-Makingin the Human BrainM.G. PHILIASTIDES, J.A. DIAZ AND S. GHERMAN

Introduction 137

Factors Affecting Perceptual Decision-Making 139

Conclusion 145

References 145

12. Perceptual Decision-Making: What DoWe Know, and What Do We Not Know?C. SUMMERFIELD AND A. BLANGERO

Introduction 149

What Are Perceptual Decisions? 149

Aims and Scope of This Chapter 150

Decision Optimality 150

Q1: How Is Information Integrated DuringPerceptual Decision-Making? 150

Q2: What Computations Do Cortical NeuronsPerform During Perceptual Decisions? 153

Q3: How Can We Study PerceptualDecision-Making in Humans? 155

Q4: How Do Observers Decide Whento Decide? 157

Q5: How Are Perceptual Decisions Biasedby Prior Beliefs? 158

Conclusions and Future Directions 160

Acknowledgments 160

References 160

13. Neural Circuit Mechanismsof Value-Based Decision-Making andReinforcement LearningA. SOLTANI, W. CHAISANGMONGKON AND X.-J. WANG

Introduction 163

Representations of Reward Value 164

Learning Reward Values 165

CONTENTSvi

Stochastic Dopamine-Dependent Plasticity forLearning Reward Values 167

Foraging With Plastic Synapses 167

Random Choice and Competitive Games 170

Probabilistic Inference With Stochastic Synapses 171

Concluding Remarks 173

Acknowledgments 174

References 174

IIISOCIAL DECISION NEUROSCIENCE

14. Social Decision-Making in NonhumanPrimatesM. JAZAYARI, S. BALLESTA AND J.-R. DUHAMEL

Introduction 179

Behavioral Studies on Social Decision-Making 180

Neuronal Correlates of Decision-Making in aSocial Context 182

Conclusions and Perspectives 185

References 185

15. Organization of the Social Brain inMacaques and HumansM.P. NOONAN, R.B. MARS, F.X. NEUBERT, B. AHMED,

J. SMITH, K. KRUG AND J. SALLET

Introduction 189

Medial Prefrontal Cortex 190

Superior Temporal Sulcus and Temporal ParietalJunction 193

A Social Brain Network 194

Summary and Perspectives 195

References 196

16. The Neural Bases of Social Influence onValuation and BehaviorK. IZUMA

Introduction 199

The Effect of Mere Presence of Others on ProsocialBehavior 199

The Effect of Others’ Opinions on Valuation 203

Concluding Remarks 206

References 206

17. Social Dominance Representations inthe Human BrainR. LIGNEUL AND J.-C. DREHER

Introduction 211

Learning Social Dominance Hierarchies 212

Interindividual Differences and SocialDominance 216

Neurochemical Approaches to SocialDominance and Subordination 218

Conclusion 222

Acknowledgments 222

References 222

18. Reinforcement Learning and StrategicReasoning During Social Decision-MakingH. SEO AND D. LEE

Introduction 225

Model-Free Versus Model-Based ReinforcementLearning 226

Neural Correlates of Model-Free ReinforcementLearning During Social Decision-Making 227

Neural Correlates of Hybrid ReinforcementLearning During Social Decision-Making 227

Arbitration and Switching Between LearningAlgorithms 228

Conclusions 230

References 230

19. Neural Control of Social Decisions:Causal Evidence From Brain StimulationStudiesG. UGAZIO AND C.C. RUFF

Introduction 233

Noninvasive Brain Stimulation Methods Used forStudying Social Decisions 234

Brain Stimulation Studies of Social Emotions 235

Brain Stimulation Studies of Social Cognition 236

Brain Stimulation Studies of Social BehavioralControl 238

Brain Stimulation Evidence for Social-SpecificNeural Activity 240

Conclusions 241

References 242

20. The Neuroscience of Compassion andEmpathy and Their Link to ProsocialMotivation and BehaviorG. CHIERCHIA AND T. SINGER

Introduction 247

The “Toolkit” of Social Cognition 248

The Neural Substrates of Empathy 249

The Psychological and Neural Bases of Compassion 250

Conclusion 255

References 255

CONTENTS vii

IVHUMAN CLINICAL STUDIES

INVOLVING DYSFUNCTIONS OFREWARD AND DECISION-MAKING

PROCESSES

21. Can Models of Reinforcement LearningHelp Us to Understand Symptoms ofSchizophrenia?G.K. MURRAY, C. TUDOR-SFETEA AND P.C. FLETCHER

Introduction 261

Reward Processing in Schizophrenia: A HistoricalPerspective 262

Dopamine, Schizophrenia, and ReinforcementLearning 263

The Possible Importance of Glutamate 263

Studies of Reward Processing/ReinforcementLearning in Psychosis: Behavioral Studies 264

Studies of Reward Processing/ReinforcementLearning in Psychosis: Neuroimaging Studies 265

Can an Understanding of Reward and DopamineHelp Us to Understand Symptoms ofSchizophrenia? 269

Summary 272

Acknowledgments 273

References 273

22. The Neuropsychology of Decision-Making: A View From the Frontal LobesA.R. VAIDYA AND L.K. FELLOWS

Introduction 277

Lesion Evidence in Humans 278

Decision-Making and the Frontal Lobes 279

Component Processes of Decision-Making 279

Summary 286

Acknowledgments 286

References 286

23. Opponent Brain Systems for Reward andPunishment Learning: Causal Evidence FromDrug and Lesion Studies in HumansS. PALMINTERI AND M. PESSIGLIONE

The Neural Candidates for Reward andPunishment Learning Systems 294

Evidence From Drug and Lesion Studies 296

Conclusions, Limitations, and Perspectives 300

References 300

24. Decision-Making and Impulse ControlDisorders in Parkinson’s DiseaseV. VOON

Introduction 305

The Role of Dopaminergic Medications andIndividual Vulnerability in Parkinson’s Disease 305

Reinforcing Effects and Associative Learning 307

Learning From Feedback 308

Risk and Uncertainty 309

Impulsivity 310

Summary 311

References 312

25. The Subthalamic Nucleus in ImpulsivityK. WITT

Introduction 315

Anatomy, Physiology, and Function of CorticobasalGanglia Circuits 315

The Subthalamic Nucleus and Decision-Making:Evidence From Animal Studies 317

The Subthalamic Nucleus and Impulsivity:Evidence From Behavioral Observations 317

The Subthalamic Nucleus and Decision-Making:Evidence From Neuropsychological Studies 319

A Model of the Impact of the Subthalamic Nucleuson Decision-Making 321

References 322

26. Decision-Making in Anxiety and itsDisordersD.W. GRUPE

Introduction 327

Summary and Conclusions 335

References 336

27. Decision-Making in Gambling Disorder:Understanding Behavioral AddictionsL. CLARK

Introduction: Gambling and Disordered Gambling 339

Loss Aversion 340

Probability Weighting 341

Perceptions of Randomness 342

Illusory Control 344

Conclusion 345

Disclosures 345

References 345

CONTENTSviii

VGENETIC AND HORMONAL

INFLUENCES ON MOTIVATION ANDSOCIAL BEHAVIOR

28. Decision-Making in Fish: Genetics andSocial BehaviorR.D. FERNALD

Social System of the African Cichlid Fish,Astatotilapia burtoni 351

Domains of Astatotilapia burtoni Social Decisions 353

Summary 358

References 358

29. Imaging Genetics in Humans: MajorDepressive Disorder and Decision-MakingU. RABL, N. ORTNER AND L. PEZAWAS

Introduction 361

Major Depressive Disorder as a Disorderof Decision-Making 361

Imaging Genetics of Major Depressive Disorder 364

Conclusion 365

References 367

30. Time-Dependent Shifts in NeuralSystems Supporting Decision-Making UnderStressE.J. HERMANS, M.J.A.G. HENCKENS, M. JOELS AND

G. FERNANDEZ

Introduction 371

Large-Scale Neurocognitive Systems and Shiftsin Resource Allocation 372

Salience Network and Acute Stress 373

Executive Control Network and Acute Stress 376

Salience Network and Executive Control Networkand Recovery From Stress 377

Summary and Conclusion 378

References 380

31. Oxytocin’s Influence on SocialDecision-MakingA. LEFEVRE AND A. SIRIGU

Introduction 387

Oxytocin and Perception of Social Stimuli 388

Oxytocin and Social Decisions 389

Oxytocin and Social Reward 390

Oxytocin, Learning, and Memory 392

Perspectives 392

Acknowledgments 393

References 393

32. Appetite as Motivated Choice:Hormonal and Environmental InfluencesA. DAGHER, S. NESELILER AND J.-E. HAN

Introduction 397

Appetitive Brain Systems Promote Food Intake 398

Self-Control and Lateral Prefrontal Cortex: Role inAppetite Regulation and Obesity 401

Interaction Between Energy Balance Signals andDecision-Making 402

Conclusion 404

References 404

33. PerspectivesJ.-C. DREHER, L. TREMBLAY AND W. SCHULTZ

Identifying Fundamental ComputationalPrinciples: Produce Conceptual Foundations forUnderstanding the Biological Basis of MentalProcesses Through Development of NewTheoretical and Data Analysis Tools 411

Understanding the Functional Organization of thePrefrontal Cortex and the Nature of theComputations Performed in Various Subregions:Value-Coding Computations 412

Demonstrating Causality: Linking Brain Activity toBehavior by Developing and Applying PreciseInterventional Tools That Change NeuralCircuit Dynamics 413

Maps at Multiple Scales: Generate CircuitDiagrams That Vary in Resolution FromSynapses to the Whole Brain 414

The Brain in Action: Produce a Dynamic Picture ofthe Functioning Brain by Developing andApplying Improved Methods for Large-ScaleMonitoring of Neural Activity 415

The Analysis of Circuits of Interacting Neurons 415

Develop Innovative Technologies andSimultaneous Measures to Understand How theBrain Makes Decisions 416

Advancing Human Decision Neuroscience:Understanding Neurological/PsychiatricDisorders and Treating Brain Diseases 417

Conclusions 418

Reference 418

Index 419

CONTENTS ix

This page intentionally left blank

List of Contributors

B. Ahmed University of Oxford, Oxford, United Kingdom

B.W. Balleine University of Sydney, Camperdown, NSW,Australia

S. Ballesta Centre National de la Recherche Scientifique,Bron, France; Universite Lyon 1, Villeurbanne, France; TheUniversity of Arizona, Tucson, AZ, United States

S. Bernardi Columbia University, New York, New York,United States

A. Blangero University of Oxford, Oxford, United Kingdom

L.A. Bradfield University of Sydney, Camperdown, NSW,Australia

W. Chaisangmongkon King Mongkut’s University ofTechnology Thonburi, Bangkok, Thailand; New YorkUniversity, New York, NY, United States

G. Chierchia Max Planck Institute for Human Cognitive andBrain Sciences, Leipzig, Germany

L. Clark University of British Columbia, Vancouver, BC,Canada

A. Dagher McGill University, Montreal, QC, Canada

J.W. Dalley University of Cambridge, Cambridge, UnitedKingdom

J.A. Diaz University of Glasgow, Glasgow, United Kingdom

J.-C. Dreher Institute of Cognitive Science (CNRS), Lyon,France

J.-R. Duhamel Centre National de la Recherche Scientifique,Bron, France; Universite Lyon 1, Villeurbanne, France

N. Eshel Harvard University, Cambridge, MA, United States

L.K. Fellows McGill University, Montreal, QC, Canada

R.D. Fernald Stanford University, Stanford, CA, UnitedStates

G. Fernandez Radboud University Medical Centre,Nijmegen, The Netherlands

P.C. Fletcher University of Cambridge, Cambridge, UnitedKingdom

J.M. Fuster University of California Los Angeles, LosAngeles, CA, United States

S. Gherman University of Glasgow, Glasgow, UnitedKingdom

P.W. Glimcher New York University, New York, NY, UnitedStates

D.W. Grupe University of WisconsineMadison, Madison,WI, United States

S.N. Haber University of Rochester School of Medicine,Rochester, NY, United States

J.-E. Han McGill University, Montreal, QC, Canada

M.J.A.G. Henckens Radboud University Medical Centre,Nijmegen, The Netherlands

E.J. Hermans Radboud University Medical Centre,Nijmegen, The Netherlands

K. Izuma University of York, York, United Kingdom

M. Jazayari Centre National de la Recherche Scientifique,Bron, France; Universite Lyon 1, Villeurbanne, France

M. Joels University Medical Center Utrecht, Utrecht,The Netherlands

T. Kahnt Northwestern University Feinberg School ofMedicine, Chicago, IL, United States

K. Krug University of Oxford, Oxford, United Kingdom

D. Lee Yale University, New Haven, CT, United States

A. Lefevre Institut des Sciences Cognitives Marc Jeannerod,UMR 5229, CNRS, Bron, France; Universite Claude BernardLyon 1, Lyon, France

R. Ligneul Institute of Cognitive Science (CNRS), Lyon,France

K. Louie New York University, New York, NY, United States

R.B. Mars University of Oxford, Oxford, United Kingdom

G.K. Murray University of Cambridge, Cambridge, UnitedKingdom

S. Neseliler McGill University, Montreal, QC, Canada

F.X. Neubert University of Oxford, Oxford, United Kingdom

M.P. Noonan University of Oxford, Oxford, UnitedKingdom

N. Ortner Medical University of Vienna, Vienna, Austria

S. Palminteri University College London, London, UnitedKingdom; Ecole Normale Superieure, Paris, France

M. Pessiglione Institut du Cerveau et de la Moelle (ICM),Inserm U1127, Paris, France; Universite Pierre et MarieCurie (UPMC-Paris 6), Paris, France

L. Pezawas Medical University of Vienna, Vienna, Austria

M.G. Philiastides University of Glasgow, Glasgow, UnitedKingdom

U. Rabl Medical University of Vienna, Vienna, Austria

T.W. Robbins University of Cambridge, Cambridge, UnitedKingdom

C.C. Ruff University of Zurich, Zurich, Switzerland

Y. Saga Institute of Cognitive Sciences (CNRS), Lyon, France

J. Sallet University of Oxford, Oxford, United Kingdom

xi

D. Salzman Columbia University, New York, New York,United States; New York State Psychiatric Institute, NewYork, New York, United States

W. Schultz University of Cambridge, Cambridge, UnitedKingdom

H. Seo Yale University, New Haven, CT, United States

T. Singer Max Planck Institute for Human Cognitive andBrain Sciences, Leipzig, Germany

A. Sirigu Institut des Sciences Cognitives Marc Jeannerod,UMR 5229, CNRS, Bron, France; Universite Claude BernardLyon 1, Lyon, France

J. Smith University of Oxford, Oxford, United Kingdom

A. Soltani Dartmouth College, Hanover, NH, United States

C. Summerfield University of Oxford, Oxford, UnitedKingdom

J. Tian Harvard University, Cambridge, MA, United States

P.N. Tobler University of Zurich, Zurich, Switzerland

L. Tremblay Institute of Cognitive Science (CNRS), Lyon,France

C. Tudor-Sfetea University of Cambridge, Cambridge,United Kingdom

N. Uchida Harvard University, Cambridge, MA, UnitedStates

G. Ugazio University of Zurich, Zurich, Switzerland

A.R. Vaidya McGill University, Montreal, QC, Canada

V. Voon University of Cambridge, Cambridge, UnitedKingdom; Cambridgeshire and Peterborough NHSFoundation Trust, Cambridge, United Kingdom

X.-J. Wang New York University, New York, NY, UnitedStates; NYU Shanghai, Shanghai, China

K. Witt Christian Albrecht University, Kiel, Germany

LIST OF CONTRIBUTORSxii

Preface

Decision Neuroscience: an Integrative Perspective ad-dresses fundamental questions about how the brainmakes perceptual, value-based, and more complex de-cisions innonsocial andsocial contexts. This bookpresentsrecent and compelling neuroimaging, electrophysiolog-ical, lesional, and neurocomputational studies, in combi-nationwith hormonal and genetic studies, that have led toa clearer understanding of the neural mechanisms behindhow the brain makes decisions. The neural mechanismsunderlying decision-making processes are of criticalinterest to scientists because of the fundamental rolethat reward plays in a number of cognitive processes(such as motivation, action selection, and learning) andbecause they have theoretical and clinical implications forunderstanding dysfunctions of major neurological andpsychiatric disorders.

The idea for this book grew up from our edition of theHandbook of Reward and Decision Making (Academic Press,2009). We originally thought to revise and reedit thisbook, addressing one fundamental question about thenature of behavior: how does the brain process rewardand makes decisions when facing multiple options?However, given the developments in this active area ofresearch, we decided to feature an entirely different bookwith new contents, covering results on the neural sub-strates of rewards and punishments; perceptual, value-based, and social decision-making; clinical aspects suchas behavioral addictions; and the roles of genes andhormones in these various aspects. For example, anexciting topic from the field of social neuroscience is toknow whether the neural structures engaged withvarious forms of social interactions are cause or conse-quence of these interactions (Fernald, Chapter 28).

A mechanistic understanding of the neural encodingunderlying decision-making processes is of great interestto a broad readership because of their theoretical andclinical implications. Findings in this research field arealso important to basic neuroscientists interested in howthe brain reaches decisions, cognitive psychologistsworking on decision-making, as well as computationalneuroscientists studying probabilistic models of brainfunctions. Decision-making covers a wide range of topicsand levels of analysis, from molecular mechanisms toneural systems dynamics, neurocomputational models,and social system levels. The contributions to this book

are forward-looking assessments of the current andfuture issues faced by researchers. We were fortunate toassemble an outstanding collection of experts whoaddressed various aspects of decision-making processes.The book is divided into five parts that address distinctbut interrelated topics.

STRUCTURE OF THE BOOK

Adecision neuroscience perspective requiresmultiplelevels of analyses spanning neuroimaging, electrophysi-ological, behavioral, and pharmacological techniques, incombination with molecular and genetic tools. Theseapproaches have begun to build a mechanistic under-standing of individual and social decision-making. Thisbook highlights some of these advancements that haveled to the current understanding of the neuronal mech-anisms underlying motivational and decision-makingprocesses.

Part I is devoted to animal studies (anatomical,neurophysiological, pharmacological, and optogenetics)on rewards/punishments and decision-making. In theirnatural environment, animals face a multitude of stimuli,very few of which are likely to be useful as predictors ofreward or punishment. It is thus crucial that the brainlearns topredict rewards,providinga critical evolutionaryadvantage for survival. This first part of the book offers acomprehensive view of the specific contributions ofvarious brain structures as the dopaminergic midbrainneurons, the amygdala, the ventral striatum, and theprefrontal cortex, including the lateral prefrontal cortexand the orbitofrontal cortex, to the component processesunderlying reinforcement-guided decision-making, suchas the representation of instructions, expectations, andoutcomes; the updating of action values; and the evalua-tion process guiding choices between prospective re-wards. Special emphasis is made on the neuroanatomy ofthe reward system and the fundamental roles of dopa-minergic neurons and the basal ganglia in learningstimulusereward associations.

Chapter 1 (Haber SN) describes the anatomy andconnectivity of the reward circuit in nonhuman primates.It describes how corticalebasal ganglia loops are

xiii

topographically organized and the key areas of conver-gence between functional regions.

Chapter 2 describes three novel electrophysiologicalproperties of the classical dopamine reward-predictionerror (RPE) signal (Schultz W). Studies have identifiedthree novel properties of the dopamine RPE signal. Inparticular, concerning its roles in making choices, thedopamine RPE signal may not only reflect subjectivereward value and formal economic utility but could alsofit into formal competitive decision models. The RPEsignal may code the chosen value suitable for updatingor immediately influencing object and action values.Thus, the dopamine utility prediction error signalbridges the gap between animal learning theory andeconomic decision theory.

Chapter 3 focuses on the electrophysiological prop-erties of another important component of the rewardsystem in primates, namely the amygdala (Bernardi Sand Salzman D). The amygdala contains distinct appe-titive and aversive networks of neurons. Processing inthese two amygdalar networks can both regulate and beregulated by diverse cognitive operations.

Chapter 4 extends the concept of appetitive andaversive motivational processes to the striatum (Saga Yand Tremblay L). This chapter describes how the ventralstriatum and the ventral pallidum, two parts of thelimbic circuit in the basal ganglia, are involved not onlyin appetitive rewarding behavior, as classically believed,but also in negative motivational behavior. These resultscan be linked with the control of approach/avoidancebehavior in a normal context and with the expression ofanxiety-related disorders. The disturbance of thispathwaymay induce not only psychiatric symptoms, butalso abnormal value-based decision-making.

Chapter 5 (Tian J, Uchida N, and Eshel N) highlightsnew advances in the physiology, function, and circuitmechanism of decision-making, focusing especially onthe involvement of dopamine and striatal neurons. Usingoptogenetics in rodents, molecular techniques, andgenetic techniques, this chapter shows how these toolshave been used to dissect the circuits underlyingdecision-making. It describes exciting new avenues tounderstand a circuit, by recording from neurons withknowledge of their cell type and patterns of connectivity.Furthermore, the ability to manipulate the activity ofspecific neural types provides an important means to testhypotheses of circuit function.

Chapter 6 (Bradfield L and Balleine B) describes theneural bases of the learning and motivational processescontrolling goal-directed action. By definition, the per-formance of such action respects both the current valueof its outcome and the extant contingency between thataction and its outcome. This chapter identifies the neuralcircuits mediating distinct processes, including theacquisition of action-outcome contingencies, the

encoding and retrieval or incentive value, the matchingof that value to specific outcome representations, andfinally the integration of this information for action se-lection. It also shows how each of these individual pro-cesses are integrated within the striatum for successfulgoal-directed action selection.

Chapter 7 (Robbins TW and Dalley JW) describes an-imal models (mostly in rodents) of impulsivity and riskychoices. It reviews the neural and neurochemical basis ofvarious forms of impulsive behavior by distinguishingthree main forms of impulsivity: waiting impulsivity,risky choice impulsivity, and stopping impulsivity. Itshows that dopamine- and serotonin-dependent func-tions of the nucleus accumbens are implicated in waitingimpulsivity and risky choice impulsivity, as well ascortical structures projecting to the nucleus accumbens.For stopping impulsivity, dopamine-dependent functionsof the dorsal striatum are implicated, as well as circuitryincluding the orbitofrontal cortex and dorsal prelimbiccortex. Differences and commonalities between the formsof impulsive responding are highlighted. Importantly,various applications to human neuropsychiatric disor-ders such as drug addiction and attention deficit hyper-activity disorder are also discussed.

Chapter 8 (Fuster JM) proposes that the neuralmechanisms of decision-making are understandableonly in the structural and dynamic context of theperceptioneaction cycle, defined as the biocyberneticprocessing of information that adapts the organism to itsenvironment. It presents a general view of the role of theprefrontal cortex in decision-making, in the generalframework of the perceptioneaction cycle, includingprediction, preparation toward decision, execution, andfeedback from decision.

Part II covers the topic of the neural representation ofmotivation, perceptual decision-making, and value-based decision-making in humans, mostly combiningneurocomputational models and brain imaging studies.

Chapter 9 (Tobler P and Kahnt T) reviews severaldefinitions of value and salience, and describes humanneuroimaging studies that dissociate these variables.Value increases with the magnitude and probability ofreward but decreases with the magnitude and proba-bility of punishment, whereas salience increases with themagnitude and probability of both reward and punish-ment. At the neural level, value signals arise in striatum,orbitofrontal and ventromedial prefrontal cortex, andsuperior parietal areas, whereas magnitude-basedsalience signals arise in the anterior cingulate cortexand the inferior parietal cortex. By contrast, probability-based salience signals have been found in the ventro-medial prefrontal cortex.

Chapter 10 (Louie K and Glimcher PW) reviews anapproach centered on basic computations underlying

PREFACExiv

neural value coding. It proposes that neural informationprocessing in valuation and choice relies on computa-tional principles such as contextual modulation anddivisive normalization. Divisive normalization is anonlinear gain control algorithm widely observed inmultiple sensory modalities and brain regions. Identifi-cation of these computations sheds light on how theunderlying neural circuits are organized, and neuralactivity dynamics provides a link between biologicalmechanism and computations.

Chapter 11 (Philiastides M, Diaz J, and Gherman S)introduces the general principles guiding perceptualdecision-making. Perceptual decisions occur whenperceptual inputs are integrated and converted to form acategorical choice. It reviews the influence of a number offactors that interact and contribute to the decision pro-cess, such as prestimulus state, reward and punishment,speedeaccuracy trade-off, learning and training, confi-dence, and neuromodulation. It shows how these deci-sion modulators can exert their influence at variousstages of processing, in line with predictions derivedfrom sequential-sampling models of decision-making.

Chapter 12 (Summerfield C) reviews the neural andcomputational mechanisms of perceptual decisions. Itaddresses current controversial questions, such as howwe decide when to draw our decisions to a conclusion,and how perceptual decisions are biased by priorinformation.

Chapter 13 (Soltani A, Chaisangmongkon W, andWang XJ) presents possible biophysical and circuitmechanisms of valuation and reward-dependent plas-ticity underlying adaptive choice behavior. It reviewsmathematical models of reward-dependent adaptivechoice behavior, and proposes a biologically plausible,reward-modulated Hebbian synaptic plasticity rule. Itshows that a decision-making neural circuit endowedwith this learning rule is capable of accounting forbehavioral and neurophysiological observations in avariety of decision-making tasks.

Part III of the book focuses on the rapidly developingfield of social neuroscience, integrating neurosciencedata from both nonhuman primates and humans. Pri-mates are fundamentally social animals, and they mayshare common neural mechanisms in diverse forms ofsocial behavior. Examples of such behavior includetracking intentions and beliefs from others, beingobserved by others during prosocial decisions, orlearning the social hierarchy in a group of individuals. Itis also likely that at the macroscopic level, importantdifferences exist concerning social brain structures andconnectivity, and there is a need to directly comparebetween species to answer this fundamental question.Indeed, studies in both humans and monkeys report notonly an increase in gray matter density of specific brain

structures relative to the size of our social network, butalso species differences in prefrontaletemporal brainconnectivity. Furthermore, this part of the book presentsneurocomputational approaches starting to provide amechanistic understanding of social decisions. Forexample, reinforcement learning models and strategicreasoning models can be used when learning social hi-erarchies or during social interactions.

A social neuroscience understanding requires multi-ple approaches, such as electrophysiology and neuro-imaging in both monkeys (Chapters 14, 15, 19) andhumans (Chapters 16, 18, 20), as well as causal (Chapter21), neurocomputational (Chapters 17e19), endocrino-logical, genetics, and clinical approaches (Part V).

Chapter 14 (Duhamel JR and colleagues) presentsmonkey electrophysiological data revealing that theorbitofrontal cortex is tuned to social information. Forexample, in one experiment, macaque monkeys workedto collect rewards for themselves and two monkeypartners. Single neurons encoded the meaning of visualcues that predicted the magnitude of future rewards, themotivational value of rewards obtained in a socialcontext, and the tracking of social preferences and part-ner’s identity and social rank. The orbitofrontal cortexthus contains key neuronal mechanisms for the evalua-tion of social information. Moreover, macaque monkeystake into account the welfare of their peers when makingbehavioral choices bringing about pleasant or unpleasantoutcomes to a monkey partner. Thus, this chapter revealsthat prosocial decision-making is sustained by anintrinsic motivation for social affiliation and controlledthrough positive and negative vicarious reinforcements.

Chapter 15 (Sallet J and colleagues) reviews the sim-ilarities between monkeys and humans in the organiza-tion of the social brain. Using MRI-based connectivitymethods, they compare human and macaque socialareas, such as the organization of the medial prefrontalcortex. They revealed that the connectivity fingerprint ofmacaque area 10 best matched that of the human frontalpole, suggesting that even high-level areas share featuresbetween species. They also showed that animals housedin large social groups had more gray matter volume inbilateral mid-superior temporal sulcus and rostral pre-frontal cortex. Beyond species similarities, there are alsodistinct differences between human and macaqueprefrontaletemporal brain connectivity. For example,functional connections between the temporal cortex andthe lateral prefrontal cortex are stronger in humanscompared to connections with the medial prefrontalcortex in humans, but the opposite pattern is observed inmacaques.

Chapter 16 (Izuma K) focuses on two forms of socialinfluence, the audience effect, which is an increasedprosocial tendency in front of other people, and socialconformity, which consists in adjusting one’s attitude or

PREFACE xv

behavior to those of a group. This chapter discusses fMRIfindings in healthy humans in these two types of socialinfluence and also shows how reputation processing isimpaired in individuals with autism. It also links socialconformity and reward-based learning (reinforcementlearning).

Chapter 17 (Ligneul R and Dreher JC) examines howthe brain learns social dominance hierarchies. Socialdominance refers to relationships wherein the goals of oneindividual prevail over the goals of another individual in asystematic manner. Dominance hierarchies have emergedas a major evolutionary force to drive dyadic asymmetriesin a social group. This chapter proposes that the emer-gence of dominance relationships are learned incremen-tally, by accumulating positive and negative competitivefeedbacks associated with specific individuals and othermembers of the social group. It considers such emergenceof social dominance as a reinforcement learning probleminspired by neurocomputational approaches traditionallyapplied to nonsocial cognition. This chapter also reportshow dominance hierarchies induce changes in specificbrain systems, and it reviews the literature on interindi-vidual differences in the appraisal of social hierarchies, aswell as the underlying modulations of cortisol, testos-terone, and serotonin/dopamine systems, which mediatethese phenomena.

Chapter 18 (Seo H and Lee D) describes reinforcementlearning models and strategic reasoning during socialdecision-making. It shows that dynamic changes inchoices and decision-making strategies can be accountedfor by reinforcement learning in a variety of contexts. Thisframework has also been successfully adopted in a largenumber of neurobiological studies to characterize thefunctions of multiple cortical areas and basal ganglia. Forcomplex decision-making, including social interactions,this chapter shows that multiple learning algorithmsmayoperate in parallel.

Chapter 19 (Ugazio G and Ruff C) reports brainstimulation studies on social decision-making, which testthe causal relationship between neural activity anddifferent types of processes underlying these decisions,including social emotions, social cognition, and socialbehavioral control.

Chapter 20 (Chierchia G and Singer T) shows that twoimportant social emotions, empathy and compassion,engage distinct neurobiological mechanisms, as well asdifferent affective and motivational states. Empathy forpain engages a network including the anterior insula andanterior midcingulate cortex, areas associated withnegative affect; compassionate states engage the medialorbitofrontal cortex and ventral striatum and are associ-atedwith feelings ofwarmth, concern, and positive affect.

Part IV of the book focuses on clinical aspectsinvolving disorders of decision-making and of the

reward system that link together basic research areas,including systems, cognitive, and clinical neuroscience.Dysfunction of the reward system and decision-makingis present in a number of neurological and psychiatricdisorders, such as Parkinson’s disease, schizophrenia,drug addiction, and focal brain lesions. The study ofpathological gambling, for example, and other motivatedstates associated with, and leading to, compulsivebehavior provides an opportunity to learn about thedysfunctions of reward system activity, independent ofdirect pharmacological activation of brain reward cir-cuits. On the other hand, because drugs of abuse directlyactivate brain systems, they provide a unique challengein understanding how pharmacological activation in-fluences reward mechanisms leading to persistentcompulsive behavior.

Chapter 21 (Murray GK, Tudor-Sfetea C, and FletcherPC) shows that principles of reinforcement learning areuseful to understand the neural mechanisms underlyingimpaired learning, reward, and motivational processesin schizophrenia. Two symptoms characteristic of thisdisease is considered in this framework, namely de-lusions and anhedonia.

Chapter 22 (Vaidya AR and Fellows LK) takes aneuropsychological approach to review focal frontal lobedamage effects on value-based decisions. It reveals thenecessary contributions of specific subregions (ventro-medial, lateral, and dorsomedial prefrontal cortex) todecision-making, and provides evidence as to the disso-ciability of component processes. It argues that theventromedial frontal lobe is required for optimal learningfrom reward under dynamic conditions and contributesto specific aspects of value-based decision-making. It alsoshows a necessary contribution of the dorsomedialfrontal lobe in representing action-value expectations.

Chapter 23 (Palminteri S and Pessiglione M) reviewsreinforcement learning models applied to reward andpunishment learning. These studies include fMRI andneural perturbation following drug administration and/or pathological conditions. They propose that distinctbrain systems are engaged, one in reward learning(midbrain dopaminergic nuclei and ventral prefrontos-triatal circuits) and another in punishment learning,revolving around the anterior insula.

Chapter 24 (Voon V) discusses decision-making im-pairments and impulse control disorders in Parkinson’sdisease. The author reports enhancement of the gainassociated with levodopa, reinforcing properties ofdopaminergic medications, and enhancement of delaydiscounting in these patients. Lower striatal dopaminetransporter levels preceding medication exposure, anddecreased midbrain D2 autoreceptor sensitivity, mayunderlie enhanced ventral striatal dopamine release andactivity in response to salient reward cues, anticipatedand unexpected rewards, and gambling tasks.

PREFACExvi

Impairments in decisional impulsivity (delay discount-ing, reflection impulsivity, and risk taking) implicate theventral striatum, orbitofrontal cortex, anterior insula,and dorsal cingulate. These findings provide insight intothe role of dopamine in decision-making processes inaddiction and suggest potential therapeutic targets.

Chapter 25 (Witt K) reports that motor control is theresult of a balance between activation and inhibition ofmovement patterns. It points to a central role of thesubthalamic nucleus within the indirect basal gangliapathway, acting as a brake on the motor system. Thissubthalamic nucleus function occurs when an automaticresponse must be suppressed to have more time tochoose between alternative responses.

Chapter 26 (Grupe DW) discusses value-baseddecision-making as one of a key behavioral symptomspresent in anxiety disorders. This chapter highlights al-terations to specific processes: decision representation,valuation, action selection, outcome evaluation, andlearning. Distinct anxious phenotypes may be charac-terized by differential alterations to these processes andtheir associated neurobiological mechanisms.

Chapter 27 (Clark L) presents a conceptualization ofdisordered gambling as a behavioral addiction driven byan exaggeration of multiple psychological distortionsthat are characteristic of human decision-making, andunderpinned by neural circuitry subserving appetitivebehavior, reinforcement learning, and choice selection.The chapter discusses the neurobiological basis of path-ological gambling behavior in loss aversion, probabilityweighting, perceptions of randomness, and the illusionof control.

Part V focuses on the roles of hormones and genesinvolved in motivation and social decision-making pro-cesses. The combination of molecular genetic, endocri-nology, and neuroimaging has provided a considerableamount of data that help in the understanding of thebiological mechanisms influencing decision processes.These studies have demonstrated that genetic and hor-monal variations have an impact on the physiologicalresponse of the decision-making system. These varia-tions may account for some of the inter- and intra-individual behavioral differences observed in socialcognition.

Chapter 28 (Fernald RD) presents an originalapproach for cognitive neuroscientists by focusing on thedifficult question of how an animal’s behavior orperception of its social and physical surroundings shapesits brain. Using a fish model system that depends oncomplex social interactions, this chapter reports how thesocial context influences the brain and, in turn, alters thebehavior and neural circuitry of animals as they interact.Gathering of social information vicariously producesrapid changes in gene expression in key brain nuclei and

these genomic responses prepare the individual tomodify its behavior to move into a different social niche.Both social success and failure produce changes inneuronal cell size and connectivity in key brain nuclei.This approach bridges the gap between social informa-tion gathering from the environment and the levels ofcellular and molecular responses.

Chapter 29 (Rabl U, Ortner N, and Pezawas L) ex-amines the use of imaging genetics to explore the re-lationships between major depressive disorder anddecision-making.

Chapters 30e32 report neuroendocrinological find-ings in social decision-making, likening variations in thelevels of different types of hormones (cortisol, oxytocin,ghrelin/leptin) to brain systems engaged in social de-cisions and food choices. Chapter 30 (Hermans EJ andcolleagues) integrates knowledge of the effects of stressat the neuroendocrine, cellular, brain systems, andbehavioral levels to quantify how stress-related neuro-modulators trigger time-dependent shifts in the balancebetween two brain systems: a “salience” network, whichsupports rapid but rigid decisions, and an “executivecontrol” network, which supports flexible, elaborate de-cisions. This simple model elucidates paradoxical find-ings reported in human studies on stress and cognition.

Chapter 31 (Lefevre A and Sirigu A) reviews evidencefor a role for oxytocin in individual and social decision-making. It discusses animal and human studies to linkthe behavioral effects of oxytocin to its underlyingneurophysiological mechanisms.

Chapter 32 (Dagher A, Neseliler S, and Han JE) ex-amines the neurobehavioral factors that determine foodchoices and food intake. It reviews findings on the in-teractions between brain systems that mediate feedingbehavior and the gut and adipose peptides that signalthe current state of energy balance.

Chapter 33 (Dreher, Tremblay, and Schultz) concludesthis decision neuroscience book by integrating perspec-tives from all contributors.

We anticipate that while some readers may read thevolume from the first to the last chapter, other readersmay read only one or more chapters at a time, and notnecessarily in the order presented in the book. This iswhy we encouraged an organization of this volumewhereby each chapter can stand alone, while makingreferences to others andminimizing redundancies acrossthe volume. Given the consistent acceleration of ad-vances in the various approaches described in this bookon decision neuroscience, you are about to be dazzled bya first look at the new stages of an exciting era in brainresearch. Enjoy!

Jean-Claude DreherLeon Tremblay

PREFACE xvii

This page intentionally left blank

C H A P T E R

33

PerspectivesJ.-C. Dreher1, L. Tremblay1, W. Schultz2

1Institute of Cognitive Science (CNRS), Lyon, France; 2University of Cambridge, Cambridge, United Kingdom

AbstractDiscovering how the brain makes decisions is one of themost exciting challenges of neurosciences that has emergedin recent years. The evolution of the field of decisionneuroscience has benefited from the advance of novel tech-nological capabilities in neurosciences, and the pace at whichthese capabilities have been developed has accelerateddramatically in the past decade.

Discovering how the brain makes decisions is one ofthe most exciting challenges of neurosciences that hasemerged in recent years. The evolution of the field of de-cision neuroscience has benefited from the advance ofnovel technological capabilities in neurosciences, andthe pace at which these capabilities have been devel-oped has accelerated dramatically since 2005.

It is certainly difficult to predict what will be the mostexciting developments in decision neuroscience inthe future and somewhat arbitrary to organize potentialperspectives along a coherent line. We have asked thecontributors to this book to give us their respectiveperspectives for developments in their research do-mains. We have taken the liberty to build these perspec-tives based on these views and along lines outlined in aneuroscience report from the Brain Research throughAdvancing Innovative Neurotechnologies (BRAIN)Initiative [1]. This BRAIN Initiative should help usdevelop and apply new tools and technologies to under-stand the brain at multiple levels. In parallel, theHuman Brain Project from the EU, the Brain/MINDSProject from Japan (BrainMapping by Integrated Neuro-technologies for Disease Studies), CanadaBrain, and anational brain project under way in China should alsofoster technological innovation for great discoveriesthat should lead to a revolution in our understandingof how the brain makes decisions, from a multilevelperspective.

IDENTIFYING FUNDAMENTALCOMPUTATIONAL PRINCIPLES:

PRODUCE CONCEPTUAL FOUNDATIONSFOR UNDERSTANDING THE

BIOLOGICAL BASIS OF MENTALPROCESSES THROUGH DEVELOPMENTOF NEW THEORETICAL AND DATA

ANALYSIS TOOLS

Theory and mathematical modeling are advancingour understanding of complex, nonlinear brainfunctions where human intuition fails. New kinds ofdata are accruing at increasing rates, mandatingnew methods of data analysis and interpretation. Toenable progress in theory and data analysis, decisionneuroscience will need to foster collaborationsbetween experimentalists and researchers from physics,mathematics, engineering, and computer science.

One general approach widely used in the field offMRI research and more recently applied to monkeyelectrophysiology is to use a so-called model-basedapproach, allowing us to identify the computationsperformed by a given brain region. This approach se-lects the best model fitting behavior among a set ofmodels and allows us to regress brain activity withoutput parameters from these models. One classicalmodel-based fMRI approach concerning learning ofbasic stimulusereinforcer associations used predictionerrors as regressors. Similar model-based fMRI ap-proaches have been used to study social learning,such as learning social hierarchies based on victoriesand defeats in a competitive game, or modeling of stra-tegic reasoning (see Ligneul and Dreher, Chapter 17;Palminteri and Pessiglione, Chapter 23; and Lee, Chap-ter 18).

One fundamental theoretical view about the brain,put forward by leading researchers such as Karl Friston

411Decision Neuroscience

Copyright © 2017 Elsevier Inc. All rights reserved.http://dx.doi.org/10.1016/B978-0-12-805308-9.00033-6

and Rajesh Rao, is that the brain performs Bayesiancomputations in general, and particularly when makingdecisions. According to this view, decision-making andaction selection are treated as an inference problemsolving the problem of selecting behavioral sequencesor policies. Choices are based upon beliefs about alter-native policies, whereby the most likely policy mini-mizes the difference between attainable and desiredoutcomes. Policies are then selected under the priorbelief that they minimize the difference (relative en-tropy) between a probability distribution over statesthat can be reached and states that agents believethey should occupy. Future developments in the fieldof decision neuroscience will be to test this Bayesianview of the brain in various contexts, not only forperceptual and value-based decisions but also for socialdecision-making. Perspectives for research include thearticulation of neurocomputational definitions of valuecoding with a general Bayesian brain perspective. Thepioneering work of Karl Friston has been developedalong this line but will need to be extended to find pre-dictable experimental validations.

Yet another perspective for future research is toextend classical reinforcement learning approachesto social decision-making (see Ligneul and Dreher,Chapter 17) and strategic reasoning (see Lee, Chapter18). Social interactions are often repeated in a particularsetting, making it possible for decision-makers toimprove their strategies through experience. Therefore,the exact nature of learning algorithms utilized duringiterative social decision-making and the correspondingneural substrates are important topics for psychologicaland neurobiological research. Previous research hasshown that humans and nonhuman primates rely ona dynamic mixture of multiple learning algorithms forboth social and nonsocial decision-making. For simple,model-free reinforcement learning, strategies arerevised exclusively based on the observed outcomesfrom previously chosen actions, whereas for model-based reinforcement learning and belief learning,observed behaviors of other decision-makers and infer-ences about them also influence future choices. Thesedifferent types of learning algorithms might be imple-mented in different regions of the association cortexand basal ganglia, but how the neuronal activity ineach of these brain areas contributes to the specifictypes of computations for learning remains poorly un-derstood. For example, how the brain can update thevalues for multiple actions through mental simulationis difficult to study, because the activity related tosuch simulation may not be tightly linked to anyobservable sensory and motor events. In addition,although the brain must continuously support multiplelearning algorithms, how the outputs of various

learning algorithms are combined and how potentialconflicts between them get resolved need to be investi-gated further.

UNDERSTANDING THE FUNCTIONALORGANIZATION OF THE PREFRONTALCORTEX AND THE NATURE OF THECOMPUTATIONS PERFORMED IN

VARIOUS SUBREGIONS: VALUE-CODINGCOMPUTATIONS

The subdivisions of the prefrontal cortex and thecomputations performed by these subregions will bekey to providing a mechanistic understanding ofdecision-making. For example, the roles of subdivi-sions of the medial and lateral orbitofrontal cortexwill need to be specified, together with their participa-tion in multiple modulatory loops with other impor-tant structures such as the nucleus accumbens,ventral pallidum, amygdala, and hypothalamus, aswell as modulation with autonomic input from thegut (see Dagher, Chapter 32).

Similarly, functional divisions of the dorsolateral pre-frontal cortex will need to be further characterized. Thisbrain region, considered the highest level of the execu-tive hierarchy, temporally coordinates the perceptioneaction cycle by means of its cognitive executive func-tions upon the posterior cortex (see Fuster, Chapter 8).In addition, the dorsolateral prefrontal cortex has thecapability of anticipating (predicting) perception, action,and outcome; this confers to that cortex the functions ofplanning and preadapting that are critical for effectivedecision-making. The ventromedial prefrontal cortexcloses the perceptioneaction cycle by collecting neuralfeedback from reward, monitoring of outcome, andrisk assessment; it also has predictive capability of antic-ipated reward. All our decisions are to some degreeBayesian, based on the updating of prior hypotheses ofperception, action, or outcome, whether their “data-base” is conscious, unconscious, or intuitive; therefore,any reasonable computational neuroscience ofdecision-making should include probability as an essen-tial variable.

One key organizing concept in the field of decisionneuroscience is the concept of value. Understandingthe computational principles of value coding in thebrain has received considerable attention from re-searchers to understand the neurobiological basis ofdecision-making. This progress has illuminated bothwhere decision processing occurs in the brain andwhat information is represented in relevant neural ac-tivity. For example, neurophysiological and neuroi-maging studies have identified specific brain areas

33. PERSPECTIVES412

V. GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

involved in option valuation and selection, includingthe frontal and parietal cortices, amygdala, and basalganglia including midbrain dopamine neurons, ventralstriatum, and pallidum. Neural activity in these areashas been shown to correlate with diverse decision vari-ables relevant to choice behavior, such as rewardmagnitude, risk, ambiguity, and delay to reinforce-ment. A central principle derived from this researchis that information about the idiosyncratic subjectivevalue of choice options is represented in the neuralactivity of decision-related brain areas and that it isthis idiosyncratic representation that appears to driveactual choice behavior. However, a critical aspect ofdecision-making processes (and one oddly relevant toeconomics and psychology) remains largely unex-plored: how neural circuits represent value informa-tion. In an information-processing system, the form ofinformation representation is a key intermediate levelmediating the link between low-level implementationand high-level goals. Information coding is a particu-larly significant issue for biological systems, whichface inherent constraints such as energetic costs andbiophysical limitations. Because such constraints limitthe information-coding capacity of neural systems,they require a transformation between the input(the variable to be encoded) and the output (the neuralactivity representing that variable) of a neural circuit.For example, the representation of the vast range of po-tential rewarding outcomes with the finite dynamicrange of neural activity necessitates a compressiveinputeoutput computation that can havesignificant implications for what we choose and whenwe choose it.

Thus, to understand decision-related inputeoutputfunctions (i.e., value coding), it will be critical for studiesnot simply to demonstrate correlation between neuralactivity and value but to quantify the precise relation-ships between the two. Experimental studies havebegun to quantify these neural value-coding computa-tions in brain regions such as the orbitofrontal and pos-terior parietal cortices. A notable finding of this initialwork is that these value inputeoutput functions are flex-ible and dynamic, changing in very specific ways inresponse to contextual influences such as the architec-ture of the choice set a decision-maker faces and the his-tory of past rewards encountered by that decision-maker. Importantly, this contextual value coding is, atleast in part, mediated by well-described computationssuch as divisive normalization that are prominent insensory processing, an observation arguing for a generalmechanism for information coding in the brain.

In addition to identifying, quantifying, and modelingthese value-coding computations, two specific direc-tions are important targets for future research. First,

how are value-coding computations related to thestructure and connectivity of the underlying biologicalcircuits? One important approach to answering thisquestion will be the examination of various circuitcomponents, including cells with different functionalroles (i.e., excitation versus inhibition), laminar loca-tions, and connectivity patterns. New devices andtechniques, such as large electrode arrays and optoge-netics, will be crucial to this process. Another promisingapproach in this direction is a dynamical analysis ofneural activity, focusing on fast-timescale (i.e., milli-second level) changes in firing rates rather than activityaveraged over long windows; such dynamics can revealkey details about the functional connectivity of neuralcircuits and the resulting patterns of information flow.Second, how do value-coding computations affectchoice behavior? Given the inherent constraints ofinformation processing in neural circuits, biologicaldecision-making can never reach the optimality pre-dicted by normative models that have no real biologicalconstraints. Quantifying the relationship between value-coding computations and choice behavior will illumi-nate both the constraints faced by biological choicesystems and how neural computational algorithmscompensate for those constraints.

DEMONSTRATING CAUSALITY:LINKING BRAIN ACTIVITY TO

BEHAVIOR BY DEVELOPING ANDAPPLYING PRECISE INTERVENTIONAL

TOOLS THAT CHANGE NEURALCIRCUIT DYNAMICS

To enable the immense potential of circuit manipula-tion, a new generation of tools for optogenetics, chemo-genetics, and biochemical and electromagneticmodulation should be developed for use in animalsand eventually in human patients.

Since the pioneering work by Wolfram Schultz andcolleagues, we have learned a great deal about thenature of dopamine responses during learning. Inparticular, we know that dopamine neurons signalreward-prediction error, or the difference between thereward that an animal expects and the reward it actu-ally receives. This signal is thought to reinforcerewarding actions and suppress alternative actions,potentially through corticobasal ganglia loops definedby expression of different dopamine receptors. Howev-er, we are only at the beginning stages of understand-ing how dopamine neurons calculate these responses.Given the number of different possible sources ofinput, how do dopamine neurons converge on suchsimilar prediction error responses? In what ways are

V. GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

DEMONSTRATING CAUSALITY: LINKING BRAIN ACTIVITY TO BEHAVIOR BY DEVELOPING 413

dopamine neurons homogeneous versus heteroge-neous? Are they involved in learning from punish-ments as well as rewards? Furthermore, there aremany unanswered questions about how dopaminerelease affects downstream circuits in vivo. What arethe differential roles of phasic versus tonic dopaminefiring in motivating learning and behavior? What isthe effect of dopamine release on striatal and corticalneurons in vivo? How do striatal D1 and D2 neuronsinteract during behavior? What types of learningrequire dopamine, and what types are dopamine inde-pendent? To address these fundamental questions,newly developed molecular, genetic, and recordingtechniques will be critical.

By directly activating and inhibiting populations ofneurons in a behavioral context, neuroscience is pro-gressing from correlative measures to understandingof causal brain regions [transcranial magnetic stimula-tion (TMS), neuropsychology]. Methods such as TMSor transcranial direct current stimulation (tDCS) arethus likely (see Ruff, Chapter 19) to establish causalmechanisms for a given brain region, complementingclassical neuropsychological approaches in patientswith focal brain lesions (see Fellows, Chapter 22). Acentral challenge for a neuropsychological perspectiveon the role of the prefrontal cortex in value-baseddecision-making is to continue to dissect decisionprocesses at the level of brain mechanisms. We havegeneral guides to this now: clearly there are specificregions within the frontal lobes, for example, contrib-uting in specific ways to value-based choice. However,the mechanisms that are engaged remain unclear, inpart because the component processes of decision-making remain ill-defined, with likely multiple routesto decision-making in any given situation. We need totake advantage of converging methods to providerobust tests of well-specified, mechanistic models ofdecision-making. This of course is true for cognitiveneuroscience in general, but it seems particularly truefor decision neuroscience, in which, for the most part,models remain very general. Progress in the neuropsy-chological study of decision-making requires goodbehavioral measures of the constructs of interest.Although the past several years of work now betterequip us in this regard, there is still much to be done.Creative approaches that go beyond button-presschoices and reaction times, such as eye tracking, auto-nomic measures, and assessments of physical andcognitive effort, hold promise for uncovering the“microbehaviors” underlying value assessment andchoice. It is also increasingly clear that we cannottake an isolationist perspective on decision-making.Decision behaviors do not emerge fully formed fromsome specialized “economic” module of the brain,but rather are interlinked with attention, memory,

socialeemotional, and action-selection processes. Abetter understanding of these interactions will accel-erate advances, particularly as many of these relatedprocesses are much more thoroughly studied. Finally,decision neuroscience must aim to understand value-based choice broadly construed: in economic, but alsopolitical, social, and esthetic contexts. Testing the gen-erality of explanatory models across the whole gamutof motivated behavior will, in the end, yield the mostpowerful insights.

Finally, causality can also be assessed using compu-tational models, which allow researchers to assessprobabilistic causality in humans. Building on theoriesof nonlinear dynamical systems, whole-brain computa-tional models have been used to efficiently characterizenetwork-level communication across distributed sets ofbrain areas (i.e., functional connectivity) to investigatethe spatiotemporal dynamics of brain organizationand complex cognitive architectures [2]. This dynamiccharacterization can incorporate time-dependent activ-ity operating on varying timescales, which may capturea more complete picture of the spatiotemporal proper-ties inherent to decision-making.

MAPS AT MULTIPLE SCALES: GENERATECIRCUIT DIAGRAMS THAT VARY IN

RESOLUTION FROM SYNAPSES TO THEWHOLE BRAIN

It is increasingly possible to map connected neuronsin local circuits and distributed brain systems, enablingan understanding of the relationship between neuronalstructure and function. It is now possible to envisionimproved technologiesdfaster, less expensive,scalabledfor anatomic reconstruction of neural circuitsat all scales, from noninvasive whole human brain imag-ing to dense reconstruction of synaptic inputs and out-puts at the subcellular level.

For example, understanding of the circuit diagramsthat underlie impulsivity, risky choice, and impulse con-trol disorders is now possible to attain based on animalmodels. Impulsivity has emerged as a major dimen-sional construct in psychiatry with relevance to a rangeof disorders from addiction to attention deficit hyperac-tivity disorder (ADHD) and from Parkinson’s diseaseto depression, mania, and dementia. As a heritable,disorder-associated trait, impulsivity is broadlyacknowledged to affect the quality of decision-makingthrough effects on risk sensitivity, subjective value-based judgments (e.g., temporal discounting of delayedrewards), and cognitive control mechanisms responsiblefor the inhibition of ongoing behavior. Several decadesof research in humans and experimental animals haverevealed divergent but often interacting neural circuitry

V. GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

33. PERSPECTIVES414

that underlies various impulsivity phenotypes,including the inability to await rewards, inability toterminate initiated behavior, preference for risky choice,or tendency to incompletely process information prior todecision-making. Yet formidable challenges lie ahead.For example, at present we lack a detailed understand-ing of the biological origins and neural circuitry of traitimpulsivity, including environmental interactions,and how these collectively contribute to poor impulsecontrol. Addressing this shortfall requires preclinicalscientists to study predictive biomarkers and neurode-velopmental trajectories for impulsivity in muchyounger animals. By continuing to explore the behav-ioral diversity of impulsivity and adopting translationalneural imaging, genomic, and objective behavioral ap-proaches, we expect to see further advances in our un-derstanding of trait impulsivity. This work requires adetailed dimensional analysis of impulsivity, character-ized in aggregate by variation in genes, molecules, andcircuits, in addition to a therapeutic focus away frombrain monoaminergic systems (e.g., in the form of medi-cation with Ritalin for ADHD) toward novel brain mech-anisms and hence new neuropharmacological targets.

THE BRAIN IN ACTION: PRODUCE ADYNAMIC PICTURE OF THE

FUNCTIONING BRAIN BY DEVELOPINGAND APPLYING IMPROVED METHODSFOR LARGE-SCALE MONITORING OF

NEURAL ACTIVITY

One important challenge in the future will be to re-cord dynamic neuronal activity from densely sampleddand in some test cases completedneural networks, overlong periods of time, in all areas of the brain, in bothmammalian systems and diverse model organisms,while making various types of decisions. There arepromising opportunities both for improving existingtechnologies and for developing entirely new technolo-gies for neuronal recording, including methods basedon electrodes, optics, molecular genetics, and nano-science and encompassing various facets of brainactivity.

The combination of existing techniques using multi-modal neuroimaging approaches in both nonhumanand human primates is also likely to bring insightsinto how the brain makes decisions. For example, thecombination of intracranial EEG (iEEG) recordings inpatients with epilepsy (whether with single cells or mac-roelectrodes) and fMRI, or single/multiple-cell record-ings combined simultaneously with fMRI in monkeys,should bring a better understanding of the precisetemporal dynamics at the systems level. Similarly, thenew PETefMRI scanners, which allow us to map

simultaneously both radiotracers and to acquire bloodoxygen level-dependent (BOLD) responses during deci-sion-making tasks, should bring exciting new findingsto the community. Converging approaches using thesame paradigms with different imaging modalities(e.g., EEG or MEG) and fMRI, together with physiolog-ical measures (e.g., pupil dilation, heart beat, etc.)should allow us to specify the dynamics of decisionstogether with a broader view at the neurophysiologicallevel.

Another interesting perspective from our field comesfrom the observation that the social environment shapesneural structures and processes, and vice versa. InChapter 28, Fernald gave a few examples of these inter-relationships using genetics and social behavior in ani-mals. Social animals interact with others and theirenvironment to survive and reproduce if possible. Todo this, animals acquire, evaluate, and translate infor-mation about their social and physical situation into de-cisions about what to do next. The information gatheredand the resulting decisions can profoundly alter both thebehavior and the physiology of an animal. These choicesin the brain are both produced by and result in a diversearray of cellular and molecular actions. The challenge isto discover where decisions are made and, in particular,what information is used to guide specific choices. Withnew genetic techniques, animal studies directed at un-derstanding how the brain decides are not restricted toa limited number of “model organisms” but any animalwith an interesting decision-making behavior.

THE ANALYSIS OF CIRCUITS OFINTERACTING NEURONS

The circuits of interacting neurons are particularlyrich in research opportunities, with potential for revolu-tionary advances. This area of research represents a realknowledge gap. We can now study the brain at veryhigh resolution by examining individual genes, mole-cules, synapses and neurons, or we can study large brainareas at low resolution with whole-brain imaging. Thechallenge remaining is what lies in betweendthe thou-sands and millions of neurons that constitute functionalcircuits.

One example is to understand the essential circuitrythat mediates the neural bases of goal-directed action.Bradfield and Balleine point that current research inneuroscience is predominantly technique driven and,as a consequence, it can be a challenge to maintainthe balance between doing what is expedient andasking questions that are worth answering. Not allrecently developed techniques are equally useful instudying complex psychological capacities, somethingthat is particularly true of studies investigating goal-

V. GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

THE ANALYSIS OF CIRCUITS OF INTERACTING NEURONS 415

directed action in animals. In such experiments theevents to which the nervous system is exposed are pre-dominantly under the animal’s control rather than theexperimenters’, meaning, therefore, that, because theinitiating and terminating conditions for actions arefluid, the dynamics of the neural processes thatmediate both acquisition and subsequent performancecan be very complex. The challenge for the future isto bring this complexity under control. To the extentthat is achieved it may become possible to addressone of the most important open questions: it is stillnot known with any precision what learning rulesmediate the acquisition of goal-directed actions. Estab-lishing the essential circuitry supporting this learningprocess should help in that regard but there are impor-tant behavioral constraints to bear in mind. Forexample, different learning processes appear to beengaged at different rates by different schedules ofreward: ratio schedules generate more consistentgoal-directed learning and higher rates of performancethan interval schedules even when parameters areselected that match rates of reward delivery or interres-ponse times. Whether such distinctions can be capturedin associative or computational terms is still an openquestion. A number of researchers have recentlyclaimed that goal-directed learning is best captured,computationally, by model-based reinforcementlearning, using which a model of the environment isconstructed to ensure that action selection maximizeslong-run future reward. However, the performance ofgoal-directed actions respects the causal value of an ac-tion with respect to its specific outcome, and causalvalue does not necessarily coincide with reward maxi-mization. Indeed, considerable evidence suggests thatanimals prefer causal actions to both equally rewardingnoncausal actions and to performing no actions at all.Establishing the essential circuitry that mediates goal-directed action and the computational processes imple-mented in that circuit that make such actions possibleis one of the most important research problems andmost difficult challenges for future research.

DEVELOP INNOVATIVE TECHNOLOGIESAND SIMULTANEOUS MEASURES TO

UNDERSTAND HOW THE BRAIN MAKESDECISIONS

Consenting humans who are undergoing diagnosticbrain monitoring or receiving neurotechnology for clin-ical applications provide an extraordinary opportunityfor scientific research. This setting enables research onhuman brain function, the mechanisms of humanbrain disorders, the effect of therapy, and the value of di-agnostics. Seizing this opportunity requires closely

integrated research teams performing according to thehighest ethical standards of clinical care and research.New mechanisms are needed to maximize the collectionof this priceless information and ensure that it benefitsboth patients and science.

Examples include linking hormones and BOLDresponse during behavioral tasks (see Hermans andFernandez, Chapter 30, and Lefbvre and Sirigu, Chap-ter 31). Another related example concerns the effectsof acute stress on decision-making, which are just begin-ning to be understood. Such stress-induced shift from“reflective” to “reflexive” behavior may map twodistinct large-scale neural systems. This mapping isbased on a vast body of animal findings of effects ofstress-related neuromodulators within individual brainregions. It is essential that this cross-species inference iscorroborated in humans. There is, however, a paucity ofhuman pharmacological work detailing region-specificeffects and time-dependent effects of catecholaminessuch as dopamine and norepinephrine. In particular,we highlight the lack of human work on stress-induced dopamine release, which to our knowledge islimited to one seminal paper showing increased dopa-mine release using PET. Understanding the specificroles of dopamine and norepinephrine in the centralresponse to stressors will be critical to developing anunderstanding not only of immediate effects ondecision-making processes, but also of the specific vul-nerabilities that occur in response to acute stress in therealm of psychopathology. Regarding corticosteroids, afruitful road for further exploration will be to specifythe role of corticosteroids in limiting or terminatingthe acute response to stressors and promoting “reflec-tive” types of decision-making to enhance long-termadaptation. In particular, this role of corticosteroidshas not been explored fully in relation to stress-relatedpsychopathology. In investigating this, it will be impor-tant to distinguish the roles of baseline shifts and phasicresponses to stressors. One particularly promisingavenue is to further explore the potential of corticoste-roids in enhancing various forms of extinction-basedtherapy. Another large gap in our knowledge is howrapid and comprehensive shifts in neural activity aregenerated across large-scale neural systems. We high-lighted the potential contribution of stress-related neu-romodulators to this process, but these probably havedownstream effects on the balance between excitatoryand inhibitory neurotransmitters, which remain poorlyunderstood. Finally, the combination of basic neurosci-ence work with network-level analyses using functionalneuroimaging in humans has yielded important newinsights about the architecture of human cognition andits regulation at various levels of stress and arousal.One important future challenge is to translate thesenetwork-level findings back to basic neuroscience, in

V. GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

33. PERSPECTIVES416

which these network-level effects can be studied inmuch more spatiotemporal detail using, for instance,in vivo electrophysiological recordings and optogeneticmanipulations.

ADVANCING HUMAN DECISIONNEUROSCIENCE: UNDERSTANDING

NEUROLOGICAL/PSYCHIATRICDISORDERS AND TREATING BRAIN

DISEASES

Clinical developments coming from the field ofdecision neuroscience and reward processing are vastand likely to bring new promises. For example, inParkinson’s disease, the main current treatment is thedopamine precursor drug, L-dopa, but its efficacydecreases over time while severe side effects increase.Understanding the brain’s motor circuits and decisionalsystem with deep brain stimulation, which can restoremotor circuit function in patients with Parkinson’s dis-ease for up to several years, may also help to understandhow we form a decision. Which factors specificallyinvolve the inhibitory cortical network interacting withsubthalamic nucleus (STN) function in the decision-making process? Is it the decision conflict per se or otherfactors such as choice difficulty, appetitive/aversivevalence of the choices, or information integration that in-fluence STN activity and adjustment of response thresh-olds? Changing dynamically the response thresholdmight be a universal function in decision conflict ormight be task specific. Therefore it has to be shown ifdifferent neuronal circuits/mechanisms are involved,for example, adopting risk-taking strategies or actingunder time pressure along the line of an accuracyespeed trade-off. In a clinical perspective the exact elec-trode position in relation to changes in inhibitory controlshould give us further insights into the exact fiber tractsthat are involved in the adjustment of responsethreshold. High-frequency stimulation has a negativeimpact on decision threshold. In analogy it should beclarified if low-frequency stimulation improves thedecision-making process, reflecting the other side.Similar research concerning deep brain stimulation ofvarious areas into brain circuits for mood and emotionhave the potential to advance psychiatry in similarways.

As noted previously, reinforcement learning com-bined with model-based fMRI has proven a valuabletool to reveal the brain regions computing predictionerrors during learning stimulusereward/punishmentassociations. It is now possible to use such tool to under-stand various neurological and psychiatric diseases,such as schizophrenia. Critically, this perspective linksclinical observations to a vibrant and rapidly developing

cognitive neuroscience field. More complex andsophisticated models of reinforcement learning arebeginning to demonstrate the importance of adaptationsin key parameters such as prediction error and learningrate. By explicitly studying this adaptivity and how itmay be perturbed in mental illness, we are likely todevelop an ever-richer explanatory link between keysymptoms of mental illness and alterations in brain,behavior, and cognition. Progress in refining our under-standing in this regard could ultimately pave the wayfor the introduction of precision medicine (scientificallybased, individually tailored treatment) interventions inpsychiatry.

Similarly, understanding the neuronal bases ofnegative motivational behavior including avoidancewill be crucial points to elucidate aversive behaviorrelated to psychiatric disorders such as anxiety.Nonhuman primate models would be essential for pre-clinical study. It would be required to find a neuronalcircuit for aversive behavior and observation of itsabnormal state. These processes would pave the wayto understanding psychiatric disorders and developingtreatments.

One example comes from the field of anxiety disor-ders. Research on psychiatric disorders has increasinglyfocused on broad biological and psychological mecha-nisms that can confer risk for psychopathology gener-ally speaking, with specific manifestations of disordersinfluenced by environmental factors experienced atdifferent developmental time points. A huge challengecurrently faced by the field is delineating what thesekey domains of functioning are that may confer suchbroad risk when disrupted, and how these disruptionsare neurobiologically characterized, all to better under-stand who may develop these conditions and treat orideally prevent clinical anxiety. In the search for thesebroad underlying mechanisms of anxiety disorders,the research domain of decision-making has beenlargely ignored, with most human neuroimagingstudies focusing instead on the passive elicitationof fear or anxiety. While phenomenologically valid,this approach falls short in demonstrating the adaptiveor maladaptive behavioral consequences of anxiety,including the choices one makes between potentiallyrewarding and punishing outcomes. Along withemerging investigations of value-based decision-making in anxiety and its disorders, extant data thatdo not explicitly probe decision-making processes pro-vide evidence for disruptions to neurobiological mecha-nisms throughout the decision-making process. Futureresearch that systematically explores alterations to spe-cific aspects of the decision-making process and associ-ated changes in brain function or structure, and linksthese changes with symptoms of anxiety and associatedpsychopathology, has the potential to advance our

V. GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

ADVANCING HUMAN DECISION NEUROSCIENCE: UNDERSTANDING NEUROLOGICAL/PSYCHIATRIC DISORDERS 417

ability to diagnose, treat, and prevent the emergence ofanxiety disorders.

Our understanding of brain mechanisms underlyingdecision-making is also likely to bring new knowledgeto the understanding of drug and behavioral addictions.For example, gambling serves as a real-world exampleof risky decision-making and an activity that becomesexcessive for some people. Chapter 27 by Clark exploreswhat we currently know about decision-making and itsunderlying brain basis in gambling, with a focus ongambling disorder, the first recognized behavioraladdiction in the Diagnostic and Statistical Manual ofMental Disorders, fifth edition. Despite long-standingdiscussion in behavioral economics as to why peopleplay such games, given their negative expected value,it is only recently that researchers have begun to inves-tigate phenomena like loss aversion and the illusion ofcontrol in groups of participants separated in terms ofgambling involvement.

CONCLUSIONS

Collectively, the chapters from this book, DecisionNeuroscience, illustrate that: (1) theories and experimentsin neuroscience are helping to illuminate the mecha-nisms underlying decisions; (2) much remains to bedone regarding complex decisions; (3) social decisionneuroscience offers a special challenge of addressingmore complex problems that depend on predicting theintentions of others; (4) the social environment shapesneural structures and processes, and vice versa; and(5) new experimental methods (optogenetics) or nonin-vasive causal methods (e.g., TMS, tDCS) will help re-searchers to decipher the necessary brain regionsengaged in specific processes when making differenttypes of decisions.

To conclude, this book opens up three mainperspectives:

1. Pursue human studies and nonhuman models in parallel.The goal is to understand the human brain, but manymethods and ideas are developed first in animalmodels, both vertebrate and invertebrate.Experiments should take advantage of the uniquestrengths of diverse species and experimentalsystems. The research on animals has been and willremain crucial to determining the neural basis of theunderlying mechanisms of decision-making.

2. Cross boundaries in interdisciplinary collaborations. Nosingle researcher or discovery will solve the brain’smysteries. The most exciting approaches will bridgefields, linking experiments to theories, biology toengineering, tool development to experimentalapplication, human neuroscience to nonhumanmodels in innovative ways.

3. Integrate spatial and temporal scales. A unified view ofthe brain will cross spatial and temporal levels,recognizing that the nervous system consists ofinteracting molecules, cells, and circuits across theentire body, and important functions can occur inmilliseconds or minutes, or take a lifetime.

The most important perspective of the field of deci-sion neuroscience will be a comprehensive, mechanisticunderstanding of how the brain makes decisions thatemerges from synergistic applications of new technolo-gies and conceptual structures.

Reference

[1] Jorgenson LA, et al. The BRAIN Initiative: developing technologyto catalyse neuroscience discovery. Philos Trans R Soc 2015;370(1668).

[2] Deco G, Tononi G, Boly M, Kringelbach ML. Rethinking segrega-tion and integration: contributions of whole-brain modelling. NatRev Neurosci July 2015;16(7):430e9.

V. GENETIC AND HORMONAL INFLUENCES ON MOTIVATION AND SOCIAL BEHAVIOR

33. PERSPECTIVES418

Index

‘Note: Page numbers followed by “f” indicate figures, “t” indicate tables and “b” indicate boxes.’

AAcute stressbiphasic-reciprocal model, 379e380,

380fexecutive control network,

376e377behavioral effects, 376brain systems level effects,376e377

recovery, 377e378, 377finduction/hydrocortisone

administration, 379, 379flarge-scale neurocognitive systems,

372e373, 372fneural resources reallocation, 379e380,

380foverview, 371e372resource allocation shifts, 372e373salience network, 373e376behavioral effects, 374e375brain systems-level effects, 375e376cellular effects, 373e374neuroendocrine, 373e374recovery, 377e378, 377f

Amygdala, 179e180, 184, 184f, 193, 299,330, 340, 362e365, 374

overview, 33e41valence, 34e36cognitive processes, 39e41, 41fcognitive regulation, 36e39, 38fextinction, 36e39, 38flearning, 36e39, 38f

Anxiety disordersaction selection, 332e333learning, 334e335neural and behavioralmechanisms,

328e330, 329foutcome evaluation, 333e334overview, 327e335, 328frepresentation, 329f, 330e331valuation, 329f, 331e332

Appetitive brain systemsamygdala, 398e399decisionmaking, 402e404energy balance signals, 402e404food intake, 398e401, 399fghrelin, 403e404glucagon-like peptide 1, 404hippocampus (HC), 398e399insula, 399insulin, 402e403leptin, 403orbitofrontal cortex (OFC), 400

overview, 397e398, 398fpeptide YY, 404prefrontal regions, 400e401, 401fsatiety hormones, 404self-control and lateral prefrontal cortex,

401e402striatum, 400

Astatotilapia burtoni, 351e353, 352fconsequences, 353deceptive males, 353e354, 354ffemale mate choice consequences,

357e358, 358thierarchy, 354e355, 355fmale behavior, 355e356social system, 353transitive inference (male), 356e357,

356fAttention deficit hyperactivity disorder

(ADHD), 414e415Autism spectrum disorders (ASD), 201,

203Aversive behaviorsdirect and indirect pathways,

49e50pathological behaviors, 55ventral pallidum, 51e55ventral striatum, 51e55

BBalloon Analogue Risk Task (BART),

310, 331Basal gangliaappetitive approach behavior, 51aversive behaviors, 49e55cortex, 48f, 49overview, 47e49positive motivation, 51single-unit recording, 50ventral pallidum, 51e53avoidance preparation and anxietylevel controlling, 53e55

local inhibitory dysfunction,51e53

ventral striatumavoidance preparation and anxietylevel controlling, 53e55

local inhibitory dysfunction,51e53

Bayesian ideal observer, 150Biased competition, 99Blood oxygen level-dependent (BOLD),

189, 192, 306, 415e417

Brain Research through AdvancingInnovative Neurotechnologies(BRAIN), 411

Brain stimulation studiesoverview, 233e234social behavioral control, 238e240,

240fsocial cognition, 236e238, 238fsocial decisions, 234e235, 235fsocial emotions, 235e236, 237fsocial-specific neural activity, 240e241,

241f

CCatechol-O-methyltransferase (COMT),

144, 365Certainty equivalent (CE), 26Channelrhodopsin (ChR2), 35e36, 61Choice behavior, 121e123, 123fChosen value, 27e28Classical receptive field (CRF), 126Compassionempathy, 249prosocial motivation, 250

neural bases, 251e253, 253foverview, 247prosocial behavior, 253e255, 254fpsychological bases, 250e251, 251fsocial cognition, 248e249emotional contagion, 248e249mentalizing, 248mimicry, 248e249

Conditioned stimulus (CS), 34e35Corpus callosum (CC), 73e74Corticalebasal ganglialateral habenula, 14e15pedunculopontine tegmental nucleus,

15raphe serotonergic systems, 15subthalamic nucleus, 14thalamus, 14e15

DDBS. See Deep brain stimulation (DBS)Decision circuits, 123e125, 124fDeclarative, rational discounting

component, 88e89Deep brain stimulation (DBS), 315,

317e321Default mode network (DMN), 192Designer receptors exclusively activated

by designer drugs (DREADDs), 74

419

Diagnostic and Statistical Manual ofMental Disorders, fifth edition(DSM-5), 340, 361e362

Divisive normalization, 127e129DMF. See Dorsomedial frontal damage

(DMF)Dopamineblocking, 61fcircuitry, 62fe63fdecision-makingdecision variables updating, 27e28,28f

direct influences on choices, 28e29,29f

defined, 59e64function, 61e62neurons, 29overview, 21physiology, 59e61reinforcement learning, 263reward-prediction error, 21e23response components, 23reward learning, 23e25, 24f

schizophrenia, 263striatum, 64e67adaptive decision-making, 66circuits, 66e67habits, 64e66, 65finitiating movement, 64e66, 65fmotor skills, 64e66, 65fvalue representations, 66

subjective value, 25e26risk, 26utility, 26e27

Dorsal anterior cingulate cortex (DACC),4, 4f, 201, 204f, 332e333

Dorsolateral prefrontal cortex (DLPFC),138e139, 235e237, 239e241,362e364

Dorsomedial frontal damage (DMF), 279,279f

Dorsomedial striatum (DMS),72, 75f

Dynamic normalization model, 131,132fe133f

EElectrodermal activity (EDA), 341fElectroencephalography (EEG), 137e138,

199EsbereHaselgrove model, 113e114

FFeedback-related negativity (FRN), 205Five-choice serial reaction time task

(5CSRTT), 81e87, 90, 317, 318bFrontal lobesdecision-making, 279, 279fcomponent processes, 279e286, 280fdecisions over time, 285e286decision strategies, 284e285learning value, 282e283option identification, 280value and attention, 283e284, 283f

value construction, 280e284humans lesion evidence, 278designs, 278

overview, 277

GGambling disorderillusory control, 344e345loss aversion, 340e341, 341foverview, 339e340probability weighting, 341e342randomness perceptions, 342e343, 343f

Gambling rats, 90Gonadotropin-releasing hormone 1

(GnRH1), 351Good-based value, 165Gray matter (GM), 194, 195f

HHebbian rule, 165Hybrid reinforcement, 227e228, 229fHypothetical competitive decision

model, 29

IIllusory control, 344e345Immediate-early gene (IEG), 357Impulse control disorders (ICDs)associative learning, 307e308dopaminergic medications, 305e307,

306fimpulsivity, 310e311individual vulnerability, 305e307, 306foverview, 305parkinson disease, 305e307, 306f, 309freinforcement learning, 309freinforcing effects, 307e308risk, 309e310uncertainty, 309e310

Impulsivity, 310e311, 317e319defined, 81five-choice serial reaction time task

(5CSRTT), 81e82premature responding, 83

impulsive choice, 82e83motor impulsivity, 90neural basis, 84reward temporal discountingamygdala, 88e89corticolimbic influences, 88e89nucleus accumbens, 87

risky choice, 89e90stop-signal inhibition, 90stop-signal reaction time (SSRT), 83e84waiting impulsivity neural substrates5CSRTT premature responding,84e87, 85f

Intraparietal sulcus (IPS), 115f, 116Iowa Gambling Test (IGT), 264, 330

LLateral habenula (LHb), 63e64Lateral intraparietal area (LIP), 123e125,

124f, 127, 129e130, 130f

Lateral intraparietal region (LIP), 153Lateral prefrontal cortex (LPFC), 185Learning/motivational processesaction-outcome learning, neural bases

of, 72e74, 73fdorsomedial striatum convergence,

77e79incentive learning, 74e76, 75foutcome retrieval and choice, 76e77overview, 71e72prelimbic cortexemediodorsal

thalamus disconnection, 73fLinear decoder, 39

MMajor depressive disorder (MDD)decision-making, 361e364, 364fdefined, 361imaging genetics, 364e365

Medial frontal cortex (MPFC), 201Medial portion of the orbitofrontal cortex

(mOFC), 72, 76Mediodorsal thalamus (MD), 72Medium spiny neurons (MSNs), 64e65Midbrain dopamine neurons, 11e13, 11fafferent projections, 11f, 12efferent projections, 11f, 12striatonigrostriatal network, 13, 13f

Mixed selectivity, 99Model-free reinforcement, 227Monetary Incentive Delay Task (MIDT),

204e205Multi-Source Interference Task (MSIT),

204e205

NNash equilibrium, 225Neural computations, 127e130, 130fNeuromodulation, 144e145

OOne-stage theory, 270Optimal decision models, 151fOptogenetic stimulation, 66Optogenetic tagging, 60, 60fOrbitofrontal cortex (OFC), 37e38, 38f, 127Oxytocindefined, 387e388learning, 392memory, 392mode of release, 387e388, 388fnonsocial reward, 391e392, 392fperspectives, 392e393social brain, 387social decisions, 389e390, 389fcooperative behaviors, 389e390empathy, 390prosociality, 390social norms, 390trust games, 389e390

social reward, 390e392, 391fe392fsocial stimuliemotion perception, 388, 389fsensory perception, 388

INDEX420

PParkinson disease (PD), 59, 81, 297, 298f,

305e307, 306f, 309f, 315, 366e367,417

Perceptioneaction (PA) cycledecision execution, 101e102decision feedback, 102e103decision-making, 97e99, 98fdefined, 96e97, 96fe97fprediction and preparation, 99e101,

100fe101fPerceptual decision-makingaims and scope, 150beliefs, 158e160adaptive gain control, 159e160predictive coding, 159probability cues, 159

cortical neurons computations,153e154

mixed selectivity, 154posterior parietal cortex neurons,153e154

rodent, 153e154decision optimality, 150defined, 137e139, 149e150dorsolateral prefrontal cortex (DLPFC),

138e139electroencephalography (EEG),

137e138factors affecting, 139e145confidence, 143e144learning and training, 142e143neuromodulation, 144e145prestimulus state, 139e140reward and punishment, 140e141speed versus accuracy trade-off,141e142

human studies, 155e157BOLD signals, 155discrete computational stages,155e157, 156f

EEG signals, 155, 156finformation integration, 150e153lateral intraparietal region firing rates,153

observers sample informationsequentially, 150e152

sequential probability ratio test(SPRT), 150e152, 151f

observers, 157e158optimal bound collapses, 158speed and accuracy, 157e158, 157ftime-varying urgency signal, 157f, 158

overview, 149Posterior cingulate cortex (PCC), 192e193,

233, 235, 285e286, 321, 401fPosterior parietal cortex (PPC), 96e97, 112,

123e124, 153e154Posttraumatic stress disorder (PTSD), 53Prediction error, 99, 216Predictive machine, 102Prefrontal cortex, 39, 412e413decision execution, 101e102decision feedback, 102e103decision-making, 97e99, 98f

defined, 96e97, 96fe97fprediction and preparation, 99e101,

100fe101fPrimatessocial decision-makingamygdala, 184behavioral studies, 180e182, 181flateral prefrontal cortex (LPFC), 185neuronal correlates, 182e185, 183fOFC, 182e183, 183foverview, 179e180, 180f

Probability weighting, 341e342

RReinforcement learning (RL), 21e22,

33e34, 36, 121e122, 143, 163e176,203e204, 204fe205f, 212e216, 308,309f, 365e366, 412

computational models, 291be292b, 293fhybrid, 227e228model-free versus model-based, 226psychosisbehavioral studies, 264e265neuroimaging studies, 265e269,266fe267f

schizophrenia, 263social decision-making, 227

RescorlaeWagner models, 22, 27e28, 36,164, 215e216

Reward circuitcomplex network features, 15corticalebasal ganglialateral habenula, 14e15pedunculopontine tegmentalnucleus, 15

raphe serotonergic systems, 15subthalamic nucleus, 14thalamus, 14e15

midbrain dopamine neurons, 11e13, 11fafferent projections, 11f, 12efferent projections, 11f, 12striatonigrostriatal network, 13, 13f

overview, 3e4, 4fprefrontal cortex, 4e5ventral pallidum, 10e11, 10fventral striatum, 5e10amygdala/hippocampalprojections, 9

connections, 6e10, 8fcortical projections, 6e9efferent projections, 9e10features, 6, 7fthalamic projections, 9

Reward-prediction error (RPE) signal, 164,166e167, 166f, 291b, 294, 297

Reward processingpsychosisbehavioral studies, 264e265neuroimaging studies, 265e269,266fe267f

schizophrenia, 262Reward/punishment learninganterior insula and dorsal striatum,

299, 299fbehavioral tasks, 293b

dopaminergic opponent system, 295e296drug and lesion studies, 296e300, 298flimitations, 300neural candidates, 294e296opponent system, 295other opponent systems, 296overview, 291e292, 293f, 295fperspectives, 300serotoninergic opponent system, 296

Rewards valuecompetitive games, 170e171dopamine, 164flearning, 165e167overview, 163e164plastic synapses, foraging with, 166f,

167e170, 169frandom choice, 170e171representations, 164e165stochastic dopamine-dependent

plasticity, 166f, 167stochastic synapses, probabilistic

inference with, 171e173, 172fRight supramarginal gyrus (rSMG), 236,

237f

SSaliencebehavioral function, 111f, 113e114brain regions encoding, 115fcompound stimuli, 114fconfounds, 114e115defined, 113e114human brain signals, 116e117probability-based salience, 117fVMPFC, 117, 117f

Salvelinus fontinalis, 357Schizophreniaanhedonia, 271e272, 272fdelusions, 269e270dopamine, 263glutamate, 263e264overview, 261e262reinforcement learning psychosisbehavioral studies, 264e265neuroimaging studies, 265e269,266fe267f

reward processing psychosisbehavioral studies, 264e265neuroimaging studies, 265e269,266fe267f

Sensorimotor circuits, 123, 124f, 127Sequential probability ratio test (SPRT),

150e152, 151fSkill-chance confusion, 344Social brainhypothesis, 212medial prefrontal cortex (MPFC),

190e193, 190fe191fnetwork, 194e195, 195foverview, 189e190, 190fperspectives, 195e196superior temporal sulcus (STS),

193e194, 194ftemporal parietal junction (TPJ),

193e194, 194f

INDEX 421

Social conformity, 199, 203e206Social decision-makingreinforcement learningalgorithm, 228e229, 230fhybrid reinforcement, 227e228, 229fmodel-free reinforcement, 227model-free versus model-based, 226overview, 225e226

Social dominanceinterindividual differences, 216e222appraisal intercultural differences,216e218, 217f

social status and personality, 218neurochemical approaches, 219adrenocortical systems, 219e221, 220fdopamine, 220f, 221e222neural systems, 219e221, 220freproductive systems, 219e221, 220fserotonin, 220f, 221e222subordination, 219

reinforcement learning, 212e216social cognition, 212e215subordination, 214fe215f, 215e216

social hierarchieshealth and well-being, 211societies evolutionary pressure andfeature, 212, 213f

Social influenceoverview, 199prosocial behavior, 199e203autism observer effect, 201e203, 202fsocial reward processing, 199e201

valuation opinionsfuture directions, 206social conformity, 203e206, 204f

Social network size (SNS), 189Somatic marker hypothesis (SMH), 330SPRT. See Sequential probability ratio test

(SPRT)Standard economic theories, 226Subcallosal anterior cingulate cortex

(sACC), 362e364Subthalamic nucleus (STN), 417

corticobasal ganglia circuits, 316fanatomy, 315e317function, 315e317physiology, 315e317

decision-makinganimal studies, 317, 318bimpact model, 321e322, 322fneuropsychological studies, 319e321

direct pathway, 315e316impulsivity, 317e319indirect pathway, 315e316overview, 315

TTemporal difference (TD), 36, 163e165Temporal Experience of Pleasure Scale

(TEPS), 272, 272fTemporoparietal junction (TPJ), 115f,

116e117, 233, 238TEPS. See Temporal Experience of Pleasure

Scale (TEPS)Theory of mind/mentalizing, 199e200Three-circuit network model, 172e173Tonically active neurons (TANs), 66Transcranial direct current stimulation

(tDCS), 216, 234e235, 235f, 239e241,240fe241f, 414

Transcranial electrical stimulation (tES),234e235

Transcranial magnetic stimulation (TMS),199, 234e236, 235f, 414

UUnconditioned stimulus (US), 34e36,

38e39, 53e54, 54f

VValuebehavioral function, 110e111, 111fbrain regions encoding, 115fcompound stimuli, 114fdefined, 110e111

human brain signals, 111e113, 112foverview, 109e110ventral striatum, 112f

Value-based decision-making, 55, 140,163e176, 226, 277, 279, 327e328,376e377

Value codingbrain/behavior context dependence,

125e127, 126f, 128fe129fchoice behavior, 121e123, 123fcomputations, 412e413decision circuits, 123e125, 124fmechanism and behavior, 121, 122fneural computations, 127e130, 130foverview, 121temporal dynamics and circuit

mechanisms, 130e133, 132fe133fVentral striatum, 5e10amygdala/hippocampal projections, 9connections, 6e10, 8fcortical projections, 6e9efferent projections, 9e10features, 6, 7fthalamic projections, 9

Ventral tegmental area (VTA), 60e62, 306Ventrolateral prefrontal cortex (VLPFC),

193, 212Ventromedial frontal damage (VMF), 279,

279f, 283fVentromedial prefrontal cortex (VMPFC),

112e113, 117, 117f, 200e201, 330,332, 334

VMF. See Ventromedial frontal damage(VMF)

VMPFC. See Ventromedial prefrontalcortex (VMPFC)

Voxel-based lesion-symptom mapping,278, 283f

WWhite matter (WM), 194, 195f

INDEX422