constraints on decision making: implications from...

Constraints on decision making: Implications from genetics,personality, and addiction

Travis E. Baker & Tim Stockwell & Clay B. Holroyd

Published online: 9 May 2013# Psychonomic Society, Inc. 2013

Abstract An influential neurocomputational theory of thebiological mechanisms of decision making, the “basalganglia go/no-go model,” holds that individual variabilityin decision making is determined by differences in themakeup of a striatal system for approach and avoidancelearning. The model has been tested empirically with theprobabilistic selection task (PST), which determines wheth-er individuals learn better from positive or negative feed-back. In accordance with the model, in the present study weexamined whether an individual’s ability to learn from posi-tive and negative reinforcement can be predicted by geneticfactors related to the midbrain dopamine system. We alsoasked whether psychiatric and personality factors related tosubstance dependence and dopamine affect PST performance.Although we found characteristics that predicted individualdifferences in approach versus avoidance learning, these ob-servations were qualified by additional findings that appearinconsistent with the predictions of the go/no-gomodel. Theseresults highlight a need for future research to validate the PSTas a measure of basal ganglia reward learning.

Keywords Individual differences . Addiction . Personality .

Midbrain dopamine system . Basal ganglia . Reinforcementlearning . Decision making . Probabilistic selection task

Neuroimaging studies reveal that normal performance ondecision making tasks is associated with widespread activa-tions of the basal ganglia, midbrain dopamine system, andconnected structures. These neural pathways are integralcomponents of complex functional neuroanatomical loopsunderlying reinforcement learning and decision-making thatappear critical for several cognitive, motor, and emotional

functions (Packard & Knowlton, 2002). Individual variabil-ity related to genetics (Frank, Doll, Oas-Terpstra, & Moreno,2009; Frank & Hutchison, 2009; Klein et al., 2007), person-ality traits (Bornovalova et al., 2009; DeYoung et al., 2010;Jentsch & Taylor, 1999; Simon et al., 2010; Zermatten, Vander Linden, d’Acremont, Jermann, & Bechara, 2005), andcomplex psychiatric disorders (Beste, Saft, Andrich, Gold,& Falkenstein, 2006; Frank, Samanta, Moustafa, &Sherman, 2007; Frank, Scheres, & Sherman, 2007; Frank,Seeberger, & O’Reilly, 2004; Maia & Frank, 2011;Moustafa, Cohen, Sherman, & Frank, 2008; Moustafa,Sherman, & Frank, 2008; Solomon, Smith, Frank, Ly, &Carter, 2011; Steele, Kumar, & Ebmeier, 2007; Waltz,Frank, Robinson, & Gold, 2007; Waltz, Frank, Wiecki, &Gold, 2011; Wiecki & Frank, 2010) are associated with biasesin basal ganglia function in such tasks, reflecting idiosyncraticdifferences in our abilities to learn and make choices. Yet theneural mechanisms that underlie individual differences indecision making remain poorly understood.

According to an influential neurocomputational model ofdecision making, “the basal ganglia go/no-go model,” dopa-minergic signaling in the basal ganglia facilitates or sup-presses action representations during reinforcement learningtasks: Phasic bursts of dopamine activity facilitate rewardlearning by reinforcing striatal connections that express D1receptors (the “go/approach” pathway), whereas phasic dipsin dopamine activity facilitate avoidance learning byreinforcing striatal connections that express D2 receptors(the “no-go/avoidance” pathway; Frank et al., 2004).Empirically, the model predictions are typically tested withthe probabilistic selection task (PST), a trial-and-error learningtask in which participants are required to learn three concur-rent discriminations (stimulus pairs AB, CD, and EF),rewarded with schedules of 80 %/20 %, 70 %/30 %, and60 %/40 %, respectively. In a subsequent test phase, theparticipants are asked to select between novel combinationsof the original stimuli without receiving feedback. Participants

T. E. Baker (*) : T. Stockwell : C. B. HolroydDepartment of Psychology, University of Victoria, P.O. Box 3050,STN CSC, Victoria, British Columbia V8W 3P5, Canadae-mail: [email protected]

Cogn Affect Behav Neurosci (2013) 13:417–436DOI 10.3758/s13415-013-0164-8

who are more accurate at picking the stimulus that was mostfrequently rewarded (the “good stimulus”) are classified as“positive learners,” whereas participants who are more accu-rate at avoiding the stimulus that was most frequentlypunished (the “bad stimulus”) are classified as “negativelearners” (Frank, D’Lauro, & Curran, 2007).

The PST has provided insight into individual differencesrelated to reinforcement learning (Cohen & Frank, 2008),genetics (Frank, D’Lauro, & Curran, 2007; Frank et al.,2009; Frank & Hutchison, 2009; Frank, Moustafa, Haughey,Curran, & Hutchison, 2007), normal aging (Frank & Kong,2008), “top-down” modulation by orbital frontal cortex andanterior cingulate cortex (Frank & Claus, 2006; Paulus &Frank, 2006), pharmaceutical manipulations (Frank &O’Reilly, 2006), and psychiatric conditions (especiallyParkinson’s disease, attention-deficit hyperactivity disorder,and schizophrenia; for a review, see Maia & Frank, 2011).Consistent with the predictions of the go/no-go model, thisempirical work indicates that reinforcement learning signalscarried by the midbrain dopamine system are instrumental tothe decision making function implemented by the basalganglia. Here, we replicate and extend these previous findingsby utilizing the PST to investigate dopamine-related individ-ual differences in genetic makeup, substance use history andpersonality. Our motivation for examining each of these fac-tors is specifically as follows.

Genetics

The replicable effect of single genes on PST performanceacross multiple independent samples (Doll et al., 2011;Frank et al., 2007a, b; Jutras-Aswad et al., 2012)suggests that the task indexes a trait-level variable (i.e., areliable and valid phenotype). The ability to learn frompositive or negative reinforcement can be predicted by var-iability related to single-nucleotide polymorphisms (SNPs)affecting D1 and D2 gene expression in the go/approach andno-go/avoid pathways (Doll et al., 2011; Frank et al., 2009;Frank & Hutchison, 2009; Frank et al., 2007a, b).Specifically, the go/no-go model predicts that reducedstriatal D2 density should be associated with impaired ac-curacy on avoid trials, together with spared accuracy onapproach trials in the PST (Frank, D’Lauro, C., & Curran,2007; Frank et al., 2009; Frank & Fossella, 2011; Frank &Hutchison, 2009). Consistent with this prediction, Klein andcolleagues (2007) demonstrated that male carriers of the A1allele (A1/A1 and A2/A1 combined) of the Taq1A SNP ofthe DRD2 gene, which is associated with reduced D2 ex-pression (Thompson et al., 1997; but see Zhang et al., 2007),were selectively impaired at avoiding the bad stimulus dur-ing the test phase. Frank and Hutchison (2009) initiallyreported similar results in both males and females

combined, but found that effects of the Taq1A polymor-phism on accuracy on avoid trials were due to indirectassociation with the C957T SNP of the DRD2 gene; partic-ipants carrying the C allele of the C957T SNP, which hasalso been identified to cause a reduction in striatal D2receptor expression and binding potential (Hirvonen et al.,2004; Hirvonen, Laakso, et al., 2009; Hirvonen, Lumme, etal., 2009; but see Duan et al., 2003; Zhang et al., 2007),were relatively inaccurate on avoid trials but performedabout normally on approach trials (Frank et al., 2007a, b;Frank & Hutchison 2009; see also Doll et al., 2011;Jutras-Aswad et al., 2012). On the other hand, Frankand Hutchison (2009) revealed that carriers of the C allele(minor) of promoter SNP (C/T) (rs12364283) were selec-tively inaccurate on avoid trials. Because this polymor-phism is associated with greater D2 density (Zhang etal., 2007), this finding appears inconsistent with thego/no-go model.

The go/no-go model further predicts that good perfor-mance on approach trials relative to avoidance trials shouldbe associated with enhanced efficacy of striatal D1 recep-tors, as for example modulated by the PPP1R1B gene(Frank et al., 2007b; Frank et al., 2009). The M12 polymor-phism (rs907094) of the PPP1R1B gene codes for theDARPP-32 protein, which is highly concentrated in thestriatum, regulates the sensitivity of D1 striatal neurons toglutaminergic excitation and dopaminergic modulation, andis required for D1-receptor mediated synaptic plasticity andreward learning (Meyer-Lindenberg et al., 2007;Svenningsson et al., 2004). A previous study found thatAA compared to G carriers of this polymorphism wererelatively better at choosing the good stimulus comparedto avoiding the bad stimulus in the PST test phase, a resultthat implicates striatal D1 receptors in reward learning(Frank et al., 2007b).

It has also been proposed that SNPs related to the DRD4receptor should relate to performance on high conflict trialsin the PST (Frank & Fossella, 2011). For example, an fMRIstudy by Fan, Fossella, Sommer, Wu, and Posner (2003)found that individuals homozygous for the G allele of theDRD4-1217G displayed a relatively strong blood oxygena-tion level-dependent (BOLD) response in the anterior cin-gulate cortex to task conflict. This finding suggests thatcarriers of the G allele of the DRD4-1217G should performbetter on high conflict trials on the PST, consistent with aprediction by Frank and Fossella. However, to our knowl-edge the impact of the gene on PST performance has neverbeen investigated.

In the present study we sought to replicate and extendthese previous findings on the effects on PST performanceof genetic polymorphisms related to the expression of do-pamine receptors (D1, approach accuracy; D2, avoidanceaccuracy; D4, conflict accuracy).

418 Cogn Affect Behav Neurosci (2013) 13:417–436

Substance dependence

The go/no-go model has also provided insight intopsychiatric and neurological disorders that involve dis-turbances of the midbrain dopamine system and basalganglia, in particular, Parkinson’s disease, attention-deficit hyperactivity disorder, and schizophrenia. Themodel predicts that disruption in dopaminergic signalingin the basal ganglia Go and no-go pathways can selec-tively impair approach and avoidance learning on thePST (Maia & Frank, 2011). For example, the modelpredicts that people with Parkinson’s disease will bemore accurate on avoid than approach trials of thePST while off medication due to a diminished dopaminesignal, and more accurate on approach than avoid trialswhile on medication due to an enhanced dopaminesignal, findings that have been confirmed empirically (Franket al., 2007c, d; Frank et al., 2004). Furthermore, Maia andFrank (2011) have recently proposed that the go/no-go modelcan provide important insights into addiction. Because alladdictive drugs potentiate the reinforcing effects of themidbrain dopamine system (Di Chiara & Imperato, 1988)and upset the normal function of its neural targets(Hyman, Malenka, & Nestler, 2006), addiction is a naturalcandidate for investigating the go/no-go model using thePST.

Consistent with this proposal, undergraduate partici-pants classified as positive learners who were substancedependent displayed reduced accuracy on avoid trials ofthe PST together with normal accuracy on approachtrials, relative to positive learners who were not depen-dent (Baker, Stockwell, Barnes, & Holroyd, 2011). Bycontrast, substance-dependent negative learners showedthe reverse effect. These findings initially suggested twopathways to addiction, one characterized by impairedapproach learning and another by impaired avoidancelearning, but the findings failed to replicate in afollow-up study (Baker et al., 2010). For this reason,we reexamined this issue by integrating PST data gath-ered over three experiments (see the Materials andMethod section) and by testing a clinical populationon the PST before and after treatment following a 7-to 8-week interval. The control group in the clinicalstudy also afforded us with an opportunity to investigatethe reliability of the PST, which to our knowledge hasnever been explicitly examined (Ragland et al., 2012;Ragland et al., 2009).

Personality

Though less explored, the go/no-go model has also proveduseful for understanding individual differences in

personality. For example, a recent study attributed enhancedaccuracy on avoidance trials of the PST by depressed in-dividuals to hypersensitivity to punishment (Cavanagh,Bismark, Frank, & Allen, 2011). Furthermore, Frank andcolleagues (2007c, d) attributed impulsive decision-makingto an inability to self-modulate decision times as a functionof conflict, a consequence of a reduced coupling betweencognitive control regions (i.e., anterior cingulate cortex) andbasal ganglia output. Other personality traits associated withdifferent styles of reinforcement learning would also appearpromising for investigation using the PST. For instance,individuals high in sensation seeking are less sensitive tothe negative consequences of their actions, make relativelymore risky choices, and perform poorly on decision makingtasks (Noël et al., 2011; Zermatten et al., 2005; Zuckerman,1988; Zuckerman & Kuhlman, 2000). As well, anxiousindividuals appear to be hypersensitive to negative conse-quences following their actions and are risk-averse (Maner& Schmidt, 2006; Meyer-Lindenberg, 2010; Mitte, 2008;Raghunathan & Pham, 1999). Perhaps surprisingly, thesepersonality traits have all been associated with individualdifferences in dopamine expression (Hamidovic, Dlugos,Skol, Palmer, & de Wit, 2009; Laine, Ahonen, Rasanen, &Tiihonen, 2001; Lawford, Young, Noble, Kann, & Ritchie,2006; Montag et al., 2008), reinforcement sensitivity(Maner & Schmidt, 2006), and/or vulnerability to addiction(Conrod & Woicik, 2002). Hence in the present study weutilized the PST to elucidate individual differences in deci-sion making related to personality.

In sum, the go/no-go model proposes that reinforcementlearning signals carried by the midbrain dopamine systemare modulated by dopamine-related genetic polymorphisms.Furthermore, in accordance with the model’s conceptualframework, we propose that individual differences associat-ed with personality and addiction may be reflected in PSTperformance. In the present study we therefore sought toreplicate and extend previous findings on the effects of geneticpolymorphisms related to the expression of dopamine D1, D2,and D4 receptors on PST performance, to investigate theimpact of substance use on PST performance, and to investi-gate the association between personality traits and decisionmaking as reflected in PST performance.

The success of this investigation depends both on thego/no-go theory and on the utility of the PST for testingthe theory. Critically, the extensive empirical support thatthe PST has provided for the go/no-go model of the basalganglia strongly suggests that individual differences in PSTperformance should be stable over time, but to our knowl-edge this prediction has never been explicitly tested(Ragland et al., 2012; Ragland et al., 2009). Thus, as asubgoal of the study, we examined whether various mea-sures of PST performance were consistent within individ-uals over time.

Cogn Affect Behav Neurosci (2013) 13:417–436 419

Materials and method

Participants and procedures

Across three experiments, we collected survey data (sub-stance use history, personality risk factors associated withaddiction, and family history) and PST data from 499 par-ticipants. Of these, 378 were undergraduate students at theUniversity of Victoria, 70 of whom were part of a previousaddiction study (hereafter, “Study 1”; Baker et al., 2011) and196 participated as part of a genetics study on the neuralmechanisms of reward processing and substance use (here-after, “Study 2”; Baker et al., 2010), the data of which arereanalyzed here. The remaining participants were involvedin a third study, wherein we collected survey and PST dataover two sessions from undergraduate students (n=112) andfrom individuals who were currently seeking treatment at alocal treatment center (n=121; “Study 3”).

All participants were asked to complete the computer-based questionnaire followed by the PST. PST proceduresacross these three studies were identical, except for Study 3in which PST data were collected for all groups at twodifferent times (Time 1 and Time 2) spanning a 7- to 8-week interval. Furthermore, participants in Studies 1 and 2,but not in Study 3, had electroencephalography recordedduring a separate reward task. And participants in Study 2but not in Studies 1 and 3 had genetic samples taken fromthem. For Study 3, the treatment population was recruitedfrom individuals currently seeking addiction treatment atEdgewood Addiction Treatment center in Nanaimo, BritishColumbia, Canada (www.edgewood.ca). Edgewood is aprivate abstinence-based detox and drug rehab center, ori-ented on a 12 Step Foundation with an average length oftreatment of 7–8 weeks. Inclusion criteria for this study didnot involve any particular demographic group; the mix ofparticipants was expected to reflect the age, sex, and ethniccomposition of the general population of first- and second-year undergraduate students at the University of Victoriaand patients at Edgewood. All participants provided in-formed consent as approved by the Office of the Vice-President of Human Research, University of Victoria, andthe ethics board at Edgewood. The study was conducted inaccordance with the ethical standards prescribed in the 1964Declaration of Helsinki.

Questionnaire The computer-based survey was comprisedof several separate inventories (Baker et al., 2011), but forthe purpose of this study, we focused on the Alcohol,Smoking and Substance Involvement Screening Test(ASSIST; Humeniuk et al., 2008), a validated screening testdeveloped by the World Health Organization for identifyingthe degree of problematic substance use (i.e., tobacco,alcohol, cannabis, cocaine, amphetamine-type stimulants,

sedatives, hallucinogens, inhalants, opioids, and “otherdrugs”), and the Substance Use Risk Profile Scale(SURPS; Conrod & Woicik, 2002), a 23-item self-reportquestionnaire that provides measures on four dimensionsof personality—depression proneness, anxiety sensitivity, im-pulsivity, and sensation seeking—that are risk factors forsubstance use. Both inventories were scored according to theirguidelines; the scores served as continuous variables and wereused for group identification (see below).

Behavioral task: The probabilistic selection task The par-ticipants were asked to position their hand and forearms sothat both fingertips of the index fingers rested on buttons 1and 0 of a standard computer keyboard placed in front ofthem, and were provided with written and verbal instruc-tions explaining the procedure. On each trial of the task theyviewed a fixation cross (green circle, 1 s) followed by a pairof visual stimuli that are not easily verbalized by mostEnglish speakers (i.e., Japanese hiragana characters)presented in black on a white background in 72-point font(Fig. 1). They then pressed the key corresponding to thestimulus that they believed to be correct. Visual feedbackwas provided following each choice: the word “Correct!”printed in blue or “Incorrect” printed in red (1 s). If noresponse was made within 6 s then the words “no responsedetected” were displayed in red (1 s). During an initialLearning Phase participants were exposed to three pairs ofstimuli presented in random order (AB, CD, EF; Fig. 1, top).The response mappings were probabilistic such that one stim-ulus in each of the three pairs was rewarded on 80 %, 70 %,and 60 % of the trials, respectively, with the remaining stim-ulus in each pair rewarded on the complementary percentageof trials (see Fig. 1, top). Stimulus probability assignmentswere counterbalanced across participants.

Participants learned by trial-and-error to choose the morefrequently rewarded stimulus over the alternative in each pair,namely, by selecting stimuli A, C, and Emore often than B, D,and F. Critically, they could do so either by learning thatstimuli A, C, and E were associated with relatively morepositive feedback, by learning that stimuli B, D, and F wereassociated with relatively more negative feedback, or both.Participants advanced to the test phase of the task if after anyblock of 60 trials they satisfied performance criteria for thethree stimulus pairs (65 % A in AB, 60 % C in CD, 50 % E inEF), or after six blocks (360 trials) of training if these criteriawere not met. During the test phase participants were exposedto all possible combinations of these stimuli (i.e., AB, CD, EF,AC, AD, AE, AF, BC, BD, BE, BF, CE, and DF) in a randomorder. As before, participants were required to select thesymbol in each pair that they believed to be correct, butwithout receiving any feedback about their choices. Theywere told to use “gut instinct” whenever they did not knowhow to respond. Each test pair was presented six times.


http://www.edgewood.ca

Behavioral measures of interest consisted only of test phaseaccuracy and reaction time. Data from the training phase wasnot used for the statistical analyses. As in previous studies,each participant’s accuracy and reaction time data were takenfrom their performance in the test phase on “approach trials,”which involved the A stimulus (the “good stimulus”; AC, AD,AE, and AF); on “avoid trials,”which involved the B stimulus(the “bad stimulus”; BC, BD, BE, and BF); and on “high-conflict trials,” which involved test pairs with similar rein-forcement values (e.g., 80 % vs. 70 %; high conflict). For thesubsequent analysis, participants who tended to pick the stim-ulus that was most frequently rewarded during the learningphase (the “good stimulus”)—behavior that depends on learn-ing from positive reinforcement—were classified as “positivelearners,” whereas participants who tended to avoid the stim-ulus that was most frequently punished during the learningphase (the “bad stimulus”)—behavior that depends on learn-ing from negative reinforcement—were classified as “nega-tive learners.” The data of participants who could speak, read,and/or write Japanese, and/or of participants who did notperform better than chance on test phase trials consisting ofthe easiest stimulus pair (AB), were eliminated from furtheranalysis.

The participants in Study 3 completed the PST (and ques-tionnaire) over two sessions. For the treatment group, sessionone began within three days following intake assessment andthey were asked to complete the questionnaire and PST overtwo sessions (Times 1 and 2) spanning a 7–8 week interval.Importantly, for Study 3, different stimuli were used in Session2 to minimize potential learning effects between sessions. AnEdgewood staff nurse provided the description of the experi-mental procedure and presented the consent form to each

participant. Individuals who provided their consent were askedto complete the computer based questionnaire, followed by thePST. Session 2 commenced 7–8 weeks following initial intake,and the procedures were identical to the first. Following thecompletion of the second session, participants were providedwith a debriefing form. For both sessions, an Edgewood staffnurse was on hand to go through the procedures, consent anddebriefing instructions, and answered any questions from theparticipating individuals. For the undergraduate individuals, allprocedures were identical to that of the treatment group, withthe exception of environment, treatment provided, and studyadministrator (research assistant vs. Edgewood staff nurse). Inall cases, the participants’ privacy was ensured via a number-ing system.

Group assignment

Genotype (Study 2 only) For Study 2, following the PST,purified DNA was obtained from 196 participants bycollecting and processing saliva samples using OrageneOG-500 DNA Collection Kits (DNA Genotek, Ottawa,Canada), as per manufacturer’s instructions. Each partici-pant’s sample was processed at the Center for BiomedicalResearch DNA Sequencing Facility, University of Victoria.For the purpose of this study, eight separate regions ofgenomic DNA, each containing a polymorphism of interestthat appears to influence dopaminergic expression and func-tioning, were amplified by polymerase chain reaction(PCR), and purified. In particular, allele groups were basedon SNPs related to the DRD2 gene (Taq1A, rs1800497;C957T, rs6277; SNP2 rs12364283), PPP1R1B gene (M04,rs879606; M12, PPP1R1B rs907094), and the DRD4 gene

Fig. 1 Probabilistic learning task. (Top) Stimuli and reward probabilities (percentages of positive feedback). (Middle) Schematic of an exampletrial during the learning phase. (Bottom) Schematic of an example trial during the test phase


(C-521T, rs1800955; VNTR, and DRD4 -127 Indel).1

Genotyping of the DRD2 Taq1A, PPP1R1B rs879606,PPP1R1B rs907094, DRD4 C-521T, and DRD4 -127 IndelSNPs was accomplished by digestion with a restrictionendonuclease appropriate to each polymorphism (Taq1A,AvaII, MseI, FspI, and BstEII, respectively), and subsequentseparation by agarose gel electrophoresis. Genotyping of theDRD2 C957T, and DRD2 rs12364283 polymorphisms wasaccomplished by tetra-primer amplification refractory muta-tion system-polymerase chain reaction (tetra-primer ARMS-PCR), and subsequent separation by agarose gel electropho-resis. Genotyping of the DRD4 VNTR polymorphism wasaccomplished by separation of the amplification fragmentsby agarose gel electrophoresis. Each genotyping method foreach SNP target was verified by performing DNA sequenceanalysis (LI-COR 4200 Genetic Analyzer, Lincoln, NE) ona representative subset of samples. Alleles of interest are asfollows:

DRD2 (avoidance performance) The A1 allele of the TaqI(A1/A2) SNP (rs1800497), (Thompson et al., 1997; but seeZhang et al., 2007), the C allele of the C957T (C/T) SNP(rs6277) (Hirvonen et al., 2004; Hirvonen et al., 2009a, b;but see Duan et al., 2003; Zhang et al., 2007), and the Tallele of the promoter Zhang_SNP-2 (C/T) (rs12364283;Zhang et al., 2007) of the DRD2 gene are associated withreduced striatal D2 receptor expression and binding poten-tial. Note that due to the small prevalence of the A1A1genotype (3 %–5 % of healthy Caucasians), A1/A1 andA1/A2 participants are commonly grouped as A1+ partici-pants, as in Klein et al. (2007). By contrast, our relativelylarge sample size allowed for comparing the A1/A1, A2/A1,and A2/A2 alleles separately. Similarly, although the C/Callele of the C957T gene are typically grouped with C/Tallele (e.g., Frank et al., 2009; Frank et al., 2007b; but seeDoll et al., 2011, and Jutras-Aswad et al., 2012, whogrouped T/T with C/T), our sample size allowed for com-paring the C/C, C/T, and T/T alleles separately.

DARPP-32, PPP1R1B gene (approach performance) DARPP-32 is a phosphoprotein that is stimulated by D1 receptoractivation and that modulates dopamine-dependent syn-aptic plasticity (Svenningsson, Nairn, & Greengard,2005; Svenningsson et al., 2004; Svenningsson et al.,2002), and a 7-SNP haplotype of the PPP1R1B geneaffects DARPP-32 expression levels (Meyer-Lindenberget al., 2007).

DRD4 (conflict performance) We considered three SNPs ofthe DRD4 receptor gene [the promoter -521 (C/T) SNP(rs1800955), the indel -1217G ins/del (-/G) (rs12720364),and the variable number of tandem repeats (VNTR) polymor-phism (long/short) in exon III] that regulate dopaminergicmodulation of cognitive control mechanisms in frontal cortex.In particular, the T allele of the promoter -521 (C/T) SNP(rs1800955) (Okuyama, Ishiguro, Toru, & Arinami, 1999) ofthe DRD4 gene is associated with reduced D4 expression. The“long” of the variable number of tandem repeats (VNTR)polymorphism in exon III allele (VNTR-L=7 or more repeats,VNTR-S=6 or less repeats) is associated with a bluntedintracellular response to dopamine, appears to bind dopamineantagonists and agonists with great affinity, and is associatedwith attenuated inhibition of intracellular signal transduction(Oak, Oldenhof, & Van Tol, 2000). The DRD4 (-1217Gins/del) polymorphism appears to modulate the conflict mon-itoring function of ACC (Fan et al., 2003) but its effects on D4expression is currently unknown.

Substance dependence Participants across all three studies(Study 1, n=70; Study 2, n=196; Study 3, n=233) wereclassified according to their scores on the Global Continuumof Substance Risk (GCR) scale of the ASSIST. The GCRscores (GCRs) provide a measure of patterns of drug usethat impose a significant cost on the individual, are difficultto interrupt, and are likely to recur following interruption.Specifically, participants with GCR scores falling within thebottom (GCRs < 15), middle (GCRs = 38!16), and top (GCRs> 39) quartiles of our undergraduate student sample (collapsedacross studies, N=396) were classified as nondependent(ND), moderately dependent (MD), and substance dependent(SD), respectively. These scores are comparable to the cutoffsestablished in previous validation studies of the ASSIST fornondependence (score < 15) and dependence (score > 39.5)(Newcombe, Humeniuk, & Ali, 2005). Furthermore, asubstance-dependent treatment group (SDTx) was composedof the individuals seeking treatment, all but six of whomscored higher than the top cutoff. Because these six individ-uals were under treatment for nonsubstance related addictions(e.g., gambling, sex), their data were excluded from the treat-ment group (SDTx).

Personality The SURPS data of participants across all threestudies (Study 1, n=70; Study 2, n=196; Study 3, n=233)were used for this analysis. For the purpose of thisstudy, we computed the z-scores for each SURPssubscales—depression-proneness, anxiety-sensitivity, im-pulsivity, and novelty seeking (Conrod & Woicik,2002). Because the undergraduate and treatment groupswere categorically different in terms of their demographicsand other traits, z scores were calculated separately for the twogroups.

1 We also collected data for a gene that regulates the expression of thecatechol-O-methyltransferase (COMT) enzyme, the primary mecha-nism for dopamine inactivation in prefrontal cortex, but these datawere not analyzed.


Statistical analysis

Test–retest reliability analysis Undergraduate students inStudy 3 (n=112) were asked to complete the PST and ques-tionnaire over two sessions (Time 1 and Time 2) spanning a 7-to 8-week interval. First, Pearson correlation coefficients werecomputed to quantify the reliability of task performance. Anacceptable reliability was determined as r>.6 (McGraw &Wong, 1996). Next, we computed the kappa coefficient toexamine the consistency in which individuals were classifiedas positive, neutral, and negative learners across time. Valuesof Kappa from .40 to .59 are considered moderate, .60 to .79substantial, and .80 outstanding (Landis & Koch, 1977).

Genetic analysis We selected genetic variants of threedopamine-related genes (DRD2, DRD4, and PPP1R1B) toexamine genetic effects on PST performance (Table 1).First, we sought to replicate previous findings that D2-related genetic polymorphisms modulate avoidance behavioron the PST. A multiple regression analysis was conducted on

accuracy and reaction time data on avoidance trials with thedopaminergic genotypes DRD2-C957T, DRD2-Taq1A, andDRD2-SNP2 as predictor variables. Second, we sought toreplicate previous findings that D1-related genetic polymor-phisms modulate approach behavior on the PST. A multipleregression analysis was conducted on accuracy and reactiontime data on approach trials with the dopaminergic genotypesM04 and M12 of the PPP1R1B gene as predictor variables.Finally, to investigate the association between dopaminergicmodulation in frontal cortex and conflict performance, a mul-tiple regression analysis was conducted using accuracy andreaction time data on high-conflict trials with the DRD4variants (VNTR, C521T, and -1217G) as predictor variables.

Substance dependence analysis We sought to replicate aprevious observation of an interaction between learner typeand substance use by applying a repeated measures analysis ofvariance (ANOVA) on test phase accuracy and reaction timedata with Dependence Group (ND, MD, SD, SDTx) andStimulus Type (positive, negative) as factors. Furthermore,

Table 1 Genotype characteristics of the research sample population with PST accuracy and reaction

Gene SNP (ID) Allele Phenotype Freq (n) Approach Avoidance

ACC RT ACC RT

DRD2 TaqA1 (rs1800497) A2/A2 !!* 55 % (99) 77 % 1,075 70 % 1,250

A2/A1 !"* 39 % (71) 74 % 1,112 74 % 1,278

A1/A1 ""* 5 % (9) 57 % 1,277 56 % 1,333

C957T (rs6277) T/T !!* 27 % (49) 76 % 1,032 79 % 1,178

C/T !"* 54 % (97) 72 % 1,223 72 % 1,366

C/C ""* 18 % (33) 72 % 1,131 74 % 1,263

SNP2 (rs12364283) T/T "" 83 % (149) 72 % 1,259 76 % 1,333

C/T !" 18 % (35) 76 % 965 67 % 1,205

PPP1R1B (DARPP-32 protein) M04 (rs907094) G/G !! 71 % (113) 72 % 1,204 74 % 1,302

A/G !" 25 % (40) 73 % 1,111 78 % 1,273

A/A "" 3 % (5) 84 % 1,309 71 % 1,674

M12 (rs1800955) A/A !! 62 % (98) 72 % 1,188 74 % 1,282

A/G !" 32 % (50) 73 % 1,149 76 % 1,298

G/G "" 10 % (6) 76 % 1,317 77 % 1,587

Conflict

ACC RT

DRD4 C-521 T (rs1800955) C/C !! 23 % (42) 66 % 1,249C/T !" 53 % (95) 61 % 1,310

T/T "" 23 % (42) 65 % 1,327

Indel (rs12720364) G/G ** 40 % (73) 67 % 1,282

-/G ** 43 % (78) 62 % 1,312

-/- ** 16 % (16) 59 % 1,305

VNTR Short !! 54 % (98) 65 % 1,258

Long "" 45 % (81) 61 % 1,344

SNP, single nucleotide polymorphism. !", dopaminergic expression (i.e., D2/D4 density, DARPP-32 protein). * Controversy in the literature.** Phenotype currently unknown


for the Study 3 participants only, treatment effects on PSTperformance were analyzed by submitting PST accuracy andreaction time data as functions of dependence group (ND,MD, SD, SDTx), time (Session1, Session 2), and stimulustype (positive, negative) to repeated measures ANOVA.

Personality analysis A multiple regression analysis wasconducted on accuracy and reaction time data on approachtrials with the computed z scores of the four SURPssubscales—depression-proneness, anxiety-sensitivity, im-pulsivity, and novelty seeking—as predictor variables.

All analyses calculations were conducted with the statisti-cal analysis program SPSS (Version 18). The Benjamini–Hochberg procedure (Thissen, Steinberg, & Kuang, 2002)was applied separately to each analysis to correct for multipletesting. The effects of sex and age on performance werestatistically controlled using a stepwise approach in the re-gression analyses and added as covariates in the ANOVAs.

Results

Participants

Within the student population, the data of seven participants thatcontained coding errors; of two participants who could read,write, and/or speak Japanese; and of 51 participants (13.5 %)who failed to learn the task, were excluded, leaving those of 318participants. Of the 121 individuals currently seeking treatment,data were excluded from 51 participants for containing codingerrors and from 16 participants (13.2 %) who failed to learn thetask, leaving the data of 54 participants. Of these participants,only 25 agreed to complete the PST at Time 2. Of those in-dividuals who did not complete Session 2, nine individuals lefttreatment, 44 completed treatment but did not agree to partici-pate after Session 1, and eight were not reported. Notably, theGCRs for those individuals who left treatment (mean GCRs=108.6, SE=24) were relatively high as compared to those of theindividuals who completed treatment (mean GSR score=83.6,SE=4), t(111)=–1.8, p<.05. As well, the percentages of partic-ipants who failed to learn the task were about equal between theundergraduate and treatment populations, and were similar tothose from previous work (Doll et al., 2011; Frank&Hutchison,2009; Frank et al., 2007a, b).

PST test–retest reliability

To determine the reliability of the PSTover time, we conducteda test–retest reliability analysis on PST performance on 90undergraduate students in Study 3 who completed the PSTover two sessions spanning a 7- to 8-week interval. To oursurprise, accuracy on approach and avoidance trials failed todemonstrate adequate test–retest reliability (r=.09, p=.42, and

r=–.08, p=.33, respectively). Similar results were obtainedwhen a partial correlation was computed controlling for age,gender, GCRs, and personality traits: approach (r=.13, p=.24)and avoidance (r=–.09, p=.42). As well, reaction times forapproach and avoidance trials displayed poor test–retest reli-ability (r=.269, p<.01, and r=.257, p<.01, respectively).Similar results were reported with the partial correlation, ap-proach (r=.302, p<.01) and avoidance (r=.312, p<.005). Allother PST measures failed to demonstrate adequate test–retestreliability (r<.4). As a check, we conducted a test–retest anal-ysis on other variables: GCRs (r=.91, p<.001), depressionproneness (r=.721, p<.001), anxiety (r=.675, p<.001), impul-sivity (r=.791, p<.001), and novelty seeking (r=.862, p<.001)all demonstrated good test–retest reliability, in agreement withthe range prescribed by Kuntsi and colleagues (2001).

Next, we computed the kappa coefficient to examine theconsistency in which individuals were classified as positive,neutral, and negative learners across time. Values of Kappafrom .40 to .59 are considered moderate, .60 to .79 substantial,and .80 outstanding (Landis & Koch, 1977). The reliability forthe learner type was found to be Kappa=–.02 (p=.801), indi-cating inconsistency in learning bias across time. As a check,we removed individuals classified as neutral learners at Time 1or at Time 2. Again, the reliability for the learner type wasfound to be poor, Kappa=–.02 (p=.851) (Fig. 2).

Taken together, the present findings are suggestive ofregression towards the mean. To confirm this observation,we calculated the percentage of regression to the mean inboth the positive learner and negative learner groups usingthe formula PRTM=100(1 – r), where PRTM is defined as thepercent of regression to the mean, and r is the correlationbetween the two measures (Trochim & Donnelly, 2007).The positive learner group moved 81 % and 95 % of thedistance to the population mean for approach and avoidanceaccuracy, respectively. The negative learner group moved92 % and 70 % of the distance to the mean of the populationfor approach and avoidance accuracy, respectively. Thesefindings suggest that the initial response biases for the twogroups may stem from random variability in the populationrather than from trait-level differences in learning style.

PST and genetics

The above results suggest that learner type on the PST consti-tutes an unstable individual differences measure of decisionmaking, at least for a typical undergraduate student popula-tion. Alternatively, Doll et al. (2011) proposed that partici-pants can learn aspects of the task structure at Time 1 thatinfluence performance at Time 2, negating any consistency inperformance across the two sessions. If this reasoning iscorrect, then previous findings that dopamine-related geneticpolymorphisms contribute to individual differences in perfor-mance on the first exposure to the PST (Frank et al., 2009;


Frank & Hutchison, 2009; Frank et al., 2007a, b; Klein et al.,2007) should be replicable. For this reason we examinedwhether genetic determinants of D1 and D2 receptor expres-sion in the striatum differentially modulate the ability to learnfrom negative and positive feedback in the PST. For explor-atory purposes, we also examined the effects of genes thatregulate the expression of the D4 receptor—which mediatedopaminergic modulation of the control functions of prefron-tal cortex (Bilder, Volavka, Lachman, & Grace, 2004; Oak etal., 2000)—on PST conflict performance.

Genetic data were only collected for Study 2 (Table 1),which consisted of 196 undergraduate student participants(148 female, 48 male; mean age=20.4 years, SD=4.1, range=18–51). In total, 16 participants were excluded for performingbelow chance on the testing phase, and two participants wereexcluded due to a DNA sequencing errors. The majority of the

sample (161 participants: 90 %) classified themselves asCaucasian, ten as Asian, two as members of First Nations,one as African-American, one as Indian, and three as “other.”With the exception of the PPP1R1B polymorphisms (p<.05),2

our sample showed no deviation from Hardy–Weinberg equi-librium (Hardy, 1908; Weinberg, 1908) expectations at any ofthe selected loci for the overall sample (p>.05). These genotypefrequencies are consistent with previous studies of primar-ily Caucasian and unselected samples. Sex, age, andethnicity were controlled for using a stepwise approachin our regression analysis. Further, all results remained

2 Because the PPP1R1B polymorphisms displayed no deviation fromHardy–Weinberg equilibrium when minority subgroups were exclud-ed, we used only data from Caucasians for this genetic associationanalysis.

Fig. 2 Learning bias across time. (Left) Undergraduate student test phaseaccuracy on the probabilistic selection task (PST) for Time 1 (solid line)and Time 2 (dotted line), ahown separately for the “choose good” and“avoid bad” trials and for positive learners (top, n=39), negative learners(middle, n=41), and neutral learners (bottom, n=16), as classified at Time

1. Note that chance accuracy is 50 %. (Right) Learning bias (positive ornegative accuracy) derived from individual undergraduate student PSTaccuracy data, shown separately for positive and negative learners acrossTimes 1 and 2. Individuals’ learning biases were inconsistent from Time 1to Time 2 and are suggestive of regression toward the mean


consistent when regression models were tested with andwithout minority subgroups, so it is unlikely that popu-lation stratification confounded the results.

DRD2 and avoidance performance The go/no-go modelpredicts that reduced striatal D2 density (Frank & Fossella,2011; Frank & Hutchison, 2009) ! or increased striatal D2affinity (Doll et al., 2011)—should be associated with im-paired avoidance relative to approach trial performance inthe PST. Accordingly, we examined whether decreased D2expression as coded by several DRD2-related genetic poly-morphisms would replicate previous findings of relatively pooravoidance learning in these individuals. Specifically, we fo-cused on three genetic polymorphisms that affect D2 expres-sion: (1) the Taq1A (A1/A2) SNP (rs1800497), (2) the C957T(C/T) SNP (rs6277), and (3) the promoter SNP (C/T)(rs12364283) (“promoter SNP2”: Zhang et al., 2007). To con-firm the association between reduced striatal D2 density andimpaired performance on avoidance trials, a multiple regressionanalysis was conducted using these DRD2 SNPs as predictorvariables and accuracy and reaction time data on avoidancetrials as dependent measures. Type I errors were statisticallycontrolled following Benjamin and Hochberg (1995; B–H)with a corrected significance level of !=.0167. The regressionyielded a predictive model, F(3, 178)=3.5, p<.015, thatexplained approximately 5 % of individual variance inavoidance trial accuracy. Notably, the promoter SNP2uniquely contributed to the prediction of avoidance trialaccuracy, Beta=–.179, t=–2.5, p<.01 (corrected for multi-ple comparisons, B–H, p<.0167). Neither the DRD2-C957T (p=.17) nor the DRD2-Taq1A (p=.35) polymor-phisms showed an association with accuracy on avoidancetrials (Fig. 3). No associations were found for reaction time onavoidance trials, F(3, 178)=1.1, p=.354.

As a check, we also analyzed DRD2 alleles as categoricalvariables in order to replicate exactly the statistical methodsused in previous studies. First, male participants with the

A1/A2 and A2/A2 alleles of the DRD2-Taq1A performednearly identically on avoidance trials (p=.602), and no differ-ence were observed between approach and avoidance accura-cy within the A1/A2 allele group (p=.981)—a result that failsto replicate that of Klein and colleagues (2007). In addition,when the CT and CC alleles of the DRD2-C957T SNP wereanalyzed both separately and combined, the interaction wasnot significant (p=.854, p=.620) thus failing to replicate theresults of Frank and colleagues (2007b). Similarly, whenalleles were grouped according to Doll and colleagues(2011), TT/CT versus CC, the interaction failed to reachsignificance as well (p=.911).

PPP1R1B (gene coding for the DARPP-32 protein) andapproach performance The go/no-go model predicts thatbetter approach relative to avoidance trial performanceshould be associated with enhanced striatal D1 efficacy, ascoded for example by the PPP1R1B gene (Frank et al.,2007b). However, a multiple regression analysis on ap-proach performance with two polymorphisms associatedwith the PPP1R1B gene—the M12 (rs907094) SNP andthe M04 (rs879606) SNP—failed to yield a predictive mod-el for accuracy, F(2, 161)=0.74, p=.487, and reaction time,F(2, 161)=0.825, p=.440 (Fig. 4).3

As a check, we also ran a gene-dose analysis on therelative difference between approach and avoidance perfor-mance (approach accuracy – avoid accuracy) in order toreplicate exactly the statistical methods used in Doll et al.(2011). The analysis failed to replicate the previous gene-dose effect (p=.989) reported in Doll et al. (2011). Finally,to replicate the interaction observed in Frank et al. (2007b),we also analyzed PPP1R1B alleles as categorical variablesusing repeated measures ANOVAs. No interactions wereobserved between test pair condition and PPP1R1B geno-types (rs907904, p=.897; also, rs879606, p=.639), thusfailing to replicate the findings of Frank and colleagues(2007a, b).

Fig. 3 Undergraduate studentaccuracy on the probabilisticselection task (PST) in Study 2and in previous studies (dataadapted from [a] Frank &Hutchison, 2009, [b] Frank et al.,2007b, and [c] Klein et al., 2007)according to promotor SNP2(Left), C957T (Middle), andTaq1A (Right) for the “avoidbad” trials. Bars indicatestandard errors of the means. Forindividual allele performance,please refer to Table 1


DRD4 and conflict performance Because D4 receptor ex-pression in the frontal cortex is implicated in cognitive controlfunction, including response conflict and top down controlover the basal ganglia and other motor structures (Frank &Claus, 2006), we examined whether three SNPs related to theDRD4 gene—the promoter -521 (C/T) SNP (rs1800955), theindel -1217G ins/del (-/G) (rs12720364), and the variable num-ber of tandem repeats (VNTR) polymorphism (long/short) inexon III—predicts PST conflict performance. Type I errors werestatistically controlled following Benjamini and Hochberg(1995) with a corrected significance level of !=.0167. A regres-sion analysis identified a statistical trend between the DRD4SNPs and accuracy on high-conflict trials, F(3, 178)=2.3,p=.075, explaining approximately 4.5 % of individual vari-ance in accuracy on high-conflict trials. Notably, the DRD4-1217G uniquely contributed to the prediction of accuracy onhigh-conflict trials, Beta=.195, t=2.4, p=.019 (Fig. 5).However, these findings did not remain statistically significantwhen corrected for multiple comparisons, B–H, p<.0167.Other DRD4 SNPs were not associated with accuracy on highconflict trials. No associations were found for reaction time onhigh-conflict trials, F(3, 178)=1.1, p=.347

Substance dependence and PST

As noted above, PST performance might be reliable inspecific populations, especially in people with disorders thatdirectly impact the midbrain dopamine system. For exam-ple, two previous studies found that PST performancedepends on medication status in people with Parkinson’sdisease (Frank, 2005; Frank et al., 2007c, d; Frank et al.,2004). Evidently in these individuals the nonmedicateddisease state (characterized by dopamine system deterioration)and the medicated state (characterized by dopamine systemoveractivation) overwhelm other sources of variability indopamine system expression. We reasoned that chronicdrug use might similarly affect PST performance as alldrugs of abuse exert their addictive properties by actingdirectly on the midbrain dopamine system. However, thespecific direction of the effect is less clear.

Here we reexamined our previous PST findings insubstance-dependent individuals (Study 1: Baker et al.,2011)3 with an increased sample that included participantsfrom the Genetics study (Study 2) and the Treatment Study(Study 3). Data were excluded for 67 participants whoperformed below chance on the test phase and for 61 par-ticipants with coding errors, leaving the data of 371 partic-ipants in total. We applied a repeated measures ANOVA onaccuracy and reaction time data with Dependence Group(SDTx, SD, MD, ND) and Stimulus Type (approach, avoid-ance) as factors. Type I errors were statistically controlledfollowing Benjamini and Hochberg (1995) with a correctedsignificance level of !=.025 level. This analysis revealed amain effect of dependence group, F(3, 371)=4.9, p<.005.Planned comparisons indicated that ND individuals weremore accurate overall (78 %, n=99) compared to SD(73 %, n=76, p<.05), MD (71 %, n=152, p<.001), andSDTx (67 %, n=50, p<.001) individuals (Fig. 6).4

PST and personality

PST performance might also be more consistent in subpopu-lations with personality traits characterized by behavioral

Fig. 5 PST test phase accuracy associated with the DRD4-1217G genein Study 2. Data are shown for the -/-, G/-, and G/G allele groups for thehigh-conflict conditions. Bars indicate standard errors of the means

3 Following Baker and colleagues (2011), we also examined test phaseaccuracy by running separate ANOVAs on negative or positive learnersby dependence group. A significant interaction was detected for pos-itive learners, F(3, 158)=3.3, p<.05, but not for negative learners. Posthoc analysis indicated that positive learners tended to choose the goodstimulus about equally often across groups, p>.05, but ND participantstended to avoid choosing the bad stimulus more often (66 %) than didthe SDTx (57 %, p<.05), MD (56 %, p<.05), and SD (58 %, p=.07)participants. However, this result was not statistically significant fol-lowing the B–H correction (p<.025).4 A two-way ANOVA with repeated measures on PST accuracy andreaction time with Time (Time 1, Time 2), Test Condition (approach,avoid), and Dependence Group (SDTx, SD, MD, ND) as factorsrevealed only a main effect of group, F(1, 105)=8.5, p<.005. No othermain effects or interactions were detected.

Fig. 4 Undergraduate student accuracy on the probabilistic selectiontask (PST) in Study 2 and in a previous study ([a] data adapted fromFrank et al., 2007b), according to PPP1R1B M12 (rs907094) and M04(rs879606) for the “choose good” trials. Bars indicate standard errorsof the means


differences in learning and decision making. For example, arecent study demonstrated that enhanced sensitivity to pun-ishment in depressed individuals contributes to better perfor-mance on avoidance trials of the PST (Cavanagh et al., 2011).To investigate this possibility we focused on personality traitsthat are associated with decision making and addiction: im-pulsivity, novelty seeking, depression and anxiety. Overall, aregression analysis indicated that the SURPs measures mod-estly predicted participants’ accuracy on avoidance trials,F(4, 389)=2.1, p=.08. Within this model, only novelty seek-ing, Beta=.127, t=2.3, p=.02, and impulsivity, Beta=–.107,t=–2.0, p=.04, predicted accuracy on avoidance trials,whereas depression and anxiety were not statisticallysignificant. Although these results remained significantwhen sex, age, and GCR score were entered into theregression model, (p<.05), the results were not statisti-cally significant following the B–H correction (p<.0125).Accuracy for approach trials and reaction time for approach andavoid trials were not reliably predicted by the SURPs, p>.05. Asa check, we followed up with an analysis on the relative differ-ence in PST performance (avoidance – approach accuracy). Thisregression analysis indicated that the SURPs measures modestlypredicted participants’ relative PST accuracy, F(4, 389)=2.0,p=.09. Interestingly, within this model only novelty seekingpredicted the relative performance, Beta=.147, t=2.6, p<.009,indicating that high novelty seekers were selectively better atavoidance relative to approach performance. This result survivedcorrection for multiple comparisons, B–H correction (p<.0125).

Discussion

The go/no-go neurocomputational model of the basalganglia has been extremely influential, providing insightinto the neural mechanisms of a host of individual

differences and psychiatric disorders related to the functionof the midbrain dopamine system (Frank & Fossella, 2011).The model has been validated largely with the PST, whichprovides a means for differentiating between individualswho learn better from positive or negative feedback(Frank, Loughry, & O’Reilly, 2001; Frank et al., 2007c, d;Maia & Frank, 2011). Yet despite the fact that the PST hasbeen widely adopted for this purpose, to our knowledge itsreliability has heretofore not been determined—as is thecase for many widely used measures of cognitive function(Kunsti et al., 2001). We addressed this issue by examiningthe test–retest reliability of the PST data in Study 3 whereinthe control participants completed the task twice across a 7-to 8-week time span. To our surprise, the PST data failed todemonstrate adequate test–retest reliability in this sample.

In a recent study, Doll and colleagues (2011) alsoreported inconsistent results across two iterations of thePST. In particular, genetic effects related to D1 and D2expression were examined across two successive sessionsof the PST. Results for the first session were very similar tothose of Frank and colleagues (2007b), such that T alleles ofthe DRD2-C957T SNP were associated with enhancedavoidance accuracy, and the DARPP-32 AA allele groupwas associated with improved approach accuracy. However,these findings did not replicate the second time the test wasadministered to the same participants. To account for theseresults, the authors argued that “experience in the first taskiteration may have allowed participants to learn the structureof the task, potentially changing the strategies they used andoverwhelming individual differences in the basic reward-learning functions” (Doll et al., 2011). This reasoning sug-gests that genetic effects on PST performance should repli-cate across studies for the first exposure to the PST, despitebeing unreliable within participants across subsequent ex-posures to the task. We investigated this possibility in ourown data. Our sample of participants who provided geneticdata were tested on the PST only once, so the results do notspeak to the question of within-participants reliability, butthey do elucidate whether previous genetic findings associ-ated with the PST are replicable across studies.

D2-related genetic polymorphisms

The go/no-go model proposes that impaired performance onavoidance trials of the PST test phase results from a dimin-ished negative reinforcement learning signal, either directlyfrom reduced phasic dips in dopamine or indirectly fromreduced striatal D2 density (Frank & Hutchison, 2009;Frank et al., 2007b; see the review in Frank & Fossella,2011) and/or increased striatal D2 affinity (Doll et al., 2011).Thus, we considered three genetic polymorphisms that af-fect the expression of D2 receptors and, by extension,avoidance performance.

Fig. 6 Performance on the probabilistic selection task (PST): Accura-cy in the test phase of the PST for the substance-dependent (SD),moderately dependent (MD), nondependent (ND), and substance-de-pendent-in-treatment (SDTx) groups for overall PST performance.Student participants were grouped across all three studies (Time 1 forStudy 3), and SDTx participants were grouped at Time 1 for Study 3.Bars indicate standard errors of the means


DRD2-C957T It has been previously shown that partici-pants carrying the C allele of the DRD2-C957T SNPperformed relatively worse on avoidance trials relative toapproach trials, as revealed by an interaction between testphase accuracy (avoid vs. approach trials) and DRD2-C957T allele group (CC/CT vs. TT; Frank et al., 2007b,).This finding was followed with a gene–dose analysis thatfound that increasing numbers of C alleles were associatedwith worse avoidance performance and with no effect onapproach performance (Frank et al., 2007b; see also Doll etal., 2011). In the present study we failed to replicate thesefindings (i.e., the interaction and gene–dose effects), despiteusing a relatively large sample size and adhering to a similarstatistical approach. At first blush these null effects areperhaps surprising, but a closer examination of the literaturereveals that evidence in favor of these predictions are mixed.Notably, the critical statistical interaction between geneticgroup and learning style has not been replicated since Frankand colleagues’ original report. Further, only one study hasreplicated the gene-dose effect, and the result wasinterpreted according to a novel hypothesis about the gene(Doll et al., 2011; see below).

To be specific, four studies have re-examined these re-sults. First, Frank and Hutchison (2009) reanalyzed the datain Frank and colleagues (2007b) to demonstrate that theeffect was independent of other DRD2 SNPs (e.g.,Taq1A); given that they did not collect new data, this studydoes not constitute a replication (although it is sometimescited as such; e.g., Jutras-Aswad et al., 2012; Whitmer &Gotlib, 2012). Second, Frank and colleagues (2009) used anentirely different task, the temporal utility integration task(TUIT), to compare avoidance and approach performancebetween C957T allele groups (TT vs. CT/CC). They foundthat individuals carrying the T allele showed slower reactiontimes for conditions associated with increased expectedvalue, which they interpreted as indicative of enhancedavoidance learning. Because this study utilized a differenttask (TUIT vs. PST), analyzed a different performancemeasure (reaction time vs. accuracy), and yielded only sta-tistically marginal effects (p=.07), these results reflect apartial conceptual replication. Furthermore, it is not clearwhether these two tasks share a common learning mech-anism. For example, future studies should determinewhether individuals who perform poorly on avoidancetrials of the TUIT also perform poorly on avoidancetrials of the PST. The go/no-go model should also beelaborated to predict when individual differences inperformance should be seen in accuracy (as with thePST) or reaction time (as with the TUIT).

Third, although Doll et al. (2011) replicated a follow-upgene-dose analysis by Frank et al. (2007b) on the effects ofthe C957T on PST accuracy, they did not conduct anANOVA on test condition (avoid vs. approach trials) and

DRD2 group. This is a critical omission because the inter-action effect confirms that the impairment reflects a bias inindividual learning style (i.e., relatively better at avoidingthan approaching) as opposed to a global performance def-icit (e.g., better at the task overall). Fourth, Jutras-Aswadand colleagues (2012) utilized the PST to investigate wheth-er the C957T SNP is associated with cannabis dependence.In this study, the authors applied separate ANOVAs toapproach and avoidance accuracy for a categorical analysis(TT/CT vs. CC) of C957T data. They reported that theTT/CT group outperformed the CC allele group on avoid-ance trials, and interpreted this as a replication of Frank andcolleagues (2007b). However, the allele groups were cate-gorized differently across the two studies (Jutras-Aswad etal., 2012, CC vs. CT/TT; Frank et al., 2007b, CC/CT vs.TT). Jutras-Aswad and colleagues (2012) also did not reportthe critical interaction observed in Frank and colleagues(2007b), namely, whether participants carrying the T alleleperformed selectively better on avoidance as compared toapproach trials. Finally, unlike Frank and colleagues(2007b), a gene-dose analysis was not performed, and in-dividuals performing below chance in the test phase appearnot to have been excluded from the analysis. For thesereasons these results also do not constitute an exact replica-tion of Frank and colleagues (2007b).

Interpretation of these results is further complicatedby the fact that the impact of the C957T gene on D2expression and its association with avoidance perfor-mance is still a matter of contention. In particular,Doll and colleagues (2011) revised their theoretical pre-diction of this gene effect and their interpretation of theresults, relating their findings to D2 affinity rather thanD2 density. But their interpretation is in line with recentpositron emission tomography studies that have revealedthat the C allele is associated with enhanced striatal D2receptor binding potential (increased affinity) with noeffect on density (Hirvonen et al., 2004; Hirvonen,Lumme, et al., 2009). Note that ligands used thus farfor in vivo PET studies of dopamine D2 receptors inhumans do not discriminate between the low- and high-affinity states of the D2 receptor, and therefore can givemisleading impressions about dopamine utilization(Seeman et al., 2005). On the other hand, an in vitrostudy by Duan and colleagues (2003) found the oppo-site pattern of results: the T allele was associated withreduced mRNA translation and stability, leading to decreasedDRD2 expression in T allele carriers. Furthermore, Zhangand colleagues (2007) demonstrated that the C957Tpolymorphism is not in fact directly responsible forchanges in D2 receptor expression. Until these resultscan be replicated across studies, it is difficult to inferhow the C allele is associated with poor accuracy onavoidance trials.


DRD2-Taq1A Klein and colleagues (2007) demonstratedthat male carriers of the A1 allele (A1/A1 and A2/A1combined) of the Taq1A SNP of the DRD2 gene, in whichthe A1 allele is associated with reduced D2 expression(Thompson et al., 1997; but see Zhang et al., 2007), wereselectively impaired at avoiding the bad stimuli during thetest phase, thus supporting the go/no-go model. Frank andHutchison (2009) analyzed the data of both male and femalecarriers and reported similar results to Klein and colleagues(2007), but later found that when both the Taq1A and C957Twere analyzed together, the effect of Taq1A vanished,suggesting that the Taq1A effects on avoidance learning is aresult of an indirect association with the C957T SNP of theDRD2 gene (Frank & Hutchison, 2009). In the present study,we failed to replicate the findings reported by Klein andcolleagues (2007) (Fig. 3), which may be less surprising giventhat several studies have failed to find an association betweenallele groups of the Taq1A SNP and D2 expression in thestriatum (Zhang et al., 2007; see also Laruelle, Gelernter, &Innis, 1998; Lucht & Rosskopf, 2008).

DRD2 promoter SNP2 Frank and Hutchison (2009) reportedthat carriers of the C allele of DRD2 promoter SNP2 (Zhanget al., 2007) were selectively impaired at avoidance learn-ing, even when the effects of other DRD2 SNPs werestatistically controlled. We replicated this finding here suchthat C allele carriers relative to TT carriers were associatedwith poor accuracy on avoidance trials (Fig. 3). However, aseminal study that analyzed 23 polymorphisms within theDRD2 gene in terms of their effects on D2 receptor mRNAexpression in postmortem brain tissue demonstrated that the Callele of DRD2 promoter SNP2 enhances promoter activityover the T allele (Zhang et al., 2007). Thus, contrary to thego/no-go prediction, the C allele is associated with more D2receptors, even though participants carrying the C allele arerelatively inaccurate on avoidance trials in the PST. Thisobservation indicates that our results have actually replicateda finding that is inconsistentwith the predictions of the go/no-go model, at least as the theory is normally interpreted.

Nevertheless, an alternative reading of the go/no-go the-ory suggests the following interpretation: Greater numbersof D2 receptors may allow for greater inhibition of the no-gopathway. According to this proposal, larger numbers of D2receptors would require longer pauses in dopamine neuronfiring following negative feedback to disinhibit the systemsufficiently to facilitate avoidance learning. In other words,the greater the D2 receptor density, the less sensitive the D2system is to transient dips in dopamine. This mechanismwould result in a diminished negative learning signal fol-lowing negative feedback and ultimately to impaired accu-racy on avoidance trials, as we observed here. This proposalprovides a consistent account of these PST findings in thecontext of the previous investigations of the C allele of

promoter SNP2 by Zhang and colleagues (2007) and the Callele of C957T by Duan and colleagues (2003): impairedaccuracy on avoidance trials associated with increasedstriatal D2 receptor expression.


The go/no-go model predicts that enhanced striatal D1 recep-tor efficacy should improve performance on approach relativeto avoidance trials of the PST (Frank et al., 2007b). Striatal D1receptors have relatively low dopamine affinity, so their stim-ulation is hypothesized to depend on phasic dopamine burstswith larger bursts producing greater neural plasticity andbetter performance on approach trials of the PST. By exten-sion, impairments on these trials are proposed to result from adiminished positive reinforcement learning signal, either di-rectly from reduced phasic bursts in dopamine as seen inpeople with Parkinson’s disease or indirectly from reducedD1 efficacy as modulated by the PPP1R1B gene (Frank et al.,2007b). This gene codes for variation of the DARPP-32protein, which is partly responsible for regulating the sensi-tivity of D1 striatal neurons to glutaminergic excitation anddopaminergic modulation (Meyer-Lindenberg et al., 2007;Svenningsson et al., 2004). Consistent with this idea, a previ-ous study demonstrated that AA compared to G carriers of theM12 SNP were selectively better at approach relative toavoidance performance (Frank et al., 2007b; see also Doll etal., 2011). Similarly, Frank and colleagues (2009) used theTUIT to demonstrate that the AA allele group compared to theGG/GA group of the M12 variant responded faster during thelast block of a condition associated with decreased expectedvalue, which they interpreted as indicative of enhanced ap-proach learning. Despite these previous observations our in-vestigation of the M12 (rs907094) and the M04 (rs879606)polymorphisms of the PPP1R1B gene did not reveal anyallelic effects on approach trial accuracy (Fig. 4) or reactiontime, nor on the relative difference between approach andavoidance accuracy.

These results appear inconsistent with both the straight-forward predictions of the go/no-go model and the previousPST findings. Perhaps the role of DARPP-32 in rewardlearning is more complex than previously thought. TheDARPP-32 protein has been described as a major integratorof glutamate and dopamine signaling underlying synapticplasticity for reinforcement learning (Lindskog, Kim,Wikstrom, Blackwell, & Kotaleski, 2006). However, othershave argued that DARPP-32 may function to integrate in-formation processing in multiple brain regions via a varietyof neurotransmitters, neuromodulators, neuropeptides, andsteroid hormones (Svenningsson et al., 2004). DARPP-32also mediates effects of D2 receptor stimulation and plays acrucial role in the induction of both long-term depressionand long-term potentiation (Calabresi et al., 2000). For these


reasons, we suggest that evidence linking the PPP1R1Bgene variants, DARPP-32 expression, and approach perfor-mance should be interpreted with caution. Perhaps futuregenetic studies should select candidate genes that are moretightly linked to D1 receptor function such as the DRD1SNP (rs686) to investigate the role of D1 expression andapproach performance.


It has been proposed that the go/no-go pathways are regu-lated by top-down control from prefrontal cortex and ante-rior cingulate cortex (Cockburn & Frank, 2011; Frank &Claus, 2006). For instance, the G allele of the DRD4-1217GSNP was previously seen to correlate with anterior cingulatecortex fMRI BOLD activation to response conflict (Fan etal., 2003; Fossella et al., 2002). A previous study also foundthat deep brain stimulation of the subthalamic nucleus inpeople with Parkinson’s disease reduces coupling betweencognitive control regions in the frontal midline and basalganglia output nuclei, resulting in impulsive decision mak-ing (Frank et al., 2007c, d). Based on this evidence weproposed the D4 receptor—which is preferentiallyexpressed in frontal cortex—should be associated with highconflict performance on PST trials. Contrary to our expec-tations, of the three D4 SNPs we analyzed only the DRD4-1217G polymorphism pointed towards a trend in the pre-dicted direction, such that individuals carrying the G alleledisplayed better conflict performance (Fig. 5). Although thisfinding requires future confirmation, these observations aresuggestive of a functional link between conflict-related ac-tivity in the anterior cingulate cortex and top down controlover the basal ganglia during decision making (Cockburn &Frank, 2011).

Substance dependence and personality

We supposed that the functional and structural changes tothe go/no-go pathways induced by chronic drug use shouldalso be evident in PST performance, much as Parkinson’sdisease together with medication status overwhelms othersources of variability on the task. In fact, we previouslyfound that substance-dependent individuals were less accu-rate at PST accuracy in comparison to nondependent partic-ipants (Baker et al., 2011). Further analysis revealed thatsubstance-dependent individuals classified as positivelearners were less accurate at avoidance trials whileexhibiting normal accuracy on approach trials in compari-son to nondependent participants, whereas substance-dependent negative learners showed the opposite pattern,in addition to higher levels of depression proneness (Bakeret al., 2011). However, in a follow-up genetic study (Bakeret al., 2010) using a larger cohort of participants, we failed

to replicate these findings. Here we reexamined these find-ings by grouping data across the first two studies with datacollected in a third study, including that of a clinical popu-lation. Overall, although our findings demonstrated that thatnondependent individuals performed better overall on thePST compared to moderate users and both substance-dependent groups, our individual difference results relatedto learner type (positive vs. negative learners; Baker et al.,2011) were not fully replicated: Dependent positive learnerscompared to nondependent positive learners were relativelyinaccurate at their nonpreferred strategy, but this finding didnot survive correction for multiple comparisons.

A number of explanations could account for the overallimpairment in substance-dependent individuals on the PST.For example, the global performance deficit could resultfrom general neurological impairment rather than from aspecific disruption of the striatal reinforcement learningsystem. It is also possible that the dependent group was lessmotivated to engage in the task. In fact, a recent finding alsoobserved disrupted sensitivity to reinforcement in the PST(“blunting”) associated with anhedonia in depressed indi-viduals (Chase et al., 2010; see also Steele et al., 2007).Given these observations, we tentatively suggest that thereduced hedonic capacity associated with substance depen-dence (Koob, 1996; Koob & Le, 1997, 2008) may blunt theeffects of reinforcement on PST performance (Chase et al.,2010; Leventhal et al., 2006). This insensitivity to reinforce-ment may ultimately drive the decision making system towithdraw control over behaviors that should be inhibited(impaired avoidance learning) and impede behaviors thatshould be facilitated (impaired reward learning).

Although to our knowledge the go/no-go model has notbeen utilized to predict individual differences in PST per-formance related to personality per se, previous research hasdemonstrated how traits such as depression, impulsivity,anxiety, and novelty seeking are reflected in decision mak-ing. Further clues to the relationship between personalitytraits and PST performance are supported by a clinical studyon depression that showed that depressed individuals arebetter able to avoid the bad stimuli than approach the goodstimuli during the test phase of the PST (Cavanagh et al.,2011; but see Chase et al., 2010). Here we found thatavoidance accuracy was predicted by traits associated withaddiction, namely novelty seeking and impulsivity.Although the statistical relationship with novelty seekingsurvived correction for multiple comparisons, the relation-ship with impulsivity did not.

Previous work indicates that novelty seekers demonstratea negative bias in which positive feedback relative to neg-ative feedback exerts a smaller influence over behavior(Krebs, Schott, & Duzel, 2009). For instance, novelty seek-ing has been conceptualized as a dopamine-mediated heri-table tendency towards exploration and excitement in


response to novel stimuli (Cloninger, Svrakic, & Przybeck,1993): individuals high in novelty seeking show a heighteneddopaminergic response to novel events relative to rewardingevents, possibly indicating that these individuals may be rel-atively insensitive to natural rewards and more sensitive tohighly novel or stimulating rewards (Krebs et al., 2009). Herewe found that high novelty seeking predicted better accuracyat avoid trials relative to approach trials. Perhaps in theseparticipants, positive feedback failed to elicit a sufficientlypositive reward signal to bias performance in favor ofapproach trials. In support of this idea, animal studiesindicate that rats selectively bred for high reactivity tonovelty are characterized by elevated levels of extracel-lular dopamine in the striatum (Hooks et al., 1994;Piazza et al., 1991). Given that increased tonic activity cancontribute to decreased phasic activity of the dopamine systemin subcortical regions (Bilder et al., 2004), the increased tonicdopamine activity may contribute to a smaller reward signal inindividuals characterized by high novelty seeking.

Although we found that high impulsivity was also asso-ciated with poor accuracy on avoid trials, the result did notsurvive correction for multiple comparisons. Nevertheless,previous work supports the prediction that negative feed-back should have a small influence on behavior in impulsiveindividuals. For instance, individuals with orbital frontaldamage, who are often described as impulsive (Antonucciet al., 2006), show impaired ability to learn from negativefeedback in the PST (Wheeler & Fellows, 2008), suggestingdysregulated top-down control over basal ganglia no-gopathway. This idea is further supported by the finding thatdeep brain stimulation of the subthalamic nucleus, whichreduces coupling between cognitive control regions (anteri-or cingulate cortex) and basal ganglia output, results inimpulsive decision making (Frank et al., 2007c, d). Giventhat the frontal system implements control functions relatedto impulse regulation and top-down control over the no-gopathway, individuals displaying impaired inhibitory controlmight be expected to perform particularly worse on avoidtrials of the PST. In this context the relationship betweenimpulsivity and PST performance warrants further investi-gation in future studies.

Limitations and future directions

Our study has several limitations. First, of the 121 treatmentgroup participants who initially participated in Session 1 ofStudy 3, half agreed to complete the questionnaire in Session2, and of these, half agreed to engage in the PST. As is notuncommon for studies of clinical populations, the responseand participation rates in this study were not ideal. Variousinstitutional (e.g., scheduling problems, treatment commit-ments, etc.) and individual (e.g., lack of interest) factors mayhave contributed to nonparticipation. Future studies may

require larger sample sizes to identify treatment-relatedchanges to decision making. Furthermore, the lack of a strongassociation between personality traits depression and anx-iety and PST performance could be related to theSURPs, which was designed to assess individual per-sonality traits related to drug reinforcement, and as suchmay be insensitive to clinical symptoms of depressionand anxiety. Future studies could adopt more refinedcriteria for characterizing these personality traits.

More importantly, our observation that the PST perfor-mance measures were unreliable in our undergraduate sam-ple should be further examined in future studies. Althoughindividuals in the general population may not in fact bebiased toward learning from either positive or negativefeedback, it is also possible that the bias is real but that thetask requires further optimization to assess it. For this reasonwe recommend that the following measures be explored. (1)The stability of test phase performance across sessions couldbe improved if participants were given the opportunity toengage in a practice phase that included both a block of thelearning phase followed by a block of the test phase, en-abling participants to better understand the purpose of thetask. (2) Providing a monetary incentive during learningmight also increase participant engagement and perfor-mance stability (Chiew & Braver, 2011; Hakyemez,Dagher, Smith, & Zald, 2008; Savine, Beck, Edwards,Chiew, & Braver, 2010; Savine & Braver, 2010). (3)Because the test phase typically utilizes only a few testingpairs (each pair being tested six times), future studies couldidentify an optimal number of test-phase trials—or sets oflearning and test phase blocks—that would reduce individ-ual variability and avoid ceiling effects. (4) Averaging re-sults across three separate iterations of the task could yieldmore reliable results by lessening other sources of variancein the data (e.g., Frank et al., 2005). (5) A test of constructvalidity between the TUIT and PST is warranted; futurestudies should examine whether individuals who performpoorly on avoidance trials of the TUIT also perform poorlyon avoidance trials of the PST. (6) Lastly, investigations intothe genetic determinants of decision making should (a)establish psychometric properties of the performance mea-sures (as they relate, for example, to test–retest reliability; topractice effects; to sensitivity to clinical, developmental orexperimental change; and to construct validity), (b) catego-rize genotypes according to the hypothesis under investiga-tion, and (c) replicate statistical and methodologicalprocedures across studies.

Conclusion

The results presented here strongly suggest that PST perfor-mance measures—both in this study and others—should be


evaluated with caution. Of greatest concern, our data failedto show adequate test–retest reliability on any measure ofPST performance for a sample of nearly 100 undergraduatestudent participants. Task reliability is considered by manyto be a necessary (though not sufficient) precondition fortask validity (Cook & Beckman, 2006) and is believedespecially important for detecting differences across geneticgroups (to establish reliable endophenotypes), between ages(to reveal developmental trajectories) and within and/orbetween clinical manipulations (to assess treatment effects).Given its widespread use in atypical populations includingpeople with Parkinson’s disease, attention-deficit hyperac-tivity disorder, or schizophrenia, it is thus perhaps surprisingthat the validity of the PST has not yet been established foruse with normal controls. In this context, it is of even greaterconcern that most of the genetic results reported here failedto replicate previous findings. Exploratory analyses relatedto dopaminergic mechanisms of substance dependence andpersonality also did not yield positive results. Although thePST as a means for investigating the go/no-go model ofdecision making is supported by sound theoretical work atthe forefront of computational cognitive neuroscience re-search on decision making, these empirical findings under-score the need for future studies to validate the task. Theseobservations should be evaluated in the context of a broaderdiscussion in the psychology literature on the need for repli-cating high-impact results (Schimmack, 2012; Yong, 2012).

Author note This research was supported by Canadian Institutes ofHealth Research Operating Grant No. 97750. The first author wassupported by Doctoral Awards from the Integrated Mentor Programin Addictions Research Training (IMPART) and from the CanadianInstitutes of Health Research (No. 195501). We are grateful to theboard of directors and staff members of Edgewood Rehab Center,Nanaimo British Columbia, for their supportive collaboration on thisresearch, Mike Hunter and Gordon Barnes for their consultation on thisproject, Roderick Haesevoets for his work on genetic analysis, as wellas Marie Clipperton, Somayyeh Montazer-Hojat, and Elizabeth Plant,and the research assistants of the Learning and Cognitive ControlLaboratory for help with data collection.

References

Antonucci, A. S., Gansler, D. A., Tan, S., Bhadelia, R., Patz, S., &Fulwiler, C. (2006). Orbitofrontal correlates of aggression andimpulsivity in psychiatric patients. Psychiatry Research, 147,213–220.

Baker, T. E., Stockwell, T., Barnes, G., Haesevoets, R., Macleod, P., &Holroyd, C. B. (2010). Genetics, drugs, and cognitive control:Individual differences underlying substance dependence. Posterpresented at the Symposium on. Oxford, UK: Motivational &Cognitive Control.

Baker, T. E., Stockwell, T., Barnes, G., & Holroyd, C. B. (2011).Individual differences in substance dependence: At the intersec-tion of brain, behaviour and cognition. Addiction Biology, 16,458–466.

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discoveryrate: A practical and powerful approach to multiple testing.Journal of the Royal Statistical Society, Series B, 57, 289–300.

Beste, C., Saft, C., Andrich, J., Gold, R., & Falkenstein, M. (2006).Error processing in Huntington’s disease. PLoS One, 1, e86.doi:10.1371/journal.pone.0000086

Bilder, R. M., Volavka, J., Lachman, H. M., & Grace, A. A. (2004).The catechol-O-methyltransferase polymorphism: Relations to thetonic-phasic dopamine hypothesis and neuropsychiatric pheno-types. Neuropsychopharmacology, 29, 1943–1961.

Bornovalova, M. A., Cashman-Rolls, A., O’Donnell, J. M., Ettinger,K., Richards, J. B., deWit, H., & Lejuez, C. W. (2009). Risktaking differences on a behavioral task as a function of potentialreward/loss magnitude and individual differences in impulsivityand sensation seeking. Pharmacology Biochemistry and Behavior,93, 258–262. doi:10.1016/j.pbb.2008.10.023

Calabresi, P., Gubellini, P., Centonze, D., Picconi, B., Bernardi, G.,Chergui, K., . . . Greengard, P. (2000). Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-termdepression and long-term potentiation, opposing forms of synap-tic plasticity. Journal of Neuroscience, 20, 8443–8451.

Cavanagh, J. F., Bismark, A. J., Frank, M. J., & Allen, J. J. (2011).Larger error signals in major depression are associated with betteravoidance learning. Frontiers in Cognition, 2, 331. doi:10.3389/fpsyg.2011.00331

Chase, H. W., Frank, M. J., Michael, A., Bullmore, E. T., Sahakian, B.J., & Robbins, T. W. (2010). Approach and avoidance learning inpatients with major depression and healthy controls: Relation toanhedonia. Psychological Medicine, 40, 433–440.

Chiew, K. S., & Braver, T. S. (2011). Monetary incentives improveperformance, sometimes: Speed and accuracy matter, and somight preparation. Frontiers in Cognition, 2, 325. doi:10.3389/fpsyg.2011.00325

Cloninger, C. R., Svrakic, D. M., & Przybeck, T. R. (1993). A psy-chobiological model of temperament and character. Archives ofGeneral Psychiatry, 50, 975–990.

Cockburn, J., & Frank, M.J. (2011). Reinforcement learning, conflictmonitoring, and cognitive control: An integrative model of cin-gulate-striatal interactions and the ERN. R. Mars, J. Sallet, M.Rushworth & N. Yeung (Eds.), Neural Basis of Motivational andCognitive Control (311–331). Cambridge: The MIT Press.

Cohen, M. X., & Frank, M. J. (2008). Neurocomputational models ofbasal ganglia function in learning, memory and choice.Behavioural Brain Research, 199, 141–156.

Conrod, P. J., & Woicik, P. (2002). Validation of a four-factor model ofpersonality risk for substance abuse and examination of a briefinstrument for assessing personality risk. Addiction Biology, 7,329.

Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity andreliability for psychometric instruments: Theory and application.American Journal of Medicine, 119, 166, e7–16.

DeYoung, C. G., Hirsh, J. B., Shane, M. S., Papademetris, X.,Rajeevan, N., & Gray, J. R. (2010). Testing predictions frompersonality neuroscience. Brain structure and the big five.Psychological Science, 21, 820–828.

Di Chiara, G., & Imperato, A. (1988). Drugs abused by humanspreferentially increase synaptic dopamine concentrations in themesolimbic system of freely moving rats. Proceedings of theNational Academy of Sciences, 85, 5274–5278.

Doll, B.B., Hutchison, K.E., & Frank, M.J. (2011). Dopaminergicgenes predict individual differences in susceptibility to confirma-tion bias. Journal of Neuroscience, 31, 6188–6198.

Duan, J., Wainwright, M. S., Comeron, J. M., Saitou, N., Sanders, A. R.,Gelernter, J., & Gejman, P. V. (2003). Synonymous mutations in thehuman dopamine receptor D2 (DRD2) affect mRNA stability andsynthesis of the receptor. Human Molecular Genetics, 12, 205–216.


http://dx.doi.org/10.1371/journal.pone.0000086

http://dx.doi.org/10.1016/j.pbb.2008.10.023

http://dx.doi.org/10.3389/fpsyg.2011.00331




Fan, J., Fossella, J., Sommer, T., Wu, Y., & Posner, M. I. (2003).Mapping the genetic variation of executive attention onto brainactivity. Proceedings of the National Academy of Sciences, 100,7406–7411.

Fossella, J., Sommer, T., Fan, J., Wu, Y., Swanson, J. M., Pfaff, D. W.,& Posner, M. I. (2002). Assessing the molecular genetics ofattention networks. BMC Neuroscience, 3, 14.

Frank, M. J. (2005). Dynamic dopamine modulation in the basalganglia: A neurocomputational account of cognitive deficits inmedicated and nonmedicated Parkinsonism. Journal of CognitiveNeuroscience, 17, 51–72.

Frank, M. J., & Claus, E. D. (2006). Anatomy of a decision: Striato–orbitofrontal interactions in reinforcement learning, decision mak-ing, and reversal. Psychological Review, 113, 300–326.

Frank, M.J., Woroch, B.S., & Curran, T. (2005). Error-related negativ-ity predicts reinforcement learning and conflict biases. Neuron,47, 495–501.

Frank, M. J., D’Lauro, C., & Curran, T. (2007a). Cross-task individualdifferences in error processing: Neural, electrophysiological, andgenetic components. Cognitive, Affective, & BehavioralNeuroscience, 7, 297–308. doi:10.3758/CABN.7.4.297

Frank, M. J., Doll, B. B., Oas-Terpstra, J., & Moreno, F. (2009).Prefrontal and striatal dopaminergic genes predict individual dif-ferences in exploration and exploitation. Nature Neuroscience, 12,1062–1068.

Frank, M. J., & Fossella, J. A. (2011). Neurogenetics and pharmacologyof learning, motivation, and cognition. Neuropsychopharmacology,36, 133–152.

Frank, M. J., & Hutchison, K. (2009). Genetic contributions toavoidance-based decisions: Striatal D2 receptor polymorphisms.Neuroscience, 164, 131–140.

Frank, M. J., & Kong, L. (2008). Learning to avoid in older age.Psychology and Aging, 23, 392–398.

Frank, M. J., Loughry, B., & O’Reilly, R. C. (2001). Interactionsbetween frontal cortex and basal ganglia in working memory: Acomputational model. Cognitive, Affective, & BehavioralNeuroscience, 1, 137–160. doi:10.3758/CABN.1.2.137

Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., &Hutchison, K. E. (2007b). Genetic triple dissociation revealsmultiple roles for dopamine in reinforcement learning.Proceedings of the National Academy of Sciences, 104, 16311–16316.

Frank, M. J., & O’Reilly, R. C. (2006). A mechanistic account of striataldopamine function in human cognition: Psychopharmacologicalstudies with cabergoline and haloperidol. BehavioralNeuroscience, 120, 497–517.

Frank, M. J., Samanta, J., Moustafa, A. A., & Sherman, S. J. (2007c).Hold your horses: Impulsivity, deep brain stimulation, and med-ication in parkinsonism. Science, 318, 1309–1312. doi:10.1126/science.1146157

Frank, M. J., Scheres, A., & Sherman, S. J. (2007d). Understandingdecision-making deficits in neurological conditions: Insights frommodels of natural action selection. Philosophical Transactions ofthe Royal Society B, 362, 1641–1654.

Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot orby stick: Cognitive reinforcement learning in Parkinsonism.Science, 306, 1940–1943. doi:10.1126/science.1102941

Hakyemez, H. S., Dagher, A., Smith, S. D., & Zald, D. H. (2008).Striatal dopamine transmission in healthy humans during a pas-sive monetary reward task. NeuroImage, 39, 2058–2065.

Hamidovic, A., Dlugos, A., Skol, A., Palmer, A. A., & de Wit, H.(2009). Evaluation of genetic variability in the dopamine receptorD2 in relation to behavioral inhibition and impulsivity/sensationseeking: An exploratory study with d-amphetamine in healthyparticipants. Experimental and Clinical Psychopharmacology,17, 374–383. doi:10.1037/a0017840

Hardy, G. H., (1908). Mendelian proportions in a mixed population.Science 28, 49–50

Hirvonen, M., Laakso, A., Någren, K., Rinne, J. O., Pohjalainen, T., &Hietala, J. (2004). C957T polymorphism of the dopamine D2receptor (DRD2) gene affects striatal DRD2 availability in vivo.Molecular Psychiatry, 9, 1060–1061.

Hirvonen, M. M., Laakso, A., Nagren, K., Rinne, J. O., Pohjalainen, T.,& Hietala, J. (2009a). C957T polymorphism of dopamine D2receptor gene affects striatal DRD2 in vivo availability by chang-ing the receptor affinity. Synapse, 63, 907–912.

Hirvonen, M. M., Lumme, V., Hirvonen, J., Pesonen, U., Någren, K.,Vahlberg, T., . . . Hietala, J. (2009). C957T polymorphism of thehuman dopamine D2 receptor gene predicts extrastriatal dopa-mine receptor availability in vivo. Progress in Neuro-Psychopharmacology & Biological Psychiatry, 33, 630–636.doi:10.1016/j.pnpbp.2009.02.021

Hooks, M. S., Juncos, J. L., Justice, J. B., Jr., Meiergerd, S. M.,Povlock, S. L., Schenk, J. O., & Kalivas P. W.(1994). Individuallocomotor response to novelty predicts selective alterations in D1and D2 receptors and mRNAs. Journal of Neuroscience, 14,6144–6152.

Humeniuk, R., Ali, R., Babor, T. F., Farrell, M., Formigoni, M. L.,Jittiwutikarn, J., . . . Simon, S. (2008). Validation of the Alcohol,Smoking and Substance Involvement Screening Test (ASSIST).Addiction, 103, 1039–1047. doi:10.1111/j.1360-0443.2007.02114.x

Hyman, S. E., Malenka, R. C., & Nestler, E. J. (2006). Neural mech-anisms of addiction: The role of reward-related learning andmemory. Annual Review of Neuroscience, 29, 565–598.

Jentsch, J. D., & Taylor, J. R. (1999). Impulsivity resulting fromfrontostriatal dysfunction in drug abuse: Implications for the controlof behavior by reward-related stimuli. Psychopharmacology, 146,373–390.

Jutras-Aswad, D., Jacobs, M. M., Yiannoulos, G., Roussos, P., Bitsios,P., Nomura, Y., . . . Hurd, Y. L. (2012). Cannabis-dependence riskrelates to synergism between neuroticism and proenkephalinSNPs associated with amygdala gene expression: Case-controlstudy. PLoS ONE, 7:e39243. doi:10.1371/journal.pone.0039243

Klein, T. A., Neumann, J., Reuter, M., Hennig, J., von Cramon, D. Y.,& Ullsperger, M. (2007). Genetically determined differences inlearning from errors. Science, 318, 1642–1645.

Koob, G. F. (1996). Hedonic valence, dopamine and motivation. Mol.Psychiatry, 1, 186–189.

Koob, G. F., & Le, M. M. (1997). Drug abuse: hedonic homeostaticdysregulation. Science, 278, 52–58.

Koob, G. F., & Le, M. M. (2008). Review. Neurobiological mecha-nisms for opponent motivational processes in addiction. Philos.Trans. R. Soc. Lond B Biol.Sci., 363, 3113-3123.

Krebs, R. M., Schott, B. H., & Duzel, E. (2009). Personality traits aredifferentially associated with patterns of reward and novelty pro-cessing in the human substantia nigra/ventral tegmental area.Biological Psychiatry, 65, 103–110.

Kuntsi, J., Stevenson, J., Oosterlaan, J.,. & Sonuga-Barke, E.J.S.(2001). Test-retest reliability of a new delay aversion task andexecutive function measures. British Journal of DevelopmentalPsychology, 19(3), 339–348.

Laine, T. P., Ahonen, A., Rasanen, P., & Tiihonen, J. (2001). Dopaminetransporter density and novelty seeking among alcoholics.Journal of Addictive Diseases, 20, 91–96.

Landis, J. R., & Koch, G. G. (1977). The measurement of observeragreement for categorical data. Biometrics, 33, 159–174.

Laruelle, M., Gelernter, J., & Innis, R. B. (1998). D2 receptors bindingpotential is not affected by Taq1 polymorphism at the D2 receptorgene. Molecular Psychiatry, 3, 261–265.

Lawford, B. R., Young, R., Noble, E. P., Kann, B., & Ritchie, T.(2006). The D2 dopamine receptor (DRD2) gene is associatedwith co-morbid depression, anxiety and social dysfunction in


http://dx.doi.org/10.3758/CABN.7.4.297

http://dx.doi.org/10.3758/CABN.1.2.137

http://dx.doi.org/10.1126/science.1146157



http://dx.doi.org/10.1037/a0017840

http://dx.doi.org/10.1016/j.pnpbp.2009.02.021

http://dx.doi.org/10.1111/j.1360-0443.2007.02114.x

http://dx.doi.org/10.1371/journal.pone.0039243

untreated veterans with post-traumatic stress disorder. EuropeanPsychiatry, 21, 180–185.

Leventhal, A. M., Chasson, G. S., Tapia, E., Miller, EK., & Pettit, J. W.(2006). Measuring hedonic capacity in depression : a psychomet-ric analysis of three anhedonia scales. Journal of ClinicalPsychology 62, 1545–1558.

Lindskog, M., Kim, M., Wikstrom, M. A., Blackwell, K. T., &Kotaleski, J. H. (2006). Transient calcium and dopamine increasePKA activity and DARPP-32 phosphorylation. PLoSComputational Biology, 2, e119.

Lucht, M., & Rosskopf, D. (2008). Comment on “Genetically deter-mined differences in learning from errors. Science, 321, 200.

Maia, T. V., & Frank, M. J. (2011). From reinforcement learningmodels to psychiatric and neurological disorders. NatureNeuroscience, 14, 154–162.

Maner, J. K., & Schmidt, N. B. (2006). The role of risk avoidance inanxiety. Behavior Therapy, 37, 181–189.

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about someintraclass correlation coefficients. Psychological Methods, 1, 30–46. doi:10.1037/1082-989X.1.1.30

Meyer-Lindenberg, A. (2010). Behavioural neuroscience: Genes andthe anxious brain. Nature, 466, 827–828.

Meyer-Lindenberg, A., Straub, R. E., Lipska, B. K., Verchinski, B. A.,Goldberg, T., Callicott, J. H., . . . Weinberger, D. R. (2007). Geneticevidence implicating DARPP-32 in human frontostriatal structure,function, and cognition. Journal of Clinical Investigation, 117, 672–682. doi:10.1172/JCI30413

Mitte, K. (2008). Memory bias for threatening information in anxietyand anxiety disorders: A meta-analytic review. PsychologicalBulletin, 134, 886–911.

Montag, C., Buckholtz, J. W., Hartmann, P., Merz, M., Burk, C., Hennig,J., & Reuter, M. (2008). COMT genetic variation affects fear pro-cessing: Psychophysiological evidence. Behavioral Neuroscience,122, 901–909. doi:10.1037/0735-7044.122.4.901

Moustafa, A. A., Cohen, M. X., Sherman, S. J., & Frank, M. J.(2008a). A role for dopamine in temporal decision making andreward maximization in parkinsonism. Journal of Neuroscience,28, 12294–12304. doi:10.1523/JNEUROSCI.3116-08.2008

Moustafa, A. A., Sherman, S. J., & Frank, M. J. (2008b). A dopami-nergic basis for working memory, learning and attentional shiftingin Parkinsonism. Neuropsychologia, 46, 3144–3156. doi:10.1016/j.neuropsychologia.2008.07.011

Newcombe, D. A., Humeniuk, R. E., & Ali, R. (2005). Validation ofthe World Health Organization Alcohol, Smoking and SubstanceInvolvement Screening Test (ASSIST): Report of results from theAustralian site. Drug and Alcohol Review, 24, 217–226.

Noël, X., Brevers, D., Bechara, A., Hanak, C., Kornreich, C.,Verbanck, P., & Le Bon, O. (2011). Neurocognitive determinantsof novelty and sensation-seeking in individuals with alcoholism.Alcohol and Alcoholism, 46, 407–415. doi:10.1093/alcalc/agr048

Oak, J. N., Oldenhof, J., & Van Tol, H. H. (2000). The dopamine D(4)receptor: One decade of research. European Journal ofPharmacology, 405, 303–327.

Okuyama, Y., Ishiguro, H., Toru, M., & Arinami, T. (1999). A geneticpolymorphism in the promoter region of DRD4 associated withexpression and schizophrenia. Biochemical and BiophysicalResearch Communications, 258, 292–295.

Packard, M. G., & Knowlton, B. J. (2002). Learning and memoryfunctions of the basal ganglia. Annual Review of Neuroscience,25, 563–593.

Paulus, M. P., & Frank, L. R. (2006). Anterior cingulate activitymodulates nonlinear decision weight function of uncertain pros-pects. NeuroImage, 30, 668–677.

Piazza, P. V., Rouge-Pont, F., Deminiere, J. M., Kharoubi, M., Le, M.M., & Simon, H. (1991). Dopaminergic activity is reduced in theprefrontal cortex and increased in the nucleus accumbens of rats

predisposed to develop amphetamine self-administration. BrainResearch, 567, 169–174.

Raghunathan, R., & Pham, M. T. (1999). All negative moods are notequal: Motivational influences of anxiety and sadness on decisionmaking. Organizational Behavior and Human DecisionProcesses, 79, 56–77.

Ragland, J. D., Cohen, N. J., Cools, R., Frank, M. J., Hannula, D. E., &Ranganath, C. (2012). CNTRICS imaging biomarkers final taskselection: Long-term memory and reinforcement learning.Schizophrenia Bulletin, 38, 62–72. doi:10.1093/schbul/sbr168

Ragland, J. D., Cools, R., Frank, M., Pizzagalli, D. A., Preston, A.,Ranganath, C., & Wagner, A. D. (2009). CNTRICS final taskselection: Long-term memory. Schizophrenia Bulletin, 35, 197–212. doi:10.1093/schbul/sbn134

Savine, A. C., Beck, S. M., Edwards, B. G., Chiew, K. S., & Braver, T. S.(2010). Enhancement of cognitive control by approach and avoid-ance motivational states. Cognition & Emotion, 24, 338–356.

Savine, A. C., & Braver, T. S. (2010). Motivated cognitive control:Reward incentives modulate preparatory neural activity duringtask-switching. Journal of Neuroscience, 30, 10294–10305.

Schimmack, H. (2012). The ironic effect of significant results on thecredibility of multiple-study articles. Psychological Methods, 17,551–566. doi:10.1037/a0029487

Seeman, P. et al. (2005). Dopamine supersensitivity correlates withD2High states, implying many paths to psychosis. Proc. NatlAcad. Sci. USA. 102, 3513–3518

Simon, J. J., Walther, S., Fiebach, C. J., Friederich, H. C., Stippich, C.,Weisbrod, M., & Kaiser, S. (2010). Neural reward processing ismodulated by approach- and avoidance-related personality traits.NeuroImage, 49, 1868–1874. doi:10.1016/j.neuroimage.2009.09.016

Solomon, M., Smith, A. C., Frank, M. J., Ly, S., & Carter, C. S. (2011).Probabilistic reinforcement learning in adults with autism spec-trum disorders. Autism Research, 4, 109–120.

Steele, J. D., Kumar, P., & Ebmeier, K. P. (2007). Blunted response tofeedback information in depressive illness. Brain, 130, 2367–2374.

Svenningsson, P., Nairn, A. C., & Greengard, P. (2005). DARPP-32mediates the actions of multiple drugs of abuse. The AAPSJournal, 7, E353–E360. doi:10.1208/aapsj070235

Svenningsson, P., Nishi, A., Fisone, G., Girault, J. A., Nairn, A. C., &Greengard, P. (2004). DARPP-32: An integrator of neurotransmis-sion. Annual Review of Pharmacology and Toxicology, 44, 269–296.

Svenningsson, P., Tzavara, E. T., Liu, F., Fienberg, A. A., Nomikos, G.G., & Greengard, P. (2002). DARPP-32 mediates serotonergic neu-rotransmission in the forebrain. Proceedings of the NationalAcademy of Sciences, 99, 3188–3193. doi:10.1073/pnas.052712699

Thissen, D., Steinberg, L., & Kuang, D. (2002). Quick and easyimplementation of the Benjamini–Hochberg procedure for con-trolling the false discovery rate in multiple comparisons. Journalof Educational and Behavioral Statistics, 27, 77–83.

Thompson, J., Thomas, N., Singleton, A., Piggott, M., Lloyd, S., Perry,E. K., . . . Court, J. A. (1997). D2 dopamine receptor gene(DRD2) Taq1 A polymorphism: Reduced dopamine D2 receptorbinding in the human striatum associated with the A1 allele.Pharmacogenetics, 7, 479–484.

Trochim, W. M. K., & Donnelly, J. P. (2007). The research methodsknowledge base. Mason, OH: Thomson.

Waltz, J. A., Frank, M. J., Robinson, B. M., & Gold, J. M. (2007).Selective reinforcement learning deficits in schizophrenia supportpredictions from computational models of striatal–cortical dys-function. Biological Psychiatry, 62, 756–764.

Waltz, J. A., Frank, M. J., Wiecki, T. V., & Gold, J. M. (2011). Alteredprobabilistic learning and response biases in schizophrenia:Behavioral evidence and neurocomputational modeling.Neuropsychology, 25, 86–97.

Weinberg, W., (1908). Über den Nachweis der Vererbung beimMenschen. Jahresh. Ver. Vaterl. Naturkd. Württemb. 64, 369–382


http://dx.doi.org/10.1037/1082-989X.1.1.30

http://dx.doi.org/10.1172/JCI30413

http://dx.doi.org/10.1037/0735-7044.122.4.901

http://dx.doi.org/10.1523/JNEUROSCI.3116-08.2008

http://dx.doi.org/10.1016/j.neuropsychologia.2008.07.011

http://dx.doi.org/10.1016/j.neuropsychologia.2008.07.011

http://dx.doi.org/10.1093/alcalc/agr048

http://dx.doi.org/10.1093/schbul/sbr168

http://dx.doi.org/10.1093/schbul/sbn134

http://dx.doi.org/10.1037/a0029487

http://dx.doi.org/10.1016/j.neuroimage.2009.09.016

http://dx.doi.org/10.1208/aapsj070235

http://dx.doi.org/10.1073/pnas.052712699

Wheeler, E. Z., & Fellows, L. K. (2008). The human ventromedialfrontal lobe is critical for learning from negative feedback. Brain,131, 1323–1331.

Wiecki, T. V., & Frank, M. J. (2010). Neurocomputational models ofmotor and cognitive deficits in Parkinson’s disease. Progress inBrain Research, 183, 275–297.

Whitmer A. J., & Gotlib I. H., (2012) Depressive rumination and theC957T polymorphism of the DRD2 gene. Cogn Affect BehavNeurosci. 12, 741-747

Yong, E. (2012). In the wake of high profile controversies,pyschologists are facing up to problems with replication.Nature, 483, 298–300.

Zermatten, A., Van der Linden, M., d’Acremont, M., Jermann, F., &Bechara, A. (2005). Impulsivity and decision making. TheJournal of Nervous and Mental Disease, 193, 647–650.

Zhang, Y., Bertolino, A., Fazio, L., Blasi, G., Rampino, A., Romano, R., .. . Sadée, W. (2007). Polymorphisms in human dopamine D2receptor gene affect gene expression, splicing, and neuronal activityduring working memory. Proceedings of the National Academy ofSciences, 104, 20552–20557. doi:10.1073/pnas.0707106104

Zuckerman, M. (1988). Sensation seeking and behavior disorders.Archives of General Psychiatry, 45, 502–504.

Zuckerman, M., & Kuhlman, D. M. (2000). Personality and risk-taking:Common biosocial factors. Journal of Personality, 68, 999–1029.


http://dx.doi.org/10.1073/pnas.0707106104

constraints on decision making: implications from...

Documents