multiple reward systems and the prefrontal cortex

Neuroscience & Biobehavioral Reviews. Vol. 13. pp. 163-170. © Pergamon Press plc, 1989. Printed in the U.S.A. 0149-7634/89 $3.00 + .00

Multiple Reward Systems and the Prefrontal Cortex

A N N R O B E R T S O N

Department o f Psychology, University o f Alberta, Edmonton, Alberta T6G 2E9 Canada

ROBERTSON, A. Multiple reward systems and the prefrontal cortex. NEUROSCI BIOBEHAV REV 13(2/3) 163-170, 1989.- Electrical stimulation of the major divisions of the prefrontal cortex, the mediodorsal and sulcal areas, can serve as a reinforcing stimulus. Studies of self-stimulation of the prefrontal cortex have produced behavioral, anatomical and pharmacological evidence that the substrate of these rewarding effects can be dissociated from that subserving self-stimulation of ventral diencephalic sites such as the lateral hypothalamus. Other studies indicate that within the prefrontal cortex itself, self-stimulation of the medial and sulcal divisions can be attributed to dissociable processes. These observations suggest the existence of multiple, largely autonomous prefrontal subsystems involved in reinforcement. This raises the question of the functional significance of such systems, and of their organization. An approach to this problem is to consider the relationship between the behavioral functions of the prefrontal divisions and the characteristics of stimulation-induced reward obtained at each site. Studies of the effects of restricted prefrontal lesions indicate that the medial and sulcal divisions can be dissociated according to their involvement in the control of distinct types of sensory and motor events. Further experiments indicate that damage to each division causes selective deficits in the learning of stimulus-reinforcer and response-reinforcer relations, depending in part on the nature of the reinforcing event. Conditioning experiments further show that the rewarding effects produced by stimulation of these areas are preferentially associated to sensory events which correspond to the functional specialization of each division. These data are interpreted to suggest that different rewarding events and/or different attributes of rewarding stimuli are processed by distinct systems which are reflected by the organization of dissociable self-stimulation pathways.

Medial prefrontal cortex Sulcal prefrontal cortex Self-stimulation Conditioned place preference Corticocortical connections Reward

Conditioned taste preference Reinforcement

IT is generally recognized that the concept of reinforcement as a causal process in learning was introduced to psychology by Thorndike's statement of the Law of Effect (69), which held that reward and punishment play a central role in the shaping of behavior by virtue of their unique ability to influence the likelihood of responses preceding their occurrence. It is less frequently acknowledged that in order to avoid circularity in the Law of Effect, Thorndike explicitly defined rewards by referring to biological processes. This he did by equating the action of a rewarding stimulus to a specific physiological effect, defined as " . . . the unknown reaction of neurones which is aroused by the satisfier and which strengthens connections upon which it impin- ges" (70). The earliest definition of reinforcement thus also implied that rewards are stimuli that share the property of activating specific neural elements involved in effecting the process underlying associative learning.

The discovery of electrical self-stimulation by Olds and Milner (44) established that reward could indeed occur as a consequence of the stimulation of brain tissue and could therefore be studied physiologically. Self-stimulation (SS) of the brain offered the promise of identifying the circuitry subserving the process by which behavior is modified by its consequences. However, the task of defining such a reward system by delineating the anatomical extent of sites in the central nervous system that support SS has proven to be a most difficult one. This is in no small part related to the fact that SS has been obtained from a remarkable variety of brain sites, widely distributed throughout all levels of

the neuraxis. This widespread distribution of SS sites constitutes a serious impediment to attempts to describe the substrates of SS in the context of an anatomically coherent functional circuit assumed to mediate the effects of all rewards on behavior. The alternative view is that separable substrates of reward exist [e.g., (45)]. However, a significant difficulty besetting the analysis of brain stimulation reward along these lines is the relative paucity of information on the properties of SS derived from many of the sites known to support SS. It is the case even today that a majority of SS experiments have used stimulation sites restricted to the portion of the ventral diencephalon delineated rostrally by the basal forebrain and caudally by the ventral tegmentum and including the medial forebrain bundle (MFB) and lateral hypothalamic complex.

The present paper focusses on the prefrontal cortex (PFC), a brain area which, although long known to support SS (57), has only recently been studied more systematically. Even so it appears clear that, on both anatomical and pharmacological grounds, the substrates of SS of the PFC are dissociable from those of SS of the MFB. The first section below reviews these data. The following section focusses on the significance of multiple reward-related pathways for an analysis of the substrates of central reinforcement mechanisms by examining SS of the PFC in more detail. Evidence pertaining to the organization of the PFC reward system is presented and argued to show that dissociation can be demonstrated even within this structure. In the third section, it is suggested that a similar pattern of dissociation of function in other non-SS behaviors can be demonstrated within the PFC. In the final

163

164 ROBERTSON

Pt.

kid

Ald~

FIG. 1. The organization of the prefrontal areas in the rat [adapted from (33) and (79)]. Left: coronal sections from anterior (top) to posterior (bottom) illustrating the relative position of the prefrontal subdivisions. Right: the same areas depicted on a lateral (top) and midsaginal section (bottom). Abbreviations: AC. anterior cingulate; Aid, dorsal agranular insular area; Alv, ventral agranular insular area; IL, infralimbic area; OF, orbitofrontal area; PL, prelimbic area.

section, data are presented to suggest that this apparent parallel is of significance in trying to account for the existence of multiple SS substrates.

INDEPENDENCE OF THE PFC FROM MFB REWARD MECHANISMS

The PFC in mammals is traditionally defined as the cortical areas near the frontal poles that receive direct projections from the mediodorsal nucleus of the thalamus (33). In the rat, it can be divided in two rather broadly-defined zones. One zone occupies the dorsal bank of the rhinal sulcus and is therefore referred to as the sulcal PFC [e.g., (2)]. On cytoarchitectural and anatomical grounds, this area can be further divided into the anterior and ventral agranular insular cortices and the orbitofrontal cortex (33,79). The other zone lies along the rostral part of the medial hemispheric wall. This area is often referred to as the medial PFC (33) and can be subdivided into both prelimbic and infralimbic constituent portions (see Fig. 1).

In the rat, the most salient effects of electrically stimulating either the sulcal or the medial PFC are related to the inhibition of ongoing behavior. Noncontingent stimulation of sites widely distributed throughout the medial and sulcal PFC can disrupt the performance of instrumental responses such as bar pressing as well as locomotion and orienting responses involving head and body movements (13, 65, 75-77). Stimulation of both the medial and sulcal PFC also produces autonomic activation manifested primarily by bradycardia and vasopressor responses (30,40). Stimulation of some sites within the medial PFC can induce head movements caused mainly by neck extension (14).

In contrast, noncontingent electrical stimulation of the rodent MFB is associated with a very different pattern of effects which predominantly feature the performance of a wide variety of species-typical goal-directed responses as well as marked in-

creases in overall activity (27, 28, 37, 71, 72). These differences are also reflected in the topography of

self-stimulation behavior seen at each site: whereas SS along the MFB is typically performed at high rates and is accompanied by much activity and a range of consummatory-like responses directed at the operant manipulandum, SS of the PFC is acquired very slowly (16) and yields a pattern of responding strikingly different; animals press at low constant rates and do not display any hyperactivity [(10) and personal observations].

Consistent with these behavioral differences between the effects of electrical stimulation of the MFB and the PFC, there is a substantial body of evidence that the rewarding effects of electrical stimulation of the two areas do not involve activation of the same neural circuits. In a seminal experiment, Huston and Borbely (31) showed that SS of the MFB does not require the participation of cortical elements since removal of most of the telencephalon did not prevent animals from performing simple operant responses in order to obtain MFB stimulation. These findings were extended in other experiments (32,47) demonstrating that even the relatively more complex behavior of lever pressing could be maintained by MFB stimulation in the absence of the neocortex ipsilateral to the electrode and with the major commissures cut. These results argue strongly that the critical elements underlying SS of the MFB are not of cortical origin and are likely not to require access to cortical processes.

A similar conclusion was reached in other experiments where the rewarding efficacy of MFB stimulation was assessed before and after extensive damage to the PFC. Stellar et al. (66) compared the parameters of the reward summation function (which is thought to provide a reliable and artifact-free index of the rewarding value of the stimulation) in two groups of rats, one of which had received large aspiration lesions of the frontal poles. No significant differences between the two groups were seen in the slope or scale of the function, indicating that the lesions did not alter the animals' sensitivity to shifts in the rewarding magnitude of the stimulation. A somewhat similar result was obtained by Cole and Wise (8) who performed large ablations of the frontal lobe and tested animals implanted with electrodes in the MFB ipsilateral and contralateral to the cortical lesions. They found a slight increase in frequency thresholds ipsilateral to the lesion and a small decrease in the same measure from electrodes contralateral to the lesions, suggesting that the lesions had a marginal effect on the effectiveness of MFB stimulation.

Corbett et al . (11) did the converse experiment. They examined the effects of large electrolytic lesions of the MFB on SS of the medial PFC. Despite damage to the entire MFB and neighbouring lateral hypothalamic area and internal capsule, SS was not adversely affected.

It is well established that MFB SS is particularly sensitive to treatments affecting the integrity of dopaminergic transmission within the mesolimbic projections [e.g., (36)]. The same does not seem to be true of SS of the PFC. For instance, Phillips and Fibiger (46) performed 6-hydroxydopamine lesions of the MFB and examined SS from the medial PFC and nucleus accumbens. A significant dopamine depletion was associated with a lasting decrease in SS only from the latter site. In a similar experiment, Clavier and Gerfen (7) showed that injections of 6-hydroxydopamine within the sulcal PFC around the point of electrical stimulation only produced a transient disruption of SS. Moreover, Simon et al . (63) performed a bilateral thermocoagulation of the ventral tegmental area and showed that, despite a significant depletion of dopamine levels with the PFC, SS of the medial PFC was not affected. These results indicate that SS of the PFC does not depend upon intact dopaminergic projections to the structure.

In an innovative experiment involving 2-deoxyglucose autora- diography (78), the pattern of changes in metabolic activity

PREFRONTAL CORTEX AND REWARD 165

associated with electrical stimulation (of identical parameters to those which supported SS) was different when PFC and MFB stimulation sites were compared. The conclusion reached by the authors of this study was that brain stimulation at parameters sufficient to support SS of each site recruits nonoverlapping circuits.

Yet another method by which to compare the substrates of SS from different sites is the use of psychophysical procedures to derive information relevant to the physiological characteristics of the reward-related elements being stimulated. These behavioral methods can provide estimates of such physiological properties as refractory periods, chronaxie, and speed and direction of conduction [see (22)]. Schenk and Shizgal (59) investigated the recovery from refractoriness at MFB and medial PFC sites and showed that the directly-stimulated elements at the latter site possessed refractory periods that were significantly longer than those associated with SS of the MFB. The authors extended this finding by measuring the degree of summation between MFB and PFC SS electrodes using a variation of the CT-interval procedure (62) which can demonstrate connectivity and functional linkages between distant stimulation sites. They reported that rewarding stimulation delivered to MFB and medial PFC sites displayed little summation or evidence of collision effects and interpreted these results to indicate that the directly stimulated reward-related elements mediating SS at both sites are different.

A number of experiments have examined the effects of administration of the indirect dopamine agonist amphetamine. In gen- erai, as with the lesion studies, the results suggest a differential role for dopamine in SS of the MFB and SS of the PFC. Amongst the first investigations was a report by Carey et al. (5) suggesting that the facilitatory consequences of systemic administration of amphetamine could not be seen with SS of the medial PFC. Very similar conclusions were drawn in another study by Robertson et al. (51) from observations of the effects of amphetamine on SS maintained by a variable interval schedule; animals responding for medial PFC stimulation failed to increase their lever-pressing rate in response to doses of amphetamine which produced large increases in the rate of MFB SS. Such differential effects were argued to reflect the relative insensitivity of medial PFC substrates to enhancement of dopaminergic transmission. This was put directly to the test in an experiment by Hand and Franklin (29) in which animals chose between medial PFC stimulation and MFB stimulation following different doses of amphetamine. The results indicated that amphetamine significantly increased preference for MFB stimulation over medial PFC stimulation in a dose-dependent manner. Since the initial preferences for SS of either site were arbitrarily equated by manipulating the stimulation parameters, the increase in time spent responding for MFB relative to PFC stimulation was argued to demonstrate that amphetamine differ- entially increases the rewarding value of the former but not the latter. In line with these conclusions, analyses of drug-induced shifts in the frequency thresholds of SS have shown that dopamine agonist treatments such as amphetamine produce large decreases in threshold for MFB (19) but not PFC (65) stimulation.

The same picture emerges from studies of the effects of dopamine antagonist drugs on SS of the medial PFC. In an early study, Robertson and Mogenson (55) observed that doses of spiroperidol (injected into the area around the SS electrode) which significantly attenuated SS of the nucleus accumbens did not affect SS of the medial PFC. More recently, Corbett (9) demonstrated that systemic injections of cis-flu-penthixol produced differential changes in the frequency threshold of MFB and medial PFC SS, which were interpreted to suggest again that medial PFC SS is relatively unaffected by interference with dopamine processes within the medial PFC or elsewhere. In contrast, frequency thresholds for SS of the MFB are dramatically and dose-depen-

dently increased by treatment with dopamine antagonists (19,21). Taken together, these experiments examining the consequences

of pharmacological or anatomical (lesion) manipulations on SS and examining the physiological characteristics of its substrate strongly imply that the processes underlying the production of rewarding effects from medial PFC stimulation are critically dependent on circuitry which is both anatomically and pharmaco- logically dissociable from that involved in reward produced by MFB stimulation. The scope and consistency of the data provide a strong empirical argument against the existence of a single process mediating the rewarding effects produced by brain stimulation. But if SS substrates cannot be reduced to a single system, it becomes important to account for the basis of their diversity. An obvious starting point is to consider the possibility that intracranial rewards may differ qualitatively and that the existence of indepen- dent SS substrates reflects the fact that distinct classes of reinforcing events affect instrumental performance through the action of dissociable neural processes.

CHARACTERISTICS OF SS OF THE PFC

The conclusion that PFC SS depends on the activation of a substrate seemingly different from that likely to mediate the rewarding effects of stimulation of the MFB raises the question of the nature and extent of the processes giving rise to the rewarding properties of cortical stimulation. One answer may lie in the remarkable differences in the topography of SS behavior previ- ously noted. The characteristically slow acquisition of SS of the PFC has been suggested to occur as the consequence of a sensitization process by which rewarding effects are dependent upon a stimulation-induced plasticity process. Corbett et al. (10) demonstrated that, in a standard lever-pressing task reinforced with medial PFC stimulation, the appearance of SS depends critically on the cumulative exposure to the effects of the stimulation. This is because a program of noncontingent stimulation of the medial PFC, delivered before SS training, caused a significant decrease in the number of training sessions required to establish SS. Since the facilitatory pretraining stimulation was delivered in the absence of any learning requirements, it is unlikely that the gradual onset of SS of this site normally occurs as the result of stimulation-induced interference with the process of learning the lever-pressing response. This suggests that medial PFC stimulation may produce rewarding effects only as the outcome of a stimulation-dependent process.

The apparent dependence of the onset of medial PFC SS on the effects of repeated electrical stimulation suggests an analogy with the well-known kindling phenomenon, in which the repeated induction of epileptiform activity by local depolarization leads to progressive and lasting changes in seizure thresholds (25). This possibility was first suggested by work done with SS of the hippocampus (4). That rewarding effects could also occur as a result of the repeated activation of prefrontal elements, perhaps through some physiological mechanism comparable to that underlying kindling, was further supported by the results of an experiment by Robertson et al. (53). They showed that diazepam and phenobarbital, two antiepileptic agents known to reduce the magnitude of stimulation-induced after discharge activity, could antagonize the facilitatory effects of pretraining stimulation of the medial PFC on the subsequent acquisition of SS.

It is important to determine whether the characteristics of the acquisition of PFC SS do in fact reflect changes specifically related to the ability of the stimulation to produce rewarding effects. An alternative view would hold that the process of acquisition of SS is constrained by factors not related to the motivational consequences of PFC stimulation and is thus unin- formative as to the nature of SS substrates. One obvious possibility

166 ROBERTSON

is that the acquisition of PFC SS reflects the process by which animals habituate to or learn to cope with motoric impairments generated by the stimulation independently of its rewarding effects. Corbett and Stellar (13) tested this possibility by examining whether extended experience with SS could lead to a reduction in the potency with which medial PFC stimulation could interfere with the performance of simple orienting response. They found no evidence that such was the case and suggested that the repeated experience of PFC stimulation therefore seemed to exert little influence on its subsequent ability to disrupt orienting reactions. Apparently different results were obtained when the effects of changing response requirements on the speed of SS acquisition were examined (12): when nose-poking was made contingent upon PFC stimulation, high rates of responding were seen very quickly, which suggested that the slow onset of SS seen to occur in lever-pressing tasks depends on factors related to the complexity of the required response. However, another study using nose- poking reinforced with PFC stimulation reported that SS was acquired slowly, over a time scale comparable to that seen in lever-pressing experiments (42). The effect of operant selection on the acquisition of PFC SS is thus incompletely understood.

In any case, the argument that different rates of acquisition of SS might follow the use of topographically different responses does not rule out the possibility that PFC reward is dependent on the induction of some necessary changes in the physiological consequence of the stimulation. Instead, such differences might simply indicate that not all response-generating mechanisms are equally accessible to the substrate of PFC SS and that responses more relevant to the nature of the rewarding consequences of PFC stimulation can be established more quickly and performed for weaker intensities of reward. The observation that frequency thresholds for medial PFC SS are lower for nose-poking than for lever pressing (12) may therefore indicate that the former response is performed for less rewarding PFC stimulation because of its greater associability to the reinforcer rather than because it represents a simpler, less difficult action.

This interpretation finds support in the observations of Shet- tleworth (60), who investigated the ability of lateral hypothalamic stimulation to reinforce different action patterns in hamsters, and found that the frequency of seemingly simple responses such as body scratching and face washing did not increase markedly when stimulation was made contingent upon their performance. This was despite the fact that the same stimulation was an effective reinforcer of other responses such as digging or scrabbling, which might be said to be more complex inasmuch as they require coordinated forepaw activity directed at external objects.

The differential susceptibility of instrumental responses to a brain stimulation reinforcer thus cannot depend solely on differences in the effort or complexity of the movements involved. Rather, such effects suggest that the associability of different classes of responses to the rewarding consequences of brain stimulation is determined by the compatibility between the instrumental response and the nature of the reinforcing effects induced by the stimulation. Such a view of reinforcement processes therefore suggests that the features of response acquisition can be useful in defining the scope of a reinforcer's control over behavior. Accordingly, the established dependence of PFC SS on the repeated delivery of stimulation in at least some situations may be a useful feature in defining functional relations between the medial PFC and other sites which could be related to it in the mediation of SS.

Although by far the majority of work done on SS of the PFC has employed electrode placements in the medial zone (Fig. 2A), the sulcal zone also supports SS (52,57). On a number of grounds, SS of the sulcal PFC appears to be very similar to SS of the medial

PFC" it is also acquired very slowly, and this normally slow acquisition can be hastened by prior exposure to noncontingent stimulation through the SS electrode (54). Other studies of the factors controlling the acquisition of SS of the sulcal and medial PFC suggest more directly that there may be a common substrate, as pretraining noncontingent stimulation of the medial PFC can greatly facilitate the subsequent acquisition of SS of the sulcal cortex, and vice versa (52,54). Furthermore, the direct corticocortical connections between the two zones seem to play a crucial role in the maintenance of SS obtained from the medial PFC (I 1) because when these connections are severed by bilateral knife cuts, SS of the medial PFC is apparently eliminated. This suggests that SS of the medial PFC is critically dependent on its connections to or from the sulcal PFC. The reverse did not seem to be the case, however, as SS of the sulcal PFC is unaffected by large bilateral electrolytic lesions of the medial zone (54).

Subsequent experiments qualified this conclusion somewhat (54). It has been demonstrated that transection of the corticocortical connections between the medial and sulcal zones does not result in a permanent loss of SS of the medial PFC, but rather produces a decrement in SS which can be overcome by the same type of repeated stimulation that is effective in establishing the response in normal animals. However, in animals with bilateral knife cuts, one can no longer observe the proactive facilitatory effects of noncontingent stimulation of the sulcal PFC on acquisition of SS of the medial PFC.

These data were interpreted to suggest that, in the normal rat, a prepotent relationship exists between SS of the medial PFC and SS of the sulcal PFC mediated at least in part by the corticocortical fibres linking the two areas. The nature of the relationship suggests that in the normal rat, positive reinforcement derived from electrical stimulation of the medial PFC proceeds through activation of the medial to sulcal efferents but, when this connection is severed (in animals with knife cuts), positive reinforcement proceeds through activation of a now dissociated substrate. The key notion is that of a disassociation of normally associated substrates because in the normal rat, one c a n observe a facilitation of SS of the medial PFC by pretraining stimulation applied to the sulcal zone, and one c a n observe a decrement (albeit a temporary one) in SS when the connections between the two areas are severed. These data therefore suggest that SS of the PFC subdivisions can be viewed as involving dissociable but normally interacting substrates.

FUNCTIONAL SPECIALIZATION WITHIN THE PFC

The apparent impossibility of equating intracranial reward derived from stimulation of the PFC with the activation of an anatomically defined specific or unitary system raises the question of the behavioral correlates of whatever systems may be found to underlie SS in this structure. One possibility is that the existence of dissociable reward-related pathways simply maps out the existence of dissociable neural systems involved in different types or classes of reinforcement.

This sort of analysis is aided by the fact that there is now an extensive literature dealing with the effects of restricted damage to the subfields of the PFC on different types of learning tasks [(33,56) for review]. Furthermore, there is evidence to suggest that the functional differences between the two major prefrontal areas may be related to the exclusive involvement of each area in the control of different sensorimotor processes. This suggestion comes from repeated observations of dissociable behavioral deficits following damage to the medial and sulcal PFC areas (18, 26, 30, 56, 68).

Lesions of the sulcal PFC most reliably affect a variety of

PREFRONTAL CORTEX AND REWARD 167

A 13

Aid

FIG. 2. A: Wet mount photograph of an electrode placement (arrow) in the medial PFC (prelimbic area). B: Example of a parasagittal knife cut through the projection path of the corticocortical connections between the medial (PL) and sulcal (Aid) prefrontal areas. Top: Schematic course of the corticocortical fibers. Bottom: photograph of a coronal hemisection showing the extent of a knife cut (arrows). Luxol Blue--Neutral Red counterstain.

behaviors having the common property of involving responses controlled by or directed at stimuli of a gustatory or olfactory nature (33). This is probably related to the fact that the cortical tissue stretching along the dorsal portion of the rhinal fissure is thought to be involved in the processing of olfactory, gustatory, visceral and autonomic processes (6, 58, 61, 73). Thus several studies have reported that lesions of portions of the anterior agranular insular cortex attenuate the learning of a conditioned taste aversion (CTA) induced by lithium chloride toxicosis (3, 17, 39). This deficit appears to occur as a consequence of interference with taste-illness associative processes rather than of sensory dysfunction since animals with similar lesions appear to possess normal taste reactivity and unimpaired abilities to effect simple taste or olfactory discriminations (38). Along similar lines, Eichen- baum et al. (18) observed deficits in the acquisition of a odor- mediated conditional go-no go discrimination following lesions of the rostral portion of the sulcal PFC.

An indication that the sulcal PFC may be thought of as effecting a rather broad control over behavior guided by olfactory/ gustatory cues is further suggested by the observation that damage to the sulcal PFC can prevent incentive contrast effects involving the gustatory quality of food reinforcement (49). This was done by observing the effects of changing the palatability of the reinforcer on the rate of lever pressing in normal rats and in animals given lesions of the medial or sulcal PFC. Normal animals and rats with medial PFC lesions exhibited a contrast effect by displaying exaggerated shifts in responding after a change in the quality of the reinforcement when their performance was compared to that of animals receiving the same reward from the beginning of training. Rats with sulcal PFC damage failed to show this overreaction despite the fact that their preference for the more palatable reward was unimpaired when assessed in choice tests. An additional experiment was performed to determine whether the lack of contrast effects in the lesioned animals could occur because of the

168 ROBERTSON

presence of response perseveration caused by the lesion [e.g., (34)]. Rats with sulcal damage were shown to be as capable as unoperated controls in suppressing lever pressing when response omission was systematically reinforced after a period of initial training. Thus lesions of the sulcal PFC reduced the behavioral consequences of gustatory manipulations of reward but did not prevent behavior from being appropriately modified when the relative availability of response-contingent reward was systematically changed (49).The results of these experiments and those highlighting the involvement of the sulcal PFC in learning (3, 17, 33, 39) are therefore consistent with the hypothesis that the sulcal PFC is an important element in the processes that subserve the response to changes in reinforcement contingencies specifically involving events of a gustatory/olfactory nature.

An equally specialized role can be suggested for the medial PFC. A central element of behavior affected by medial PFC damage appears to be related to the inhibition of responding and/or the sequencing of responding in time. Thus rats with damage to the medial PFC display perseverative responding in a number of operant tasks [e.g., (41)] and appear to be impaired at learning tasks requiring the shifting of response patterns such as those required in delayed alternation problems (15, 33, 43).

In view of the suppressive effects of stimulation of the medial PFC on behavior, it is perhaps not surprising that the effects described above have traditionally been related to a loss of inhibitory control over instrumental or species-typical responding. In fact, motor perseveration is frequently invoked to explain some of the behavioral consequences of PFC lesions on operant performance (33,43). However it is not entirely clear under what conditions perseverative deficits are observed. For instance, Kolb et al. (34) failed to find much evidence for response perseveration after bilateral medial PFC lesions. Similarly, Robertson and Laferriere (48) observed no deficit in the ability of rats with medial PFC lesions to alter their rate of lever pressing in response to changes in response-reinforcer contingencies, in a task identical to that described above for testing perseveration in rats with lesions of the sulcal PFC. Perseverative deficits consequent to medial PFC damage may thus reflect the failure of processes controlling responding in specific situations more than they do a motoric incapacity per se. This is further suggested by the repeated observation that medial PFC lesions produce sensory neglect of visual, auditory and tactile stimuli (1,14), interfere with habitua- tion to novel stimuli of the same type (33), and severely interfere with rats' ability to learn spatial relations (34, 35, 67). Thus attempts to formulate a comprehensive description of medial PFC function have emphasised the fact that lesions of the area appear to produce their most reliable effects in behavioral situations requiring the organization of responding controlled by exteroceptive stimuli distant in time and space (33, 56, 68).

Little is known of the role of the medial PFC in associative processes in rats. The observation that medial PFC lesions facilitate the autoshaping of a lever-press response (74) would indicate that the structure is not critical for the establishment of stimulus-reinforcer associations. However, preliminary studies using the conditioned place paradigm and drug reinforcement indicate that PFC lesions can be shown to prevent the association of contextual stimuli with the rewarding effects of dopaminergic agents (unpublished observation). These results thus further strengthen the notion that the medial PFC is primarily and specifically involved in the processing of exteroceptive stimuli.

This functional description of the medial PFC can be readily complemented by a similar definition of the specialization of the sulcal PFC in terms of the dominant sensory modalities within which it performs similar integrative activities, since the sulcal PFC may be primarily involved in the organization of responses aimed at or directed by proximal stimuli of a gustatory/olfactory

nature. The available evidence therefore suggests that the prefrontal divisions in the rat may be characterized as performing similar integrative functions over segregated sensorimotor channels. This raises the question of whether some sensory attributes of the rewarding effects of PFC stimulation may be similarly distinguish- able. We have followed up this question by examining SS of the medial and sulcal PFC, and by looking at what classes of stimuli are most easily associated with rewarding electrical stimulation of either area.

THE PROPERTIES OF POSITIVE REINFORCEMENT FROM THE PFC

Based upon the literature outlined above, it seemed possible to hypothesize that the rewarding effects induced by medial or sulcal PFC stimulation should be specifically relatable to the particular processing functions subserved by each division. Moreover, as described in a previous section, since the corticocortical connections between the two areas may serve to mediate interactions between dissociable SS systems by defining a hierarchy of prefrontal reward-related pathways, the nature of these rewarding effects should be dissociable in rats lacking this corticocortical link. In order to test this notion, Robertson and Laferriere (50) examined the ability of rewarding medial PFC stimulation to impart secondary reinforcing effects to various classes of stimuli, using both a conditioned taste preference (CTP) paradigm and a conditioned place preference (CPP) paradigm. It was found that SS of the medial PFC could induce a CTP but not a CPP. Knife cuts intersecting the trajectory of the corticocortical fibres (Fig. 2B) reversed this apparent associative bias. The effects could not be accounted for in terms of a general associative deficit caused by the cuts, or by effects on the rewarding magnitude of the stimulation. This is because sulcal SS in animals given bilateral knife cuts could sustain the formation of a strong CTP and also because SS of the medial PFC was as intense in lesioned rats as in unoperated animals self-stimulating the PFC at the same parameters. These observations were argued to show two important features of PFC SS; first, that not all types of sensory cues are equally associable to the reinforcing effects of medial PFC stimulation; and second, that the direct connections between the medial and sulcal PFC appear to control the type of stimulus most easily associated with the rewarding consequences of PFC stimulation. Furthermore, the pattern of associative effects seen to follow the knife cuts suggests that the differential effects of pairing flavor or place cues with medial PFC SS is consistent with the functional specialization of each prefrontal division inferred from lesion studies: in intact (no lesion) animals, medial PFC rewarding effects are more readily associated with a stimulus relevant to the processing domain of the sulcal PFC. In lesioned rats, the disruption of the corticortical connections is accompanied by a change in the associative bias of the rewarding effects towards the types of stimuli which serve to define a specific location and are thus relevant to the processing domain of the medial PFC. In other words it is possible that in the normal rat, a prepotent relationship exists between the medial and sulcal PFC, mediated by the corticocortical association fibres, which determines the dominant sensory properties of the reinforcing effects produced by medial PFC stimulation.

The existence of biases in the associability of stimuli and responses to different types of reinforcing events is a well- documented feature of learning phenomena. A well known example of this was provided by the seminal work of Garcia and colleagues (23). In one demonstration in rats, it was shown that gustatory but not visual and auditory cues are rapidly associated with the consequences of gastrointestinal toxicosis and that the reverse is true when shock is used as the negative reinforcer. The above observations on the conditioning of PFC SS indicate that

P R E F R O N T A L C O R T E X AND R E W A R D 169

analogous processes are likely to apply to brain stimulation reward.

CONCLUSION

Two major conclusions may be drawn from the present paper. First, the analysis o f the determinants of PFC SS provides strong evidence for the existence of distinct and dissociable substrates of brain stimulation reward, both within the PFC itself and in comparison with reward loci within the MFB. Second, it is possible to draw parallels between the functional specialization o f PEC subdivisions inferred from lesion studies and the organization of reward-related substrates inferred from studies of PFC SS.

An important implication of the analysis o f PFC SS is that the attributes of the rewarding effects induced by PFC stimulation can vary depending on the locus of stimulation. This would indicate that reward produced by brain stimulation does vary qualitatively and that the experiential consequences of brain stimulation must be considered in attempts to make sense of the diversity and organi-

zation of reward-related substrates. In the context o f prefrontal mechanisms of SS, the evidence reviewed suggests that such features are related to the specific behavioral functions of the medial and sulcal PFC. The dissociability of reward-related circuits within the PFC argues for the presence of parallel but interacting pathways. PFC SS behavior may thus map the organization of separate sensorimotor processes involved in distinctive classes of adaptive functions, a view which is finding growing support from anatomical and hodological studies of the PFC (18,26) and has figured prominently in some accounts of motivated behavior (20,24). The study of SS behavior may therefore still provide important insights on the determinants o f behavior, some thirty-five years after its discovery.

ACKNOWLEDGEMENTS

This work was supported by a grant from the Natural Sciences and Engineering Research Council (NSERC) Canada to the author. The author would like to thank Andr6 Laferri~re for his invaluable assistance in the preparation of this manuscript.

REFERENCES

1. Barth, T. M.; Parker, S. M.; Sinnamon, H. M. Unilateral lesions of the anteromedial cortex in the rat impair approach to contralateral visual cues. Physiol. Behav. 25:141-147; 1982.

2. Beckstead, R. M. An autoradiographic examination of the corticocortical and subcortical projections of the mediodorsal-projection (prefrontal) cortex in the rat. J. Comp. Neurol. 184:43-62; 1979.

3. Braun, J. J.; Kiefer, S. W.; Ouellet, J. V. Psychic ageusia in rats lacking gustatory neocortex. Exp. Neurol. 72:711-716; 1981.

4. Campbell, K. A.; Milgram, N. W. Mechanisms underlying the plasticity of hippocampal stimulation induced reward. Behav. Neuro- sci. 99:209-219; 1985.

5. Carey, R. J.; Goodall, E.; Lorens, S. A. Differential effects of amphetamine and food deprivation on self-stimulation of the lateral hypothalamus and medial frontal cortex. J. Comp. Physiol. Psychol. 88:224-230; 1975.

6. Cechetto, D. F.; Saper, C. B. Evidence for a viscerotopic sensory representation in the cortex and thalamus of the rat. J. Comp. Neurol. 262:27--45; 1987.

7. Clavier, R. M.; Gerfen, C. Intracranial self-stimulation from the sulcal prefrontal cortex in the rat: the effect of 6-hydroxydopamine or kainic acid lesions at the site of stimulation. Brain Res. 224:291-304; 1981.

8. Colle, L. M.; Wise, R. A. Opposite effects of unilateral forebrain ablations on ipsilateral and contralateral hypothalamic self-stimulation. Brain Res. 407:285-293; 1987.

9. Corbett, D. Differences in sensitivity to neuroleptic blockade: MFB vs. frontal cortex ICSS. Can. Psychol. 29(2A):155; 1988; (abstract).

10. Corben, D.; Laferri~re, A.; Milner, P. M. Plasticity of the medial prefrontal cortex: Facilitated acquisition of intracranial self-stimulation by pretraining stimulation. Physiol. Behav. 28:531-534; 1982.

11. Corbett, D.; Laferri~:re, A.; Milner, P. M. Elimination of medial prefrontal cortex self-stimulation following transection of efferents to the sulcal cortex in the rat. Physiol. Behav. 29:425--431; 1982.

12. Corbett, D.; Silva, L. R.; Stellar, J. R. An investigation of the factors affecting development of frontal cortex self-stimulation. Physiol. Behav. 34:89-95; 1985.

13. Corbett, D.; Stellar, J. R. Neurological reactivity during medial prefrontal cortex stimulation: Effects of self-stimulation experience. Physiol. Behav. 31:771-776; 1983.

14. Corwin, J. W.; Kanter, S.; Watson, R. T.; Heilman, K. N.; Valenstein, E.; Hashimoto, A. Apomorphine has a therapeutic effect on neglect produced by unilateral dorsomedial prefrontal cortex lesions in rats. Exp. Neurol. 94:683-698; 1986.

15. Divac, I. Frontal lobe system and spatial reversal in the rat. Neu- ropsychology 9:175-183; 197 I.

16. Douglin, D. C.; Glassman, R. B. Gradual increase in self-stimulation response rates: effects of electrode loci. Physiol. Psychol. 7:135-138; 1979.

17. Dunn, L. T.; Everitt, B. J. Double dissociations of the effects of amygdala and insular cortex lesions on conditioned taste aversion, passive avoidance, and neophobia in the rat using the excitotoxin ibotenic acid. Behav. Neurosci. 102:3-23; 1988.

18. Eichenbaum, H.; Clegg, R. A.; Feeley, A. Reexamination of functional subdivisions of the rodent prefrontal cortex. Exp. Neurol. 79:434--451; 1983.

19. Franklin, K. Catecholamines and self-stimulation: Reward and performance effects dissociated. Pharmacol. Biochem. Behav. 9:813- 820; 1978.

20. Gallistel, C. R. The organization of action: A new synthesis. Hillsdale, N.I; Lawrence Erlbaum Assoc.; 1980.

21. Gallistel, C. R.; Davis, A. J. Affinity for the dopamine D2 receptor predicts neuroleptic potency in blocking the reinforcing effects of MFB stimulation. Pharmacol. Biochem. Behav. 19:867-872; 1983.

22. Gallistel, C. R.; Shizgal, P.; Yeomans, J. A portrait of the substrate for self-stimulation. Psychol. Rev. 88:228--273; 1981.

23. Garcia, J.; Koelling, R. A. Relation of cue to consequence in avoidance learning. Psychonom. Sci. 4:123-124; 1966.

24. Glickman, S. E.; Schiff, B. B. A biological theory of reinforcement. Psychol. Rev. 74:81-109; 1967.

25. Goddard, C. V.; Mclntyre, G.; Leech, C. A permanent change in brain function resulting from daily electrical stimulation. Exp. Neurol. 25:295-330; 1969.

26. Goldman-Rakic, P. S. Topography of cognition: Parallel distributed networks in primate association cortex. Annu. Rev. Neurosci. 11: 137-156; 1988.

27. Gratton, A.; Wise, R. A. Comparisons of refractory periods for medial forebrain bundle fibers subserving stimulation-induced feeding and brain stimulation reward: a psychophysical study. Brain Res. 438:256-263; 1988.

28. Gratton, A.; Wise, R. A. Comparisons of connectivity and conduction velocities for medial forebrain bundle fibers subserving stimulation induced feeding and brain stimulation reward. Brain Res. 438: 264-270; 1988.

29. Hand, T. H.; Franklin, K. B. J. The influence of amphetamine on preference for lateral hypothalamic versus prefrontal or ventral tegmental area self-stimulation. Pharmacol. Biochem. Behav. 18:695- 699; 1983.

30. Hurley-Guis, K. M.; Neafsey, E. J. The medial frontal cortex and gastric motility: microstimulation results and their possible significance for the over-all pattern of organization of rat frontal and parietal cortex. Brain Res. 365:241-248; 1986.

31. Huston, J. P.; Borbely, A. A. Operant conditioning in forebrain ablated rats by use of rewarding hypothalamic stimulation. Brain Res. 50:467--472; 1973.

32. Huston, J. P.; Ornstein, K.; Lehner, R. The diencephalic peninsula: self stimulation after unilateral precollicular transection and removal

170 R O B E R T S O N

of the telencephalon. Brain Res. 245:187-191 ; 1982. 33. Kolb, B. Functions of the frontal cortex of the rat: A comparative

review. Brain Res. Rev. 8:65-98; 1984. 34. Kolb, B.; Nonneman, A. J.; Singh, R. K. Double dissociation of

spatial impairments and perseveration following selective prefrontal lesions in rats. J. Comp. Physiol. Psychol. 87:772-780; 1974.

35. Kolb, B.; Sutherland, R. J.; Whishaw, I. Q. A comparison of the contributions of the frontal and parietal association cortex to spatial localization in rats. Behav. Neurosci. 97:13-27; 1983.

36. Koob, G. F.; Fray, P. J.; Iversen, S. D. Self-stimulation of the lateral hypothalamus and locus coeruleus after specific unilateral lesions of the dopamine system. Brain Res. 146:123-140; 1978.

37. Lammers, J. H. C. M.; Meelis, W.; Kruk, M. R.; van der Poel, A. M. Hypothalamic substrates for brain stimulation-induced grooming, digging and circling in the rat. Brain Res. 418:1-19; 1987.

38. Lasiter, P. S. Gastrointestinal reactivity in rats lacking anterior insular neocortex. Behav. Neural Biol. 39:149-154; 1983.

39. Lasiter, P. S.; Glanzman, D. L. Cortical substrates of taste aversion learning: Dorsal prepiriform (insularl lesions disrupt taste aversion learning. J. Comp. Physiol. Psychol. 96:376-392; 1982.

40. Lofving, B. Cardiovascular adjustments induced from the rostral cingulate gyrus. Acta Physiol. Scand. Suppl. 184:1-84; 1961.

41. McDonough, J. H.; Manning, F. J. The effects of lesions in amygdala or dorsomedial frontal cortex on reinforcement omission and noncontingent reinforcement in rats. Physiol. Psychol. 7:167-172: 1979.

42. Nassif, S.; Cardo, B.; Liberstat, F.; Velley, L. Comparison of deficits in electrical self-stimulation after ibotenic acid lesions of the lateral hypothalamus and the medial prefrontal cortex. Brain Res. 332: 247-257; 1985.

43. Neill, D. B. Frontal-striatal control of behavioral inhibition in the rat. Brain Res. 105:89-103; 1976.

44. Olds, J.; Milner, P. M. Positive reinforcement produced by electrical stimulation of septal area and other regions of the rat brain. J. Comp. Physiol. Psychol. 47:419--427: 1954.

45. Phillips, A. G. Brain reward circuitry: A case for separate systems. Brain Res. Bull. 12:195-201; 1984.

46. Phillips, A. G.; Fibiger, H. C. The role of dopamine in maintaining intracranial self-stimulation in the ventral tegmentum, nucleus accumbens and medial prefrontal cortex. Can. J. Psychol. 32:58-66: 1978.

47. Pritzel, M.; Huston, J. P.; Bussher, W. Hypothalamic self-stimulation in rats with one hemisphere isolated anterior to the midbrain and the other hemisphere devoid of the telencephalon. Exp. Neurol. 81: 426--445; 1983.

48. Robertson, A.: Howarth, T.; Laferrirre, A. Prefrontal cortex lesions and response-reinforcement contingencies. Can. Psychol. 29:663; 1988; (abstract).

49. Robertson, A.; Laferrirre, A. Insular prefrontal cortex lesions prevent incentive contrast effects in the rat. Soc. Neurosci. Abstr. 14(2): 1230; 1988.

50. Robertson, A.; Lafem~re, A. Disruption of the connections between the medial and sulcal prefrontal cortices alters the associability of rewarding medial cortical stimulation to place and taste stimuli in rats. Behav. Neurosci.; in press.

51. Robertson, A.: Laferri~re, A.; Franklin, K. B. J. Amphetamine and increases in current modulate reward in the hypothalamus and sub- stantia nigra but not in the prefrontal cortex. Physiol. Behav. 26:809-813; 1981.

52. Robertson, A.; Laferrirre, A.; Milner, P. M. Development of brain stimulation reward in the medial prefrontal cortex: facilitation by prior electrical stimulation of the sulcal prefrontal cortex. Physiol. Behav. 28:869-872; 1982.

53. Robertson, A.; Laferri~re, A.; Milner, P. M. Treatment with anticon- vulsant drugs retards the development of brain stimulation reward in the prefrontal cortex. Physiol. Behav. 29:275-280; 1982.

54. Robertson, A.; Laferri/~re, A.; Milner, P. M. The role of corticocortical projections in self-stimulation of the prelimbic and sulcal prefrontal cortex in rats. Behav. Brain Res. 21:129-142; 1986~

55. Robertson, A.; Mogenson, G. Evidence for a role of dopamine in self-stimulation of the nucleus accumbens of the rat. Can. J. Psychol. 32:67-76; 1978.

56. Rosvold, H. E. The frontal lobe system: cortical-subcortical interre- lationships. Acta Neurobiol. Exp. 32:439-460; 1972.

57. Routtenberg, A.; Sloan, M. Self-stimulation in the frontal cortex in Rattus norvegicus. Behav. Biol. 7:567-572; 1972.

58. Saper, C. B. Convergence of autonomic and limbic connections in the insular cortex of the rat. J. Comp. Neurol. 210:261-273; 1982.

59. Schenk, S.; Shizgal, P. The substrates for lateral hypothalamic and medial prefrontal cortex self-stimulation have different refractory periods and show poor spatial summation. Physiol. Behav. 28: 133-138; 1982.

60. Shettleworth, S. J.; Juergensen, M. R. Reinforcement and the organization of behavior in golden hamsters: Brain stimulation reinforcement for seven action patterns. J. Exp. Psychol. [Anim. Behav.] 6:352-375; 1980.

61. Shipley, M. T.; Sanders, M. S. Special senses are really special: Evidence for a reciprocal bilateral pathway between insular cortex and nucleus parabrachialis. Brain Res. Bull. 8:493-501; 1982.

62. Shizgal, P.; Bielajew, C.; Corbett, D.; Skelton, R.; Yeomans, J. Behavioral methods for inferring anatomical linkage between rewarding brain stimulation sites. J. Comp. Physiol. Psychol. 94:227-237; 1980.

63. Simon, H.; Stinus, L.; Tassin, J. P.; Lavielle, S.; Blanc, G.; Thierry, A. M.; Glowinski, J.; LeMoal, M. Is the dopaminergic mesocorti- colimbic system necessary for intracranial self-stimulation. Behav. Neural Biol. 27:125-145; 1979.

64. Sinnamon, H. M.; Galen, B. S. Head movements elicited by electrical stimulation of the anteromedial cortex of the rat. Physiol. Behav. 33:185-190; 1984.

65. Spence, S. J.; Silverman, J. A.; Corbett, D. Cortical and ventral tegmental systems exert opposing influences on self-stimulation from the prefrontal cortex. Behav. Brain Res. 17:117-124; 1985.

66. Stellar, J. R.; Illes, J.; Mills, L. E. Role of ipsilateral forebrain in lateral hypothalamic stimulation reward in rats. Physiol. Behav. 29:1089-1097; 1982.

67. Sutherland, R. J.; Kolb, B.; Whishaw, I. Q. Spatial mapping: definitive disruption by hippocampal or medial frontal cortex damage in the rat. Neurosci. Lett. 31:271-276; 1982.

68. Teuber, H.-L. Unity and diversity of frontal lobe function. Acta Neurol. Exp. 32:625-656; 1972.

69. Thorndike, E. L. Animal intelligence. New York: Macmillan; 1911. 70. Thorndike, E. L. A theory of the action of the after-effects of a

connection upon it. Psychol. Rev. 40:434--439; 1933. 71. Valenstein, E. S.; Cox, V. G.; Kakolewski, J. W. Modification of

motivated behavior elicited by electrical stimulation of the hypothalamus. Science 159:1119-1121; 1968.

72. Valenstein, E. S.: Cox, V. G.; Kakolewski, J. W. Reexamination of the role of the hypothalamus in motivation. Psychol. Rev. 77:16-31; 1970.

73. Van Der Kooy, D.; Kodo, L. V.; McGinty, J. F.; Gerfen, C. R.; Bloom, F. E. Visceral cortex: a direct connection from prefrontal cortex to the solitary nucleus in rat. Neurosci. Lett. 33:123-127; 1982.

74. van Haaren, F.; van Zijderveld, G.; van Hest, A.; deBruin, J. P. C.; van Eden, C. G.; van der Poll, N. E. Acquisition of conditional associations and operant delayed spatial response alternation: Effects of lesions in the medial prefrontal cortex. Behav. Neurosci. 102: 481--488; 1988.

75. Wilcott, R. C. Medial and orbital cortex and the suppression of behavior in the rat. Physiol. Behav. 27:237-241; 1981.

76. Wilcott, R. C. Prefrontal cortex and bulbar reticular formation and behavioral inhibition in the rat. Brain Res. Bull. 12:63-69; 1984.

77. Wilcott, R. C.; Sabol, B. A.; Yurchesen, R. P. Frontal cortex and response suppression in the rat. Brain Behav. Evol 13:116-124; 1976.

78. Yadin, E.; Guarini, V.; Gallistel, C. R. Unilaterally activated systems in rats self-stimulating at sites in the medial forebrain bundle, medial prefrontal cortex, or locus coeruleus. Brain Res. 266:39-50; 1983.

79. Ziles, K.; Wree, A. Cortex: A real and laminar structure. In: Paxinos, G., ed. The nervous system, vol. 1. North Ryde, NSW Australia: Academic Press; 1985.

multiple reward systems and the prefrontal cortex

Documents