instrumental conditioning of the rat cardiac control systems

Instrumental Conditioning of the Rat Cardiac Control SystemsAuthor(s): Craig FieldsSource: Proceedings of the National Academy of Sciences of the United States of America,Vol. 65, No. 2 (Feb. 15, 1970), pp. 293-299Published by: National Academy of SciencesStable URL: http://www.jstor.org/stable/59637 .

Accessed: 04/05/2014 16:47

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

National Academy of Sciences is collaborating with JSTOR to digitize, preserve and extend access toProceedings of the National Academy of Sciences of the United States of America.

http://www.jstor.org

This content downloaded from 130.132.123.28 on Sun, 4 May 2014 16:47:36 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=nas

http://www.jstor.org/stable/59637?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Proceedings of the National Academy of Sciences Vol. 65, No. 2, pp. 293-299, February, 1970

Instrumental Conditioning of the Rat Cardiac Control Systems*

Craig Fields

THE ROCKEFELLER UNIVERSITY, NEW YORK

Communicated by Neal E. nMiller, November 21, 1969

Abstract. The PR, PP, and RR intervals of the rat EKG were instrumentally conditioned using a variety of reinforcement schedules under computer control. The PR and PP intervals could be conditioned independently. Individual rats demonstrated either a steady shift of their response distribution or an abrupt change of an all-or-none variety. Some response distributions shifted continu- ously; others, through the growth of new response patterns in a manner reminiscent of hypothesis-testing behavior of human subjects. A study of the effects of single reinforcements showed that each reinforcement contributed to the learning behavior of the subject, in a well-defined but nonlinear fashion.

In recent years, the study of simple systems has proved to be a profitable ap- proach for the study of the physiological bases of memory and learning. For

example, work on conditioning of single cells, isolated ganglia,2 and whole animal

preparations3 of Aplysia have demonstrated that some forms of learning can be correlated with specific synaptic facilitations dependent upon appropriate reinforcement. The present study concerns a simple mammalian system-the cardiac control system of the curarized albino rat.

A great deal of experience has been gathered with the general methodology4 and dynamics of conditioning5 of this system. The response is easily recorded and quantified, making possible the application of a variety of conditioning schedules to study the progress of learning under reinforcement. The experiments reported were designed to answer several questions: (1) Can the PR and PP intervals be conditioned independently? (2) How do the parameters of reinforcement schedules determine the detailed dynamics of instrumental condi-

tioning? (3) How do individual reinforcements contribute to the accumulated

learning of the subject? (4) Is learning in this system a simple, gradual process, or can it show complex learning properties, such as all-or-none learning?

Materials and Methods. Preparation: Male albino rats, 250-350 gm, were used.

They were curarized and respirated,6 and made as comfortable as possible before the

experiment was begun. After local application of Xylocaine, needle electrodes, made from no. 27 hypodermic needles, were inserted subcutaneously in appropriate positions to

produce large amplitude P and R waves of the EKG. In those experiments in which tail

shock was used as the reinforcement, a tail electrode7 was attached; all other experiments used electrical stimulation of the brain as a reinforcement. Monopolar electrodes were

made from plastic-coated insect pins and were implanted bilaterally in a reward area.

The location and technique have been described elsewhere.- 293



294 PSYCHOLOGY: C. FIELDS PRoc. N. A. S.

Electronics: The electrocardiogram was amplified by a 100 X preamplifier9 and a Tektronix 502A oscilloscope, which also served as a continuous monitor of both EKG and stimuli. The vertical outputs of the 502A were the signLal source for an on-lilne computer. Reinforcements (either tail-shock or electrical stimnulation- of the brainl) were adminiistered for 40 msec by a constannt-curr-ent stimulator:1O 0.3 inna for tail shock and apprloximately 40 ,a for electrical brain stimulation.

Computer facilities: Experiments were controlled on-line with a Linc-8 (I)igital Equipment Corp.) computer. Simple signal processing was also p)erformed to provide the experimenter with a constant monitor of the respolnse pattern of the subject. After the completion of the experiment, data were transferred to a Control Data Corporation 160G computer for preliminary analysis, and to a Control Data Corporation 6600 com-

puter for final analysis (located at the Courant Institute of Mathematics, New York

University). For certain applications, the following facilities were also used: PDP-IO

(Applied Logic Corp., Princeton), PDP-10 (donated by Digital Equipment Corp., Maynard, Mass.), IBM 360/65,360/67,7094 (Information Processing Center, Massa- chusetts Institute of Technology, Cambridge).

Analysis: In most experiments, simultaneous measurements were retained for the

PR, PP, and RR intervals of the rat EKG. These were averaged in groups of 512 to calculate the general learning curve and sequential histograms for the subject. In addi-

tion, autocorrelations and appropriate cross-correlations were calculated for each block of 512 heart beats.

A record was kept of which responses received reinforcement, and a study was made of a sequence of 50 heart beats following each reinforced one. Each reinforced interval was characterized by its length and by its time of occurrence in the experiment. At each time of occurrence of a response, the "running" histogram stored in memory had a

particular average. Therefore, each reinforced response was classified by two numbers: its interval length and the current histogram average. For each reinforced response, the 50 intervals following the reinforcement were separated from the rest of the sequence and sorted on the basis of the two parameters described. Those sequences with similar sets of parameters were averaged to produce a set of "averaged evoked responses" for each

experiment. General experimental design: After the rat had been prepared as previously

described and allowed to adapt for 30 min, the computer program to be used was loaded into the Line-8 and begun. Different programs were used depending on the interval to be conditioned (PR, PP, or RR) and the type of conditioning schedule to be used. In

general, the program would gather 512 samples of the interval of choice and display a

histogram. The experimenter could then enter parameters relevant to the current sched-

ule, and begin the actual conditioning. At any time during the experiment the condi-

tioning could be stopped to reset the reinforcement parameters. Periodically during the conditioning, analog samples of the EKG were sampled and stored for later checking for possible abnormalities of shape which could bias the results.

All computer programs sequentially sampled and stored all three intervals, keeping the last 512 samples of the interval to be conditioned in a running histogram. As each interval was measured a decision was made as to whether or not to reinforce that response. The subroutine which performed this decision was the basic source for the variety of reinforcement schedules. In addition, if reinforcement was to be applied, a further selection was made whether to apply it at once or after a particular time delay.

Electrocardiographic Results. Four subjects were used to gather basic data re-

garding the control of the rat electrocardiogram under curare, one was used while under Nembutal, and one was used while under ether. These six rats were not

conditioned, but their EKG intervals were measured and stored for approxi-

mately three hours. Table 1 presents the means and standard deviations for the

PR, PP, and RR intervals using each drug. The values reported for curare are the averages from the four animals. In all cases, the distributions are rather



VOL. 65, 1970 PSYCHOLOGY: C. FIELDS 295

TABLE 1. Mean interval lengths, in msec, for the electrocardiogram intervals which were studied. Standard deviations appear in parentheses.

PR Interval PP Interval RR Interval

Curare 49.2 (3.1) 131.4 (9.6) 130.7(10.3) Nembutal 52.7 (4.3) 1.35.9 (10.6) 136.8(11.8) Ether 55.6 (12.6) 141.0(17.9) 142.1(18.4)

"narrow," indicating rapid control of the cardiac rhythms of anesthetized or

paralyzed rats. Autocorrelations were calculated for each time series. An example can be

seen in Figure 1. The autocorrelations clearly revealed a periodicity which can be attributed to the breathing cycle of the respirated rat (70/min).

1.0

FIG. 1.-Sample autocor- 0 - relation of PR intervals fromn the curarized rat. Calculated from of 512 consecutive ob- servations. Similar results seen with ether and Nembutal. _,L L

5 10 15 20 25

Log (intervol)

It has been commonly accepted that the PR and PP intervals of the EKG are

highly positively correlated. In fact, this is quite untrue in the rat. Appreci- able negative correlations were obtained, and verified with scatter diagrams as in

Figure 2. At times, the negative correlations disappear almost completely, in-

dicating independence of the control systems for the PP and PR intervals. The PP and RR intervals are always highly correlated.

Conditioning Results. Seventy subjects were used in a variety of conditioning

experiments. This report will be concerned with those experiments investi-

gating the effects of time delay of reinforcement and two simple types of rein-

forcement schedules: (1) The simplest situation is one in which a histogram of

intervals is initially calculated, and a percentage of either the upper or lower "tail" chosen. Whenever a response is found to fall within the tail, a reinforce-

ment is administered. For example, if tail shock is used as punishment and the

upper tail of the distribution is chosen, one expects that after a time the distri-

bution will shift toward shorter intervals, thus reducing the number of aversive

reinforcements. This type of schedule will be called a constant criterion sched-

ule; it was used with both tail shock and electrical stimulation of the brain to

shift distributions toward either larger or smaller intervals. (2) A more com-

plicated schedule required the calculation of a running histogram of the last 512

intervals measured. At each response, the histogram was consulted to deter-



296 PSYCHOLOGY: C. FIELDS Poc. N. A. S.

2 I I

I I 1 2 2 3 I I

I I I I 121 1

II I 21 13 31 I I I 12 252 1 34 211 1 13 1

2 12 1 3 1 2 I I I I I I I I I I II I 22 I 122 2 II

212 I I 35 I 2 22 131 1 31 11213214 3 31 22

1 223122223 112 I 2 1222 1 1 1

12 2313422411 II 1 332 1 1 1 422333121 2 1 31 2232 1

I 333231421 1 1 14141 3

II 523322 43 1 3323233 1 I 135212 121473 22221 1

i 2 222 2 II 11212531121 2 I I I 14141661223 1

1 21 322

1 12

Correlation -0.585

FIG. 2.-Scatter diagram plotting PP interval against the enclosed PR interval. Numbers indicate the frequency of observation at each point. Results for curare, ether, and Nembutal were similar.

mine if the response was above or below the percentage of the distribution which had been chosen. For example, 10 might be the chosen percentage, in which case, if an interval were found to be below 90 per cent of the distribution, an aversive reinforcement would be applied. This would cause the response distribution to shift toward longer intervals. This will be referred to as a variable criteria schedule.

Both constant criteria and variable criteria experiments were performed, using both electrical brain stimulation and tail shock, attempting to condition animals to shift their distribution toward longer or shorter intervals. Each type of experiment was used to condition PR, PP, and RR inltervals. In all cases, the PR interval could be conditioned independently of the PP intervals. Conditioning was continued for 90 minutes in most experiments. The percentage of change of the conditioned interval from baseline was always greater with variable criteria schedules than with constant criteria schedules.

The conditioning period was followed by 60 minutes of extinction, which was usually sufficient to return the responnse to its initial value, regardless of the conditioning scheme used. Distributions could be shifted toward longer or shorter intervals with equal ease. In general, tail shock was a much more efficient reinforcement than electrical stimulation of the brain. One reason for this is that brain stimulation produced appreciable unconditioned effects, which eventually necessitated time-out periods that were similar to those previously used.5 Both the PR and PP intervals could be conditioined to produce similar percentage of



VOL. 65, 1970 PSYCHOLOGY: C. FIELDS 297

changes. The most important parameter associated with the variable criteria and constant criteria reinforcement schedules was the percentage of the distribution to be reinforced. Figure 3 shows the average percentage of change to be

expected with each criterion placement. It is clearly rather critical, so that one can appreciate some of the difficulties which may be encountered with hand

shaping of a subject. Additional experiments were performed to determine the optimal time delay

between response and reinforcement. I't is quite clear that the introduction of almost any time delay seriously retards the learning process. In particular, de-

lays of 0.5 second were tried, and almost no learning could be produced. Delays of as little as 10 msec could effectively reduce the percentage of change.

Learning curves were calculated for all subjects. Sixty per cent of the animals showed curves similar to that shown in Figure 4. This gradual learning curve is one expected from learning in a simple system such as the cardiac control system of the rat. However, 40 per cent of the subjects showed curves similar to those in Figure 5. This curve is of an all-or-none variety, and clearly differs from the

gradual learning curve previously presented. The curves presented are not two

samples from a continuum; no intermediate learning curves were ever encountered. Figure 6 is a histogram of the maximum slope of the learning curves of

l00.0 -

22.50 -

20.35 -

18.20

59.60

n- 13G.9 n-- e \

7.45

5.30 - i Co nditioning i Extinction

3.15

1. 00 I I n - n - n - j n - L n - n - i O n - C n - n - n - n - I n - n - I

i 9.oo -

50 60 65 751 n 85 90n95-99 5 10 15 20 25 30 35 40

Criteria (percent of histograms) Time (min/5)

FrIG. 3.-For each variable criteria schedule FIG. 4.--Gradual learning curve for cu- experiment, a particular percentage of the re-- rarized rat, tail shock as reinforcement, PR sponse distribution was chosen for reinforcement. interval conditioned down. The extinction The complement of this percentage is plotted section of the experiments differed from against the maximum percentage of change dur-- the conditioning section in that no rein- ing learsning experiments, comparing initial forcements were delivered-all other stim- baseline with plateau value at about 90 min. uli remained constant. Similar results were observed for PR, PP, and RR intervals, and for conditioning up or down.



298 ISYCHOLOGY: (. FIELDS Pioc. N. A. S.

57. 12

4

End of

conditioning 2 -

42.55.. ....-

5 15 25 35 45 55 65 75 85 0.1 0.5 1.0 5 20 2.5

Time (min) rsec/512 intervals

FIG. 5.-A learning curve of the all-or- FIG. 6.-Histogram of the maximum none type for a curarized rat, PR interval slope of the learning curves of 46 conditioned up, tail shock as reward. The subjects. Slope was calculated by extinction section of this graph (not shown) fitting each learn-ing curve to a family did not differ from that for gradual learn- of S-shaped curves by a least squares ing (Fig. 4). procedure.

46 subjects. The bimodality is evident. Apparently, this elementary system is able to learni in at least two modes, although it is not clear at this time whether these modes are qualitatively or merely quantitatively different.

At least 40 per cent of the animals tested did not learn by a shift of their entire response distribution in the appropriate direction. Instead, after a few reinforcements, they would begin to produce scattered responses in small numbers, retaining the starting distribution. Eventually, one of the "scatter points" would become a focus for response, and a new distribution would grow up at that value, while the old distribution would "shrink" and disappear. Duriing this process the interval would produce a distinctly bimodal distribution. An example of such a distribution is shown in Figure 7.

To summarize, learning curves tend to be either gradual or all-or-nlonie. In addition, learning occurs either by a continuous shift of response distribution or by a discontinuous displacement reminiscent of hypothesis testing in humans. The type of learning and distribution shift were relatively indepenrdent.

Averaging of responses after a reinforcement produced results such as those

69 - New response distribution

U, Old response ---

o distribution FIG. 7.-Inltermnediate response distlibution during a conldition-

o90^~~ /~ t\ ~i \ iing experiment (PR interval, (512

D! \I J \ sample) tail shock as reward, con- n- a I ditioned up).

5 10 15 20 25 30 35 40 45 50 55 60

PR interval (msec)



VOL. 65, 1970 PSYCIOLOGY: C. FIELDS 299

I.00 --

FIG. 8.-A typical average evoked

response from a PR inverval condi-

tioning experiment, conditioning DOWN, tail shock as reinforcement. E

Computer roundoff error caused linear sections of the plot from 25 to 50 heart beats.

-I1.00n n I - n -----L----I- 5 10 15 20 25 30 35 40 45 50

Time (heartbeats)

shown inr Figure 8. If one were conditioning in a gradual fashion, one might ex-

pect gradually rising or falling averages. In fact, one observes effects with very rapid rise and exponential decay, the amplitude depending on several factors, in-

cluding the direction of the attempted conditioning. These averages are pres- ently being analyzed in greater detail.

A serious disadvantage accompanying the use of simple systems in the study of

learning is that the system may learn poorly or not at all, and perhaps will not exhibit the complex phenomena of higher systems. The cardiac control system of the rat is sufficiently simple to analyze the learning process in great detail. In addition, there is a high rate of data acquisition, and electrophysiological and

pharmacological methods may be applied for further investigation. It also demonstrates a variety of quantifiable and reliable behavior not available with other simple systems.

* I wish to acknowledge the continuing assistance and advice of Dr. Neal Miller and Dr. Leo DiCara. This work was supported by USPHS grant MH 13189.

1 Kandel, E. H., and L. Tauc, J. Physiol. (London), 181, 1 (1965). 2 Kandel, E. R., in The Neurosciences: A Study Program, ed. G. C. Quarton, T. Melnechuk,

and F. O. Schmitt (New York: Rockefeller University Press, 1967), p. 666. 3 Lickey, M. E., and R. W. Berry, Physiologist, 9, 230 (1966). 4 DiCara, L. V., and N. E. Miller, J. Comp. Physiol. Psychol., 65, 8 (1968). 5 Miller, N. E., and L. V. DiCara, J. Comp. Physiol. Psychol., 63, 12 (1967). 6 DiCara, L. V., Behav. Res. Methods & Instru., in press. An analysis of arteral blood gases

in the artificially vespirated, curavized rat. 7 Weiss, J. M., J. Exptl. Anal. Behav., 10, 85 (1967). 8 Trowill, J. A., J. Comp. Physiol. Psychol., 63, 7 (1967).

. Amplifier designed and constructed by M:r. Herbert Longenecker at The Rockefeller

University. 10 Weiss, J. M., J. Comp. Physiol. Psychol., 65, 251 (1968).



instrumental conditioning of the rat cardiac control systems

Documents