single trial-based prediction of a go/no-go decision in

Neural Networks 19 (2006) 1223–1232www.elsevier.com/locate/neunet

2006 Special Issue

Single trial-based prediction of a go/no-go decision in monkey superiorcolliculus

Ryohei P. Hasegawaa,b,∗, Yukako T. Hasegawaa,b, Mark A. Segravesb

a Neuroscience Research Institute, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Ibaraki 305-8568, Japanb Department of Neurobiology and Physiology, Northwestern University, Evanston, IL 60208, USA

Received 13 January 2006; accepted 8 May 2006

Abstract

While some decision-making processes often result in the generation of an observable action, for example eye or limb movements, othersmay prevent actions and occur without an overt behavioral response. To understand how these decisions are made, one must look directly at theirneuronal substrates. We trained two monkeys on a go/no-go task which requires a saccade to a peripheral cue stimulus (go) or maintenance offixation (no-go). We performed binary regressions on the activity of single neurons in the superior colliculus (SC), with the go/no-go decision as apredictor variable, and constructed a virtual decision function (VDF) designed to provide a good estimation of decision content and its timing in asingle trial decision process. Post hoc analyses by VDF correctly predicted the monkey’s choice in more than 80% of trials. These results suggestthat monitoring of SC activity has sufficient capacity to predict go/no-go decisions on a trial-by-trial basis.c© 2006 Elsevier Ltd. All rights reserved.

Keywords: Monkey; Saccade; Superior colliculus; Decision-making; Prediction; Brain–machine interface

1. Introduction

Recent advances in both science and technology provide theability to read neuronal signals for the purpose of controllinga variety of external devices (Andersen, Musallam, & Pesaran,2004; Donoghue, 2002; Mussa-Ivaldi & Miller, 2003; Nicolelis,2003; Schwartz, 2004), or decoding visual scenes (Kamitani &Tong, 2005; Matsumoto, Sugase-Miyamoto, & Okada, 2005;Stanley, Li, & Dan, 1999). It is widely recognized that suchbrain–machine interfaces (BMI) have enormous potential toaid patients suffering from neurological disorders/impairments.In order to produce an effective BMI for motor control, it isimportant to be able to extract neural signals that reflect motorplanning in advance of the actual movement. Previous studieshave shown that real-time recording of ensemble activity frommultiple neurons can be used for arm movement control(Chapin, Moxon, Markowitz, & Nicolelis, 1999; Wessberg

∗ Corresponding address: Neuroscience Research Institute, National Instituteof Advanced Industrial Science and Technology (AIST), 1-1-1 Umezono,Tsukuba, Ibaraki 305-8568, Japan. Tel.: +81 029 861 5176; fax: +81 029 8615849.

E-mail address: [email protected] (R.P. Hasegawa).

0893-6080/$ - see front matter c© 2006 Elsevier Ltd. All rights reserved.doi:10.1016/j.neunet.2006.05.035

et al., 2000). In addition to the execution of movements,cognitive aspects, for example, the decision-making processfor an upcoming movement have also attracted the interestof neuroscientists (Glimcher, 2002; Leon & Shadlen, 1998;Schall, 2001). The decision- making process is entirely internaland does not necessarily produce motor behavior. In addition,the meaning of the cue that triggers the decision of thesubject can, to some extent, be assigned arbitrarily (Hasegawa,Sawaguchi, & Kubota, 1998; Hasegawa, Blitz, Geller, &Goldberg, 2000; Hasegawa, Matsumoto, & Mikami, 2000). Inorder to emulate such flexibility for cognitive BMI techniqueswe focused on the oculomotor system as a model “decision-maker” in the brain. The superior colliculus (SC), locatedon the dorsal surface of the midbrain, is important for thecontrol of saccadic eye movements (Sparks, 2002). Althoughmany investigations of decision-making processes for saccadetarget selection focus upon parietal and prefrontal cortical areas(Glimcher, 2002; Schall, 2001), recent studies have shownthat the SC also contributes to these cognitive processes(Basso & Wurtz, 1997; Glimcher & Sparks, 1992; Horwitz &Newsome, 1999; Ignashchenkova, Dicke, Haarmeier, & Thier,2004; Krauzlis & Dill, 2002; Li & Basso, 2005; McPeek &Keller, 2002; Ratcliff, Cherian, & Segraves, 2003). It has not

http://www.elsevier.com/locate/neunet

mailto:[email protected]

http://dx.doi.org/10.1016/j.neunet.2006.05.035

1224 R.P. Hasegawa et al. / Neural Networks 19 (2006) 1223–1232

Fig. 1. (A) Oculomotor go/no-go task. Go (saccade) or no-go (maintenance of fixation) response is required immediately following the appearance of a peripheralcue. The concurrent disappearance of the fixation spot signals the time for a response. The colour of the cue is green (go) or red (no-go). Dashed circle indicatesdesired eye position. Blue arrow signifies a correct saccade. (B) Hypothetical virtual decision function (VDF). This function is constructed to reflect the developmentof neuronal processing related to a go/no-go decision, beginning with the appearance of a cue stimulus. (For interpretation of the references to colour in this figurelegend, the reader is referred to the web version of this article.)

been demonstrated, however, whether the information contentof the activity of a small number of neurons during singletrials is sufficient to predict the decision-making process forsaccades.

This report tests the possibility that the single trial activity ofSC neurons can be used to predict a simple go/no-go decisionprocess. We recorded from 59 single neurons in the SC whilea monkey performed an oculomotor go/no-go task. Here, wepresent the results of a post hoc prediction of a go/no-godecision by single trial-based activity. We found that even theactivity of single neurons could match the monkey’s choicebehavior on each trial. The timing of these predictions was fastenough to predict a choice well in advance of the required motorresponse.

2. Methods

2.1. Animal and surgery

Two adult female Rhesus monkeys (Macaca mulatta)were prepared for this study. Surgery to provide a meansfor head restraint, eye position recording with a magneticsearch coil, and recording of neuronal activity, was donewith the monkey anesthetized, using standard sterile surgicaltechniques. Northwestern’s Animal Care and Use Committeeapproved all animal protocols, which were in compliance withthe NIH Guide for the Care and Use of Laboratory Animals.

2.2. Behavioral task

We trained each monkey on an oculomotor go/no-go task(Fig. 1A). A trial was initiated by fixation of a centralfixation spot (white spot; 0.5◦ diam.). Next, the fixation spotdisappeared and simultaneously a peripheral stimulus (“cue”;1◦ by 1◦ square) appeared. The monkey was trained to responddifferently to the stimulus depending upon the colour of cue,that is, either make a saccade toward the stimulus within 800ms (green cue; go trial) or maintain fixation for longer than 800ms (red cue; no-go trial). A correct response was rewarded witha drop of water. During training, we presented the green or redcue at 24 locations that were in 8 directions (45◦ spacing) at3 eccentricities (5, 10 and 15◦ away from the fixation point).

All conditions were run in a pseudorandom fashion. Duringthe recording sessions, we first presented all of these possibleconditions to define the response fields of isolated neurons.Then, we presented the green/red cue only at the preferredlocation (near the center of the response field) and at theopposite location (rotated 180◦ with the same amplitude). Thisreduction in the number of conditions allowed us to record moretrials with the responses of interest (usually more than 20 trialsfor each condition).

2.3. Recording of neuronal activity

When the monkey’s performance reached a criterion of>85% correct over 3 successive days, we started recordingfrom the SC. The location of the SC was confirmed bystereotaxic coordinates, the response properties of isolatedneurons, and the characteristics of its topographically organizedvisual/motor map. The recording of single and multiunitactivity was done with tungsten microelectrodes (A-MSystems, Inc.) introduced through stainless steel guide tubesthat pierced the dura, using a Crist grid system (Crist,Yamasaki, Komatsu, & Wurtz, 1988). For multielectroderecordings, electrodes were introduced through separate guidetubes. These electrodes were manipulated by one or twoNarishige microdrives. A 16-channel Plexon System wasdedicated to accepting the input from up to 16 electrodes,and could isolate 2 neurons’ waveforms from each of thoseelectrodes. We usually used 2–4 electrodes for recordingmaximally from 8 neurons. In this study, we focused on thebuildup (or prelude) type of SC neurons (Basso & Wurtz, 1998;Glimcher & Sparks, 1992; Munoz & Wurtz, 1995). We testedthe isolated neurons in separate blocks of a “delay-version” ofthe go task with an additional 800 ms response delay insertedbetween cue onset and fixation spot disappearance. The neuronswere classified as buildup if they exhibited a significant increasein delay period activity on the delayed go task (Wilcoxon Sign-Rank test, P < 0.05, compared to the precue-fixation activity).We used the REX system (Hays, Richmond, & Optican, 1982)running on a Dell Pentium II computer for behavioral controland eye position monitoring. Visual stimuli were generated bya second PC, which was controlled by the REX machine andrear-projected onto a tangent screen in front of the monkey by

R.P. Hasegawa et al. / Neural Networks 19 (2006) 1223–1232 1225

a CRT video projector (Sony VPH-D50, 75 Hz non-interlacedvertical scan rate, 1024 × 768 resolution).

2.4. Analysis and model

We analyzed data offline using programs written in Matlab.We performed a regression analysis on the ensemble activityof SC neurons in order to generate a virtual decision function(VDF, Fig. 1B) which was designed to reflect the progress ofgo/no-go decision making. As the decision process is entirelyinternal, we could not determine if the VDF function representsthe actual time of the monkey’s decision. Our goal was to makethe VDF a reasonably good approximation that was indicatedby a high correspondence to the actual decision content at anappropriate time (for example, before actual saccades on gotrials).

For each neuron we first calculated a spike density functionby convolving single trial activity with a Gaussian kernel(Richmond, Optican, Podell, & Spitzer, 1987) with a sigmaof 10 ms (corresponding to the tail becoming almost zero at±36 ms). We measured the activity over a 500 ms interval,starting 100 ms before the cue onset and ending 400 ms afterit. We used the least squares method to fit a logistic model tothe data for each millisecond (t) of activity:

y(t) =2

1 + exp(c(t) − w(t)x(t))− 1 (1)

where the independent variables, x(t) was discharge rateof a neuron at a time t , w(t) was a regression coefficient(weight), and c(t) was a constant term at the moment oft . Dependent/predicted variable, y(t) was a desired go/no-goscore; its actual data (type of trial), z was set to either 1 on gotrials or to −1 on no-go trials, and was held constant throughouta given trial.

z =

{1: go trial−1: no-go trial.

(2)

A set of appropriate weights of the model were determinedby the least square method which minimized the sum of thesquared regression errors

SE (t) =

m∑j=1

(z j − y j (t)

)2 (3)

where zi and yi (t) were respectively z (actual data) and y(t)(fitted data) on each trial ( j , trial ID; m, total number of trials).SE (t) represents the variation which is unexplained by theregression equation. On the other hand, the sum of squarederrors from the mean of y(t) or y(t)

ST (t) =

m∑j=1

(z j (t) − y(t)

)2 (4)

represents the total variation. The fitness of the model wasevaluated by the coefficient of determination (R2)

R2(t) = 1 −SE (t)

ST (t)(5)

which represents the proportion of variation that is explainedby the model. R2 may vary from 0 (no predictive power) to 1(perfect prediction).

In this study, we repeated this analysis over the 500 msinterval beginning 100 ms before cue onset to determineweights w, constant c and R2 at a time t (sampled at 1 kHz).We used R2(t) as well as y(t) to produce a VDF(t) that wasdesigned to reflect a single trial-based decision-making processfor go (saccade) or no-go (maintenance of fixation) time-lockedby the appearance of cue stimulus.

VDF(t) = y(t)R2(t) (6)

R2 represents how well the expected data fit the real data.It can be a good index for the confidence level of predictionbut R2 itself does not indicate the essence of the prediction(go vs. no-go). Instead, y(t) is the actual predictor although itlacks the information about “how well”. Therefore, y(t) and R2

complement each other when those functions are multiplied.To evaluate the success or failure of the prediction on each

trial, we used a pair of criteria for the “go decision” and forthe “no-go decision” (e.g., +0.3 for go and −0.3 for no-go). Ifthe VDF reached either criterion, the neurons were consideredto have made a prediction. If the neuronal prediction matchedthe observed behavior, that is, the monkey’s decision, such atrial was considered to be one with a good prediction. Foreach recording session, we set the criteria (threshold to reachthe estimated go/no-go decision level) by simulations; out of20 candidates of a pair of criteria (i.e., ±0.05, ±0.1, ±0.15. . . ±1.0), we chose the “optimal” pair so that the number ofgood predictions was maximized. We defined decision time asthe first moment when the function reached either the go orno-go criterion. A mismatch of neuronal prediction to actualbehavior or (rarely) a trial where no neural prediction wasmade was considered to be a trial with a bad prediction. Itwas also incorrect if the VDF became 1 or −1 before the cuewas presented, since it was unlikely that the monkey formeda decision before the instruction was presented. Therefore,we counted those trials (including the trials with the neuralprediction made within 50 ms after the cue onset) as bad ones.We also set a time limitation of up to 400 ms after the cuepresentation for inclusion of neuronal data in the analysis.

For the prediction above, we used the leave-one-out method,in which we remove single trials from the dataset (n − 1) toconstruct the model and tested the model against the removedtrial. This procedure was repeated n times.

3. Results

3.1. Recording and analysis of a single neuron activity

Fig. 2 shows examples of the single trial activities of anSC neuron. This neuron exhibited a strong response when the“go” cue was presented within its response field located in thecontralateral visual field 15◦ away from the fixation spot (Fig. 2left). The neuron also briefly increased its firing rate when the“no-go” cue was presented at the same contralateral location,but this activity decayed quickly (Fig. 2, middle). Conventional


Fig. 2. Concept of single trial prediction. Left and middle columns: examples of go and no-go trials. Right column: an unknown trial. Cartoons in the top row showthe cue location (green or red square), saccade direction (blue arrow) and response field (yellow area). A raster display of spike timing with eye position, and aspike-density function is shown for each trial. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of thisarticle.)

Fig. 3. Responses of an SC neuron in the go/no-go task. Top and middle rows: spike-density functions of single trial activity on go (top row) and no-go (middle row)trials. Cartoons to the left of these rows show the cue location (green or red square), saccade direction (blue arrow) and response field (yellow area). Bottom row:Average of spike density functions on go (green) and no-go (red) trials. Blue cross indicates saccade onset time on single trials (top row) or average saccade onsettime on combined trials (bottom row). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

neurophysiological studies often examine the property of singleneurons by comparing the average spike densities acrossexperimental conditions. For the case of the neuron in Fig. 2,we first collected data from as many go and no-go trials aspossible (Fig. 3, top and middle) then calculated an averagefor each condition (Fig. 3, bottom). These analyses revealedthat the spike density functions generally take higher values on

go than no-go trials, and that the neuron had a combination ofvisual and buildup activities. The comparison of time courseof activities between go and no-go trials suggests that, onaverage, the subject might begin to discriminate between goand no-go decisions at about 80 ms after cue onset (Fig. 3,bottom). Nevertheless, the discrimination time is different indifferent neurons. In addition, due to the inherent noisiness of


Fig. 4. An example of regression analysis of neuronal activity. (A) Single trial activity with associated regression curves at selected time intervals (−50 to 300 mswith an increment of 50 ms). X - and Y -axes correspond to neuronal activity level and go/no-go score (monkey’s choice). For each plot a solid blue line indicates thelogistic function that was fitted to all of the data. Horizontal green and red lines indicate go/no-go scores of +1 (go) and −1 (no-go). (B) Time course of a weight(unstandardized regression coefficient), a constant, and a coefficient of determination (R2) aligned to the cue onset. These data are from the recording session shownin Fig. 3. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

single neuron activity, it is difficult to infer, from a continuousspike density function on single trials, when those decisionswere made on each trial. Furthermore, a trial with intermediateactivity such as the one depicted in Fig. 2 right makes predictioneven more difficult.

3.2. Application of regression analysis on neuronal activity

The conventional analysis provided above does not directlylead us to a reliable prediction of behavior for individualtrials. Instead, we focused upon a regression analysis. Thisanalysis is frequently used in neurophysiological studies toexplain the variation of single trial neuronal activity asa dependent/predictor variable in relationship to cognitive,motivational or motor parameters as independent/explanatoryvariables (Hasegawa, Peterson, & Goldberg, 2004; Hasegawa,Blitz, & Goldberg, 2004; Itoh et al., 2003). In this study, weused the activity (spikes/s) of single neurons as independentvariables and a binary decision outcome (go/no-go score, 1for go and −1 for no-go) as a dependent variable. Since thedependent variable is binary, we adopted a logistic modelinstead of a linear model. First, we repeatedly applied theregression analysis to single trial activity over all trials with atime step of 1 ms and duration of 500 ms ranging from 100ms before to 400 ms after the onset of the cue. Fig. 4A shows

examples of regression from an SC neuron at 8 different times.The logistic curve was well fitted to the data between 100 and300 ms after the cue onset, indicating that we could predict thego/no-go decision even from activity on a new trial. While theproduction of the predictable logistic functions at these timeswas reasonable, it was expected that up to and including theinitial time of cue onset the regression should produce a flatfunction without prediction, since y and/or R2 are small duringthis period. The data at 0 ms, however, also produced a shapeof the logistic curve suggesting that high and low activity arelinked to the go and no-go decisions, which would seem to beimpossible just at the time of cue presentation. However, thecoefficient of determination (R2) for this time period was verylow (0.03), indicating that the shape of the curve may have beendue to chance, and calling for a closer evaluation of the abilityof the logistic function to predict go/no-go decisions.

Fig. 4B shows time course of weight (w) and constant (c) aswell as R2. For this neuron, the weight term (w) became higherbetween 100 and 300 ms, indicating that the neuron’s higheractivity was linked to a higher probability of a go decision(Fig. 4B, left). The decrease in the constant term (c) during thesame 100–300 ms time period reflects the nonselective responseof this neuron to the visual cue onset regardless of the go/no-go decision (Fig. 4B, middle). The higher R2 values, which


Fig. 5. A typical example of a successful “go” prediction for the neuron in Figs. 2–4. Top row: single trial activity with raster display and a spike density function.Blue cross indicates the onset of saccade. Middle row: the regressions of go/no-go scores by the leave-one-out method at representative times (0–150 ms with anincrement of 50 ms). For each plot a dashed blue curve indicates the original logistic function that was fitted for all trials except the trial that was used to testprediction (shown in the top row). A solid blue curve indicates the logistic function modified by R2 value. Vertical dashed green line indicates the activity levelat each time. Open and filled diamonds on the vertical green line indicate the predicted go/no-go score without R2 modulation (y) or that with R2 modulation.Horizontal green and red lines indicate go/no-go scores of +1 (go) and −1 (no-go). Bottom row: single trial prediction by the VDF. Dashed blue curve indicates thetime course of y values (y(t), plot of the open diamonds against time shown in the middle row). Solid blue curve indicates the time course of the VDF (VDF(t), plotof the filled diamonds against time shown in the middle row). Horizontal green and red lines indicate go and no-go criteria (go/no-go scores of 0.45 and −0.45).Vertical green line indicates the “decision time”, in which the VDF(t) reached the go criteria, and hence the go decision was considered to be made (t = 125 ms).Vertical solid and dashed lines in the top and bottom rows indicate time slices at 0, 50, 100 and 150 ms, which are depicted in the middle row. (For interpretation ofthe references to colour in this figure legend, the reader is referred to the web version of this article.)

also appear between 100 and 300 ms, indicate that, predictionduring this time was more reliable than during the precedingperiod (Fig. 4B, right). Although there was some increase in w

even before the appearance of the cue, the R2 value during thisperiod was near zero, demonstrating that a focus upon R2 wasthe most informative way to interpret the regression results.

3.3. Go/no-go prediction by the virtual decision function(VDF)

After preparing the templates from all trials except the oneleft for testing the prediction for each recording session, wereturned to the analysis of single trials to verify the template’sability to predict the go/no-go decision. We generated a VDFfor each trial that was designed to reflect the progress of thego/no-go decision time-locked to the appearance of the cue. Tocalculate the VDF we multiplied a predictor variable (y), whichwas calculated for each millisecond of a trial by a regressionmodel, with R2 (see Methods). Fig. 5 shows a typical example

of a successful “go” prediction for the neuron in Figs. 2–4. Ona go trial, we observed strong activity between the appearanceof the cue and the saccade towards it (Fig. 5 top). For this trialwe calculated y as well as R2 values at every millisecond basedupon the regression analysis on the rest of the trials (Fig. 5,middle). We then generated a VDF for this trial (Fig. 5, bottom,solid blue line). The VDF stayed near 0 at the beginning ofthe trial, indicating that a decision had not been made. Later,however, the VDF started increasing towards 1 about 100 msafter the cue onset, indicating an increasing probability of ago decision. We judged that the subject made a go decision at125 ms, that is, when the VDF reached the go criteria (for thisneuron, +0.45). Fig. 6 shows an example of a successful no-goprediction in the same session. The VDF started decreasing ona no-go trial with a similar time course, indicating an increasingprobability of a no-go decision. Although the change of thefunction was continuous, we defined the time for go/no-godecision (“decision time”) as the earliest time at which theVDF reached either the go or no-go criterion, which was set


Fig. 6. A typical example of a successful “no-go” prediction for the neuron in Figs. 2–4. Conventions the same as in Fig. 5 except the following two points. In themiddle row vertical red line indicates the activity level at each time. In the bottom row vertical red line indicates the “decision time”, in which the VDF(t) reachedthe no-go criteria, and hence the no-go decision was considered to be made (t = 141 ms). (For interpretation of the references to colour in this figure legend, thereader is referred to the web version of this article.)

in individual sessions by simulation (see Methods). For the“go” trial, the VDF reached the criterion for “go” at 125 msafter cue onset (Fig. 5, bottom panel, green vertical line). Thisvirtual decision was accomplished 95 ms before the beginningof the saccade. For the “no-go” trial, the VDF reached the “no-go” criterion 141 ms after cue onset (Fig. 6, bottom panel, redvertical line). Note that the VDF was not affected by the initialvisual response during the first 100 ms interval after cue onset.Fig. 7 depicts all VDFs constructed in this recording session.For the 46 trials, most of the trials (N = 41) were predictedcorrectly (Fig. 6A; go: 87%, 20/23; no-go: 91%, 21/23).The remaining 5 trials were incorrect predictions (i.e., no-godecision on go trials or go decision on no-go trials; Fig. 6B).On some incorrect trials (not in this session), the VDF reachedthe criteria too early (before the cue onset or within 50 ms afterthe cue onset).

3.4. Accuracy of predictions by the VDF

For the 59 single neurons, we obtained results that weresimilar to those illustrated in Figs. 2–7. The average correctprediction percentage for all 59 neurons was 83% for both“go” and “no-go” trials (Fig. 8). If we did not introduce thegain effect by R2, the correct prediction dropped significantly(go: F(1, 58) = 13.986, P < 0.001; no-go: F(1, 58) =

60.312, P < 0.001). Regardless of whether or not R2 was

included in the prediction calculation, incorrect predictionswere primarily due to mismatches between cue and go/no-godecision (as opposed to premature predictions). This form ofincorrect predictions was made with a timing that was similarto that for trials with correct predictions (usually between 100and 150 ms after cue onset).

3.5. Speed of prediction by the VDF

While one aspect of the major aims of this study wasto evaluate the accuracy of prediction of the final decision,the other aspect was to evaluate the speed of the prediction.We compared the decision time between go and no-go trials(Fig. 9A). For both types of trials, decision times were similarlydistributed (around 75–200 ms, average of 59 median decisiontimes on go trials = 141 ms, and on no-go trials = 148 ms),though the difference was significant (Wilcoxon Sign-Ranktest, P < 0.001).

Since it cannot be predictive if the model’s decision is madeafter the beginning of the saccade, we compared the decisiontime on go trials with saccade latency (Fig. 9B) to see how earlythe VDF can predict future action. The average of 59 mediansaccade latencies was 217 ms; all of the decision times precededthe beginning of the saccade by an average of 77 ms (WilcoxonSign-Rank test, P < 0.001).


Fig. 7. VDFs on all 23 go (left column) and 23 no-go (right column) trials associated with correct (top row) and incorrect (bottom row) predictions. Data wereobtained from the same session as used in Figs. 2–6. Upward- and downward-pointing triangles indicate the timing of estimated go and no-go decision, at whichVDFs reach either the go or no-go criterion (±0.45).

Fig. 8. Population effect of R2 on prediction accuracy. Prediction model withR2 (=VDF) produced significantly (P < 0.001) better predictions than thatwithout R2 (=y only) on both go and no-go trials. Error bar indicates standarderror of mean.

4. Discussion

In this study, we addressed the issue of how well the activityof single neurons in the superior colliculus (SC) can predict,for individual trials, a monkey’s decision to make a saccade orto maintain fixation in response to a peripheral cue. To test this,we constructed a virtual decision function (VDF) by performingrepetitive regressions on every millisecond of neuronal activitytime-locked to the cue onset. We found that the VDF correctlypredicts the monkey’s decision on more than 80% of the trials,and did so around 75–200 ms after the cue onset and about77 ms on average before the saccade onset.

There are several innovative aspects to this study. First,we focused on a single trial-based prediction of the decision-making process. Most studies of the neural basis of thedecision process compare averaged activity of a neuron onmultiple trials under one behavioral condition with activityof the neuron under a different behavioral condition. Whilethis type of multi trial-based prediction yields discrimination

latency between those behavioral conditions, it cannot predictbehavior for individual trials. The single trial-based predictiondemonstrated in this report has the advantage that it producesa decision outcome (i.e., go or no-go) as well as a decisiontime for each trial. Previous studies of brain–machine interfaces(BMI) demonstrated that neural signals from motor-relatedareas could serve to control robot arms or computer cursors(Musallam, Corneil, Greger, Scherberger, & Andersen, 2004;Serruya, Hatsopoulos, Paninski, Fellows, & Donoghue, 2002;Taylor, Tillery, & Schwartz, 2002; Wessberg et al., 2000). Inthese studies, the major goal was to predict either kinematic(e.g., trajectory and joint rotation) of arm movements orgeometric (e.g., movement of an X–Y position) parameters,which can be directly observed. In contrast, the outcomeof internal mental processes such as a YES/NO decisionor perceptual categorization may not result in an event thatcan be directly observed. Our study follows the direction ofprevious BMI studies, with an expansion to cognitive BMI,allowing us to estimate decision contents and their timingsfor binary decisions including an entirely internalized “no-go”decision.

The second innovative aspect of this study was theintroduction of the Virtual Decision Function (VDF), thefunction which was designed to represent the progress ofthe internal decision process on single trials. Although weindicated that the progress from the “nothing decided” stateto a “something decided” state could be continuous, wehypothetically set 1 to the go decision and −1 to the no-godecision as a predictor variable (y). Unlike many regressionstudies, the continuous changes in the VDF were not simplyproduced by expected values from a single best-fit function.Instead, those changes were constructed based upon theintegrated results of repetitive regressions every millisecond, inwhich not only y values but also R2 values played an importantrole in determining confidence levels for a decision. Weexpected that y and R2 would complement one another when


Fig. 9. (A) Distribution of medians of decision times over 59 recording sessions. Results for go (top row) and no-go (bottom row) are separately shown. (B)Comparison of decision times on go trials and saccade latencies.

they were multiplied, and found that the average prediction wasmuch higher with R2 than without. The VDF allowed us notonly to predict the monkey’s final choice but also to estimatethe time course of the computed decision-making process. Onaverage a decision point was reached about 80 ms before thebeginning of the saccade. This rapid signal allows ample timefor the control of external robotic devices. For the purposes ofthis study, we focused on the presentation of the concept of aVDF, that is, a function based upon repetitive regressions andmodification by R2. Our study does not, however, exclude otherpossibilities for the form of the VDF. Our choice (y multipliedby R2) is just one example and other choices might producebetter prediction results. Nevertheless, it turned out that ourVDF was a reasonably good predictor with a high degree ofdecision accuracy that could reach a decision at an appropriatetime before the actual saccade.

Finally, recording of single neurons from the SC madethe single trial prediction possible. The primate SC isan important structure for the generation of saccadic eyemovements, integrating multiple cortical inputs and providingan oculomotor command to the premotor burst neurons (Sparks,2002). The deep layers of the SC primarily contain twotypes of neurons related to saccade execution: buildup andburst neurons. Previous studies have shown the activity ofSC neurons, particularly buildup neurons, to be related to theselection of saccade targets from among multiple candidatestimuli during a variety of tasks (Basso & Wurtz, 1997;Glimcher & Sparks, 1992; Horwitz & Newsome, 1999;Ignashchenkova et al., 2004; Krauzlis & Dill, 2002; Li &Basso, 2005; McPeek & Keller, 2002; Ratcliff et al., 2003).These studies suggest that the activity of SC neurons reflectsnot only the generation of a motor command but also thecognitive process for decision-making related to saccades. Ourattempt to extract a prediction signal from the SC is based onthese earlier studies demonstrating a reliable SC signal for asaccade decision. The SC has also been shown to be an areathat is involved in the inhibitory control of saccades duringan antisaccade task (Everling, Dorris, Klein, & Munoz, 1999;Munoz & Everling, 2004), a countermanding task (Pare &

Hanes, 2003) and a go/no-go task (Li & Basso, 2005). Thesetasks are similar to our go/no-go task in that the subject isrequired to suppress a saccade toward the stimulus location.Our results are consistent with these studies in which SCneurons exhibited less activity on antisaccade/stop trials thango prosaccade/control trials. These earlier findings, along withour own, suggest that the SC is an ideal candidate to serve asa reliable predictor of oculomotor decisions that can be madeavailable for real-time BMI purposes.

In conclusion, our regression analysis of the responses ofsingle neurons produced, on each trial, a virtual decisionfunction that was designed to reflect a decision-making processfor go (saccade) or no-go (maintenance of fixation) time-lockedto the appearance of a cue stimulus. Decisions reached in morethan 80% of trials were successfully predicted by this function.The results of this study support the hypothesis that single SCneurons can provide sufficient information to make accuratetrial-by-trial predictions of a monkey’s decision. In addition,the methods described in this report could easily be extendedto a multiple regression model using a population of neuronswith the expectation that prediction accuracy would be furtherimproved.

Acknowledgments

We thank Drs. Ken Ohta, Kenichiro Miura, Takio Kurita,Narihisa Matsumoto for comments on this article, AngelaNitzke for technical assistance, and the staff of Northwestern’sCenter for Comparative Medicine for Animal Care. Thisstudy was supported by the NIH (EY08212) and the AIST.Correspondence should be addressed to RPH.

References

Andersen, R. A., Musallam, S., & Pesaran, B. (2004). Selecting the signalsfor a brain–machine interface. Current Opinion in Neurobiology, 14(6),720–726.

Basso, M. A., & Wurtz, R. H. (1997). Modulation of neuronal activity by targetuncertainty. Nature, 389(6646), 66–69.


Basso, M. A., & Wurtz, R. H. (1998). Modulation of neuronal activityin superior colliculus by changes in target probability. The Journal ofNeuroscience, 18(18), 7519–7534.

Chapin, J. K., Moxon, K. A., Markowitz, R. S., & Nicolelis, M. A. (1999).Real-time control of a robot arm using simultaneously recorded neurons inthe motor cortex. Nature Neuroscience, 2(7), 664–670.

Crist, C. F., Yamasaki, D. S., Komatsu, H., & Wurtz, R. H. (1988). Agrid system and a microsyringe for single cell recording. Journal ofNeuroscience Methods, 26(2), 117–122.

Donoghue, J. P. (2002). Connecting cortex to machines: Recent advances inbrain interfaces. Nature Neuroscience, 5(Suppl.), 1085–1088.

Everling, S., Dorris, M. C., Klein, R. M., & Munoz, D. P. (1999). Role ofprimate superior colliculus in preparation and execution of anti-saccadesand pro-saccades. The Journal of Neuroscience, 19(7), 2740–2754.

Glimcher, P. (2002). Decisions, decisions, decisions: Choosing a biologicalscience of choice. Neuron, 36(2), 323–332.

Glimcher, P. W., & Sparks, D. L. (1992). Movement selection in advance ofaction in the superior colliculus. Nature, 355(6360), 542–545.

Hasegawa, R., Peterson, B., & Goldberg, M. (2004). Prefrontal neurons codingsuppression of specific saccades. Neuron, 43(3), 415–425.

Hasegawa, R., Sawaguchi, T., & Kubota, K. (1998). Monkey prefrontalneuronal activity coding the forthcoming saccade in an oculomotor delayedmatching-to-sample task. Journal of Neurophysiology, 79(1), 322–333.

Hasegawa, R. P., Blitz, A. M., Geller, N. L., & Goldberg, M. E. (2000). Neuronsin monkey prefrontal cortex that track past or predict future performance.Science, 290(5497), 1786–1789.

Hasegawa, R. P., Blitz, A. M., & Goldberg, M. E. (2004). Neurons in monkeyprefrontal cortex whose activity tracks the progress of a three step self-ordered task. Journal of Neurophysiology, 98(3), 1524–1535.

Hasegawa, R. P., Matsumoto, M., & Mikami, A. (2000). Search targetselection in monkey prefrontal cortex. Journal of Neurophysiology, 84(3),1692–1696.

Hays, A., Richmond, B., & Optican, L. (1982). A UNIX-based multiple processsystem for real-time data acquisition and control. In WESCON conf. proc.vol. 2 (pp. 1–10).

Horwitz, G. D., & Newsome, W. T. (1999). Separate signals for target selectionand movement specification in the superior colliculus. Science, 284(5417),1158–1161.

Ignashchenkova, A., Dicke, P. W., Haarmeier, T., & Thier, P. (2004). Neuron-specific contribution of the superior colliculus to overt and covert shifts ofattention. Nature Neuroscience, 7(1), 56–64.

Itoh, H., Nakahara, H., Hikosaka, O., Kawagoe, R., Takikawa, Y., & Aihara,K. (2003). Correlation of primate caudate neural activity and saccadeparameters in reward-oriented behavior. Journal of Neurophysiology, 89(4),1774–1783.

Kamitani, Y., & Tong, F. (2005). Decoding the visual and subjective contentsof the human brain. Nature Neuroscience, 8(5), 679–685.

Krauzlis, R., & Dill, N. (2002). Neural correlates of target choice for pursuitand saccades in the primate superior colliculus. Neuron, 35(2), 355–363.

Leon, M. I., & Shadlen, M. N. (1998). Exploring the neurophysiology ofdecisions. Neuron, 21(4), 669–672.

Li, X., & Basso, M. (2005). Competitive stimulus interactions within singleresponse fields of superior colliculus neurons. The Journal of Neuroscience,25, 11357–11373.

Matsumoto, M., Sugase-Miyamoto, Y., & Okada, M. (2005). Categorical sig-nals in single-trial neuron activity of inferotemporal cortex. NeuroReport,16, 1707–1710.

McPeek, R. M., & Keller, E. L. (2002). Saccade target selection in the superiorcolliculus during a visual search task. Journal of Neurophysiology, 88(4),2019–2034.

Munoz, D. P., & Everling, S. (2004). Look away: The anti-saccade task and thevoluntary control of eye movement. Nature Reviews: Neuroscience, 5(3),218–228.

Munoz, D. P., & Wurtz, R. H. (1995). Saccade-related activity in monkeysuperior colliculus. I. Characteristics of burst and buildup cells. Journal ofNeurophysiology, 73(6), 2313–2333.

Musallam, S., Corneil, B. D., Greger, B., Scherberger, H., & Andersen, R. A.(2004). Cognitive control signals for neural prosthetics. Science, 305(5681),258–262.

Mussa-Ivaldi, F. A., & Miller, L. E. (2003). Brain–machine interfaces:Computational demands and clinical needs meet basic neuroscience. Trendsin Neurosciences, 26(6), 329–334.

Nicolelis, M. A. (2003). Brain–machine interfaces to restore motor functionand probe neural circuits. Nature Reviews: Neuroscience, 4(5), 417–422.

Pare, M., & Hanes, D. P. (2003). Controlled movement processing: Superiorcolliculus activity associated with countermanded saccades. The Journal ofNeuroscience, 23(16), 6480–6489.

Ratcliff, R., Cherian, A., & Segraves, M. (2003). A comparison of macaquebehavior and superior colliculus neuronal activity to predictions frommodels of two-choice decisions. Journal of Neurophysiology, 90(3),1392–1407.

Richmond, B. J., Optican, L. M., Podell, M., & Spitzer, H. (1987). Temporalencoding of two-dimensional patterns by single units in primate inferiortemporal cortex. I. Response characteristics. Journal of Neurophysiology,57(1), 132–146.

Schall, J. D. (2001). Neural basis of deciding, choosing and acting. NatureReviews: Neuroscience, 2(1), 33–42.

Schwartz, A. B. (2004). Cortical neural prosthetics. Annual Review ofNeuroscience, 27, 487–507.

Serruya, M. D., Hatsopoulos, N. G., Paninski, L., Fellows, M. R., &Donoghue, J. P. (2002). Instant neural control of a movement signal.Nature, 416(6877), 141–142.

Sparks, D. L. (2002). The brainstem control of saccadic eye movements. NatureReviews: Neuroscience, 3(12), 952–964.

Stanley, G. B., Li, F. F., & Dan, Y. (1999). Reconstruction of natural scenesfrom ensemble responses in the lateral geniculate nucleus. The Journal ofNeuroscience, 19(18), 8036–8042.

Taylor, D. M., Tillery, S. I., & Schwartz, A. B. (2002). Direct cortical controlof 3D neuroprosthetic devices. Science, 296(5574), 1829–1832.

Wessberg, J., Stambaugh, C. R., Kralik, J. D., Beck, P. D., Laubach, M.,Chapin, J. K., et al. (2000). Real-time prediction of hand trajectory byensembles of cortical neurons in primates. Nature, 408(6810), 361–365.

single trial-based prediction of a go/no-go decision in

Documents