ieee transactions on biomedical engineering, vol. 64, …

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 64, NO. 4, APRIL 2017 935

A High-Performance Neural ProsthesisIncorporating Discrete State Selection With

Hidden Markov ModelsJonathan C. Kao∗, Student Member, IEEE, Paul Nuyujukian∗, Member, IEEE,

Stephen I. Ryu, Member, IEEE, and Krishna V. Shenoy†, Senior Member, IEEE

Abstract—Communication neural prostheses aim to re-store efficient communication to people with motor neuro-logical injury or disease by decoding neural activity intocontrol signals. These control signals are both analog (e.g.,the velocity of a computer mouse) and discrete (e.g., click-ing an icon with a computer mouse) in nature. Effective,high-performing, and intuitive-to-use communication pros-theses should be capable of decoding both analog anddiscrete state variables seamlessly. However, to date, thehighest-performing autonomous communication prosthe-ses rely on precise analog decoding and typically do notincorporate high-performance discrete decoding. In this re-port, we incorporated a hidden Markov model (HMM) intoan intracortical communication prosthesis to enable accu-rate and fast discrete state decoding in parallel with ana-log decoding. In closed-loop experiments with nonhumanprimates implanted with multielectrode arrays, we demon-strate that incorporating an HMM into a neural prosthesis

Manuscript received August 18, 2015; revised February 12, 2016 andMay 4, 2016; accepted June 3, 2016. Date of publication June 21, 2016;date of current version March 17, 2017. J.C. Kao and P. Nuyujukian con-tributed equally to this work. The work of J. C. Kao was supported bythe National Science Foundation Graduate Research Fellowship. Thework of P. Nuyujukian was supported by the Stanford Medical Schol-ars Program, Howard Hughes Medical Institute Medical Research Fel-lows Program, Paul and Daisy Soros Fellowship, and Stanford MedicalScientist Training Program. The work of S. I. Ryu and K. V. Shenoywas supported by Christopher and Dana Reeve Paralysis Foundation.The work of K. V. Shenoy was supported by Burroughs Welcome FundCareer Awards in the Biomedical Sciences, Defense Advanced Re-search Projects Agency Revolutionizing Prosthetics 2009 N66001-06-C-8005 and Reorganization and Plasticity to Accelerate Injury Recov-ery N66001-10-C-2010, U.S. National Institutes of Health National In-stitute of Neurological Disorders and Stroke Collaborative Research inComputational Neuroscience Grant R01-NS054283 and BioengineeringResearch Grant R01-NS064318 and Transformative Research Award T-R01NS076460, and U.S. National Institutes of Health EUREKA AwardR01-NS066311 and Director’s Pioneer Award 1DP1OD006409. Asteriskindicates co-first authors and Dagger indicates corresponding author.

∗J. C. Kao is with the Department of Electrical Engineering, StanfordUniversity.

∗P. Nuyujukian is with the Department of Electrical Engineering, De-partment of Bioengineering, and School of Medicine and Neurosurgery,Stanford University.

S. I. Ryu is with the Department of Electrical Engineering, StanfordUniversity, and also with the Palo Alto Medical Foundation.

†K. V. Shenoy is with the Department of Electrical Engineering, De-partment of Bioengineering, and Department of Neurobiology, StanfordUniversity, and also with the Howard Hughes Medical Institute, Stanford,CA 94305-5436 USA (e-mail: [email protected]).

This paper contains supplemental material available online at http://ieeexplore.ieee.org (File size: 6 MB).

Digital Object Identifier 10.1109/TBME.2016.2582691

can increase state-of-the-art achieved bitrate by 13.9% and4.2% in two monkeys (p < 0.01). We found that the tran-sition model of the HMM is critical to achieving this per-formance increase. Further, we found that using an HMMresulted in the highest achieved peak performance we haveever observed for these monkeys, achieving peak bitrates of6.5, 5.7, and 4.7 bps in Monkeys J, R, and L, respectively. Fi-nally, we found that this neural prosthesis was robustly con-trollable for the duration of entire experimental sessions.These results demonstrate that high-performance discretedecoding can be beneficially combined with analog decod-ing to achieve new state-of-the-art levels of performance.

I. INTRODUCTION

INTRACORTICAL neural prostheses, also known as brain–machine interfaces, decode spiking activity in motor cortex

to drive prosthetic devices, such as computer cursors or roboticarms (e.g., [1]–[11]). The ability of these devices to restoreefficient high-performance communication is critical to theirclinical viability (e.g., [12]–[14]). In the past 15 years, sub-stantial progress has been made to improve the performance of“continuous decoders,” where analog variables, such as intendedposition and velocity, are decoded to drive a computer cursoror robotic arm (e.g., [5], [8], [11], [15]–[17]). These decodersallow the user to move the prosthesis continuously through theworkspace. For example, the user may use a continuous decoderto move a cursor on a computer screen, or make a reach with arobotic arm.

In addition to an analog component, everyday control tasksincorporate discrete actions, such as clicking an icon with acomputer mouse. A neural prosthesis that decodes both analogand discrete control signals well would therefore give the userintuitive control over the neural prosthesis. Furthermore, de-coding discrete state signals in parallel with analog continuoussignals has the potential to increase the performance of neuralprostheses. Consider the example of controlling a computer cur-sor to type on a virtual keyboard. To convey the intent to typeone key out of many potentially selectable keys requires a selec-tion mechanism. In previous studies, achieving state-of-the-artcommunication rates using only a continuous decoder (ReFIT-KF), key selections were conveyed by holding a neurally drivencursor still over the desired key for a certain amount of time(e.g., [15], [16]). However, this selection mechanism has limi-tations. First, it requires a mandatory hold time to communicateselection which decreases the overall selection rate. Second, it

0018-9294 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

936 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 64, NO. 4, APRIL 2017

is prone to inadvertent selections if the user is not actively fo-cused on controlling the cursor (e.g., as a result of being underother cognitive loads, such as contemplating what to write) or ifthe user moves the cursor too slowly and accidentally holds onan incorrect key. A potentially better selection mechanism is tohave a discrete decoder, running in parallel with the continuousdecoder, detect the user’s intent to select a key. This approachcircumvents the required hold time and would reduce the num-ber of inadvertent selections that may arise from accidentallydwelling on an incorrect key while under other cognitive loads.

While it is intuitive that having a discrete decoder operat-ing in parallel with a continuous decoder should increase theperformance and utility of a neural prosthesis, much like be-ing able to click with a computer mouse greatly increases itsutility, to date, the highest bitrates (or information throughputs)achieved by communication prostheses have been implementedwith purely continuous neural prostheses with a mandatory holdtime [15], [16]. While previous studies have incorporated dis-crete decoders, the performance of these studies have not ex-ceeded the performance of mandatory hold time systems. Aprevious study which combined a continuous decoder with abinary discrete decoder observed that on average in two humanparticipants, it took 2.5 and 6.9 s, respectively, for a discretedecoder to correctly click a desired target, and moreover, thatapproximately 45% and 65% of clicks did not occur when thecontinuous decoder was over the correct target [18]. Yet anotherstudy found that using linear discriminant analysis (LDA) to de-code a discrete control signal resulted in acquisition rates on theorder of ten characters per minute [19]. A recent study reportedthat using such discrete decoders resulted in selection rates thatwere over 2× longer than simply holding the target for 500 mswith a continuous decoder [20].

A major reason why quick and accurate discrete decodingis difficult is that neural observations are noisy on single trials.Although one solution to ameliorate noise is to integrate neuralactivity for longer amounts of time, this has the negative effectof slowing selection rate. Thus, a quick and accurate discretestate decoder must 1) integrate neural activity for only tens ofmilliseconds as opposed to hundreds of milliseconds (as in [6]and [18]); and 2) be robust in the presence of substantial noise,not making frequent spurious transitions between discretestates. Furthermore, the capability of decoding an arbitrarynumber of desired discrete states, rather than just a binary state,may also substantially increase the performance and utility ofthe discrete decoder.

We propose addressing these challenges by using hiddenMarkov models (HMMs) as a framework for discrete decod-ing (e.g., [21]–[23]). In addition to its emissions model, whichdescribes the probabilities of the neural observations given adiscrete state, the HMM incorporates a transition model. Thetransition model describes the probabilities of transitioning be-tween discrete states, providing a prior on the distribution ofdiscrete states even before the observation of neural data. In-formally, this imparts a sense of “continuity” to discrete states,mitigating the effect of observation noise and reducing spuriousdiscrete state transitions. This is in contrast to discrete decoderssuch as naı̈ve Bayes classifiers (e.g., [9], [10], [24], [25]), LDAs(e.g., [19], [26]–[28]), and support vector machines (e.g., [29],

[30]), which do not incorporate a transition model and maytherefore more often switch between states due to observationnoise. As we show in this paper, it is also possible to designarbitrary probabilistic finite-state machines that govern the tran-sitions between discrete states. These more complex transitionmodels enable the HMM to decode an arbitrary number of dis-crete states.

We note that a prior offline study used an HMM to detectneural state transitions [23], but did not combine it with a con-tinuous state decoder, as proposed here. Further, we note otheroffline studies have combined continuous and discrete decoding(e.g., [31]–[34]) where a discrete decoder modulates the con-tinuous decoder. Some studies have begun to explore these de-coders online (e.g., [35]). These studies investigated how dis-crete decoders may be used to improve continuous decoders,rather than to execute discrete commands in a communicationtask. The goal of this study is to demonstrate a high-performanceneural prosthesis for use in closed-loop systems, providing in-tuitive and accurate continuous and concurrent discrete control.To achieve high-performance discrete control, we designed andincorporated an HMM into a neural prosthesis. Specifically, weevaluated if it could be used beneficially with a state-of-the-artcontinuous decoder, the ReFIT-KF [5]. We evaluated perfor-mance with rhesus macaques in closed-loop experiments anddemonstrated that discrete selection with an HMM could in-crease communication rates of a neural prosthesis. Further, wedemonstrate that the transition model of the HMM is critical toachieving high-performance discrete state selection.

II. MATERIALS AND METHODS

A. Experimental Setup and Data Acquisition

All surgical and animal care procedures were performed inaccordance with National Institutes of Health guidelines andwere approved by the Stanford University Institutional AnimalCare and Use Committee. Experiments were conducted withthree adult male rhesus macaques (J, R, and L) implanted with 96electrode Utah arrays (Blackrock Microsystems Inc., Salt LakeCity, UT) using standard neurosurgical techniques. Electrodearrays were implanted in dorsal premotor cortex (PMd) andprimary motor cortex (M1) as visually estimated from localanatomical landmarks. Monkeys J & R had two arrays, one inM1 and one in PMd, while Monkey L had one array implantedon the M1/PMd border.

The monkeys made point-to-point reaches in a 2-D plane witha virtual cursor controlled by the contralateral arm or by a neu-ral prosthetic decoder. This task has previously been describedin prior work (e.g., [5], [15], [16], [36], [37]). The virtual cur-sor and targets were presented in a 3-D environment (MSMS,MDDF, USC, Los Angeles, CA) [38]. Hand position data weremeasured with an infrared reflective bead tracking system (Po-laris, Northern Digital, Ontario, Canada). Spike counts werecollected by applying a single negative threshold, set to −4.5×root-mean-square of the spike voltage per neural channel. Be-havioral control and neural decode were run on separate com-puters using the Simulink/xPC platform (Mathworks, Natick,MA) with communication latencies of 3 ms. This enabled mil-lisecond timing precision for all computations. Neural data were

KAO et al.: HIGH-PERFORMANCE NEURAL PROSTHESIS INCORPORATING DISCRETE STATE SELECTION WITH HIDDEN MARKOV MODELS 937

Fig. 1. Rig setup and task, adapted from [15] and [16]. (a) Diagram of monkey performing reaches in a virtual task environment. (b) Diagramof the grid task with 25 (5 × 5) array of targets. The white-dashed lines denote the acceptance windows around each target, but these were notvisible to the monkey. On each trial, one target was prompted (in green). The monkey selected a target by moving the cursor (grey) and triggering a“click” with the HMM. When the HMM triggered a click, whichever target acceptance window the cursor was in would be selected, so that any clickoccurring outside the acceptance window of the green target was an incorrect selection.

initially processed by the Cerebus recording system (BlackrockMicrosystems Inc., Salt Lake City, UT) and were available tothe behavioral control system within 5 ms ± 1 ms. Visual pre-sentation was provided via two LCD monitors with refresh ratesat 120 Hz, yielding frame updates of 7 ms ± 4 ms. Two mirrorsvisually fused the displays into a single three-dimensional per-cept for the user, creating a Wheatstone stereograph [36], but alltasks were limited to 2-D. This setup is illustrated in Fig. 1(a).

For continuous decoding, we used the ReFIT-KF decode al-gorithm [5], [15], which was used in the highest-reported com-munication rate neural prosthesis to date [16]. We trained theReFIT-KF using the center-out-and-back task according to aprotocol previously described in [5]. The observations for thedecoder were binned threshold crossings counted in nonover-lapping 10-ms bins for Monkey J, 15-ms bins for Monkey Rand 50-ms bins for Monkey L. In all experiments, the monkeyswere free to move their contralateral arm, consistent with pre-vious studies [5], [11], [15]–[17], [39]. We chose this animalmodel because we believe it most closely mimics the neuralstate of a human subject who would be employing a neuralprosthesis in a clinical study [40]. This model is limited in thatproprioceptive feedback is present in the neural activity [40],[41]. However, we favor this model over a restrained arm modelwhere the monkey would generate neural activity that presum-ably largely resides in a nullspace of cortical activity [42]. Theanimal model we employ recognizes that a human subject usinga neural prosthesis could generate neural activity that wouldhave been capable of driving muscles.

B. Grid task

We used the grid task, described in [15] and illustrated inFig. 1(b), to evaluate the performance of the neural prosthesis.In the grid task, an array of mutually exclusive targets tile a24 cm × 24 cm workspace. Every target is selectable at everypoint in time; if the monkey clicks while on a correct (incorrect)target, then a correct (incorrect) selection will have been made.We enforced a 200-ms lock-out period following target selec-tion during which no target could be selected to account for thereaction time of the monkey [15]. As any target can be selected

throughout the course of a trial, selecting one correct targetuniformly chosen from N targets conveys log2(N − 1) bits ofinformation. The factor N − 1 arises due to our assumption thatone target is reserved as a backspace key. The parameters ofthe grid task were different for each monkey and experiment.In direct performance comparisons, we chose the grid size tofavor the ReFIT-KF by using previously reported grid parame-ters that are optimal for ReFIT-KF [15]. We note that the studyby Nuyujukian et al. [15] did not include the 6 × 6 grid con-figuration, which we found led to higher bitrates than the 5 × 5grid configuration in Monkey J. Therefore, we used a 6 × 6grid for Monkey J and a 5 × 5 grid for Monkey R. In long-runperformance evaluations of the HMM, we chose the grid sizefor which the HMM achieved highest performance, which wasa 7 × 7 grid for Monkey J and a 5 × 5 grid for Monkeys Rand L. For Monkey L, we were only able to perform long-runperformance evaluations of the HMM. When we attempted di-rect performance comparisons with Monkey L, we found thathis behavior had unfortunately declined over time due to ageand other factors. The 5 × 5 grid was composed of targets withsquare acceptance windows of length 4.8 cm, the 6 × 6 grid oflength 4 cm, and the 7 × 7 grid of length 3.45 cm.

To quantify performance, we evaluated the achieved bitrateof the decoder, which has previously been described in [15].Briefly, we calculated achieved bitrate conservatively by as-suming that every incorrect selection had to be compensated bya correct selection (much like incorrectly selecting a key on akeyboard requires hitting the backspace key). If in T seconds, ccorrect selections were made, while � incorrect selections weremade on a grid with N targets, then the achieved bitrate is

I =(c − �) log2(N − 1)

T, if c ≥ � (1)

and 0 if � < c, i.e., if the monkey performs the task at or lessthan 50% success rate.

C. HMM Training

The HMM, designed for a closed-loop neural prosthesis, isthe major contribution of this report. As described in Section I,we chose the HMM as a potential discrete decoder that would be


Fig. 2. Graphical representation of the HMM and transition models. (a) Graphical model of the HMM, where a discrete random variable, ck ,evolves over time through a transition matrix. The output of the HMM at each point is the projected neural activity, sk . (b) Transition model used inonline experiments for Monkey J and R incorporated two states, “move” and “stop.” (c) Transition model used in online experiments for Monkey Lincorporated an “idle” state and partitioned the movement phase into a sequence of “slow” to “fast” to “slow” transitions.

able to achieve high-performance discrete state decoding by 1)integrating neural activity over short periods of time, specificallytens of milliseconds, and 2) being robust in the presence of noisysingle-trial neural activity so as to not making frequent spurioustransitions between discrete states.

A graphical representation of the HMM is shown in Fig. 2(a).Neural observations at time k, denoted by sk , are used to infer thedistribution of a probabilistic discrete state variable, ck , whichevolves over time. We denote the collection of discrete statesas C. The distribution of ck , which we also call the probabilityvector, describes the probability of being in each discrete state inC. The neural observation sk is a 5-D projection of the observedneural binned spike counts found via principal component anal-ysis (PCA) as in previous reports (e.g., [43], [44]). As optimalparameters from offline simulations may differ from those inclosed-loop control (e.g., [36], [45], [46]), we performed a cur-sory optimization of performance as a function of the numberof PCs. We determined that using the top five PCs led to ade-quate control, although it is possible that a different number ofPCs may result in even higher performance. The HMM uses thedistribution on ck−1 , and the current neural observation, sk , tocompute the distribution of ck . This is graphically illustrated byFig. 2(a). This computation involves two components.

The first component is a time-invariant probability transi-tion model describing the probability of transitioning betweenstates, i.e., pc,c ′ = Pr(ck = c|ck−1 = c′) for all discrete statesc, c′ ∈ C. This transition model gives a prior estimate of thediscrete state probability vector in the absence of observations.For Monkeys J and R, we designed a straightforward transitionmodel comprising two states, “move” and “stop,” as shown inFig. 2(b). For Monkey L, we designed a more complex modelcomprising five states, as shown in Fig. 2(c).

The second component is a Gaussian emissions model whichapproximates the distribution of the neural observations sk whenin state c. The emissions model is parameterized by the meanand covariance matrix of the neural observations in each state,i.e., (sk |ck = c) ∼ N (μc,Σc). The transition model and theemissions model are combined to estimate the probability ofbeing in each state.

The parameters of the HMM were learned in a supervisedfashion from experimental training data during which each mon-key controlled a virtual cursor to move and hold over targets for

500 ms. In the direct performance comparisons for MonkeysJ and R, the training set virtual cursor was controlled by themonkey’s hand. In the long-run performance evaluations, thetraining set virtual cursor was controlled by a neural prosthesisfor Monkeys J and L. We found that the HMM was capable ofachieving high performance in both scenarios.

We assigned every time bin during the training set as being ina discrete state, c ∈ C. We defined the “stop” state as the periodof time 250 ms (133 ms) into the hold epoch for hand trainingsets (neural prosthesis training sets) until 100 ms after holdcompletion. We chose these boundaries to both account for thereaction time of the monkey and cursorily optimize closed-loopperformance. It is possible that these boundaries could befurther optimized to achieve higher performance. For MonkeysJ and R, the rest of the trial was classified as the “move” state,resulting in the transition model shown in Fig. 2(b). For MonkeyL, we split the “move” state into “slow” and “fast” states basedon a speed threshold. For any given trial, times when the cursorspeed was below 25% of the maximum cursor speed of that trialwere assigned to be in the “slow” state, while times when cursorspeed exceeded 25% of the maximum speed were assigned to bein the “fast” state. We found that this design increased closed-loop performance in Monkey L by preventing early transitionsinto the “stop” state. Behaviorally Monkey L was also prone toidle during the task due to inconsistent motivation, and so we in-corporated an “idle” state corresponding to when Monkey L satidly. The transition model for Monkey L is shown in Fig. 2(c).

After binning the neural activity and state sequence per trial,we learned the transition matrix and emissions process for theHMM. The transition matrix was learned by calculating the pro-portion of transitions between potential states in the training set.For example, in the transition model of Fig. 2(b), we calculatedthe proportion of transitions from “move” to “move,” “move”to “stop,” “stop” to “move,” and “stop” to “stop.” These valuescomprised the transition matrix. For the idle state in MonkeyL, we chose the transition probability into the “idle” state tobe approximately 10% of the probability of transitioning out ofthe state. To learn the emissions model, we aggregated all binsof projected neural activity corresponding to a certain discretestate. We then computed the empirical mean and covariance ofthe projected neural activity, which were treated as the emis-sions mean and emissions covariance for that discrete state. We


note that in Monkey L, the states “slow1” and “slow2” (shownin Fig. 2(c)) had the same emissions model.

To decode the probability of each discrete state at time k, weused the forward algorithm (e.g., [21], [22]). We kept track ofthe probability αk (c) = p(s1 , s2 , . . . , sk , ck = c), which is thejoint probability of observing all neural observations and thediscrete state value of ck = c. Calculating αk (c) for all c ∈ Cyields our probability vector over the discrete states at time k.Using the chain rule for probability and the graph structure ofthe HMM shown in Fig. 2(a), we can derive a forward recursionfor αk (c) as

αk (c) ∝ f(sk |ck = c)∑

c ′∈Cαk−1(c′)pc,c ′ . (2)

Here, f(sk |ck = c) is the Gaussian density over the neuralobservations. The α’s were renormalized at every time step.We note that using the forward algorithm is in contrast to aprior closed-loop study that incorporated a transition modelinto discrete state selection. Specifically, in the closed-loopexperiments of [6] and [18], the discrete decoder assumed thatck was deterministically in the most likely state in C. Thisapproach does not propagate the probability vector forward intime and loses state distribution information (i.e., how likelyeach state was) at each time step.

D. Offline Simulations

In all reported offline results, we decoded neural datarecorded from datasets where each monkey performed reacheson a center-out-and-back task with eight peripheral targetsspaced 8 cm from the center target [5], [11]. During thesereaches, we tagged each bin as either in the “move” or the“stop” state as previously described in HMM training. Whendecoding discrete state, bins were classified as “stop” if theprobability of being in the “stop” state exceeded a threshold,tstop , and in the “move” state otherwise. (We note that foroffline analyses only, we used the simple two-state transitionmodel shown in Fig. 2(b) for Monkey L.) When offline decodeerror is reported, it is the proportion of incorrect classificationsat bin resolution. Finally, all offline performance analyses wereperformed on withheld testing data.

E. Closed-Loop Experiments

In closed-loop experiments, the monkeys controlled theReFIT-KF and HMM in parallel. The ReFIT-KF controlled thevelocity of the cursor, while the HMM indicated whether or notto select a target. We call this decoder the “ReFIT-KF + HMM.”To be conservative in detecting the selection of a target (whichwe call a “click”), we enforced that a click only occur after theprobability of being in the “stop” state exceeded tstop = 0.8 forat least two consecutive time bins. Following a click, we endedthe trial and initiated a new trial where, after a 40-ms pause, anew target was presented for the monkey to acquire. Consistentwith how a computer mouse, after receiving an input such as afinger press to indicate a click, will reset to the unclicked state inthe absence of continuous input, we set the probability of beingin the “move” (“slow1”) state for Monkeys J and R (MonkeyL) to be 1 following any target selection. In this fashion, the

algorithm performs the analogous and automatic “unclick” ofa computer mouse. We note that indicating a continuous hold(e.g., as one might hold their finger down continuously on acomputer mouse) may be encoded as a separate state in theHMM and may be explored in future work.

When comparing the performance of the ReFIT-KF + HMMto the ReFIT-KF with a hold time selection mechanism, weused a hold time of 450 ms, as found from previous optimiza-tions [15]. This hold time is chosen with the intent to maximizethe performance of the ReFIT-KF. We note that, as reportedby Nuyujukian et al. [15], a shorter hold time that allows forquicker selections ultimately results in a lower achieved bitratedue to an increase in incorrect selections (e.g., inadvertentlydwelling on an incorrect target en route to the desired target).

We performed three closed-loop experiments. The first twowere performed in Monkeys J and R only, as Monkey L’sbehavior had unfortunately declined over time due to aging andother factors. The third experiment was performed by MonkeysJ, R, and L.

In the first experiment, we sought to compare the perfor-mance of the ReFIT-KF to the ReFIT-KF + HMM on the gridtask. Within an experimental day, the monkey controlled theReFIT-KF for 200 trials, followed by the ReFIT-KF + HMM for200 trials. The 200 trials were then used to compute a bitrate foreach decoder. These decoders were repeatedly tested after eachother in an A-B-A-B-A-... fashion, yielding repeated within-day measurements of ReFIT-KF performance and ReFIT-KF +HMM performance. We call one A-B segment (i.e., 200 trialsof ReFIT-KF followed by 200 trials of ReFIT-KF + HMM) anexperimental block. We always evaluated the ReFIT-KF first inthe block, so that any benefits from the HMM were not a resultof degrading motivation. Bitrates were paired in each block forstatistical testing. We also note that the parameters of the gridwere chosen to maximize the performance of the ReFIT-KF [15]and not the ReFIT-KF + HMM; as shown in the third experi-ment, it was possible to choose the grid density to achieve evenhigher performance with the ReFIT-KF + HMM.

In the second experiment, we sought to determine the impor-tance of using a transition model in discrete state selection. Webuilt a discrete decoder, termed the quadratic discriminant (QD),which performs classification using only the emissions processof the HMM. Therefore, the only difference between the QDand the HMM is that the HMM incorporates a transition model.The two decoders have the same parameters relating the neuralobservations to the discrete state. If the performance of the twodecoders are not significantly different, this indicates that theHMM achieves good performance due to its emissions processrather than the transition model; however, if the HMM performssignificantly better than the QD, this indicates that the transitionmodel is crucial to achieving high-performance discrete stateclassification. We compared the performance of the ReFIT-KF+ QD to the ReFIT-KF + HMM on the grid task in the samefashion as the first experiment, conducting within-day compar-isons where each decoder was evaluated in an experimentalblock for 200 trials to measure an achieved bitrate.

In the third experiment, we allowed the ReFIT-KF + HMMto be controlled for hours-long experimental sessions, demon-strating that the HMM is robust across an experimental session.


Fig. 3. Low-dimensional projections of neural activity. (a) Neural activity is projected into PC1 and PC2 for Monkey J. Each dot denotes theprojected neural activity in a bin. Orange corresponds to neural activity while in the “stop” state, while blue corresponds to neural activity while inthe “move” state. Histograms of the activity in each PC are also shown. We note that the PCs are not zero-centered since we applied the projectormatrix to the non-mean centered neural activity. This was to reduce the number of parameters in the model, as the emissions model incorporatesthe mean of the neural activity. (b) Same as (a) but for Monkey R. (c) Same as (a) but for PCs 3 and 4 in Monkey J. (d) Same as (a) but for PCs 3and 4 in Monkey R.

III. RESULTS

The results are organized into three sections. First, we presentneural data and offline simulations that characterize the HMM.Second, we present the results of closed-loop experimentsdemonstrating that the HMM can increase neural prostheticperformance and performs better than an equivalent classifierwith no transition model. Third, we present results where themonkeys controlled the ReFIT-KF + HMM for entire experi-mental sessions, demonstrating that high performance can besustained.

A. Offline Decode Using PCs of the Neural Activity

The emissions process of the HMM models a multivariatedistribution on the neural activity. To avoid the “curse of di-mensionality” [21], we reduced the dimensionality of the neuralactivity with PCA, especially as the dimensionality of motorcortex during simple reaching tasks is on the order of 10–20(e.g., [47], [48]). We found that a substantial proportion of vari-ance related to the “move” state versus the “stop” state wascaptured by the leading PCs of the neural activity. Fig. 3(a)–(d)shows a scatter plot of neural activity during reaching whenprojected onto the first four PCs of the neural activity for Mon-keys J and R (for Monkey L, see Supplementary Fig. 1(a) and

(b)). We note that even in PC 1, the distributions of the “move”and “stop” neural activity, though highly overlapping, are dis-tinguishable. As shown in Fig. 4(a) and (b) and SupplementaryFig. 1(c), we observed that increasing the number of PCs in-creased offline performance until about five to ten PCs wereused (Monkeys J and L) or did not significantly increase offlineperformance (Monkey R). This suggests that variance related todiscriminating “move” versus “stop” states is largely containedwithin the top ten PCs of the data.

We note that although the distributions of neural activity dur-ing the “move” and “stop” states are distinguishable, they havea high degree of overlap. This high degree of overlap may bea major reason why achieving high-performance discrete de-coding is not straightforward, as discriminating only based onthe likelihood of the noisy observations may be unreliable. Wetested this idea by comparing the performance of the HMM tothe performance of the QD, which is a classifier using only theemissions process of the HMM.

We evaluated the extent to which the transition model of theHMM can help increase discrete state selection accuracy. For5, 11, and 4 experimental days in Monkeys J, R, and L, respec-tively, we performed an offline decode of 200 cross-validationtrials per experimental day using the HMM and QD decoders.Across all of these trials, we obtained a distribution of discretestate transition times per trial, as well as a daily classification


Fig. 4. Offline decode using the HMM. (a) Offline decode error for Monkey J, measured as the proportion of incorrectly predicted states across alltime bins. We note that after five to ten PCs, performance did not increase with more PCs, indicating that a majority of useful variance associatedwith differentiating between “move” and “stop” is captured by the top PCs of the neural activity. Shading denotes the standard error of the mean. (b)Same as (a) but for Monkey R. Monkey R’s results suggest that a majority of the useful variance for decoding “move” versus “stop” is contained inPC 1. (c) Offline decode in Monkey J for reaching trials. The gray line represents whether the monkey was in the “move” or “stop” state. The redmarkers denote the decoded probability of being in the “stop” state per bin using the HMM. The purple marker denotes the probability of being inthe “stop” state per bin using the QD classifier. (d) Same as (c) but for Monkey R.

error rate for each decoder. We found that the HMM decodeswere more accurate than the QD decodes, as shown in Fig. 4(c)and (d) and Supplementary Fig. 1(d), better tracking the truestate. The decode error (percentage of bins incorrectly clas-sified) for the HMM was 24%, 20%, and 19% for MonkeysJ, R, and L, respectively, while for the QD, the errors were46%, 24%, and 21%. Therefore, the HMM decoder more accu-rately decoded the discrete state (p < 0.01 for Monkeys J andR, p = 0.011 for Monkey L, paired t-test on each experimentalday’s classification error). The time series of the offline decodein Fig. 4(c) and (d) and Supplementary Fig. 1(d) suggest that thetransition model effectively denoises ambiguous neural activitywhose distributions are not strong enough to cause transitions inthe discrete state. This is supported by our finding that, in gen-eral, the transition probabilities between states were small, i.e.,pmove,stop < 0.05 and pstop,move < 0.05, indicating that therewas strong inertia to remain in the same state.

We also evaluated the time it took to correctly transitionfrom the “move” to “stop” state to determine if the HMMcould transition as quickly as an QD decoder. Importantly, thistransition time will be a function of the probability threshold,tstop , used to detect the “stop” state, as shown in the offlinesimulations of Supplementary Fig. 3. At lower thresholds, theQD transitions more quickly than the HMM, as may be ex-pected due to the effective “momentum” imparted by the HMMtransition model. However, offline decode performance is alsopoorer at lower thresholds. Oppositely, we found that at highenough thresholds, the HMM transitions more quickly than theQD because the transition model enables the HMM to morequickly achieve high confidence in a certain state. In particular,

we found that at the probability threshold used for closed-loopexperiments (tstop = 0.8), the HMM transitioned 159 and 23 msmore quickly than the QD model in Monkeys J and L (p < 0.01,two-sample t-test across transition times for every trial acrossall experimental days), while for Monkey R, the transition timeswere not significantly different (p = 0.81). These results sug-gest that at thresholds useful for closed-loop control, the HMMtransition model does not slow transition time.

B. Performance Comparisons of ReFIT-KF andReFIT-KF + QD Versus ReFIT-KF + HMM

Quantifying neural prosthetic performance in closed-loop ex-periments is crucial for assessing clinical viability and utility,as closed-loop results may differ from offline simulation [36],[45]. To evaluate the utility and performance of the HMM ina closed-loop neural prosthesis system, we performed experi-ments where monkeys controlled the HMM in parallel with acontinuous decoder. In this section, we present two closed-loopexperiments. First, to test if incorporating the HMM into a neu-ral prosthesis could improve state-of-the-art performance, wecompared the performance of the ReFIT-KF using a mandatoryhold selection mechanism to the ReFIT-KF + HMM. Second,to test the importance of the transition model of the HMM,we compared the performance of the ReFIT-KF + QD to theReFIT-KF + HMM.

We found that the performance of the ReFIT-KF + HMMwas significantly higher than that of the ReFIT-KF usinga mandatory hold selection mechanism. We evaluated theperformance of the ReFIT-KF and ReFIT-KF + HMM in ablocked fashion (see Methods). We repeated these experiments


Fig. 5. Direct comparison of ReFIT-KF and ReFIT-KF + QD versusReFIT-KF + HMM. (a) In Monkey J, we compared the performance of theReFIT-KF (using a mandatory hold selection mechanism, blue) versusthe ReFIT-KF + HMM (red) in within-day blocks. Error bars denote thestandard error of the mean. The ReFIT-KF + HMM significantly achievedhigher performance than the ReFIT-KF alone. Datasets J_2015-12-03,J_2015-12-04, and J_2015-12-07 comprising 7075 total trials and 17comparison blocks. (b) In Monkey J, we compared the performance ofthe ReFIT-KF + HMM vs the ReFIT-KF + QD. The ReFIT-KF + HMMsignificantly achieved higher performance than the ReFIT-KF + QD, in-dicating that the transition model of the HMM is important for achievinghigh-performance discrete state selection. Datasets J_2015-12-08 andJ_2015-12-09 comprising 4803 total trials and 12 comparison blocks.(c) Same as (a) but for Monkey R. Datasets R_2015-12-14, R_2015-12-15, R_2015-12-16, and R_2015-12-17 comprising 6117 total trialsand 15 comparison blocks. (d) Same as (b) but for Monkey R. DatasetsR_2015-12-16 and R_2015-12-17 comprising 4152 total trials and tencomparison blocks.

across three experimental days in Monkey J (comprising 7075trials) and four experimental days in Monkey R (comprising6117 trials). We found that the ReFIT-KF + HMM increasedthe achieved bitrate of the ReFIT-KF by 13.9% and 4.2% inMonkeys J and R (p < 0.01, paired t-test of bitrates of all ex-perimental blocks; also significant under Wilcoxon signed-ranktest), as shown in Fig. 5(a) and (c). These results demonstratethat incorporating an HMM into a state-of-the-art decoder cansignificantly increase neural prosthetic performance, providingintuitive, fast, and accurate discrete state selection.

We also sought to understand the importance of the Marko-vian transition model of the HMM in achieving this perfor-mance increase. To this end, we evaluated the performance ofthe ReFIT-KF + QD, which is a quadratic discriminator usingthe emissions process of the HMM but crucially has no transitionmodel. This performance comparison would therefore quantifythe benefit of incorporating a transition model into a discretestate classifier. We performed experiments in the same blockedfashion as for the ReFIT-KF versus ReFIT-KF + HMM experi-ment (see Methods). We repeated these experiments across twoexperimental days in Monkey J (comprising 4803 trials) and twoexperimental days in Monkey R (comprising 4152 trials). Wefound that the ReFIT-KF + HMM achieved a higher bitrate thanthe ReFIT-KF + QD by 14.8% and 16.5% in Monkeys J and R(p < 0.01, paired t-test of bitrates of all experimental blocks;also significant under Wilcoxon signed-rank test), as shown inFig. 5(b) and (d). These results demonstrate that the transitionmodel of the HMM is crucial to achieving high performance.Indeed, the ReFIT-KF + QD achieved mean bitrates that were

either comparable to (Monkey J) or worse (Monkey R) thansimply using a mandatory hold time selection. This result isconsistent with a report that, in humans, discrete state classifi-cation with an LDA (having no transition model) was slowerthan selection than with a mandatory hold time [20].

Together, these results indicate that an HMM provides fastand accurate discrete state control, and can be used to im-prove state-of-the-art continuous decoders. Further, these re-sults demonstrate that the HMM transition model is a crucialcomponent for achieving this performance improvement.

C. Long-run Performance of a ReFIT-KF + HMM

We further evaluated if the monkeys could control the HMMfor entire experimental sessions. Across five experimental daysin Monkey J (comprising 18 825 total trials), five experimentaldays in Monkey R (comprising 29 406 trials) and four experi-mental days in Monkey L (comprising 18 791 total trials), weevaluated the long-run performance of the ReFIT-KF + HMM.As shown in Fig. 6 and Supplementary Fig. 2, we found thatMonkeys J, R, and L were able to control the ReFIT-KF +HMM for entire experimental sessions. The ReFIT-KF + HMMwas capable of being controlled at state-of-the-art levels of per-formance for the duration of the task, although performancesometimes declined over the course of the session. As furtherdiscussed below, this reduction in performance is potentiallydue to a decline in behavior resulting from fatigue. In additionto this, the monkeys performed the task until they lost motiva-tion, reflected by a sharp drop in performance at the end of theexperimental session. This sharp drop in performance is alsoobserved when the monkey controls a cursor with his arm orwith the ReFIT-KF alone [15] and does not reflect a drop inperformance of the HMM.

During these sessions, we measured the peak achieved bitrateof the ReFIT-KF + HMM. We defined the peak bitrate as themaximum achieved bitrate that was sustainable for 60 s. Thisnumber is not representative of the performance of the decoderon average, but characterizes the upper limits of decoder perfor-mance. A decoder that is able to achieve a higher peak bitratehas the capacity to achieve higher communication rates.

For Monkeys J, R, and L, we found that the ReFIT-KF + HMMachieved peak bitrates of 6.49, 5.71, and 4.74 bps, respectively.Across years-worth of experimental sessions we performed withMonkeys J, R, and L, we never observed the ReFIT-KF achieve ahigher peak bitrate than ReFIT-KF + HMM. Specifically, across899 024 trials in Monkey J (spanning 298 experimental ses-sions across 5.2 years), 26 449 trials in Monkey R (spanning22 experimental sessions across 3.3 years), and 386 563 tri-als in Monkey L (spanning 158 experimental sessions across5.3 years), the highest bitrates achieved by the ReFIT-KF (or aslightly modified variant of the ReFIT-KF) were 5.27, 4.43, and4.45 bps. Thus, the peak bitrate achieved by ReFIT-KF + HMMin the course of a few experimental sessions exceeds the highestachieved peak performance of the ReFIT-KF across years ofexperiments.

We also note that, toward the end of long-run experiments,the performance of the ReFIT-KF + HMM tended to degrade.


Fig. 6. Long-run performance of the ReFIT-KF + HMM. (a) Performance of the ReFIT-KF + HMM over entire experimental sessions for Monkey J.Each trace corresponds to one experimental day. The monkey sustained performance until he lost interest in the task, a behavior also observed whenthe monkey reaches with his native arm [15]. Datasets J_2012-03-26, J_2012-03-27, J_2012-03-30, J_2012-04-05, and J_2012-04-06 comprising18 825 total trials. (b) Same as (a) but for Monkey R. Datasets R_2015-12-20, R_2016-01-04, R_2016-01-05, R_2016-01-06, and R_2016-01-07comprising 29 406 trials.

This was likely due to fatigue, as the monkey had to sustainhigher selection rates and thus, higher average neural prosthesisvelocities than with a ReFIT-KF having a mandatory hold time.Toward the end of the session, the monkey did not as reliably“dial-in” to the target, which led to incorrect clicks on adja-cent targets. We did not observe such levels of degradation inperformance in the direct within-day comparisons, where themonkeys were allowed brief pauses, while the decoders werechanged. Hence, the average performance of long-run sessionsmay be confounded by fatigue and, as such, do not adequatelyreflect the average performance of the ReFIT-KF + HMM dur-ing normal patient use, where a patient would be capable oftaking breaks as desired.

Supplementary Movies 1, 2, and 3 demonstrate near peakperformance of the ReFIT-KF + HMM decoder for MonkeysJ, R, and L respectively, while Supplementary Movies 4, 5,and 6 demonstrate performance approximately an hour into theexperimental session.

IV. DISCUSSION

Many communications and motor tasks incorporate discretestate selection (e.g., clicking an icon with a computer cursor).Achieving high-performance discrete state control therefore hasthe potential to considerably increase both the ease-of-use andperformance of neural prostheses. Here, we demonstrated thatdiscrete decoding using an HMM could increase the perfor-mance of a neural prosthesis on a virtual keyboard communica-tion task [49]. This study represents the highest reported peakand average bitrate achieved on a communication task of anyneural prosthesis under any recording modality in non-humanprimates. Further, we demonstrated that the HMM is robust andcan be used for entire experimental sessions. Together, theseresults demonstrate that high-performance parallel continuousand discrete decoding is possible. This should enable the high-performance use of intuitive devices incorporating both analogand discrete components, such as a computer mouse.

An important component in improving the speed and accuracyof discrete decoding is the use of a transition model. Althoughneural activity is very noisy on single trials at relatively smallbin widths, the incorporation of a transition model increased

decoding accuracy in both offline (see Fig. 4(c) and (d) andSupplementary Fig. 1(d)) and online comparisons (see Fig. 5(b)and (d)). This is likely because the transition model, whichuses prior information about the frequency of state transitions,has a tendency to stay in the same state until strong evidencefrom the emissions process causes a state transition. A potentialdrawback of using a transition model, however, is that it mayprovide too much “inertia” to the discrete state, causing discretetransitions to occur too slowly. We observed that this was not thecase for appropriate thresholds, where correct state transitionsin an HMM were as quick, if not quicker than state transitionsin a QD.

We also found that the transition model could be modifiedto accommodate differing behaviors or to potentially increaseperformance. Importantly, in the HMM, the transition modelcould be flexibly designed to incorporate an arbitrary number ofdiscrete states and their transition rules. In Monkey L, who hadpoorer quality arrays and motivation, we were able to success-fully decode an “idle” state when the monkey was not engaged inperforming the task (see Fig. 2(c)). Further, we employed a tran-sition model that required the monkey to transition through threephases of a reach, move “slow” to move “fast” and then againto move “slow” before entering the “stop” state. As HMMs areflexible in design, it is possible that our results could be furtherimproved by optimizing the discrete state transition model.

We note that a previous online study [18] did also usea transition model, which took into account the probabilityPr(ck = c|ck−1 = c′). However, this study did not propagatethe probability vector forward in time. That is, at each time pointk, they chose ck to be deterministically in one state. Therefore,in calculating the distribution of ck+1 , the only information usedfrom the previous time step is the most likely state of ck ratherthan the entire distribution of ck . We note that, in human clinicaltrials, this approach to discrete decoding was at least two timesslower than selecting a target with a mandatory hold time [20].In our study, we decoded with the forward algorithm whichuses the distribution of ck to estimate the distribution of ck+1 .We believe this use of a transition model is a crucial step to-ward achieving high-performance discrete decoding. Our studydemonstrates, for the first time, an improvement in closed-loop


performance by using discrete state selection instead of select-ing targets with a mandatory hold period.

While we decoded discrete states corresponding to distincttask actions (i.e., “move” and “stop”) we note that decodingdiscrete states may also extend beyond task actions. It may bepossible to further increase performance by allowing discreteand continuous decoders to interact. For example, decoding dis-crete states may be used to modify a continuous decoder (e.g.,[31]). In a simple example, one might have two continuous de-coders, one used to make “ballistic” movements and the otherused to make “fine” movements. The continuous decoder under“ballistic” control may be optimized for making quick move-ments across the workspace, while the continuous decoder under“fine” control would be optimized for making small and precisemovements. A discrete decoder would determine the probabil-ities of being in the “ballistic” or “fine” control states, whichcould be used to choose (or mix) the corresponding continu-ous decoders. In this manner, the decoder would no longer betime-invariant, but would instead be a piecewise compositionof decoders. The resulting nonlinear decoder, whose parameterschange based on the decoded discrete state distribution, may beable to improve control during different task epochs. Existingoffline simulations along these lines suggest that this may leadto increased neural prosthetic performance (e.g., [31]–[33]).Future work may explore the extent to which such interactionsmay increase closed-loop neural prosthetic performance.

We observed that neural activity was very noisy on single tri-als so that the distribution of neural activity during the “move”and “stop” state were highly overlapping. In light of this, thereare at least two possible methods by which HMM performancemight be further improved. First, in our study, we decoded the“stop” state by modeling neural activity when the monkey washolding a target. Therefore, our “stop” state corresponded to theintention to hold the cursor still over a target. However, in clini-cal trials with human participants, other cues, such as “squeeze”or “hand open” might be used to signal a “click” [18]. If theseimagined movements have modulation that is distinct from con-trolling the cursor, performance may substantially improve be-cause the neural activity would be more separable. Second, itmight be possible to incorporate neural dynamical estimationto more robustly distinguish “move” and “stop” activity. A re-cent report demonstrates that when modeling the dynamics ofthe neural population activity, the neural activity during the“stop” state is closer to a fixed point than the “move” state inneural state space [11]. These dynamics accentuate the differ-ences in neural activity between the “move” and “stop” state,effectively denoising the observations. We predict that incor-porating this technique into an HMM would improve decodingperformance.

V. CONCLUSION

As both analog and discrete actions are a part of everydaymotor control tasks, it is important that neural prostheses becapable of decoding both signals at high levels of performance.We report that an HMM discrete decoder can significantly in-crease neural prosthetic performance by running in parallel toa continuous decoder. Our results suggest that existing neu-ral prostheses using only continuous decoders can be further

improved by incorporating a parallel HMM decoder. These ad-vances are important for further increasing the utility, usability,and clinical viability of intracortical neural prostheses.

ACKNOWLEDGMENT

The authors would like to thank M. Risch, J. Aguayo, C. Sher-man, and E. Morgan for surgical assistance and expert veterinarycare; S. Eisensee, E. Castenada, and B. Davis for administrativesupport; and B. Oskotsky for information technology support.

Author contributions: J. C. Kao and P. Nuyujukian wereresponsible for designing and conducting experiments, algo-rithm development and data analysis. J. C. Kao was responsiblefor manuscript writeup. P. Nuyujukian assisted in manuscriptreview. S. I. Ryu was responsible for surgical implantationand assisted in manuscript review. K. V. Shenoy was involvedin all aspects of experimentation, data review, and manuscriptwriteup.

REFERENCES

[1] D. M. Taylor et al., “Direct cortical control of 3D neuroprosthetic devices,”Science, vol. 296, no. 5574, pp. 1829–1832, Jun 2002.

[2] J. M. Carmena et al., “Learning to control a brain-machine interface forreaching and grasping by primates,” PLoS Biol., vol. 1, no. 2, p. E42, Nov.2003.

[3] M. Velliste et al., “Cortical control of a prosthetic arm for self-feeding,”Nature, vol. 453, no. 7198, pp. 1098–1101, Jun 2008.

[4] S.-P. Kim et al., “Neural control of computer cursor velocity by decodingmotor cortical spiking activity in humans with tetraplegia,” J. Neural Eng.,vol. 5, no. 4, pp. 455–476, Dec. 2008.

[5] V. Gilja et al., “A high-performance neural prosthesis enabled by controlalgorithm design,” Nature Neuroscience, vol. 15, no. 12, pp. 1752–1757,Nov. 2012.

[6] L. R. Hochberg et al., “Reach and grasp by people with tetraplegia using aneurally controlled robotic arm,” Nature, vol. 485, no. 7398, pp. 372–375,May 2012.

[7] J. L. Collinger et al., “High-performance neuroprosthetic control by anindividual with tetraplegia,” Lancet, vol. 381, no. 9866, pp. 557–564, Feb2013.

[8] B. Wodlinger et al., “Ten-dimensional anthropomorphic arm control in ahuman brain-machine interface: Difficulties, solutions, and limitations,”J. Neural Eng., vol. 12, no. 1, p. 016011, Dec 2014.

[9] S. Musallam, “Cognitive control signals for neural prosthetics,” Science,vol. 305, no. 5681, pp. 258–262, 2004.

[10] G. Santhanam et al., “A high-performance brain-computer interface,” Na-ture, vol. 442, pp. 195–198, Jul. 2006.

[11] J. C. Kao et al., “Single-trial dynamics of motor cortex and their appli-cations to brain-machine interfaces,” Nature Commun., vol. 6, pp. 1–12,May 2015.

[12] S. I. Ryu and K. V. Shenoy, “Human cortical prostheses: Lost in transla-tion?” Neurosurgical Focus, vol. 27, p. E5, 2009.

[13] V. Gilja et al., “Challenges and opportunities for next-generation intracor-tically based neural prostheses,” IEEE Trans. Biomed. Eng., vol. 58, no. 7,pp. 1891–1899, Jul. 2011.

[14] J. C. Kao et al., “Information systems opportunities in brain-machineinterface decoders,” Proc. IEEE, vol. 102, no. 5, pp. 666–682, May 2014.

[15] P. Nuyujukian et al., “A high-performance keyboard neural prosthesisenabled by task optimization,” IEEE Trans. Biomed. Eng., vol. 62, no. 1,pp. 21–29, Jan. 2015.

[16] P. Nuyujukian et al., “Performance sustaining intracortical neural pros-theses,” J. Neural Eng., vol. 11, no. 6, p. 066003, 2014.

[17] D. Sussillo et al., “A recurrent neural network for closed-loop intracorticalbrain-machine interface decoders,” J. Neural Eng., vol. 9, no. 2, p. 026027,Apr 2012.

[18] S.-P. Kim et al., “Point-and-click cursor control with an intracortical neuralinterface system by humans with tetraplegia,” IEEE Trans. Neural Syst.Rehabil. Eng., vol. 19, no. 2, pp. 193–203, Apr. 2011.

[19] D. Bacher et al., “Neural point-and-click communication by a personwith incomplete locked-in syndrome,” Neurorehabilitation Neural Repair,vol. 29, pp. 462–471, 2014.


[20] V. Gilja et al., “Clinical translation of a high-performance neural prosthe-sis,” Nature Med., vol. 21, no. 10, pp. 1142–1145, 2015.

[21] C. M. Bishop, Pattern Recognition and Machine Learning. New York,NY, USA: Springer, 2006.

[22] D. Koller and N. Friedman, Probabilistic graphical models, Cambridge,UK: MIT Press, 2009.

[23] C. Kemere, “Detecting neural-state transitions using hidden Markov mod-els for motor cortical prostheses,” J. Neurophysiol., vol. 100, pp. 2441–2452, 2008.

[24] K. V. Shenoy et al., “Neural prosthetic control signals from plan activity,”NeuroReport, vol. 14, no. 4, pp. 591–596, 2003.

[25] N. Achtman et al., “Free-paced high-performance brain–computer inter-faces,” J. Neural Eng., vol. 4, no. 3, pp. 336–347, 2007.

[26] K. L. Brigmann, “Optical imaging of neuronal populations duringdecision-making,” Science, vol. 307, pp. 896–901, 2005.

[27] D. Durstewitz et al., “Abrupt transitions between prefrontal neural ensem-ble states accompany behavioral transitions during rule learning,” Neuron,vol. 66, no. 3, pp. 438–448, 2010.

[28] V. Aggarwal et al., “State-based decoding of hand and finger kinematicsusing neuronal ensemble and LFP activity during dexterous reach-to-graspmovements,” J. Neurophysiol., vol. 109, no. 12, pp. 3067–3081, 2013.

[29] M. T. Kaufman et al., “Vacillation, indecision and hesitation in moment-by-moment decoding of monkey motor cortex,” eLife, vol. 4, pp. 1–21,2015.

[30] E. Stark and M. Abeles, “Predicting movement from multiunit activity,”J. Neurosci., vol. 27, no. 31, pp. 8387–8394, Aug. 2007.

[31] B. M. Yu et al., “Mixture of trajectory models for neural decoding of goal-directed movements,” J. Neurophysiology, vol. 97, no. 5, pp. 3763–3780,May 2007.

[32] X. Kang, “Decoding of finger, hand and arm kinematics using switchinglinear dynamical systems with pre-motor cortical ensembles,” in Proc.34th Annu. Int. Conf. IEEE EMBS, 2012, pp. 1732–1735.

[33] W. Wu et al., “A switching Kalman filter model for the motor corticalcoding of hand motion,” in Proc. 25th Annu. Int. Conf. IEEE EMBS, 2003,pp. 2083–2086.

[34] L. Srinivasan et al., “General-purpose filter design for neural prostheticdevices,” J. Neurophysiol., vol. 98, no. 4, pp. 2456–2475, Aug. 2007.

[35] M. M. Shanechi, “A real-time brain-machine interface combining motortarget and trajectory intent using an optimal feedback control design,”PLoS ONE, vol. 8, no. 4, pp. 23–32, 2013.

[36] J. P. Cunningham, “A closed-loop human simulator for investigating therole of feedback control in brain-machine interfaces,” J. Neurophysiol.,vol. 105, pp. 1932–1949, 2011.

[37] J. M. Fan, “Intention estimation in brain machine interfaces,” J. Neuroeng.,vol. 11, no. 1, p. 016004, 2014.

[38] R. Davoodi and G. E. Loeb, “Real-time animation software for customizedtraining to use motor prosthetic systems,” IEEE Trans. Neural Syst. Reha-bil. Eng., vol. 20, no. 2, pp. 134–142, Mar. 2012.

[39] S. D. Stavisky et al., “A high performing brainmachine interface drivenby low-frequency local field potentials alone and together with spikes,” J.Neural Eng., vol. 12, no. 3, p. 036009, 2015.

[40] P. Nuyujukian et al., “Monkey models for brain-machine interfaces: Theneed for maintaining diversity,” in Proc. 33rd Annu. Conf. IEEE EMBS,Jan 2011, vol. 2011, pp. 1301–1305.

[41] A. J. Suminski et al., “Incorporating feedback from multiple sen-sory modalities enhances brain-machine interface control,” J. Neurosci.,vol. 30, no. 50, pp. 16777–16787, 2010.

[42] M. T. Kaufman et al., “Cortical activity in the null space: Permittingpreparation without movement,” Nature Neurosci., vol. 17, no. 3, pp. 440–448, Mar 2014.

[43] M. M. Churchland et al., “Neural population dynamics during reaching,”Nature, vol. 487, no. 7405, pp. 51–56, Jul 2012.

[44] J. C. Kao, “Investigating the role of firing-rate normalization and dimen-sionality reduction in brain-machine interface robustness,” in Proc. 35thAnnu. Conf. IEEE EMBS, 2013, vol. 2010, pp. 3–7.

[45] S. Koyama et al., “Comparison of brain-computer interface decodingalgorithms in open-loop and closed-loop control,” J. Comput. Neurosci.,vol. 29, nos. 1/2, pp. 73–87, Aug. 2010.

[46] S. M. Chase et al., “Bias, optimal linear estimation, and the differencesbetween open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms,” Neural Netw., vol. 22, no. 9,pp. 1203–1213, 2009.

[47] B. M. Yu, “Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity,” J. Neurophysiol., vol. 102,pp. 612–635, 2009.

[48] J. P. Cunningham and B. M. Yu, “Dimensionality reduction for large-scaleneural recordings,” Nature Neurosci., vol. 17, no. 11, pp. 1500–1509, Aug.2014.

[49] P. Nuyujukian et al., “A non-human primate brain-computer typing inter-face,” in Proc. IEEE, in press.

Jonathan C. Kao (S’13) received the B.S.and M.S. degrees in electrical engineering fromStanford University, Stanford, CA, USA, in 2010,where he is currently working toward the Ph.D.degree in electrical engineering.

His research interests include algorithms forneural prosthetic control, neural dynamical sys-tems modeling, and the development of clinicallyviable neural prostheses.

Paul Nuyujukian (S’05–M’13) received the B.S.degree in cybernetics from the University of Cal-ifornia, Los Angeles, CA, USA, in 2006, and theM.S. and Ph.D. degrees in bioengineering andthe M.D. degree from Stanford University, Stan-ford, CA, USA, in 2011, 2012, and 2014, respec-tively.

He is currently a Postdoctoral Scholar withthe Department of Neurosurgery, Stanford Uni-versity. His research interests include the devel-opment and clinical translation of neural pros-

theses.

Stephen I. Ryu received the B.S. and M.S. de-grees in electrical engineering from StanfordUniversity, Stanford, CA, USA, in 1994 and 1995,respectively, and the M.D. degree from the Uni-versity of California at San Diego, La Jolla, CA,in 1999. He completed neurosurgical residencyand fellowship training with Stanford Universityin 2006.

He was a Postdoctoral Fellow with the Depart-ment of Neurobiology and Electrical Engineer-ing, Stanford University, from 2002 to 2006. He

was on faculty as an Assistant Professor of neurosurgery with StanfordUniversity until 2009. Since 2009, he has been a Consulting Professorof electrical engineering with Stanford University. He now practices withthe Palo Alto Medical Foundation, Palo Alto, CA. His research interestsinclude brain–machine interfaces, neural prosthetics, minimally invasiveneurosurgery, and stereotactic radiosurgery.

Krishna V. Shenoy (S’87–M’01–SM’06) re-ceived the B.S. degree in electrical engineer-ing from the University of California, Irvine, CA,USA, in 1990, and the M.S. and Ph.D. degreesin electrical engineering from the MassachusettsInstitute of Technology, Cambridge, MA, USA, in1992 and 1995, respectively.

He was a Neurobiology Postdoctoral Fellowwith Caltech from 1995 to 2001. He then joinedStanford University, Stanford, CA, where he iscurrently a Professor with the Departments of

Electrical Engineering, Bioengineering, and Neurobiology, and in theBio-X and Neurosciences Programs. He is also with the Stanford Neu-rosciences Institute and is a Howard Hughes Medical Institute Investiga-tor. His research interests include computational motor neurophysiologyand neural prosthetic system design. He is the Director of the NeuralProsthetic Systems Laboratory and co-Director of the Neural Prosthet-ics Translational Laboratory, Stanford University.

Dr. Shenoy has received the 1996 Hertz Foundation Doctoral The-sis Prize, a Burroughs Wellcome Fund Career Award in the BiomedicalSciences, an Alfred P. Sloan Research Fellowship, a McKnight Endow-ment Fund in Neuroscience Technological Innovations in NeurosciencesAward, a 2009 National Institutes of Health Director’s Pioneer Award, the2010 Stanford University Postdoctoral Mentoring Award, and the 2013Distinguished Alumnus Award from the Henry Samueli School of Engi-neering at the University of California, Irvine.

ieee transactions on biomedical engineering, vol. 64, …

Documents