direct brain stimulation modulates encoding states and...

Article

Direct Brain Stimulation M
odulates Encoding Statesand Memory Performance in Humans
Highlights

d Intracranial brain stimulation has variable effects on episodic

memory performance

d Stimulation increased memory performance when delivered

in poor encoding states

d Recall-related brain activity increased after stimulation of

poor encoding states

d Neural activity linked to contextual memory predicted

encoding state modulation

Ezzyat et al., 2017, Current Biology 27, 1–8May 8, 2017 ª 2017 Elsevier Ltd.http://dx.doi.org/10.1016/j.cub.2017.03.028

Authors

Youssef Ezzyat, James E. Kragel,

John F. Burke, ..., Richard Gorniak,

Daniel S. Rizzuto, Michael J. Kahana

[email protected]

In Brief

Direct brain stimulation is a promising

tool for modulating cognitive function.

Ezzyat et al. show that stimulation

differentially affects episodic memory

encoding depending on its timing relative

to the brain’s encoding state. The data

suggest applications for closed-loop

treatment of memory dysfunction.

mailto:[email protected]

http://dx.doi.org/10.1016/j.cub.2017.03.028

Please cite this article in press as: Ezzyat et al., Direct Brain Stimulation Modulates Encoding States and Memory Performance in Humans, CurrentBiology (2017), http://dx.doi.org/10.1016/j.cub.2017.03.028

Current Biology

Article

Direct Brain Stimulation Modulates EncodingStates and Memory Performance in HumansYoussef Ezzyat,1 James E. Kragel,1 John F. Burke,2 Deborah F. Levy,1 Anastasia Lyalenko,1 Paul Wanda,1

Logan O’Sullivan,1 Katherine B. Hurley,1 Stanislav Busygin,1 Isaac Pedisich,1 Michael R. Sperling,3 Gregory A. Worrell,5

Michal T. Kucewicz,5 Kathryn A. Davis,6 Timothy H. Lucas,7 Cory S. Inman,9 Bradley C. Lega,10 Barbara C. Jobst,11

Sameer A. Sheth,12 Kareem Zaghloul,13 Michael J. Jutras,14 Joel M. Stein,8 Sandhitsu R. Das,6 Richard Gorniak,4

Daniel S. Rizzuto,1,15 and Michael J. Kahana1,15,16,*1Department of Psychology, University of Pennsylvania, Philadelphia, PA 19104, USA2Department of Neurological Surgery, University of California, San Francisco Medical Center, San Francisco, CA 94143, USA3Department of Neurology4Department of Radiology

Thomas Jefferson University Hospital, Philadelphia, PA 19107, USA5Department of Neurology, Mayo Clinic, Rochester, MN 55905, USA6Department of Neurology7Department of Neurosurgery8Department of Radiology

Hospital of the University of Pennsylvania, Philadelphia, PA 19104, USA9Department of Neurosurgery, Emory University Hospital, Atlanta, GA 30322, USA10Department of Neurosurgery, University of Texas Southwestern, Dallas, TX 75390, USA11Department of Neurology, Dartmouth-Hitchcock Medical Center, Lebanon, NH 03756, USA12Department of Neurosurgery, Columbia University Medical Center, New York, NY 10032, USA13Surgical Neurology Branch, National Institutes of Health, Bethesda, MD 20814, USA14Department of Physiology and Biophysics, University of Washington, Seattle, WA 98195, USA15These authors contributed equally16Lead Contact*Correspondence: [email protected]


SUMMARY

People often forget information because they failto effectively encode it. Here, we test the hypo-thesis that targeted electrical stimulation canmodulate neural encoding states and subsequentmemory outcomes. Using recordings from neuro-surgical epilepsy patients with intracranially im-planted electrodes, we trained multivariate classi-fiers to discriminate spectral activity duringlearning that predicted remembering from forget-ting, then decoded neural activity in later ses-sions in which we applied stimulation duringlearning. Stimulation increased encoding-state es-timates and recall if delivered when the classifierindicated low encoding efficiency but had thereverse effect if stimulation was delivered whenthe classifier indicated high encoding efficiency.Higher encoding-state estimates from stimulationwere associated with greater evidence of neuralactivity linked to contextual memory encoding.In identifying the conditions under which stimu-lation modulates memory, the data suggeststrategies for therapeutically treating memorydysfunction.

INTRODUCTION

Memory depends on encoding processes that lay down neural

representations of experiences for long-term storage [1]. Re-

cordings taken during laboratory memory tasks demonstrate

that neural activity in the hippocampus, medial temporal lobe

(MTL) cortex, frontal lobe, and parietal lobe [2, 3] differentiates

learned information that is likely to be remembered from informa-

tion likely to be forgotten. These effects extend to other brain

areas [4] and exist both during and prior to when a to-be-remem-

bered stimulus is present [5–8]. This suggests that coordinated

activity in a distributed neural network generates states that

are responsible for effective memory encoding.

If variability in distributed neural network activity reflects

fluctuation of encoding states that leads to differences in

memory performance, then it should be possible to modulate

memory by perturbing the brain’s encoding state directly [9].

We test this hypothesis using electrical stimulation delivered

through electrodes implanted in the brains of epilepsy patients.

Direct electrical stimulation allows for targeting focal brain

structures in order to modulate activity in complex neural net-

works [10–12] and can be precisely timed to target specific en-

coding events, offering some advantages over non-invasive

methods [13].

We predicted that stimulation’s effects on memory would

depend on the brain’s encoding state at the time it is delivered.

If the memory network is operating efficiently, stimulation should

Current Biology 27, 1–8, May 8, 2017 ª 2017 Elsevier Ltd. 1

mailto:[email protected]


Recalled

Forgotten

TAPEMOLE

DADFLOOD

2+3+1

Encoding (30 sec)

Electrode

“mole” ...

Freq

uenc

y

Distractor (20 s) Recall (30 s)

“tape”

A

B

...

wTX

Pro

babi

lity

LogisticRegression

...

8+4+51+6+7

9+7+2

...

...

False Alarm Rate

Hit

Rat

e

AUC

Classifier Performance

Electrode

Freq

uenc

y

...

...

Figure 1. Experimental Design and Analysis

(A) Subjects performed delayed free recall while intracranially implanted

electrodes recorded local field potentials simultaneously across multiple re-

gions of the brain.

(B) The electrode frequency pattern of spectral power for each word-encoding

period was used as input (X) to fit a classifier to discriminate recalled from

forgotten patterns (resulting weight; w). We assessed classifier performance

using area under the receiver-operating characteristic curve (AUC).

See also Figure S1.


interfere with the encoding process and thus later memory. How-

ever, if the memory network is not operating efficiently, we pre-

dicted that stimulation should disrupt dysfunctional encoding

activity and therefore facilitate memory. A mechanism whereby

stimulation disrupts dysfunctional brain networks is thought to

explain the success in using deep brain stimulation (DBS) of tha-

lamocortical circuits in treating motor dysfunction in Parkinson’s

disease [14, 15].

To use stimulation to modulate encoding states, we first

needed to reliably identify neural activity conducive to successful

memory. There is evidence that theta activity in the hippocam-

pus and MTL cortex prior to stimulus presentation predicts

memory [5, 6, 8], and pre-stimulus theta activity has been used

to trigger learning trials and improve performance in animal

models of classical conditioning [16]. However, similar ap-

proaches in humans using MTL activity in the form of intracranial

theta [17] have not reliably modulated memory performance. We

hypothesized that we could derive a more sensitive index of

memory function by estimating encoding states that reflect

global memory function, as opposed to specific operations car-

ried out in focal brain areas.

To do so, we simultaneously measured neural activity across

the brain. We recorded intracranial electroencephalography

(iEEG) signals from subdural and depth electrodes implanted in

patients with medically refractory epilepsy undergoing clinical

monitoring to determine seizure onset foci. Subjects performed

free recall, amemory task sensitive tomany types of neurological

dysfunction [18, 19] and whose cognitive basis has been

modeled by multiple computational mechanisms [20]. We then

used multivariate classification to test whether a classifier could

predict the probability of recall success from patterns of neural

activity recorded across the brain during encoding. In this way,

we took advantage of our access to many recording channels

2 Current Biology 27, 1–8, May 8, 2017

to derive a subject-specificmodel that could differentiate encod-

ing states likely to lead to remembering from states likely to lead

to forgetting.

Using multivariate classification was advantageous in another

way. Direct electrical stimulation has multifaceted effects on

neural activity that are local and remote relative to the site of

stimulation [21, 22], and that depend on the baseline excitability

of the targeted neural population at the time stimulation is deliv-

ered [23–26]. This poses a challenge when trying to predict the

brain structures stimulation is likely to excite or inhibit and its

consequent effects on behavior. Stimulating hippocampal and

MTL cortical targets in humans, for example, leads to inconsis-

tent and modest effects on memory, with some studies suggest-

ing memory facilitation [27–29] and others showing memory

disruption [30–34].

We addressed this problem by using each subject’s classifier

trained on record-only sessions to decode patterns of neural ac-

tivity during later stimulation sessions. The classifier served as a

model that allowed us to assess evidence for the presence of

good encoding states before and after stimulation and control

periods. The estimates from the classifier integrate information

across electrodes and frequencies, which we predicted would

account for heterogeneity in stimulation’s physiological effects

across people. We targeted stimulation to electrodes placed in

nodes of the memory network: if available within the electrode

montage, we stimulated a single MTL structure (hippocampus

or entorhinal, perirhinal, or parahippocampal cortex) in a given

session. For subjects without MTL contacts, we stimulated other

structures linked to memory encoding, such as the prefrontal

and parietal cortex [3], which we selected by identifying the con-

tact that showed the largest subsequent memory effect in the

high-frequency range (70–200 Hz), a marker of successful mem-

ory encoding that correlates with multi-unit neural firing [35].

RESULTS

Multivariate Classification of Encoding Activity PredictsLater RecallOne hundred and two subjects participated in the record-only

phase of the study. Subjects performed a free recall memory

task during which they studied at least 25 lists of 12 unrelated

words, with each list followed by a 20 smental arithmetic distrac-

tor task (a subset of subjects also performed additional sessions

of free recall with categorized word lists). Subjects then freely re-

called the words from the list in any order (Figure 1A; mean

recall = 27.2% ± 1.2%; SEM). For each encoded word, we

computed the time-frequency decomposition of the iEEG signal

for each bipolar electrode pair (50 frequencies between 1 and

200 Hz; Figure 1B). We used these estimates of spectral power

at each frequency and electrode, for each encoded word, as

input for training a logistic regression classifier. We employed

L2 penalization to avoid overfitting, then assessed performance

using N � 1 cross-validation by experimental session and area

under the receiver-operating characteristic curve (AUC), a stan-

dard measure of a classifier’s ability to generate true positives

while avoiding false positives.

Figures 2A–2D show data from two subjects for eight example

encoding lists. The classifier generated higher probabilities for

recalled than forgotten items in these periods (Figures 2A and

A B

DC

GFEWord Encoding Trial

Cla

ssifi

er O

utpu

tP

roba

bilit

y

0

0.2

0.4

0.6

0.8

1

List 1 List 2 List 3 List 4 List 5 List 6 List 7 List 8

Word Encoding Trial

Cla

ssifi

er O

utpu

tP

roba

bilit

y

0

0.2

0.4

0.6

0.8

1

List 1 List 2 List 3 List 4 List 5 List 6 List 7 List 8

False Alarm Rate0.2 0.4 0.6 0.8 1

Hit

Rat

e

0.2

0.4

0.6

0.8

1

AUC = .76


Hit

Rat

e

0.2

0.4

0.6

0.8

1

AUC = .73

RegionIF

GM

FGSFG

MTLC

Hipp TCIP

CSPC

OC

Fre

quen

cy (

Hz)

2

3

5

8

13

23

40

68

116

200

Mod

el-B

ased

Act

ivat

ion

-0.5

-0.25

0

0.25

0.5

RegionIF

GM

FGSFG

MTLC

Hipp TCIP

CSPC

OCF

requ

ency

(H

z)

2

3

5

8

13

23

40

68

116

200

Sub

sequ

ent M

emor

y (t

-sta

t)

-0.8

-0.4

0

0.4

0.8


Hit

Rat

e

0.2

0.4

0.6

0.8

1

Mean AUC: 0.63t(101) = 19.2, P < 10-10

Figure 2. Classifier Performance

(A and C) Classifier output probability for an eight-list period of the delayed free recall task in two example subjects. Dashed lines indicate the optimal decision

threshold dividing recalled from forgotten trials. Red, later recalled words; blue, later forgotten words. (A) Example patient 1. (C) Example patient 2.

(B and D) AUC for both subjects was significantly greater than chance. (B) Example patient 1. (D) Example patient 2.

(E) Individual receiver-operating characteristic (ROC) curves plotted for all subjects.

(F) Forward-model-derived estimates of classification importance for each electrode region 3 frequency feature, grouped into anatomical regions of interest.

(G) Subsequent memory analysis contrasting encoding power for later recalled words with later not recalled words.

(F andG) IFG, inferior frontal gyrus;MFG,middle frontal gyrus; SFG, superior frontal gyrus;MTLC,medial temporal lobe cortex; Hipp, hippocampus; TC, temporal

cortex; IPC, inferior parietal cortex; SPC, superior parietal cortex; OC, occipital cortex. Data are multiple comparisons corrected using false discovery rate (FDR)

at q = 0.05.

See also Figures S2 and S3.


2C) and across all encoded words as measured using AUC (Fig-

ures 2B and 2D). Across subjects, classification performance ex-

ceeded chance (mean AUC 0.63 ± 0.07, t(101) = 19.2, p < 10�10;

Figure 2E), which indicates that our approach can identify

subject-specific features of brain activity during encoding that

predict memory. We next asked whether the features that

were important to classification were idiosyncratic or instead re-

flected consistent activity in similar brain regions and at similar

frequencies.

We derived a forward model [36] for each subject using clas-

sifier weights, accounting for covariance between features in

the input data, to estimate the relative importance of each region

3 frequency feature for classifier performance. This showed that

the classifier relied on widespread low-frequency power de-

creases simultaneous with high-frequency power increases

across the frontal, temporal, and occipital cortex, as well as in

the hippocampus, to predict successful recall (Figure 2F). We

observed a similar pattern when contrasting power for remem-

bered and forgotten words (Figure 2G), with theMTL and parietal

cortex also showing high-frequency power increases. This

echoes prior work in intracranial and scalp EEG [37] and sug-

gests consistency in the features that predict efficient memory

function across people.

Stimulation Has Heterogeneous Effects on MemoryPerformanceHaving established that classification discriminates encoding

states, we asked whether stimulation modulates these states

in a way that influences memory performance. On stimulation

(Stim) lists, we applied 50-Hz trains across a single pair of elec-

trodes at parameters previously used to modulate spatial mem-

ory in humans [28]. We then used each subject’s record-only

classifier to decode neural activity during stimulation sessions

(N = 52 stimulation datasets from 36 subjects). We first tested

classifier generalization to the stimulation sessions. Our experi-

mental design included lists without stimulation (NoStim lists)

to serve as a baseline for behavioral performance and for testing

between-session classifier generalization (Figure 3A). The classi-

fier significantly discriminated encoding activity for recalled

and forgotten words (mean AUC on NoStim lists 0.61 ± 0.01,

Current Biology 27, 1–8, May 8, 2017 3

Serial Position4 8 12

Pro

babi

lity

ofR

ecal

l

0

0.4

0.8

Lag-5 -3 -1 1 3 5

Con

d. R

espo

nse

Pro

babi

lity

0

0.4

0.8

Stim ListsNoStim Lists

Record-Only

Normalized Stim-No Stim Recall-75 -50 -25 0 25 50 75

# P

atie

nt S

timul

atio

n S

essi

ons

0

4

8

12

16 Mean = -6.8%P < 0.04

LOCK

FUR

Stim Lists

RING

HEN

NoStim Lists

50 Hz

A

B

D

C

Figure 3. Classifier Output Predicts the Effect of Stimulation on

Memory

(A) We applied stimulation across alternating pairs of words on Stim lists;

NoStim lists were devoid of stimulation.

(B) The effect of stimulation on memory performance varied across subjects

(SD 22.5%); mean, red dashed line.

(C and D) Recall probability as a function of serial position (C) and inter-item lag

(D) does not significantly differ as a function of stimulation condition.

See also Table S1.

LOCK FUR GOLD ...

PreS

timD

ata

...

Recalled?

Low High

Low-High State Difference

RegionIF

GM

FGSFG

MTL

Hipp TCIP

CSPC

OC

Fre

quen

cy (

Hz)

2

3

5

8

13

23

40

68

116

200

Mea

n t-

stat

-8

-6

-4

-2

0

2

4

6

8

P < 0.003

P < 0.03 P < 0.05

Stim-NoStim Classifier (t-stat)-5 0 5

N

0

2

4

6

8

10

12Mean = 0.67P < 0.01

Low States

-5 0 5

N

0

2

4

6

8

10

12Mean = -1.52

P < 0.001

High States

Stim-NoStim Classifier (t-stat)

Low High

Stim

-NoS

tim N

orm

aliz

ed R

ecal

l

-250

-150

-50

50

150

Preceding State

A

B

D E

C

Figure 4. The Effect of Stimulation Depends on Brain State

(A) Classifier decoding prior to stimulation onset allowed us to analyzememory

performance based on pre-stimulation brain state.

(B) Spectral power prior to stimulation onset was significantly lower at high

frequencies in frontal, temporal, parietal, and occipital cortex (FDR corrected

at q = 0.05).

(C) Recall performance increased if stimulation was delivered when the brain

was in a low encoding state (p < 0.03) and decreased if delivered in a high

encoding state (p < 0.05). The difference between low and high stimulation was

also significant (p < 0.003). Red bars show mean SE of the difference.

(D) Stimulation significantly increased classifier output when delivered at low

encoding states (p < 0.01).

(E) Stimulation significantly decreased classifier output when delivered at high

encoding states (p < 0.001).

See also Table S2.


t(51) = 10.4, p < 10�10), even though recall performance was

slightly higher for NoStim lists compared to record-only ses-

sions (record-only 30.6% ± 1.7%, NoStim lists 33.5% ± 1.9%,

t(51) = 2.6, p < 0.02). This suggests that the relation between

neural activity and memory states was stable from the record-

only to the stimulation sessions.

We next asked whether encoding stimulation tended to facili-

tate or disrupt recall performance. Within-subject, stimulation

significantly increased recall performance in two subjects and

decreased recall performance in six subjects (c2 test, p <

0.05). Across the group, stimulation reliably decreased memory

performance (D normalized recall �6.8% ± 3.2%, p < 0.04; Fig-

ure 3B), but there was considerable variability in stimulation’s

effects across individuals, ranging from large memory disruption

to facilitation (SD = 22.5%). This heterogeneity is consistent with

past work [27–30, 32–34], and meant that the small difference in

overall recall performance was not accompanied by specific

group-level differences in recall organization, measured using

two traditional assays of human memory performance, the serial

position curve and lag conditional response probability curve

(Figures 3C and 3D).

Stimulation’s Behavioral and Neural Effects Are StateDependentAlthough stimulation had inconsistent effects overall, we pre-

dicted the pre-stimulation encoding state would account for

some of this variability. In subjects who showed above-chance

classifier generalization (N = 39 datasets from 27 subjects), we

applied the classifiers to intervals just prior to each stimulation

train (Figure 4A) and split the resulting distribution of classifier

outputs into low and high bins, based on the optimal classifica-

tion threshold from the previous record-only sessions. Low pre-

stimulation encoding states were associated with decreased


high-frequency power in widespread brain areas that predicted

memory performance during word encoding, including the

frontal, temporal, and parietal cortex (Figure 4B).

Stimulation enhanced recall performance when delivered

just after low encoding states (t(38) = 2.26, p < 0.03) but

decreased performance when delivered just after high encoding

states (t(38) = �2.09, p < 0.05; low-high difference t(38) = 3.32,

p < 0.003; Figure 4C). We compared the classifier estimates of

the brain’s encoding state post- and pre-stimulation, which

showed that low-state stimulation increased evidence for good

encoding (p < 0.02; Figure 4D) whereas high-state stimulation

decreased evidence for good encoding (p < 0.001; Figure 4E).

The data suggest that stimulation influences memory function

by perturbing the brain’s encoding state relative to its status at

the time of delivery.

Evoked Spectral Tilt after Poor Encoding StatesWhen stimulation was delivered in low states, both recall perfor-

mance and classifier evidence increased. We next asked how

Figure 5. Correlation between the Stimulation-Related Change

in Classifier Output and the Spectral Tilt Effect: (High-Frequency

Activity t Stat) – (Low-Frequency Activity t Stat)


increased classifier evidence relates to stimulation-evoked

changes in neural activity across the brain. To measure stimula-

tion’s effect on neural activity, we used an index of the spectral

tilt, which is characterized by increased high-frequency power

simultaneous with widespread decreases in low-frequency po-

wer. These spectral modulations are thought to reflect both local

increases in multi-unit firing [17] and decreased long-range low-

frequency synchrony [38]. Evidence for these patterns correlates

with the fMRI blood-oxygen-level-dependent (BOLD) signal [39],

predicts successful memory encoding [6, 40], and is related to

core memory processes such as item-context binding [41]. We

found that the change in classifier output after stimulation was

related to how much stimulation evoked the tilt pattern (r(37) =

0.54, p < 0.001; Figure 5), suggesting that stimulation increased

classifier evidence by modulating a neural marker that has been

linked to contextual memory encoding.

DISCUSSION

We applied direct electrical stimulation to nodes of the memory

network and found that stimulation reliablymodulatesmemory in

a way that depends on the brain’s encoding state. In showing

that stimulation improves memory in low encoding states and

disrupts memory in high encoding states, the findings suggest

that stimulation alters the ongoing course of memory processing

in the brain. By using brain-state-matched trials from non-stim-

ulated lists, our data show that stimulation modulates neural ac-

tivity beyond what might be expected by regression to the mean

arising from temporal autocorrelation in the brain’s encoding

state. Our data offer insight into the inconsistent effects that

have been reported in studies of how brain stimulation modu-

lates memory performance [27–34], and suggest that using

brain-state decoding can improve the ability to influence mem-

ory outcomes with stimulation.

Our results are consistent with amodel in which targeted stim-

ulation leads to changes in network activity across brain areas

that contribute to successful memory encoding. There is

growing consensus that direct electrical stimulation is likely to in-

fluence physiology across a network connected to the targeted

site [10, 12]. In using DBS for the treatment of Parkinson’s [42],

for example, researchers have had success targeting multiple

structures within the affected motor network [12], which sug-

gests that it is more important to target the relevant functional

network rather than individual structures within the network. In

the case of episodic memory, it may therefore be possible to

enhance the effectiveness of stimulation by using measures of

connectivity to identify nodes that offer a high degree of control-

lability over the memory network [43]. Resting-state data could

be leveraged to predict the stimulation targets that are most

likely to modulate the core memory network [44].

Prior work has shown that stimulation’s effects on physiology

depend not only on the excitability of the targeted neurons [26]

but also on ongoing rhythms generated by synchronous activity

in larger populations. Hippocampal stimulation, for example,

alternately promotes long-term potentiation or long-term

depression depending on whether theta phase is at the peak

or trough at the time of stimulation delivery [24, 25]. Learning it-

self is also state dependent, as shown in classical conditioning

experiments in which animals show faster learning when trials

are triggered based on theta rhythm [16]. Our data confirm the

role of pre-stimulus brain states for upcoming learning and

show that these states can be directly modulated.

We applied classification to whole-brain iEEG to decode brain

states that predict later recall. Multivariate decoding allowed us

to overcome individual differences in neural connectivity, clinical

etiology, and electrode placement that could increase variability

in stimulation’s neural and behavioral effects. Decodingmay also

have provided a more sensitive index of the encoding state than

would be possible if using a single feature to identify good and

poor memory states [17]. We then related stimulation’s effect

on physiology to its effect on memory, extending prior work

that has used stimulation to map behavior. Using direct brain re-

cordings most likely facilitated decoding, but future work should

address the extent to which non-invasive decoding and stimula-

tion methods [45] could be combined to modulate memory

states.

We used the classifier trained on encoding data to decode

pre-stimulation states. Our approach suggests that, at a broad

level, similar whole-brain patterns of neural activity predict

successful encoding during and prior to stimulus onset. In

both training and testing our classifier, we averaged spectral

power over a temporal interval of several hundred millisec-

onds, meaning our model was sensitive to consistent spectral

power fluctuations over the pre- and post-stimulus intervals.

There is evidence that assessing neural activity at a finer

temporal scale can identify distinct patterns of pre- and

post-stimulus activity that predict encoding. Increased pre-

stimulus theta power recorded using non-invasive methods,

for example, has been shown to predict successful memory

[6, 46], and increased intracranial theta power has also been

shown to predict memory pre-stimulus [5, 47], although not

during free recall. Taken together with our data, these findings

suggest that both tonic and phasic pre-stimulus signals are

predictive of memory success. Algorithms to identify good en-

coding states could therefore be improved by incorporating

time as a feature, which would allow both sustained and tran-

sient fluctuations in spectral power to influence estimates of

the encoding state.



By testing classifier generalization across days and using the

free recall task to measure memory performance, our data sup-

port the interpretation that the decoded brain states are stable in

their neural representation over time and globally predict mem-

ory function. Free recall is a complex task that recruits multiple

core episodic memory processes [20]. We show that stimulation

increases encoding states by increasing high-frequency activity

(HFA) power and decreasing low-frequency activity (LFA) power

across the brain, a pattern that predicts behavioral measures of

item-context encoding [41]. Although such encoding processes

promote memory function, free recall is also known to depend

heavily on retrieval processes, suggesting that future work may

find success influencing memory function by applying stimula-

tion during memory search.

We show an overall reduction in verbal memory performance

when stimulating a large set of brain regions, including many

outside of the MTL. This is broadly consistent with recent work

focused on the hippocampus and entorhinal cortex that showed

that stimulation tends to impair both verbal and spatial memory

[31]. However, we further test the hypothesis that stimulation’s

effects on memory depend on timing relative to the brain’s en-

coding state [48]. Our findings therefore extend prior studies of

human intracranial brain stimulation in several ways. First, we

usemultivariate decoding of neural activity to separate pre-stim-

ulation brain states, and show that stimulation counteracts low

encoding states but disrupts high encoding states. Second, we

show that stimulation at low and high encoding states differen-

tially modulates neural activity in amanner consistent with the ef-

fect on memory. Third, we show that stimulation’s effect on the

encoding state is correlated with the spectral tilt, a biomarker

of successful memory encoding. Our work therefore identifies

situations in which stimulation increases and decreases mem-

ory, and relates stimulation’s effects on behavior to its influence

on neural activity through novel use of subject-specific multivar-

iate classification.

By showing that stimulation is most likely to improve memory

when encoding efficiency is low prior to stimulation delivery, our

data provide the foundation for future work to apply stimulation

when it is most likely to improve memory function. Non-invasive

closed-loop approaches have improved attention through

training using fMRI [49] andmaximized the benefit of restudy op-

portunities using scalp EEG [50]. Closed-loop neural decoding

could thus optimally target stimulation for treatment of memory

disorders [48, 51].

EXPERIMENTAL PROCEDURES

Participants

One hundred and two patients undergoing iEEG monitoring as part of clinical

treatment for drug-resistant epilepsy were recruited to participate in this study.

Data were collected as part of a multi-center project designed to assess the

effects of electrical stimulation on memory-related brain function. Data were

collected at the following centers: Thomas Jefferson University Hospital,

Mayo Clinic, Hospital of the University of Pennsylvania, Emory University Hos-

pital, University of Texas Southwestern Medical Center, Dartmouth-Hitchcock

Medical Center, Columbia University Medical Center, National Institutes of

Health, and University of Washington Medical Center. The research protocol

was approved by the institutional review board (IRB) at each hospital and

informed consent was obtained from each participant. Electrophysiological

data were collected from electrodes implanted subdurally on the cortical sur-

face as well as deep within the brain parenchyma. In each case, the clinical


team determined the placement of the electrodes so as to best localize epilep-

togenic regions. Subdural contacts were arranged in both strip and grid con-

figurations with an inter-contact spacing of 10mm.Most subjects (N = 83) also

had temporal lobe depth electrodes with 5 mm inter-contact spacing.

Verbal Memory Task

Each subject participated in a delayed free recall task in which they were in-

structed to study lists of words for a later memory test; no encoding task

was used. Lists were composed of 12 words chosen at random and without

replacement from a pool of high-frequency nouns (either English or Spanish,

depending on the participant’s native language; http://memory.psych.

upenn.edu/Word_Pools). Each word remained on the screen for 1,600 ms, fol-

lowed by a randomly jittered 750- to 1,000ms blank inter-stimulus interval (ISI).

Immediately after the final word in each list, participants performed a dis-

tractor task (20 s) consisting of a series of arithmetic problems of the form

A + B + C = ?, where A, B, and C were randomly chosen integers ranging

from 1 to 9. After the distractor task, participants were given 30 s to verbally

recall as many words as possible from the list in any order; vocal responses

were digitally recorded and later manually scored for analysis. Each session

consisted of 25 lists of this encoding-distractor-recall procedure. A subset

of subjects completed additional sessions of the free recall task using catego-

rized word lists, which were included in the electrophysiological analyses. The

categorized recall task is identical to the free recall task, with the exception that

the word pool was drawn from 25 semantic categories (e.g., fruit, furniture,

office supplies). Each list of 12 items in the categorized version of the task con-

sisted of four words drawn from each of three categories. In total, 41 patients

completed at least one session of the categorized recall task.

Stimulation Methods

At the start of each session, we determined the safe amplitude for stimulation

using a mapping procedure in which stimulation was applied at 0.5 mA while

a neurologist monitored for afterdischarges. This procedure was repeated,

incrementing the amplitude in steps of 0.5 mA, up to a maximum of

1.5 mA for depth contacts and 3.5 mA for cortical surface contacts. These

maximum amplitudes were chosen to be well below accepted safety limits

for charge density [52]. For each stimulation session, we passed electrical

current through a single pair of adjacent electrode contacts. Because the

electrode locations were determined strictly by the monitoring needs of the

clinicians, we used a combination of anatomical and functional information

to select stimulation sites. If available, we prioritized electrodes in the hippo-

campus, entorhinal cortex, perirhinal cortex, parahippocampal cortex, and

dorsolateral prefrontal cortex. To choose among these regions in cases in

which more than one was available, we selected the electrode demonstrating

the largest subsequent memory effect (SME) in the high-frequency range

(70–200 Hz) among these regions. In cases in which none of the aforemen-

tioned regions was available, we selected the contact with the largest

SME. We used a mapping procedure at the start of each session to deter-

mine the safe amplitude for stimulation. Stimulation was delivered using

charge-balanced biphasic rectangular pulses (pulse width = 300 ms) at

50 Hz frequency, and was applied continuously for 4.6 s while subjects en-

coded two consecutive words; stimulation was not applied for the following

two words. This alternation of stimulated and non-stimulated word pairs

continued until the end of the list. Stimulation onset was 200 ms prior to

word presentation and lasted until 200–450 ms after the offset of the next

word (the range is due to the variable ISI between words). Stimulation was

applied in 20 of the 25 lists in a session, and each stimulation list was

randomly chosen to begin with a stimulated or non-stimulated pair of words.

We randomized the order of the 20 stimulation lists and the remaining five

non-stimulation control lists within each session.

Statistics

Data are presented as mean ± SEM. Unless otherwise specified, all statistical

comparisons were conducted as two-tailed tests. Data distributions were

either visually inspected or assumed to be normal for parametric tests. For

both the record-only and stimulation samples, we included any enrolled sub-

ject who completed at least one full session of the task. In both cases, the sam-

ple sizes were chosen to at least match or exceed the sample sizes reported in

prior human intracranial record-only and stimulation studies. For stimulation

http://memory.psych.upenn.edu/Word_Pools

http://memory.psych.upenn.edu/Word_Pools


analyses, we treated all sessions collected in a single patient at a single stim-

ulated bipolar pair as our unit of observation.

All other methods are described in the Supplemental Experimental

Procedures.

Data and Software Availability

All deidentified raw data and analysis code may be downloaded at http://

memory.psych.upenn.edu/Electrophysiological_Data.

SUPPLEMENTAL INFORMATION

Supplemental Information includes Supplemental Experimental Procedures,

three figures, and two tables and can be found with this article online at

http://dx.doi.org/10.1016/j.cub.2017.03.028.

AUTHOR CONTRIBUTIONS

Y.E., D.S.R., and M.J.K. designed experiments. K.B.H., D.F.L., A.L., L.O.,

P.W., M.T.K., C.S.I., and M.J.J. performed data collection and recording.

Y.E. analyzed the data. J.E.K., J.F.B., S.B., and I.P. contributed data analysis

tools. J.M.S., S.R.D., and R.G. localized the electrodes. M.R.S., G.A.W.,

K.A.D., T.H.L., B.C.L., B.C.J., S.A.S., and K.Z. recruited participants and pro-

vided general assistance. Y.E. and M.J.K. wrote the manuscript. All authors

provided feedback on the manuscript. D.S.R. and M.J.K. supervised the

research.

ACKNOWLEDGMENTS

We dedicate this paper in loving memory of Anastasia Lyalenko, without

whose contributions this work would not have been possible. This work was

supported by the DARPA Restoring Active Memory (RAM) program (coopera-

tive agreement N66001-14-2-4032). We thank Blackrock Microsystems for

providing neural recording and stimulation equipment. We thank J.A. Wachter

for suggesting the analysis in Figure 4. We are indebted to the patients and

their families for their participation and support. The views, opinions, and/or

findings contained in this material are those of the authors and should not

be interpreted as representing the official views or policies of the Department

of Defense or the U.S. Government. B.C.J. receives research funding from

NeuroPace and Medtronic not relating to this research. M.J.K. and D.S.R.

are in the process of organizing Nia Therapeutics, LLC (‘‘Nia’’), a company in-

tended to develop and commercialize brain stimulation therapies for memory

restoration. Currently, Nia has no assets and has not commenced operations.

M.J.K. and D.S.R. each holds a greater than 5% equity interest in Nia.

Received: December 5, 2016

Revised: February 24, 2017

Accepted: March 13, 2017

Published: April 20, 2017

REFERENCES

1. Paller, K.A., and Wagner, A.D. (2002). Observing the transformation of

experience into memory. Trends Cogn. Sci. 6, 93–102.

2. Davachi, L., Mitchell, J.P., and Wagner, A.D. (2003). Multiple routes to

memory: distinct medial temporal lobe processes build item and source

memories. Proc. Natl. Acad. Sci. USA 100, 2157–2162.

3. Uncapher, M.R., and Wagner, A.D. (2009). Posterior parietal cortex and

episodic encoding: insights from fMRI subsequent memory effects and

dual-attention theory. Neurobiol. Learn. Mem. 91, 139–154.

4. Kim, H. (2011). Neural activity that predicts subsequent memory and

forgetting: a meta-analysis of 74 fMRI studies. Neuroimage 54, 2446–

2461.

5. Fell, J., Ludowig, E., Staresina, B.P., Wagner, T., Kranz, T., Elger, C.E., and

Axmacher, N. (2011). Medial temporal theta/alpha power enhancement

precedes successful memory encoding: evidence based on intracranial

EEG. J. Neurosci. 31, 5392–5397.

6. Guderian, S., Schott, B.H., Richardson-Klavehn, A., and Duzel, E. (2009).

Medial temporal theta state before an event predicts episodic encoding

success in humans. Proc. Natl. Acad. Sci. USA 106, 5365–5370.

7. Park, H., and Rugg, M.D. (2010). Prestimulus hippocampal activity pre-

dicts later recollection. Hippocampus 20, 24–28.

8. Rutishauser, U., Ross, I.B., Mamelak, A.N., and Schuman, E.M. (2010).

Human memory strength is predicted by theta-frequency phase-locking

of single neurons. Nature 464, 903–907.

9. Hanslmayr, S., Matuschek, J., and Fellner, M.C. (2014). Entrainment of

prefrontal beta oscillations induces an endogenous echo and impairs

memory formation. Curr. Biol. 24, 904–909.

10. Kim, K., Ekstrom, A.D., and Tandon, N. (2016). A network approach for

modulatingmemory processes via direct and indirect brain stimulation: to-

ward a causal approach for the neural basis of memory. Neurobiol. Learn.

Mem. 134, 162–177.

11. Kringelbach, M.L., Jenkinson, N., Owen, S.L.F., and Aziz, T.Z. (2007).

Translational principles of deep brain stimulation. Nat. Rev. Neurosci. 8,

623–635.

12. McIntyre, C.C., and Hahn, P.J. (2010). Network perspectives on themech-

anisms of deep brain stimulation. Neurobiol. Dis. 38, 329–337.

13. Matzen, L.E., Trumbo, M.C., Leach, R.C., and Leshikar, E.D. (2015).

Effects of non-invasive brain stimulation on associative memory. Brain

Res. 1624, 286–296.

14. Bronstein, J.M., Tagliati, M., Alterman, R.L., Lozano, A.M., Volkmann, J.,

Stefani, A., Horak, F.B., Okun, M.S., Foote, K.D., Krack, P., et al. (2011).

Deep brain stimulation for Parkinson disease: an expert consensus and re-

view of key issues. Arch. Neurol. 68, 165.

15. Benabid, A.L., Pollak, P., Gervason, C., Hoffmann, D., Gao, D.M.,

Hommel, M., Perret, J.E., and de Rougemont, J. (1991). Long-term sup-

pression of tremor by chronic stimulation of the ventral intermediate

thalamic nucleus. Lancet 337, 403–406.

16. Seager, M.A., Johnson, L.D., Chabot, E.S., Asaka, Y., and Berry, S.D.

(2002). Oscillatory brain states and learning: impact of hippocampal

theta-contingent training. Proc. Natl. Acad. Sci. USA 99, 1616–1620.

17. Burke, J.F., Ramayya, A.G., and Kahana, M.J. (2015). Human intracranial

high-frequency activity during memory processing: neural oscillations or

stochastic volatility? Curr. Opin. Neurobiol. 31, 104–110.

18. Grober, E., Lipton, R.B., Hall, C., andCrystal, H. (2000). Memory impairment

on free and cued selective reminding predicts dementia. Neurology 54,

827–832.

19. Gershberg,F.B., andShimamura,A.P. (1995). Impaireduseoforganizational

strategies in free recall following frontal lobe damage.Neuropsychologia 33,

1305–1333.

20. Lohnas, L.J., Polyn, S.M., and Kahana, M.J. (2015). Expanding the scope

of memory search: modeling intralist and interlist effects in free recall.

Psychol. Rev. 122, 337–363.

21. Haglund, M.M., Ojemann, G.A., and Blasdel, G.G. (1993). Optical imaging

of bipolar cortical stimulation. J. Neurosurg. 78, 785–793.

22. Suh, M., Bahar, S., Mehta, A.D., and Schwartz, T.H. (2006). Blood volume

and hemoglobin oxygenation response following electrical stimulation of

human cortex. Neuroimage 31, 66–75.

23. Borchers, S., Himmelbach, M., Logothetis, N., and Karnath, H.O. (2011).

Direct electrical stimulation of human cortex - the gold standard for map-

ping brain functions? Nat. Rev. Neurosci. 13, 63–70.

24. Hyman, J.M., Wyble, B.P., Goyal, V., Rossi, C.A., and Hasselmo, M.E.

(2003). Stimulation in hippocampal region CA1 in behaving rats yields

long-term potentiation when delivered to the peak of theta and long-

term depression when delivered to the trough. J. Neurosci. 23, 11725–

11731.

25. Pavlides, C., Greenstein, Y.J., Grudman, M., and Winson, J. (1988). Long-

term potentiation in the dentate gyrus is induced preferentially on the

positive phase of theta-rhythm. Brain Res. 439, 383–387.

26. Pollen, D.A. (1977). Responses of single neurons to electrical stimulation

of the surface of the visual cortex. Brain Behav. Evol. 14, 67–86.


http://memory.psych.upenn.edu/Electrophysiological_Data

http://memory.psych.upenn.edu/Electrophysiological_Data


http://refhub.elsevier.com/S0960-9822(17)30326-3/sref1
















































































27. Fell, J., Staresina, B.P., Do Lam, A.T., Widman, G., Helmstaedter, C.,

Elger, C.E., and Axmacher, N. (2013). Memory modulation by weak syn-

chronous deep brain stimulation: a pilot study. Brain Stimulat. 6, 270–273.

28. Suthana, N., Haneef, Z., Stern, J., Mukamel, R., Behnke, E., Knowlton, B.,

and Fried, I. (2012). Memory enhancement and deep-brain stimulation of

the entorhinal area. N. Engl. J. Med. 366, 502–510.

29. Miller, J.P., Sweet, J.A., Bailey, C.M., Munyon, C.N., Luders, H.O., and

Fastenau, P.S. (2015). Visual-spatial memory may be enhanced with theta

burst deep brain stimulation of the fornix: a preliminary investigation with

four cases. Brain 138, 1833–1842.

30. Coleshill, S.G., Binnie, C.D., Morris, R.G., Alarcon, G., van Emde Boas,W.,

Velis, D.N., Simmons, A., Polkey, C.E., van Veelen, C.W.M., and van Rijen,

P.C. (2004). Material-specific recognition memory deficits elicited by uni-

lateral hippocampal electrical stimulation. J. Neurosci. 24, 1612–1616.

31. Jacobs, J., Miller, J., Lee, S.A., Coffey, T., Watrous, A.J., Sperling, M.R.,

Sharan, A., Worrell, G., Berry, B., Lega, B., et al. (2016). Direct electrical

stimulation of the human entorhinal region and hippocampus impairs

memory. Neuron 92, 983–990.

32. Halgren, E., and Wilson, C.L. (1985). Recall deficits produced by

afterdischarges in the human hippocampal formation and amygdala.

Electroencephalogr. Clin. Neurophysiol. 61, 375–380.

33. Lacruz, M.E., Valentın, A., Seoane, J.J.G., Morris, R.G., Selway, R.P., and

Alarcon, G. (2010). Single pulse electrical stimulation of the hippocampus

is sufficient to impair humanepisodicmemory. Neuroscience170, 623–632.

34. Perrine, K., Devinsky, O., Uysal, S., Luciano, D.J., and Dogali, M. (1994).

Left temporal neocortex mediation of verbal memory: evidence from func-

tional mapping with cortical stimulation. Neurology 44, 1845–1850.

35. Manning, J.R., Jacobs, J., Fried, I., and Kahana, M.J. (2009). Broadband

shifts in local field potential power spectra are correlated with single-

neuron spiking in humans. J. Neurosci. 29, 13613–13620.

36. Haufe, S., Meinecke, F., Gorgen, K., D€ahne, S., Haynes, J.D., Blankertz,

B., and Bießmann, F. (2014). On the interpretation of weight vectors of

linear models in multivariate neuroimaging. Neuroimage 87, 96–110.

37. Long, N.M., Burke, J.F., and Kahana, M.J. (2014). Subsequent memory

effect in intracranial and scalp EEG. Neuroimage 84, 488–494.

38. von Stein, A., and Sarnthein, J. (2000). Different frequencies for different

scales of cortical integration: from local gamma to long range alpha/theta

synchronization. Int. J. Psychophysiol. 38, 301–313.

39. Winawer, J., Kay, K.N., Foster, B.L., Rauschecker, A.M., Parvizi, J., and

Wandell, B.A. (2013). Asynchronous broadband signals are the principal

source of the BOLD response in human visual cortex. Curr. Biol. 23,

1145–1153.


40. Burke, J.F., Long, N.M., Zaghloul, K.A., Sharan, A.D., Sperling, M.R.,

and Kahana, M.J. (2014). Human intracranial high-frequency activity

maps episodic memory formation in space and time. Neuroimage 85,

834–843.

41. Long, N.M., and Kahana, M.J. (2015). Successful memory formation is

driven by contextual encoding in the core memory network. Neuroimage

119, 332–337.

42. Benabid, A.L., Pollak, P., Louveau, A., Henry, S., and de Rougemont, J.

(1987). Combined (thalamotomy and stimulation) stereotactic surgery of

the VIM thalamic nucleus for bilateral Parkinson disease. Stereotact.

Funct. Neurosurg. 50, 344–346.

43. Gu, S., Pasqualetti, F., Cieslak, M., Telesford, Q.K., Yu, A.B., Kahn, A.E.,

Medaglia, J.D., Vettel, J.M., Miller, M.B., Grafton, S.T., and Bassett, D.S.

(2015). Controllability of structural brain networks. Nat. Commun. 6,

8414.

44. Fox, M.D., Buckner, R.L., Liu, H., Chakravarty, M.M., Lozano, A.M., and

Pascual-Leone, A. (2014). Resting-state networks link invasive and nonin-

vasive brain stimulation across diverse psychiatric and neurological dis-

eases. Proc. Natl. Acad. Sci. USA 111, E4367–E4375.

45. Wang, J.X., Rogers, L.M., Gross, E.Z., Ryals, A.J., Dokucu, M.E.,

Brandstatt, K.L., Hermiller, M.S., and Voss, J.L. (2014). Targeted enhance-

ment of cortical-hippocampal brain networks and associative memory.

Science 345, 1054–1057.

46. Salari, N., and Rose, M. (2016). Dissociation of the functional relevance

of different pre-stimulus oscillatory activity for memory formation.

Neuroimage 125, 1013–1021.

47. Merkow, M.B., Burke, J.F., Stein, J.M., and Kahana, M.J. (2014).

Prestimulus theta in the human hippocampus predicts subsequent recog-

nition but not recall. Hippocampus 24, 1562–1569.

48. Young, N.P., and Deisseroth, K. (2017). Cognitive neuroscience: in search

of lost time. Nature 542, 173–174.

49. deBettencourt, M.T., Cohen, J.D., Lee, R.F., Norman, K.A., and Turk-

Browne, N.B. (2015). Closed-loop training of attention with real-time brain

imaging. Nat. Neurosci. 18, 470–475.

50. Fukuda, K., and Woodman, G.F. (2015). Predicting and improving recog-

nition memory using multiple electrophysiological signals in real time.

Psychol. Sci. 26, 1026–1037.

51. Lozano, A.M., Mayberg, H.S., Giacobbe, P., Hamani, C., Craddock, R.C.,

and Kennedy, S.H. (2008). Subcallosal cingulate gyrus deep brain stimu-

lation for treatment-resistant depression. Biol. Psychiatry 64, 461–467.

52. Shannon, R.V. (1992). Amodel of safe levels for electrical stimulation. IEEE

Trans. Biomed. Eng. 39, 424–426.






















































































Current Biology, Volume 27

Supplemental Information

Direct Brain Stimulation Modulates Encoding

States and Memory Performance in Humans

Youssef Ezzyat, James E. Kragel, John F. Burke, Deborah F. Levy, Anastasia Lyalenko, PaulWanda, Logan O'Sullivan, Katherine B. Hurley, Stanislav Busygin, IsaacPedisich, Michael R. Sperling, Gregory A. Worrell, Michal T. Kucewicz, Kathryn A.Davis, Timothy H. Lucas, Cory S. Inman, Bradley C. Lega, Barbara C. Jobst, Sameer A.Sheth, Kareem Zaghloul, Michael J. Jutras, Joel M. Stein, Sandhitsu R. Das, RichardGorniak, Daniel S. Rizzuto, and Michael J. Kahana

Figure S1: Aggregate Electrode Rendering. Related to Figure 1. All bipolar (virtual) electrodes from all102 patients rendered on normalized cortical surfaces. Red spheres indicate stimulated electrodes visiblefrom lateral and medial views.

Figure S2: Classifier performance within each time-frequency bin. Related to Figure 2E. Mean perfor-mance of classifiers trained on data from individual time-frequency pixels. Vertical dashed lines indicateword onset and offset. For each of 50 frequencies and for each of 65 timebins from -500 ms relative to wordonset to 500 ms relative to word offset, we fit an L2-penalized logistic regression classifier. We used powerat each electrode, within a given time-frequency bin, as the set of features for each classifier. We fit theseclassifiers for every subject and calculated cross-validated AUC for each time-frequency bin.

Figure S3: Whole-brain Spectral Tilt vs. Full Classifier. Related to Figure 2E. Area under the curve(AUC) for the L2-penalized logistic regression (L2 LR) classifier from the main manuscript compared withthe whole-brain spectral tilt classifier. The multivariate model outperforms the tilt model, P < 10−8. Foreach subject, we averaged spectral power above (high frequency) and below (low-frequency) a given cutofffrequency, across all electrodes. We did this for each possible cutoff frequency (from 1 to 200 Hz) and tookthe high-frequency–low-frequency difference as the measure of the tilt. We then computed area under thecurve (AUC) for this measure and did so for the measure defined using each cutoff frequency. We thenselected the cutoff frequency that maximized spectral tilt AUC for that subject and used this in the acrosssubject distribution.

Subject Site Gender Age Lang. Dom. Ictal Onset Stim Location Stim Hemisphere Amp (mA) ∆ Recall (Normalized %) ∆ Recall (Raw # Words/List)1 P F 48 L R CA1 L 1.5/1.0 -2.84 -0.052 P F 49 L B Amy L 1.5 1.21 0.05

Amy L 1.5 -7.87 -0.403 P F 39 L L CA3 L 1.5 43.22 1.854 P M 47 L L DLPFC L 1.5 -16.13 -0.405 J F 48 L R CA1 R 1.5/1.0 -1.15 -0.05

CA1 R 1.5 -2.49 -0.106 J M 24 L L CA1 L 0.5/1.0 -53.57 -1.137 J M 32 L R ACg R 1.0 -11.22 -0.35

PCg R 1.5 23.10 0.858 E F 36 L L CA3 L 1.0 -14.96 -0.789 D F 24 B L EC L 0.5 -11.03 -0.38

EC L 0.5 -35.34 -1.35Precuneus L 1.5 16.06 0.70

10 J M 48 L L CA1 L 1.0 18.95 -0.51PRC L 1.0 -10.87 -0.20

11 M F 27 NA R EC R 1.0/1.5 2.75 0.25EC R 1.0 -9.16 -0.70

12 W F 33 NA L DLPFC L 3.5 -16.67 -0.50DLPFC L 3.5 -57.29 -1.1

13 J M 23 L L PHC L 0.5 16.53 0.40PHC L 0.5 -0.35 -0.01

14 M M 40 NA L PRC R 1.5 35.09 1.3315 D F 31 L R PRC L 1.5 28.85 0.45

CA3 L 1.0 -31.25 -0.8016 M M 49 R L PRC L 1.0/0.5 -18.18 -0.7717 M F 28 R R DLPRC R 3.0 -4.57 -0.3018 E F 19 NA R CA3 L 1.5 -12.50 -0.5019 M M 34 L R CA1 R 1.5 -14.81 -0.9320 M F 36 NA R DLPFC R 3.0 -9.23 -0.2721 P F 45 L L CA3 L 1.0 -66.53 -1.6522 M M 26 L L DLPFC L 2.5 -10.42 -0.50

DLPFC L 2.5 8.85 0.4023 T F 40 L L PRC R 0.50 35.71 0.9024 M M 24 NA B ACg L 1.0 13.89 0.2525 T F 26 R L CA1 R 0.50 -27.44 -0.9026 M M 20 L L MTG L 1.5 21.01 1.62

PRC L 0.75 -15.96 -1.0027 T M 47 L L Insula R 0.5 -18.44 -0.9028 J F 49 L L DG L 0.25 -2.12 -0.0529 C M 22 L R CA1 R 0.5 -3.07 -0.1530 M F 41 L R ITG L 1.0 -39.07 -1.58

MTG L 1.0 8.7 0.3531 M F 23 L L ITG L 1.0 0.57 0.03

MTG L 1.0 13.00 0.36MTG L 1.0 -10.31 -0.40

32 M M 42 NA R Lateral Occipital Cortex R 0.75 -17.09 -0.67Inferior Parietal Cortex R 0.75 6.76 0.29

33 J M 50 NA R EC L 1.5 -3.42 -0.1034 J F 22 NA R Fusiform Gyrus R 1.0 6.25 0.2535 T F 38 L R PRC L 1.0 -15.63 -0.3836 T M 35 L R PRC L 1.0 -43.64 -1.60

Clinical Site: C: Columbia, D: Dartmouth, E: Emory, J: Jefferson, M: Mayo, P: Penn, T: Texas, W: University of WashingtonLanguage Dominance/Ictal Onset: L: Left Hemisphere, R: Right Hemisphere, B: Bilateral, NA: Not Reported/UndeterminedStimulation Location: CA1/2/3: Hippocampal subfields, DG: Dentate Gyrus, DLPFC: Dorsolateral prefrontal cortex, EC: Entorhinalcortex, ITG/MTG: Inferior/Middle temporal gyrus, PRC: Perirhinal cortex, PHC: Parahippocampal cortex, PFC: Prefrontal cortex,ACg: Anterior cingulate gyrus, PCg: Posterior Cingulate Gyrus

Table S1. Patient Demographics and Raw Data. Related to Figure 3. For each patient, the clinical datacollection site, gender, age, language dominance, ictal onset hemisphere, stimulation location, stimulationhemisphere, stimulation amplitude, Normalized ∆ %Recall, and Raw ∆ Number of Words Recalled/List.Each row presents data for a unique stimulation location (bipolar electrode pair), and encompasses one ormore sessions collected at the stimulated site.

Stim Region (N) HC (10) MTL (9) TC (7) MFG (5)Low Brain State (∆t) 0.47 ± 0.40 0.78 ± 0.59 0.83 ± 0.86 1.71 ± 0.83High Brain State (∆t) -0.96 ± 0.49 -1.47 ± 0.74 -2.68 ± 0.62 -1.82 ± 0.85

Table S2. Change in Brain State as a Function of Stimulated Region. Related to Figure 4D, E. Post-pre∆ classifier output for Low State stimulation events (Top) and High State stimulation events (Bottom) as afunction of stimulated brain region. Only regions with N > 4 are shown. Numbers indicate mean ± s.e.mwithin stimulated brain region. HC = hippocampus, MTL = medial temporal lobe cortex, TC = non-MTLtemporal lobe, MFG =middle frontal gyrus.

Supplemental Experimental Procedures

Anatomical localization

Anatomical localization of electrode placement was accomplished using a two step process. First,hippocampal subfields and MTL cortices were automatically labeled in a pre-implant 2 mm thickcoronal T2-weighted MRI using the automatic segmentation of hippocampal subfields (ASHS)multi-atlas segmentation method [S1]. Following this automatic labeling procedure, a post-implantCT scan was co-registered with the MRI using Advanced Normalization Tools [S2]. Electrodes thatare visible in the CT were then localized within the MTL subregions by a pair of neuroradiologistswith expertise in MTL anatomy. The neuroradiologists confirmed the output of the automatedpipeline and provided additional detail on localization within subfields/subregions. The wholebrain cortical surface was also obtained from a pre-implant volumetric T1-weighted MRI usingFreesurfer [S3], and subdural electrodes were separately co-registered and localized to corticalregions using an energy minimization algorithm [S4]. In cases in which automatic localizationfailed, the neuroradiologists performed manual localization of the electrodes.

Electrophysiological data processing

Intracranial data were recorded using one of the following clinical electroencephalogram (EEG)systems (depending the site of data collection): Nihon Kohden EEG-1200, Natus XLTek EMU 128or Grass Aura-LTM64. Depending on the amplifier and the preference of the clinical team, thesignals were sampled at either 500, 1000 or 1600 Hz and were referenced to a common contactplaced either intracranially, on the scalp or mastoid process. Intracranial electrophysiologicaldata were filtered to attenuate line noise (5 Hz band-stop fourth order Butterworth, centeredon 60 Hz). To eliminate potentially confounding large-scale artifacts and noise on the referencechannel, we re-refereced the data using a bipolar montage [S5]. To do so, we identified all pairs ofimmediately adjacent contacts on every depth, strip and grid and took to difference between thesignals recorded in each pair. The resulting bipolar timeseries was treated as a virtual electrode andused in all subsequent analysis. We performed spectral decomposition (50 frequencies from 1-200Hz, logarithmically-spaced; Morlet wavelets; wave number = 5) for 2600 ms epochs from -500 to to2100 ms relative to word onset. An additional 1500 ms buffer included before and after the intervalof interest was discarded to remove convolution edge effects. The resulting time-frequency datawere then log-transformed, resampled in the time domain to 50 Hz, and z-scored within sessionand frequency band across word presentation events.

Multivariate classification

We included only data collected in record-only sessions as input to a logistic regression classifiertrained to discriminate encoding-related activity predictive of whether a word was later remem-bered or forgotten. We used spectral power averaged across the time dimension for each wordencoding epoch (0-1600ms relative to word onset) as the input data. Thus, the features for eachindividual word encoding observation were the average power across time, at each of the 50 ana-lyzed frequencies×N electrodes. We used L2-penalization [S6] and selected the penalty parameterfor each subject based on the best penalty parameter (in terms of maximizing area under the curve,AUC) across the entire distribution of subjects. To do this, we began with a set of 22 possiblepenalty parameters spaced logarithmically between 10−6 and 104 and, for each penalty parameter,fit the classifier using maximum likelihood and used N − 1 cross-validation to assess performance

using AUC. For the majority of subjects that completed more than one session, N − 1 was appliedover sessions; for subjects that completed a single session N − 1 was applied over lists within thesession. This procedure yielded a distribution of AUCs across subjects for each penalty param-eter. Then, for each subject we conducted a final cross-validated training in which we selectedthe penalty parameter with the highest mean AUC across the distribution of all subjects exceptthe one whose classifier was being trained. We then computed AUC [S6] to quantify classifierperformance and repeated this procedure for all subjects. AUC measures a classifier’s ability toidentify true positives while minimizing false positives, where chance AUC = 0.50. To ensurethe classifier learned equally from both classes (given the imbalance between recalled and notrecalled exemplars), we also weighted the penalty parameter in inverse proportion to the numberof exemplars of each class [S6]. We assessed the significance of each classifier within-subject usinga permutation test in which we randomized the labels of recalled/not recalled events in the trainingdata, computed AUC and repeated the randomization 1000 times to generate a null distributionof AUCs. Classifiers that exceeded the P < 0.05 level were considered reliable for the purposes ofanalyses contingent on brain-state.

In subsequent behavioral analyses of the stimulation sessions (see Analysis of stimulation’s effect onmemory performance), we used Youden’s J statistic, which identifies the classifier output thresholdthat maximizes the difference between true and false positives. To derive Youden’s J, we fit anunequal variance signal detection model to the distribution of classifier outputs across the record-only sessions and estimated Youden’s J from the best-fit curve. These curves are plotted in Figure2E.

To assess the importance of different frequency × brain region features to classification, we gener-ated a forward model for each subject using the weights from their classifier [S7]

A =ΣxWσ2

y

where Σx is the covariance matrix of the data X, W is the vector of classifier weights obtainedfollowing maximum likelihood estimation, σ2

y is the variance of the logit-transform of the vector

of classifier outputs across all events, y, and A is the vector of parameter values for the uniqueforward model corresponding to the backward model W. The magnitude and direction of the valuesin A reflect the strength of the relation between classifier output and each input feature in X.We computed A separately for each subject and plot the average across subjects in Figure 2F. Weassessed significance across the group by computing a t-statistic against zero for the activation ofeach feature and applying threshold-free cluster enhancement to the t-map. We then generated1000 null datasets by flipping the signs of the values in A for 1000 random samples of up to Nsubjects. We computed t-statistics on the null distributions, derived P values by comparing thetrue t-statistic for each feature to the distributions generated from the null data, and corrected formultiple comparisons by controlling the false discovery rate at q = 0.05 [S8].

Spectral analysis of stimulation sessions

For the analyses of the behavioral effects of stimulation contingent on the pre-stimulation encodingstate, we used wavelet decomposition (see Electrophysiological data processing) applied to the-1100 to -100 ms intervals prior to stimulation onset. We used mirrored buffers to avoid inclusion ofdata from stimulation epochs. We averaged power values over the -900 to -100 ms pre-stimulation

interval to generate input patterns for the classifier. We applied the same analysis to the post-stimulation data (and matched periods from NoStim lists) in the interval 100 to 900 ms relative tostimulation offset.

Behavioral analysis

We used the percentage of words recalled from the encoding lists as our behavioral measureof memory performance. We first analyzed the overall effects of stimulation on memory bycomparing recall performance for the Stim and NoStim conditions using two standard assaysof memory performance in free recall tasks: the serial position curve (SPC; Figure 3A,) and thelag conditional response probability curve (CRP; Figure 3B). The SPC depicts the probability ofrecalling a studied word as a function of its position within the study list. The CRP depicts theprobability of recalling two words from the study list as a function of the lag (in number of words)between the serial positions of the two words. We also computed the same measures for therecord-only data.

Analysis of stimulation’s effect on neural activity

For the data in Figure 4B, we compared power within each region of interest at each frequencybetween trials in which the classifier indicated low or high encoding state (defined by classifieroutput being below or above the decision threshold). We computed a two-sample within-subjectt-test to quantify the effect of brain state on spectral power, then computed a one-sample t-testacross the group distribution of within-subject t-statistics. We corrected for multiple comparisonsby controlling the false discovery rate at q = 0.05.

To assess stimulation’s effect on classifier output, we extracted logit-transformed classifier es-timates pre- and post-stimulation (see Spectral analysis of stimulation sessions) and computed awithin-subject paired t-test. We plot the resulting across-subject distribution of t-statistics in Fig-ures 4D and E, separately for trials in which the pre-stimulation classifier estimate was low or high.We then used a one-sample t-test to assess stimulations effect on classifier output across the group.One subject was excluded from the high-state analysis for having only one matched NoStim trial.

To analyze neural activity evoked by stimulation during the low state (Figure 5), we computed asubsequent memory analysis contrasting power for subsequently recalled and not recalled words(i.e. Figure 2G). We corrected for multiple comparisons by controlling the false discovery rate(q = 0.05) and used the resulting map to define a set of significant brain-region × frequencyclusters. To ensure this localizer was fully independent, we did this analysis only for subjectsthat not participate in the stimulation task. We then extracted power in these clusters from thepost-stim intervals in Stim lists and matched intervals in NoStim lists from stimulation task data.For each subject we computed an unpaired t-test within-subject between Stim and NoStim lists,separately for data averaged within the high-frequency clusters and within the low-frequencyclusters. We then took the difference between these two within-subject t-statistics and computedPearson’s r between this distribution and the distribution of mean differences in classifier outputbetween post-stim and matched NoStim intervals.

Analysis of stimulation’s effect on memory performance

To quantify the effect of stimulation on encoding efficiency, we applied the classifier to the pre-and post-stimulation epochs (and matched NoStim periods). We used the inverse logit transformto convert the classifier probability estimates to log-odds. For the analyses in Figures 4 and 5, wecomputed the mean difference in classifier output between Stim lists and NoStim lists. We alsocomputed the difference in memory between Stim and NoStim lists, normalized by the patient?soverall mean recall:

∆R =RS − RNS

RAll

where RX refers to mean percent recall for either Stim lists, NoStim lists, or collapsed across All lists.

To assess the effects of stimulation on recall rate as a function of the classifier output just prior tostimulation delivery, we computed∆R but included in the numerator only those trials that followedthe termination of a stimulation train (Figure 4A) in RS and only trials in matched positions inNoStim lists in RNS. We then split the trials into two groups based on the classifier output during thepre-stimulation interval (see Analysis of stimulation sessions). We used Youden’s J (see Multivariateclassification), derived from classifier output in the record-only sessions, to separate the distributionof pre-stimulation classifier outputs into low and high bins. We then computed ∆R as a functionof state in each subject and compared ∆R to each other and to zero using paired and one-samplet-tests.

Analysis of the Spectral Tilt

For the subjects that performed the stimulation task, we used within-subject repeated measures t-tests to compare power in each region × frequency bin before and after Stim and NoStim intervals.To assess how stimulation modulated the spectral tilt in our stimulation subjects, we first usedthe subset of subjects that did not participate in stimulation sessions to define the clusters of brainregion × frequency space that significantly discriminate recalled and not recalled words in therecord-only sessions (e.g. Figure 2G). Then, in stimulation subjects, we extracted the t-statisticsin these clusters from the post-pre stimulation comparison. We then averaged the all t-stats fromclusters showing HFA increases and separately averaged t-stats from the large cluster showingLFA decreases. We took the difference (HFA-LFA) as the index of the magnitude of the spectral tilt[S5].

Supplemental References

S1. Yushkevich, P.A., Pluta, J.B., Wang, H., Xie, L., Ding, S.L., Gertje, E.C., Mancuso, L., Kliot,D., Das, S.R., and Wolk, D.A. (2015). Automated volumetry and regional thickness analysis ofhippocampal subfields and medial temporal cortical structures in mild cognitive impairment.Human Brain Mapping 36, 258–287.

S2. Avants, B.B., Epstein, C.L., Grossman, M., and Gee, J.C. (2008). Symmetric diffeomorphicimage registration with cross-correlation: evaluating automated labeling of elderly and neu-rodegenerative brain. Medical Image Analysis 12, 26–41.

S3. Fischl, B., van der Kouwe, A., Destrieux, C., Halgren, E., Segonne, F., Salat, D.H., Busa, E.,Seidman, L.J., Goldstein, J., Kennedy, D., et al. (2004). Automatically parcellating the humancerebral cortex. Cerebral Cortex 14, 11–22.

S4. Dykstra, A.R., Chan, A.M., Quinn, B.T., Zepeda, R., Keller, C.J., Cormier, J., Madsen, J.R.,Eskandar, E.N., and Cash, S.S. (2012). Individualized localization and cortical surface-basedregistration of intracranial electrodes. NeuroImage 59, 3563–3570.

S5. Burke, J.F., Long, N.M., Zaghloul, K.A., Sharan, A.D., Sperling, M.R., and Kahana, M.J. (2014).Human intracranial high-frequency activity maps episodic memory formation in space andtime. NeuroImage 85, 834–843.

S6. Hastie, T., Tibshirani, R., and Friedman, J. (2001). The elements of statistical learning (NewYork: Springer-Verlag).

S7. Haufe, S., Meinecke, F., Gorgen, K., Dahne, S., Haynes, J.D., Blankertz, B., and Bießmann, F.(2014). On the interpretation of weight vectors of linear models in multivariate neuroimaging.NeuroImage 87, 96–110.

S8. Benjamini, Y. and Hochberg, Y. (1995). Controlling the False Discovery Rate: a practical andpowerful approach to multiple testing. Journal of Royal Statistical Society, Series B 57, 289–300.

direct brain stimulation modulates encoding states and...

Documents