What are the key objectives of your research?

My research will answer the question of how distributed neural circuits give rise to coordinated behaviours. To address this, I will combine multiscale electrophysiology with optical manipulations in rodents undergoing a reach and reward test to investigate the functional organisation and dynamic coordination of sensorimotor networks.

You have a unique mix of training and research experiences. How does your multidisciplinary background contribute to your current investigations?

My undergraduate studies in mathematics and computational cognitive science nurtured abstract thinking about the brain as a computing machine, while my PhD in Neuroscience grounded these ideas in the organ’s neurobiological reality and the constraints of experimental observations. Together, these training and research experiences have provided me with both quantitative tools and neurobiological intuitions to understand brain functioning.

Despite many experiments on the sensorimotor cortex, why does understanding remain limited of the functional organisation and coordination of neuronal populations? How will your research endeavour to illuminate these gaps in the literature?

One primary impediment to understanding the functional organisation and coordination of neuronal populations is a technological one. The brain is composed of myriad neuronal microcircuits that perform specific computations while being simultaneously integrated into larger networks. Investigating the activity of individual neurons and small neuronal populations (ie. microscale) has yielded exquisite insight into microcircuit mechanisms of local computations, while macroscale measurements, eg. functional magnetic resonance imaging (fMRI), have revealed global processing of entire brain

areas. However, much less is known about the brain at the intermediate- or mesoscale. This is primarily because the methods available to neuroscientists have traditionally not permitted simultaneous high spatiotemporal resolution monitoring of brain activity over large areas. These are precisely the kind of measurements that are provided by electrocorticography (ECoG) and are required to understand the functional organisation and coordination of neuronal populations.

How does ECoG aid the translation of your findings for an understanding of the human brain, its health and diseases?

Most studies of the human brain utilise methods that indirectly measure neural activity, such as fMRI or magnetoencephalography (MEG). In contrast, many studies in basic neuroscience investigate the activity of single neurons or small populations of single neurons. These are fundamentally different signals, and the relationship between them is poorly understood.

ECoG is a unique methodological bridge between basic neuroscience and human studies. By recording electrical activity directly from the cortical surface, it achieves very high spatiotemporal resolution neural recordings in humans. By employing the same recording technology used for humans in my investigations of the rodent brain, I am studying the same signal. Further understanding the ECoG signal will aid epilepsy monitoring and brain-machine interfaces. Combining ECoG with more traditional methods to record neural activity from different cortical depths will provide a multiscale view of brain activity in (approximately) 3D that is difficult to achieve over large areas with other methods.

At what stage is your current research?

We have spent the past year developing advanced neural recording technologies and computing infrastructures. In

collaboration with Dr Peter Denes of Lawrence Berkeley National Laboratory (LBNL), in the coming months we will be testing a 2,000-channel electrophysiology recording device that only has nine wires. This equipment will be connected to computing hardware for real-time extraction and visualisation of specific brain signals as they are being acquired. We have created an experimental preparation to study the functional organisation and dynamic coordination of cortical networks in head-restrained rodents performing arm reaches to a real-time controlled 3D lever that delivers a reward. Finally, in collaboration with Dr Fritz Sommer of the University of California, Berkeley and Mr Prabhat of LBNL, we will utilise state-of-the-art machine learning algorithms (eg. deep neural networks, convolutional sparse coding) implemented on the National Energy Research Scientific Computing Center’s supercomputer.

What goals do you hope to achieve in the next five years?

Understanding how diverse neural circuits give rise to whole-brain function requires engaging animals in tasks for which the brain was evolved to operate. As neurotechnologies and computational capabilities evolve, I foresee recording and manipulating thousands of sites with high temporal resolution throughout the entire brain while tracking complex behaviours in ethological environments. The ongoing developments in neurotechnology and computing at LBNL bring us closer to that aspiration.

Dr Kristofer Bouchard provides an insight into his current work that makes use of a brain imaging technique to directly measure brain activity from the cortical surface, maximising the spatiotemporal resolution and coverage achievable in both human and animal studies

Enhancing electrocorticography

56 INTERNATIONAL INNOVATION

DR KRISTOFER BOUCHARD

THE BRAIN IS made up of many different circuits of neurons that, although the fundamental unit of computation in the brain, are also integrated within complex larger networks. Scientific understanding of how these networks communicate with each other and coordinate their activities is limited. Much research on the brain is focused on understanding these neuronal circuits on an individual basis, and thus many of the analytical and imaging technologies available are only appropriate for analysing networks at this micro level. Consequently, relatively little research has been carried out with the specific aim of gaining intermediate- or mesoscale insight.

This perspective is particularly important for investigating the processes underlying the generation of human behaviours. Many everyday actions, while performed unconsciously, are in fact learned skills that require the precise and rapid coordination of numerous body parts. Grasping an object, for example, involves many joints and muscles, from the shoulder through to the fingers, each of which is spatially represented in the brain within a different region of the sensorimotor cortex. As a result, it can be supposed that this region plays a key role in organising and coordinating the different networks of neurons that are necessary to carry out skilled behaviours. As of yet, however, understanding of the precise mechanisms involved is limited, both because the majority of studies in this area tend to revolve around the dynamics of local neural populations rather than entire networks, and because linking network dynamics to their spatial representation is technologically challenging.

In California, USA, a collaborative research programme involving scientists from the Lawrence Berkeley National Laboratory (LBNL), University of California, Berkeley (UCB) and University of California, San Francisco (UCSF) is working to overcome these shortcomings. Its aim is to elucidate

how distinct neural circuits give rise to skilled motor behaviours through the study of functional organisation and dynamic coordination within the sensorimotor cortex. Specifically, the project combines the engineering of improved high-density brain recording methods (led by Dr Peter Denes, LBNL), studies of language production in patients with temporarily implanted recording electrodes (led by Drs Edward Chang, UCSF, and Kristofer Bouchard, LBNL), and the development of computational brain theories (led by Professor Fritz Sommer, UCB, and Bouchard).

Bouchard is co-leading the efforts in theory and experimentation. Alongside Sommer, Bouchard conceived the hypothesis that the concept of sparsity plays an essential role in motor activity generating behaviour. As such, Bouchard is conducting research to elucidate sparse motor representations and understand how they are dynamically coordinated across the sensorimotor cortex, and investigating brain functions across multiple spatiotemporal scales.

OPTIMISING RESOLUTIONTo achieve its research goals, the LBNL-UCB-UCSF collaborative group is developing instrumentation and computational methods to measure brain networks in high spatiotemporal resolution and perform functional brain mapping. The partnership’s research aims in this area are divided across three pillars: the first revolves around instrumentation development, which is predominantly LBNL’s responsibility; the second involves experimental mapping of cortical network function, and is the domain of UCSF; and the third aims to computationally enable data-driven discovery – an effort being undertaken jointly by UCB and LBNL.

Much of the work revolves around the advancement of electrocorticography (ECoG), a technique capable of detecting electrical activity directly from the surface of the

Scientists at the Lawrence Berkeley National Laboratory, University of California, Berkeley and University of California, San Francisco are working together to develop techniques to more fully understand how distinct neuronal circuits in the brain coordinate their activity to enable complex learned behaviours

Grasping complex behaviours

A CALIFORNIAN COLLABORATION

Dr Kristofer Bouchard provides an insight into the strengths of the three-way inter-institutional collaboration in which he participates

“The Obama BRAIN Initiative calls for bringing together expertise in diverse areas to understand the brain in health and disease. It is for precisely this reason that I am very excited about the ongoing collaborative efforts between UCB, UCSF and LBNL.

The UCSF-based partner, neurosurgeon Dr Edward Chang, is a leader in both basic and clinical neuroscience and a pioneer in understanding how the human brain produces and perceives speech; the UCB collaboration includes Professor Fritz Sommer, member and co-Founder of the Redwood Center, one of a few institutes for theoretical neuroscience that exists nationwide; and scientists at LBNL Dr Peter Denes and Prabhat excel at integrating multidisciplinary teams from engineering, chemistry, physics and computer science to solve problems of scale. Together, these institutes will develop novel neurotechnologies, apply them to basic neuroscience questions and translate them to humans. This unique nexus of skills exemplifies the collaborative spirit required to understand the brain.”

www.internationalinnovation.com 57

cortex, and can achieve high spatiotemporal resolution neural recordings. It can be used on both humans and animal models, meaning that findings are directly comparable.

REACHING RODENTSWithin this wider research programme, Bouchard’s group is dedicated to combining ECoG with more conventional imaging methods to record neural activity at different cortical depths to gain a multiscale, 3D perspective of brain activity. ECoG’s unique abilities mean it can overcome the challenges associated with linking mesoscale observations to single neuron and local microcircuit processing. As such, it represents a powerful tool for the investigations of both local and more widely distributed processing. Specifically, the team is merging micro-ECoG with laminar polytrodes (which record from single neurons at different cortical depths) and optogenic manipulations (combining genetics and optics to control well-defined events within specific cells) in its reaching rodent model to more fully understand how the coordinated activation of distributed and distinct neural circuits gives rise to coordinated behaviours.

The task of measuring and determining the sensorimotor cortex’s functional organisation across multiple spatiotemporal scales during learned, skilled behaviours is challenging. Currently, limitations involve the technological challenges associated with acquiring vast amounts of neural activity data across functionally distinct regions of the brain at high resolution. The researchers are confident, however, that ECoG can meet such a challenge.

Bouchard has invaluable experience in utilising ECoG; while conducting postdoctoral research he was part of a team that used the technique to investigate functional organisation of the vocal tract’s spatial representation during human speech. He has also collaborated on the development and use of micro-ECoG, which can simultaneously record neural activity from many electrodes placed upon a rodent cortex and reveal the functional organisation of a rat’s auditory cortex. By combining micro-ECoG with other tools, it should be possible to directly monitor and manipulate neuronal activity at both the micro- and meso-scales, and with high spatiotemporal resolution.

So far, the research programme is progressing as planned; the required advanced neural recording technologies and infrastructures are in development and the experiments are also in preparation. “In the upcoming months, we will begin testing a novel 2,000-channel, nine-wire electrophysiology recording device that enables the real-time extraction and visualisation of specific brain signals,” reveals Bouchard. “Furthermore, the team will soon begin to utilise state-of-the-art

The LBNL-UCB-UCSF collaboration will develop new recording instruments, utilise them in novel experiments and analyse the data with high-performance computing. This will result in high-resolution functional brain mapping and data driven discovery.

The LBNL E-Chip: a 44x44 channel neural recording device developed by Dr Peter Denes. This device acquires 1,936 channels of electrical recordings at 20,000 Hz per channel, covering an area of ~5 mm2 at 100 micrometre resolution, and has only nine wires.

machine learning algorithms for the analysis of the vast amount of data generated.”

MAKING AN IMPACTOverall, this work is expected to provide a multiscale understanding of the sensorimotor cortex, allowing a detailed insight into how coordinated spatiotemporal patterns of brain activity emerge from distributed local microcircuits. Such an improved understanding could have applications in pathological conditions that are thought to result from the dysfunctional coordination and routing of neural signals, such as developmental coordination disorder and schizophrenia. Additionally, the advancement of ECoG and other computational tools is likely to facilitate the translation of basic neuroscience discoveries to clinical benefits. For example, an enhanced understanding of the ECoG signal could prove useful for epilepsy monitoring and brain-machine interfaces, while the team plans to apply its novel computational tools to existing human ECoG data generated by studies of speech production, which may yield significant new insights.

Looking to the future, Bouchard predicts that current lines of investigation may eventually lead to the ability to combine data across disparate spatial scales – from the subcellular structure of individual neurons to the functioning of entire brain areas that give rise to complex behaviours. For this, it will be critical to develop mechanistic models that link the

statistical behaviour of collections of fine-grain elements to more coarse-grained observations.

The team is confident that future work will advance existing models that link different types of neural processing, combining concepts such as dynamical systems and compressed sensing, and establish methods for ‘fusing’ multimodal data. “Moving forwards, we will continue to collaborate on the development of neurotechnologies to collect neural recordings across multiple spatial scales, and interpret experimental observations in the context of theoretical frameworks,” Bouchard concludes.

58 INTERNATIONAL INNOVATION

CORTICAL COORDINATION GENERATING BEHAVIOUR

OBJECTIVETo address how distributed neural circuits give rise to coordinated behaviours by combining multiscale electrophysiology and optogenetics in rodents carrying out learned behaviours to investigate functional organisation and dynamic coordination in the sensorimotor cortex.

KEY COLLABORATORSDr Edward Chang, Department of Neurological Surgery, University of California, San Francisco, USA

Dr Peter Denes, Physical Sciences Directorate, Lawrence Berkeley National Laboratory, USA

Mr Prabhat, Data and Analytics Services Group Lead, National Energy Research Scientific Computing Center, USA

Dr Fritz Sommer, Redwood Center for Theoretical Neuroscience, University of California, Berkeley, USA

PARTNERSUniversity of California, San Francisco, USA

University of California, Berkeley, USA

FUNDINGLawrence Berkeley National Laboratory (LBNL): Laboratory Directed Research and Development: Neuro/nano Technology for BRAIN, Professor Peter Denes, Principal Investigator

CONTACTDr Kristofer Bouchard Research Scientist

Lawrence Berkeley National Laboratory1 Cyclotron Road, MS 977Berkeley, CaliforniaUSA

T +1 415 548 0026E kebouchard@lbl.gov

http://bit.ly/Bouchard_Lab

http://bit.ly/Bouchard_Lab2

DR KRISTOFER BOUCHARD gained his BA in Mathematics and Computational Cognitive Science from Brandeis University, Massachusetts, before moving

to California to complete a PhD in Neuroscience at the University of California, San Francisco. Bouchard subsequently held a postdoctoral research position at the Speech Neuroengineering Laboratory at the same university before moving to his current position in the Life Sciences and Computational Research Divisions at LBNL.

INTELLIGENCE

BRAIN-MACHINE INTERFACES

Brain-machine interfaces involve a direct communication pathway between neural signals in the brain and an external effector to assist, augment or repair human cognitive or sensorimotor function. These have a large variety of clinical and basic science applications; however, current performance is relatively modest, especially for speech

In continuing collaboration with the lab of Dr Edward Chang, a neurosurgeon at UCSF, Bouchard is working to achieve state-of-the-art single-trial decoding of speech from the human sensorimotor cortex to an external effector. This will enable a novel, mixed continuous and discrete approach to a speech prosthetic.

CONTINUOUS DECODING OF VOWEL ACOUSTICS The speech sensorimotor cortex controls the movement of the vocal tract articulators. In general, the relationship between vocal tract mechanics and produced acoustics is a mathematically degenerate problem. However, the acoustics of vowels during the steady state are more directly related to specific vocal tract configurations than for most other sounds. Therefore, the researchers utilised a novel approach of decoding the produced continuous acoustics of three cardinal vowels (/a/ (‘aa’), /i/ ‘ee’, /u/ ‘oo’) from the concurrently recorded neural activity. It was possible to predict the acoustics of produced vowels on a single-trial basis with extremely high accuracy. This is currently the state-of-the-art in published continuous speech decoding.

CONTINUOUS DECODING OF ARTICULATOR KINEMATICS Although the team has demonstrated the state-of-the-art in continuous decoding of vowel acoustics, its ability to decode the acoustics of other speech sounds, such as consonants remains relatively poor. As discussed above, this in part reflects the fact that the sensorimotor cortex controls the vocal tract articulators, and the mapping from articulators to acoustics is, in general, degenerate. However, simultaneous measurement of all vocal tract articulators is challenging, especially in the clinical setting in which ECoG recordings are taken. To overcome this challenge, the researchers pioneered a novel, multimodal system for real-time tracking of all vocal tract articulators during speech production compatible with ECoG recordings in the hospital. Preliminary analysis of lip kinematics has demonstrated the capacity to predict the lip aperture with high fidelity. This is the first time that vocal tract articulator kinematics have been directly decoded from human brain recordings.

DEEP NEURAL NETS FOR SYLLABLE CLASSIFICATION In contrast to the continuous decoding described above, the most common approach to speech prostheses in the literature treats speech as a sequence of categorical tokens, and attempts to classify these from the recorded neural activity. Most recently, the California-based researchers have been using deep neural networks to classify speech syllables. Their initial results are far surpassing the current state-of-the-art published results for speech classification. These results suggest that deep neural networks will be fruitful avenues for neural prosthetics.

The production of even simple speech sounds, such as the word ‘she’ (depicted here as a time-frequency plot) is generated by precise, coordinated control of the vocal tract articulators (ie. the lips, tongue, jaw and layrnx).

www.internationalinnovation.com 59