PowerPoint Presentation

First progress meetingBrain Control Club4-5-2017CRI

http://cri-paris.org/scientific-clubs/brain-control-club/

Organization & MembersGroup leader: Ignacio Rebollo (CRI)Communication: Hernan Anllo (CRI)Finance:Claire Baudelet (CRI)

Mentors:Dr. Stephen Whitmarsh (ENS)Dr. Guillaume Dumas (Pasteur)Prof. Robert Oostenveld (DCCN, KI)

Approx. 12 CRI students working on 4/5 projects

Brain Control ClubGoalUnderstanding theory and applications of BCIThrough hands-on creation of BCIs

ApproachGroup projects on a specific BCI applicationSharing knowledge within and between projects

SupportAutodidacticEducational tools and materialNecessary hardware and softwareHackathonsMentorshipInvited experts

Brain Control ClubGoalUnderstanding theory and applications of BCIThrough hands-on creation of BCIs

ApproachGroup projects on a specific BCI applicationSharing knowledge within and between projects

SupportAutodidacticEducational tools and materialNecessary hardware and softwareHackathonsMentorshipInvited experts

GoalStatus of each projectAccomplishments so farShare knowledge between projectsIdentify obstacles

ProjectsNirvanaMorpheusEEG-gameEEGsynth.orgEEG-composition (NEW)

Progress meeting

Project NirvanaIgnacio Claire Elizabeth Hernan (Raly)

BackgroundBodily signals (heart, stomach, respiration, brain) interact with each other during different mental/bodily states Brain alpha (10Hz) power fluctuations are associated with vigilance fluctuations, together with changes in respiration and heart rate

Understanding their coupling in real time could allow us to detect certain mental statesAlthough these processes are normally unconscious, explicit knowledge of them (e.g. sounds), allow us to exert some control over them, a technique known as biofeedback

ObjectivesDevelop tools for online tracking of mental states such as vigilance, concentration or stress

Use biofeedback to help users attain or maintain a desired mental state

Progress so farSoftware MATLAB for processing raw incoming dataEEGsynth.org modules Openbci2ft (monopolar, bipolar), EEG playback

EEG instrumentation and hardwareEEG recordingOpenBCI architectureEEG recording by Raspberry Pi

TestcasePlay the sound of the power of alpha based on pre-recorded dataset

Next stepsAcquire multimodal 20 minutes dataset (EGG, ECG, EEG) for playback

Update Openbci2ft.c to allow recording from 8 to 16 channels

Online detection of eye blinks and saccades

EGG module (online detection of 0.05Hz waveform)

Heartbeat module

Physiological connectivity modules (e.g. Heart Brain coupling)

Obstacles so farSignal quality and preprocessing

Amplifier saturates sometimes

OpenBCI GUI shows good data, but data received in MATLAB seems very noisy

Optimal acquisition parameters for the EEGsynth?

Optimal filter settings (using MATLAB)?

Project Morpheus

Background

12These are the different kinds of spontaneous EEG signals we can reliably obtain. Different sleep stages. Sleep is important for memory. Particularly crucial for memory is stage 2 of sleep (appear arrow).

Targeted memory reactivation (TMR)

Background

13Targeted memory consolidation. Hear sound / music / smell odour associated with a task. Then play sound / music / odour during sleep to bias the ongoing dreams. This leads to better memory consolidation on the reactivated memories.TMR works best in the first hour of sleep (NOT REM), before deep stages take over.

What happens in stage 2?

silenceburstingbursting

14Delta wave = silence. Spindle = crazy activity.Theories: right after delta waves, the cortex is the most plastic (this is because plasticity thresholds are *relative*. Right after silence, a little bit of activity counts *a lot*).Theories: during crazy spindles, cortex is hyperactive (due to thalamus nucleus reticularis) and does not respond to other inputs much.

Hypothesis

silenceburstingbursting

15Delta wave = silence. Spindle = crazy activity.Theories: right after delta waves, the cortex is the most plastic (this is because plasticity thresholds are *relative*. Right after silence, a little bit of activity counts *a lot*).Theories: during crazy spindles, cortex is hyperactive (due to thalamus nucleus reticularis) and does not respond to other inputs much.

Hypothesis

16The hypothesis is that TMR works differently depending on the current EEG pattern. Lets say were doing TMR, were randomly playing sounds displayed on the bottom. At the same time, theres some EEG activity shown on top. The hypothesis is that sounds that fall at the right time relative to the EEG will be effective, and the sounds that fall in the wrong time will be ineffective.

Hypothesis

TMReffective

17Further, the hypothesis postulates that TMR sounds played directly following a delta wave will come in the right window to cause plasticity and thus would be most effective. The subject will remember the parts of the task associated with these sounds.

Hypothesis

TMReffectiveTMRnon-effective

18Moreover, sounds that are played *during* spindles will have little effect on the current cortical activity (units are overly active, bursting. The brain is screaming and our sound will be drowned in the ongoing activity). These sounds will be ineffective and subjects wont remember the parts of the task associated with these sounds better than if we didnt play the sounds at all.

do task with many associated sounds

separate sounds into 4 categories and play them during sleep: sounds after deltas sounds during spindles no sounds non-specific (random) sounds

test performance on task

Hypothesis

19

PlanTask designobtain object-sound pairscode task with GUI

EEG analysisget signal in real timemake sounds respond to detected eventsrecord sleep datamake online detection work using pre-recorded data (replayed)record exact timing of sounds to future analysistest sleep session: 4 sounds only (no task) analyse and debugExperimentrecruit ~5-10 subjectstestanalyse data :))

PlanTask designobtain object-sound pairscode task with GUI

EEG analysisget signal in real timemake sounds respond to detected eventsrecord sleep datamake online detection work using pre-recorded data (replayed)record exact timing of sounds to future analysistest sleep session: 4 sounds only (no task) analyse and debugExperimentrecruit ~5-10 subjectstestanalyse data :))

Brain-based music compositionLiburn Jupolli & Stephen Whitmarsh

Accent marks in music composition staccatostaccatissimomarcatoaccenttenuto

McFadden KL, Steinmetz SE, Carroll AM, Simon ST, Wallace A, et al. (2014) Test-Retest Reliability of the 40 Hz EEG Auditory Steady-State Response. PLOS ONE

B. Ross et al. J (2005) Stimulus Induced Desynchronization of Human Auditory 40-Hz Steady-State Responses. Neurophysiol.Kaongoen, Jo (2017) The effect of selective attention on multiple ASSRs for future BCI application. 5th International Winter Conference on Brain-Computer Interface (BCI)Auditory Steady-State response

Treble (guitar)Bass (guitar)

EEGsynth

45Hz AM35Hz AMAmplitudeAmplitude35Hz EEG45Hz EEG

Schematic design

Treble (guitar)Bass (guitar)

EEGsynth

45Hz AM35Hz AMAmplitudeAmplitude35Hz EEG45Hz EEGattention

Schematic design

Treble (guitar)Bass (guitar)

EEGsynth

45Hz AM35Hz AMAmplitudeAmplitude35Hz EEG45Hz EEGattention

Schematic design

Plan:Use outputCVgate device to create 0-10v modulation at 35/45 Hz. Use VCA to create amplitude modulationAnalyze relative difference in EEG amplitude at steady-state freq. Modulate VCA amplitude with outputCVgate deviceSo far:Creating outputCVgate device

EEG-synth projectPer HuttnerJean-Louis HuhtaRobert OostenveldStephen Whitmarshand many more

Some highlights in recent progressThoroughly tested EEGsynth installation instructions for Windows, OS-X, Linux (Raspian, Mint, Redhat)Pipeline for creating EEG music composition scoresPlug-and-play(back) of EEGsynth on Raspberry PiLive light control by EEGsynth on (Neopixel and DMX)

Offline playback of control signals based on EEG

Composition & MIDI score based on EEGusing LilyPad

Plug-and-playEEG-synthWiFi networkand remote access

EEG-controlledlight in visualart installation

March 16th April 16th

EEG-controlledlight in visualart installation

March 16th April 16th

Portable EEGsynth costumeby Emilia Rota, Stockholm

http://cri-paris.org/scientific-clubs/brain-control-club/