neurophone : brain-mobile phone interface using a wireless eeg headset
DESCRIPTION
NeuroPhone : Brain-Mobile Phone Interface using a Wireless EEG Headset. Andrew T. Campbell, Tanzeem Choudhury , Shaohan Hu , Hong Lu, Matthew K. Mukerjee !, Mashfiqui Rabbi, and Rajeev D. S. Raizada Dartmouth College, Hanover, NH, USA. Motivation. - PowerPoint PPT PresentationTRANSCRIPT
NEUROPHONE: BRAIN-MOBILE PHONE INTERFACE USING A WIRELESS EEG HEADSET
Andrew T. Campbell, Tanzeem Choudhury, Shaohan Hu, Hong Lu,Matthew K. Mukerjee!, Mashfiqui Rabbi, and Rajeev D. S. Raizada
Dartmouth College, Hanover, NH, USA
Motivation• Mobile phones and neural signals are present are
accessible to many people. • Recent advances in technology has led to the
development in low-cost EEG headsets. • Smart phones are now powerful enough to run
sophisticated machine learning algorithms.• It is thus easy to interface neural signals with mobile
computing paradigms.
Introduction• This group proposed to used neural signals to control a
mobile phone. • They developed the NeuroPhone system that translates
and decodes neural signals to drive a mobile app using off-the-shelf wireless EEG headsets.
• This paper demonstrates their brain-controlled address app:• An application that uses the brain signals to select address
contacts to call.
Introduction• They implement their mobile app using two different
paradigms: P300 dialing and “Wink”-triggered dialing. • P300 signals are positive transient deflections in EEG that are
elicited in response to a rare or novel stimulus• The eye “Wink” is a type of EMG signal that is generated in
response to the contraction of skeletal muscle contraction.
Challenges• Research Grade EEG headsets
• Expensive (Often costing tens of thousands of dollars)• Offer very robust and reliable EEG signals
• Off-the-shelf EEG headsets• More affordable ($100-$500)• Electrode design and amplification are not as robust
• Results in noisy, low-quality signals.• Require more sophisticated processing techniques to classify neural
events.• Most Off-the-shelf headsets are wireless and thus encrypt the EEG
signals. • They are designed for synchronization with a computer (using wireless
dongle). • They complicate the process of developing a clean brain-mobile
interface.
Challenges• There is an energy cost for brain-mobile interfacing:
• Continuously streaming raw brain-signals wirelessly• Running classifiers on the phone introduces heavy processor
loads. • Brain-mobile phones could likely be used in applications
such as: walking, riding in a car or bicycle, shopping, etc. • Many of these cases present significant noise artifacts in the EEG
signals. • These signals will need to be filtered out to improve the brain-mobile
interface
NeuroPhone• The NeuroPhone system uses the
iPhone to display pictures of contacts in the phone’s address book.
• The pictures are displayed and flashed in random order.
• For the EEG mode, the user concentrates on a picture of the person they wish to call.
• For the wink mode, the person winks with the left or right eye to make the intended phone call
P300• Whenever the user
concentrates on a target stimulus among a pool of non-target stimulus, the target stimulus (flash) will elicit a positive peak in the EEG at around 300ms after stimulus onset (P-300).
• The P300 signal can be found on most EEG channels• Common on central and parietal
channels
NeuroPhone - P300 Paradigm
• In This case, there are 6 total stimuli on the screen (5 non-target and 1 target). The user visually attends to one of the photos while each photo is flashed in a random order. Whenever the target photo flashes, a P300 should be generated.
Wireless EEG Headset• Emotiv EPOC headset
• 14 data electrodes (2 reference electrodes)• Transmits encrypted data wirelessly to a
windows-based machine. (802.11) 2.4GHz• Low SNR• Contains build in gyroscope• ~$300
Pre-Processing• Signals were band-passed filtered to keep only the
relevant information within the P300 range. • Signal averaging was performed to increase the SNR
• This improves the quality of the signal while simultaneously adding lag to the system
Classification• To reduce complexity, only a subset of relevant channels
are used for classification. • Wink Mode
• Multivariate, naive Bayesian classifier. • P300 Mode
• Decision stump classifer
Implementation• Laptop relay is used for decoding of the encrypted Emotiv
signals• Encrypted EEG signals are sent from the phone to a laptop for
decryption (via WiFi). • Decrypted EEG signals are sent back to the phone.• Signals are sampled at 128 samples per second and transferred to
the phone at 4kbps per channel.
Wink Mode Classification• Emotiv head-set was put on
backwards to place two electrodes directly above the eyes.
• Data was collected by having the subject wink multiple times. – Data were labeled as “wink” or “non-
wink”• A Bayesian classifier was trained by
calculating the mean and variance of each wink and non-wink and building respective Gaussian models. – As can be seen, the two models do not
overlap leading to good classification
P300 Classification• The Gaussian distributions overlap too much and
therefore cannot be classified with a Bayesian classifier. • Signals from each of the six stimuli were band-passed
filtered between 0-9Hz. • The highest signal segment at around 300ms after
stimulus onset is extracted. • For classification, a decision stump is used where the
threshold is set to the maximum value of the extracted segment.
• Multiple sessions were collected on three subjects. • Subjects performed the test while sitting and while walking• The classifier was trained on five sessions from a single
subject and then tested on the remaining subjects. (I think). • Results are shown in table 1
– Precision: % of classified winks that are actual winks– Recall: % of actual winks that are classified as winks. – Accuracy: % of total events that are classified correctly
Results (Wink-Mode)
Results (P300 mode)• Data was collected with same set of subjects while sitting,
with loud background music and while standing up.
Discussion• Although data was classified using the P300 mode, large
amounts of averaging is needed to get decent classification accuracies. • This “unresponsiveness” of the system proves to be very
frustrating for the end user. • i.e. it can take 100 seconds to initiate a phone call with only 89% chance
of dialing the right person (with six to choose from).
• This System is currently not in any form to be used by subjects on a regular basis. • Looking into single trial classification techniques to speed up the
system.
Phone Loading Statistics• The CPU usage when running the application:
• 3.3% for the iPhone (iphone 3g?). • Total memory usage:
• 9.40MB memory used • (9.14MB are for GUI elements).
• Continuous streaming raw EEG channels to the phone, and processing signals lead to battery drain (no quantitative measure given)• Looking into duty cycling to solve this phone.