Adap%ve Interac%ve Learning: a Novel Approach to Training Brain-‐Computer Interface Systems
Supervised by Konstan%n Tretyakov
Ilya Kuzovkin
University of Tartu, 2013
Mental inten%on
Brain-‐Computer Interface
Mental inten%on
Brain-‐Computer Interface
Neuroimaging
Mental inten%on
Signal
Brain-‐Computer Interface
Neuroimaging
Mental inten%on
Signal
Representa%on
Brain-‐Computer Interface
Neuroimaging
Mental inten%on
Signal
Representa%on Algorithm
Brain-‐Computer Interface
Neuroimaging
Mental inten%on
Signal
Representa%on Algorithm
With 87% certainty I can say that you are thinking “LeQ” right now
Brain-‐Computer Interface
Neuroimaging
Tradi%onal Approach
• User has to pre-‐decide which thoughts he will use to control the machine • No feedback during the learning process
Two Problems 2-‐class accuracy
3-‐class accuracy
≈ 0.9
≈ 0.7
Inconsistency of the signal
Producing
Dis5nguishable mental states
Abstract Concept: Communica%on
You want to move “LeQ”?
Yes!
Establishing communica%on protocol
Adap%ve Interac%ve Learning • Data is processed in real-‐5me • AQer each sample both the user and the machine get feedback • User can adapt his behavior according to feedback • Machine can update the predic%ve model
Self-‐Organizing Map
• Unsupervised learning algorithm
• Map consists of units
• Weight vector w(a) maps units to feature space
• For a new data vector x we find the best matching unit: the one, which has weight vector w(a) closest to x
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
Predic%ve SOM • Class probability vector • Probabilis%c confusion matrix • F1 score provides the feedback for the machine • Feedback to the user is based on the class probabili%es
}
Feedback for the machine
}
Feedback for the user
The Experimenter
Tradi%onal 0.696 Tradi%onal 0.352
Adap%ve 0.85 Adap%ve 0.418
p-‐value 0.0075 p-‐value 0.06
Tradi%onal vs. Adap%ve: Real Data Facial expressions Mental states
Summary • Proposed a new approach to BCI training
• Implemented the idea using Predic%ve SOM: unsupervised online learning, one parameter
• Applica5on which embodies the new approach
• Demonstrated the advantage of the adap%ve approach on ar5ficial data
• Experimental results: • 0.07 F1 score increase for mental states • 0.15 F1 score increase for facial expressions
