deep learning penmiir - sebastian stober · 1 chim chim cheree 3/4 210 13.3 13.5 2 take me out to...
TRANSCRIPT
-
Research FocusCognitive Sciences
Learning Discriminative Features from EEGRecordings by Encoding Similarity ConstraintsSebastian Stober
MLC g Research Focus Cognitive SciencesMachine Learning in Cognitive Science Lab
MLC g
songs with / without lyrics: meter tempo length (s)1 Chim Chim Cheree 3/4 210 13.3 13.52 Take me out to the Ballgame 3/4 186 7.7 7.73 Jingle Bells 4/4 200 9.7 9.04 Mary Had a Little Lamb 4/4 160 11.6 12.2
instrumental pieces:1 Emperor Waltz 3/4 178 8.32 Harry Potter Theme 3/4 166 16.03 Imperial March (Star Wars Theme) 4/4 104 9.24 Eine Kleine Nachtmusik 4/4 140 6.9
penMIIRpublic domain dataset of EEG recordingsfor music imagery information retrieval
This research was supported by the donation of a Geforce GTX Titan X graphics card.
Classifier Features Accuracy McNemar’s test (mid-p) vs. *
SVC raw EEG 18.52% 0.0002SVC raw EEG channel mean 12.41% < 0.0001End-to-end NN raw EEG 18.15% 0.0001Neural Net (NN) SCE output 27.22% 0.8200*SVC SCE output 27.59%
Stimulus Recognition (12-class classification)• minimalistic encoder: 1 spatial convolution filter• nested cross-validation scheme:
Deep Learning for EEG Analysis?
• high-dimensional data
• limited amount of trials
• bad signal-to-noise ratio
https://github.com/sstober/openmiir
• passive listening condition• 64 EEG channels @ 512 Hz• 9 subjects x 12 stimuli x 5 trials• all trials cropped at 6.9 s
• 12 music stimuli from 8 music pieces:
540 trials (~ 62 min total) with > 225k dimensions
9-foldsubjectcross-validationtrainondatafrom8subjects(8x5x12=480trials)
testonremainingsubject(1x5x12=60trials)
Pre-Training
5-foldtrialblockcross-validation8x4x12=384trainingtrialsfrom4trialblocks
8x1x12=96validationtrialsfromremainingtrialblock
trainon50688tripletsfrom384trainingtrials
selectmodel(earlystopping)basedon21120validationtriplets(a,b,c)witha from96validationtrialsandb,c from480trainingandvalidationtrials
encoderlayer(L1)=averageoverfolds
Supe
rvise
dTraining 5-foldtrialblockcross-validation
sametraining/validationsplitsasinpre-trainingphase
trainwithCon480trainingtrials classifierlayer(L2)=meanoverfolds
LinearSupportVectorClassifier(SVC)gridsearch forCwithhighest
meanvalidationaccuracy
SimpleNeuralNetwork(NN)selectfoldmodel(earlystopping)
basedonhighestvalidationaccuracy
Chim Chim Cheree (lyrics)Take Me Out to the Ballgame (lyrics)
Jingle Bells (lyrics)Mary Had a Little Lamb (lyrics)
Chim Chim ChereeTake Me Out to the Ballgame
Jingle BellsMary Had a Little Lamb
Emperor WaltzHedwig’s Theme (Harry Potter)
Imperial March (Star Wars Theme)Eine Kleine Nachtmusik
minimizeconstraintviolations
FeatureExtraction(signalfilter)
InputTriplet
(sharedparameters)
PairwiseSimilarity(dotproduct)
Prediction(binary)
EncoderPipeline
EncoderPipeline
EncoderPipeline
Similarity
Similarity
Output
Reference
PairedTrial
OtherTrial
netw
ork
stru
ctur
e
A
B
C
• systematic errors: confusing lyrics/non-lyrics pairs• little difference between classifiers
Average Weights of the Neural Network Classifier(including pre-trained encoder filter)
Network Activation Analysis
http://www.uni-potsdam.de/mlcog
• temporal patterns in weights and activations are picking up musical events (down beats)
forward model:(regression)
• dimensionality reduction by factor 64• significant improvement of signal-to-noise ratio• more complex encoders and classifiers possible• pre-training is key to reduce over-fitting
goo.gl/4xzma3
Comparison against Baselines
Confusion Analysis
• common encoder pipeline extracts features that are representative and allow to distinguish classes
sim(A,B) > sim(A,C)
Cha
lleng
e
Dat
aset
Met
hod
Expe
rimen
t
Res
ults
for all A,B from same class and C from other class
Similarity-Constraint Encoding (SCE) Pre-Training• motivated by metric learning• find features that satisfy similarity constraints