[elearnica.ir]-improving the robustness of myoelectric pattern recognition for upper limb

1534-4320 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. Seehttp://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI10.1109/TNSRE.2015.2492619, IEEE Transactions on Neural Systems and Rehabilitation Engineering

IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING 1

Improving the Robustness of Myoelectric PatternRecognition for Upper Limb Prostheses by Covariate

Shift AdaptationMarina M.-C. Vidovic∗, Han-Jeong Hwang, Sebastian Amsuss, Janne M. Hahne, Dario Farina∗, and

Klaus-Robert Muller∗

Abstract—Fundamental changes over time of surfaceEMG signal characteristics are a challenge for myo-control algorithms controlling prosthetic devices. Thesechanges are generally caused by electrode shifts afterdonning and doffing, sweating, additional weight orvarying arm positions, which results in a change ofthe signal distribution – a scenario often referred to ascovariate shift. A substantial decrease in classificationaccuracy due to these factors hinders the possibilityto directly translate EMG signals into accurate myo-electric control patterns outside laboratory conditi-ons. To overcome this limitation, we propose the useof supervised adaptation methods. The approach isbased on adapting a trained classifier using a smallcalibration set only, which incorporates the relevantaspects of the nonstationarities, but requires only lessthan 1 min of data recording. The method was testedfirst through an offline analysis on signals acquiredacross 5 days from 7 able-bodied individuals and 4amputees. Moreover, we also conducted a three dayonline experiment on 8 able-bodied individuals and 1amputee, assessing user performance and user-ratingsof the controllability. Across different testing days, bothoffline and online performance improved significantlywhen shrinking the training model parameters by agiven estimator towards the calibration set parameters.In the offline data analysis, the classification accuracyremained above 92% over five days with the proposedapproach whereas it decreased to 75% without adapta-tion. Similarly, in the online study, with the proposedapproach the performance increased by 25 % comparedto a test without adaptation. These results indicate thatthe proposed methodology can contribute to improverobustness of myoelectric pattern recognition methodsin daily life applications.

Index Terms—Classification, EMG, myoelectric si-gnals, nonstationarities, covariate shift, adaptation,pattern recognition.

We thank the German Ministry for Education and Rese-arch (BMBF) via the Bernstein Focus Neurotechnology (BFNT)Gottingen and the European Commission via the Industrial Acade-mia Partnerships and Pathways (project AMYO, Grant No. 251555.)for their support.

∗ corresponding authorsMarina M.-C. Vidovic is with the Technical University of Berlin,

10587 Berlin, Germany, e-mail: [email protected]. Hwang is with TU Berlin.Sebastian Amsuss, Janne Hahne and Dario Farina are with Depart-

ment of Neurorehabilitation Engineering, Georg August University,37073 Gottingen, Germany

Klaus-Robert Muller is with TU Berlin and with Department ofBrain and Cognitive Engineering, Korea University, Seoul, Republicof Korea.

I. Introduction

PATTERN recognition is a promising approach incontrolling upper limb prosthetic devices with sur-

face EMG electrodes [1]–[4]. It enables the user to intui-tively control a myoelectric prosthetic hand with multipledegrees of freedom. However, despite the good laboratoryperformance of this approach, its clinical and commercialimpact is still limited. One of the reasons for this limitationis that laboratory conditions often do not include sourcesof EMG signal nonstationarity, such as electrode shiftsfollowing donning and doffing, changes in arm position,variable loads when grasping objects, muscle fatigue, andvarying electrode-skin impedance [5]. These factors have asignificant influence on the data distributions and thus onthe robustness of the system. Note that using implantedelectrodes could avoid electrode shifts, however with theknown complications of invasive methods.

Different strategies have been suggested to improve therobustness of EMG pattern recognition [6]. One approachhas been the inclusion of examples of nonstationarities inthe training set, which increases the generalization abilityof the classifier but requires a large training data set. Inaddition, there are factors that cannot be easily includedduring training, such as changes in the way the usersperform the attempted tasks. In [1], a full calibration wasperformed every day, which, however, is very demandingand time consuming for the user. To avoid this, adaptationstrategies have been explored [7], [8], where the modelparameters are updated by samples of the testing data. Ina fully unsupervised adaptive approach, the classificationaccuracy may however decrease, due to the inclusion ofmisclassified samples. To address this problem, Sensingeret al. [6] proposed to estimate the confidence of theclassifier decision, and then to use only the most reliabledecisions for adaptation. This approach was outperformedby all supervised adaptation methods, investigated in thesame study for comparison. Most of the work done onadaptation have been done offline on pre-recorded data.Although offline studies are useful to compare differentalgorithms while optimize their parameters, they reflectthe actual problem only partly. This is because offlinestudies cannot consider the fact that the user can react onthe algorithmic output and adapt his muscle contractionsto improve the outcome in a real-time application [9].

Downloaded from http://www.elearnica.ir




Hahne et al. [10] demonstrated the advantage of concur-rent adaptation between the user and the algorithm onmyoelectric control.

Extensive research on adaptive classification has beenperformed also in related fields. For example, covariateshift adaptation [11], [12] has been applied for binaryclassification problems in brain-computer interface appli-cations [13]. In this field, the adaptation method suggestedby Shenoy et al. [11] computed first a probability densityon the test set and then the expected test error overtest samples. This approach requires, however, a relativelylarge labeled calibration set, which would imply a largeeffort by the user.

In our present paper, for which preliminary results havebeen shown in [14], we address the problem of classifica-tion robustness across sessions and days by adapting amyocontrol algorithm, using a short calibration data setthat requires only less than 1 min data recording. We testthis approach on a large data set that includes both able-bodied subjects and amputees, in offline as well as onlineconditions. The results indicate that the proposed metho-dology is an appropriate compromise between a limiteduser effort in re-training and a substantial improvementin classification accuracy.

The remainder of the paper is organized as follows. InSection II we present the data sets, the mathematicalmodels, and the experimental paradigms used in thisstudy. The results were presented in Section III, and areafterwards discussed in Section IV before we conclude withsome final remarks in Section V .

II. METHODSA. Dataset

For the offline analysis, we used the five day datasetof the experiments presented in [15]. The data set wasobtained from seven able-bodied subjects (five males,two females 25.4±1.4 yrs) and four transradial amputees(males, ages 25, 28, 29, and 64 yrs). All amputees andtwo able-bodied subjects were experienced with myocon-trol, while the remaining able-bodied subjects were naive.Eight commercially available double differential electrodes(13E200=50AC Otto Bock Healthcare Products GmbH,Vienna, Austria) were used for the data recording. Theywere placed equidistantly around the forearm of the sub-jects approximately 7 cm from the olecranon. To mimic areal-life usage, for each amputee (two left and two rightside affected), the electrodes were fitted in an individu-al hard socket prosthesis. For able-bodied subjects theelectrodes were mounted on the dominant forearm by aspring-grid, which ensures that the electrodes were slightlypressed on the skin. The electrode positions were markedby a water resistant pen and the markers were used formounting the electrodes in the other days. The subjectswere seated on a chair, holding their forearm parallel tothe ground in a 90 degree angle to the upper arm and wereinstructed to perform 8 movements: wrist pronation (WP),wrist supination (WS), wrist extension (WE), wrist flexion

(WF), hand opening (HO), fine pinch (FP), key grip (KG),and no movement (NM) on five subsequent days (Mondayto Friday). To offer a more reliable and faster controllingof the prosthetic device to the user for both weak andvery strong movements, all movements were recorded atthree contraction forces (30, 60, and 90 % maximum long-term voluntary contraction (MLVC)). It has been recentlyshown [16] that training and testing at different force levelsis detrimental to classification performance, however pro-portional control is essential for good prosthesis handling.We introduce the following notations:• trial: one movement in a specific contraction level• run: all 8 movements in a specific contraction level• session: five runs in all contraction levels (3× 5× 8 =

120 trials)

MLVCDay 1 Day 4 Day 5

30

60

90

R1 R2 R3 R4 R5

• Training Data• Calibration Data• Test Data

Day 2 Day 3

30

60

90

Fig. 1: Illustration of the data segmentation, whereRn, n = 1, . . . , 5 includes the trials of all movements forone particular contraction level. The classifier was trainedon one day (exemplary on Day 1 (green)), adapted by onerun of the test day (orange) and tested on all other runs ofthe test day (blue). A reverse leave one out cross validationwas performed by taking each run once for adaptation(orange) and computing the performance on the remaining14 runs of the same day.

1) Offline experimental paradigm: Each subject perfor-med two 45-min session of data recordings on five subse-quent days. Between the sessions the amputees performeda donning and doffing, as they would do during daily use.For the able-bodied subjects, the donning and doffing wasmimicked by a lateral displacement of the electrodes by 0.8cm. Each session started with a calibration phase consistedin performing each movement for 30 s at 100% MLVC. Thereference value for normalizing the muscle activity was theaverage maximum root mean squared (RMS) value overthe eight channels RMS =

∑8p=1

√1l

∑li=1(ξp

i )2, wherel is the number of samples per time-window and ξp

l theinstantaneous EMG signal of channel p. Afterwards, eachof the 120 trials per session took 5-s, where the user follo-wed a trapezoidal force profile with a curser representingthe RMS while doing the instructed movement. For thefirst second, the user followed the trapezoidal ramp up tothe given force level, where the force level was maintainedfor 3 s before the user followed the ramp down to the No-Movement level. For the data analysis we only used the




3 s of signal at constant force. The division of the datainto training, test, and validation set was done as follows.The first session of the first day was used as training setand the first session of the subsequent days were usedas test sets. The second session of each day was used asthe validation set for the parameter optimization of ourproposed adaptation method (following standard cross-validation schemes see Lemm et al. [17]).

2) Signal acquisition and processing: The acquired rawsignals were amplified to the range 0-4.5 V and filteredin a bandwidth of 20-450 Hz. Moreover, a 50-Hz notchfilter was included in the active Otto Bock electrodes.The filtered signals were sampled at 1 kHz, digitized bya 10 bit A/D converter, and transferred via Bluetooth toa computer by the Axonmaster (Otto Bock HealthCareProducts GmbH Vienna, Austria).

3) Feature extraction: For this study we calculated thelogarithm of the signal variance (logVar), as proposed in[18], in intervals of 250 ms, which overlapped by 50 ms (15features per trial).

B. ClassificationFor the following considerations we are given a training

set X = {(xi, yi)ni=1}, with n ∈ N labeled d-dimensional

samples, thus xi ∈ Rd, d ∈ N, where yi ∈ {1, . . . , C}denotes the class allocation, respectively. Furthermore,let πc = 1/C be the prior probability for each classc ∈ {1, . . . , C} and fc(x) the class-conditional densityfunction of X, which we assume to be the multivariateGaussian distribution

fc(x) = 1√(2π)d|Σc|

exp(− 1

2 x>Σ−1

c x), (1)

where x := x− µc and µc is the class mean. In this study,we focused on two Bayesian multi-class classifiers: lineardiscriminant analysis (LDA) and quadratic discriminantanalysis (QDA) [19]. QDA determines quadratic boun-daries for class separation by using class-wise covariancematrices, whereas LDA eliminates the quadratic termsby assuming equal covariance matrices for all classes andthus uses hyperplanes for classification. We consider theproblem of a Bayesian multi-class classification: given anunknown point x, we want to allocate it to the class withthe largest posterior probability Pr(c|x) c = 1, . . . , C. [19].Circumventing the fact that the posterior probabilitiescannot be obtained directly, we estimate them by usingthe Bayes rule on the training data:

Pr(c|x) := fc(x)πc∑Cl=1 fl(x)πl

. (2)

We obtain the QDA disciminant δ1c (x) for each class c by

taking the natural logarithm of Equation (2) and rewriteit to

δ1c (x) := −1

2 log |Σc| −12 x>Σ−1

c x+ logπc. (3)

If we further assume that all classes share the samecovariance matrix Σ = 1

C

∑Cc=1 Σc, we can rearrange Eq.

(3) to the linear discriminant δ2c

δ2c (x) := x>Σ−1µc −

12µ>c Σ−1µc + logπc. (4)

An unknown point x∗ ∈ Rd is allocated by QDA (j = 1)and LDA (j = 2) to the class with the highest probability,respectively

y∗ = arg maxcδj

c(x∗), (5)

where y∗ ∈ {1, . . . , C}.These models work well under ideal laboratory conditi-

ons. However, in the presence of nonstationarities, the truedata distribution pte(x) (i.e. the test data distribution)almost never coincides with the training distribution ptr.

In the following, we propose an adaptation methodologyto adjust the training model parameters towards the para-meters of a very small calibration data set of the currentconditions.

C. AdaptationThe basis for the model adaptation is a small calibration

data set Xcal = {ti, ui}mi=1, m ∈ N, which follows

the test distribution Xcal ∼ pte, where ti ∈ Rd andui ∈ {1, . . . , C}. Let µtrc and Σtrc be the mean and thecovariance matrix of the training set X and µcalc andΣcalc the ones of the calibration set Xcal. We propose anadaptation by shrinking the training parameters towardsthe ones obtained from the calibration set [20]:

µc = (1− τ)µtrc + τµcalc, (6)

and

Σc = (1− λ)Σtrc + λΣcalc, (7)

where τ and λ ∈ [0, 1] are the regularization parameters.To get subject independent reasonable values for τ and λthey were estimated by a grid search with a step size of0.1 across all days and subjects on the validation data set(see II-D3 ). If there is only one shrinkage parameter tobe determined, the other is held at zero.

In the following analysis we denote LDA as the classifiertrained on the training data set that served as a baselinefor comparison with its adapted versions (same for QDA):

1) LDAMA: LDAM(ean)A(daptation) adapts the meanof LDA towards the mean of the calibration data set,where the covariance matrix remains unchanged.

2) LDACMA: In the LDAC(ovariance)M(ean)A(dapta-tion) method the mean and the covariance matrixof LDA are simultaneously adapted towards theparameters of the calibration data set as shown inEquation (6) and (7).

3) LDAnew: LDAnew is a new LDA classifier trainedon the calibration set only.

4) LDADEA: LDAD(ata)E(xtension)A(daptation) ad-apts the mean and the covariance matrix, equal toLDACMA, but after each daily adaptation towardsthe calibration data set, the calibration data set wasincorporated into the initially training data set and




the LDA, which is used for the ongoing adaptationwas retrained on the extended training data set.

5) LDAFA: Unlike LDACMA and LDADEA, whichused the non-adapted LDA trained on the trai-ning or extended training data set for adaptation,LDAF(urther)A(daptation) repeatedly adjusted thealready adapted LDA towards the calibration dataset.

D. Evaluation procedure1) Subject variability: To investigate the inter-subject

variability concerning false class label predictions we vi-sualized the non-diagonal elements of the confusion ma-trices, reshaped as a one-column vector for each subject.Moreover, to indicate the changes of the variability of falsepredictive class labels over days, we included the meanconfusion matrix of all days but training too.

2) General adaptation procedure: The data (first sessionof each day) were split as shown in Fig. 1, where the entiredata set of one day (green) was used as initial classifiertraining set. The other data set was further separated intoa calibration set that comprised one run (orange) and inthe testing set, consisting of the other 14 runs (blue). In afirst step, the LDA was trained on the entire training dataset. Afterwards, the LDA was adapted by the calibrationdata set. Finally the performance of the adapted LDA wascalculated on the test set. A reverse leave one out crossvalidation1 was performed by taking each run of one dayas calibration data set and computing the performance onthe remaining test set.

3) Optimal shrinkage parameter: First, we trained theLDA on the entire data set of the first day. We accomplis-hed the computation of the optimal shrinkage parameterson the validation data set, i.e. the second session of eachday, where we averaged over the daily results. To get asubject-independent adaptation of the LDA, we computedthe optimal shrinkage parameter values for τ and λ bya grid search that included the data from all subjects.We adapted the LDA by all possible combination pairsof τ and λ with elements in {0, 0.1 . . . , 1} towards onerun of the validation set (see II-D2). For the case ofLDAMA, the covariance adaptation was held to zero withλ = 0. For each parameter pair, we adapt with 15 differentruns for each day and each subject. To reveal the errorwe made for each subject by taking the average optimalshrinkage values over all participants instead of the singlesubject specific values we computed the Euclidean distancebetween both: LDACMA adapted with the subject averageregularization values and LDACMA adapted with thesubject-specific τ and λ values.

E. Online ExperimentTo evaluate the adaptation performance for a real time

application, we conducted an online experiment including

1A reverse leave one out cross validation means a data divisioninto N parts, where one is used for training and N-1 for testing.

one amputee and 8 able-bodied subjects. One able-bodiedsubject and the amputee were experienced with myocon-trol, while all other subjects were naive. The experimentconsisted of a training day (day zero) and three test days(day 1-3). The initial data recording on the training dayinvolved 3 trials for each of the 3 contraction strengthsfor the 8 movements (see II-A1) and served the LDA astraining data set.

On each test day, we recorded a small calibration dataset, where the user was instructed to perform each mo-vement in a perceived 60 % force level for 4 s once, withno feedback but a time progress bar. This data set wasused to train LDAnew and to adapt LDACMA, where thelatter was adapted by the optimal subject independentparameters found by the grid search (II-D3). To comparethe performance of LDA, LDAnew, and LDACMA, weconducted a 1-min online test, twice for each LDA va-riation. The user was unaware of the control algorithmtested in each case. During the 1-min test the user wasasked to copy a sequence of different target movements.Two pictures, side by side, were presented to the user on ascreen. The target movement was shown on the left accom-panied by a vocal cue. The subject was asked to replicatethe given target movement as fast as possible and wasprovided with visual feedback of the movement decodedby the algorithm under test (direct user feedback). A hitwas counted if the user reached the target movement andheld it for 1 s. Afterwards the next target movement wasshown on the left. In total, the user had 10-s to reach a hit.Note that in this study we use an error measure that relieson completion time, which is distinct from alternativemeasures that use completion rate [21]. After each testiteration, the subjects provided a subjective indication onthe comfort in controlling the specific system, on a scalebetween 1 and 10.

III. RESULTSFirst, we present the systematic class confusion within

the training day as well as within the subsequent daysfor all subjects. The significant performance drop on theother days motivate an adaptation. According to that, wepresent the results for the optimal shrinkage parameterchoice, which gave the basis for the further classifieradaptation computations. Subsequently, we report theoffline results on the five day data set for each adaptationstrategy. Then we present the influence of the contractionstrength to the adaptation performance and finally wegive the results of the online experiment. The informationabout accuracy and error on the figure axis are given inpercent.

A. Systematic class confusionWe investigated whether or not there exists a systematic

class dependency within the confusion matrix between thetraining day (TD) and the other days (OD). We observedan increased number of false predicted class labels in ODcompared to TD. This was also reflected in decreased




Fig. 2: Vectorial representation of the non-diagonal elements of the confusion matrix for the able-bodied subjects (HS1-7) and for the amputees (AP 1-4). The false predictive class labels are given as pair (M1, M2) on the y-axis, whereM1 is the real class and M2 the predicted class and the degree of misclassification is given by the color, where darkred indicates a high value and lighter colors a lower value of false positives. Moreover, the false predictive class labelsare shown for the training day (TD) in the first column and as average over all other days (OD) in the second columnfor each subject. The accuracy for each subject is reported on the top of each column for both TD and OD.

classification accuracy in OD (see accuracy on top of eachcolumn in Fig. 2). There was a significant dependencybetween the confusion matrix of TD and OD (Fisher’sexact test: p < 10−10). Furthermore we observed a highvariability of false predicted class labels over subjects(Fig. 2). The wrong predicated class labels of the bestperforming amputee, AP 1, is lower than the classificationaccuracy threshold of 0.5% and thus not visible. The otheramputees perform worse than the able-bodied subjects,which can be seen by the greater number of false predictedclasses.

B. Optimal shrinkage parameter choiceTo obtain subject-independent values for τ and λ we

computed the optimal shrinkage parameter by a gridsearch (Fig. 3), including all subjects as described inSection II-D3. The best values obtained for LDACMAwere τ = 0.8 and λ = 0.8. For the case of LDAMA, thecovariance adaptation was held to zero with λ = 0, whichgave an optimal mean shrinkage value of τ = 0.7.

The error between LDACMA adapted with the subjectaverage regularization values and LDACMA adapted with

the subject specific τ and λ values is shown in Fig. 4 foreach subject. For the able-bodied subjects (subjects 1 to7) the mean error was lower than 2%. For the amputee A1,the shrinkage parameters were almost perfect (err < 1%),whereas the error for the amputee A3 was comparativelylarge (err ≈ 8%). For the other two amputees the meanerrors were approximately 4%.

C. Covariate shift adaptationIn a first step, we trained a LDA classifier on the

entire data set of the first day session of the able-bodiedsubjects. Using a 5-fold cross validation, we achieved highclassification results with an accuracy > 95% when testingthe same subjects in the same day. When testing theLDA classifier on the following days without retraining,however, the performance dropped substantially by almost20%, as shown in Fig. 5 a). Further, we investigated theperformance of the proposed adaptation methods. Theoptimal shrinkage parameter values for τ and λ (see Eq.(6) and Eq. (7)) were obtained by a grid search, as shownin Fig. 3).

We first evaluated the mean adaptation, where we




0

0.2

0.4

0.6

0.8

1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

τλ

accu

racy

0.68

0.7

0.72

0.74

0.76

0.78

0.8

0.82

Fig. 3: Illustration of the grid search including the elevensubjects for finding the optimal adaptation parametersτ and λ for LDA. A continuous performance increasingwas preserved by shrinking the mean of the training setversus the mean of the calibration set, where the optimalvalue was τ = 0.7 (λ = 0). By shrinking additionally thecovariance matrix, the optima were found by τ = 0.8 andλ = 0.8, a further, but relatively small performance gainwas obtained.

1 2 3 4 5 6 7 A1 A2 A3 A4

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

subject

err

or

Fig. 4: Illustration of the subject specific errors whenadapting the LDA by the mean regularization parametervalues extracted from the grid search including all subjectsinstead of the best parameter choice for the respectivesubject. The able-bodied subjects were tagged by 1-7 andthe amputees by A1-A4.

shrank the mean of the LDA towards the mean of the ca-libration set (LDAMA: (τ = 0.7, λ = 0)). Afterwards, weextended the mean adaptation by an additional covariancematrix regularization (LDACMA: (τ = 0.8, λ = 0.8)).Finally, we trained a LDA on the small calibration setonly (LDAnew: (τ = 1, λ = 1)). Regarding the baselineLDA performance, the LDAMA significantly improved theclassification accuracy for each day. Although the LDAperformance continuously decreased from day 2 to day4, the LDAMA performance remained constant above90%. Compared to the mean adaptation, LDACMA could

not gain any further performance increase. The LDAnewalso significantly improved the LDA performance, butperformed worse than both adaptation methods (see Fig.5 a).

A similar trend was observed with the results of theamputee subjects, although the classification accuracy waslower by more than 10% compared to the able-bodied sub-jects (see Fig. 5 b). Contrary to the results on able-bodiedsubjects, LDACMA significantly outperformed LDAMA inamputees. LDAnew had worse performance than LDAMAfor the first two days but showed a similar performanceafterwards.

We performed a two-way repeated ANOVA test, in-cluding all eleven subjects, to compute the statisticaldifferences between the methods. The statistical test re-sults showed that LDAMA and LDACMA significantlyoutperform LDA (p = 0.03 and p = 0.016), whereas nosignificant difference was obtained when comparing LDAand LDAnew. We repeated the equivalent experimentfor QDA, where the optimal shrinkage parameters forthe mean adaptation (QDAMA: (τ = 0.8, λ = 0)) andthe covariance matrix adaptation (QDACMA τ = 0.7,λ = 0.9)) were extracted by a grid search (similar to theresults of the LDA grid search as shown in Fig. 3). Forthe able-bodied subjects, we observed similar behaviorsfor QDA, QDAMA, QDACMA, and QDAnew comparedto the LDA counterparts. The QDA performance droppedfrom 97% on the training day to 70%. Adapting themean, QDAMA attained a significant performance raiseof 20%. Moreover, unlike the additional LDA covariancematrix adaptation, QDACMA significantly outperformedQDAMA. QDAnew reached no significant improvement(Fig. 5 c).

From the QDA results of the amputee data shown inFig. 5 d), we observe again that QDACMA performedbest, followed by QDAMA. Compared to the LDA results,QDAnew performed significantly worse than QDAMA.Also in the case of QDA, the statistical test results on allsubjects show that QDAMA and QDACMA outperformQDA (p = 0.013 and p = 0.010), and no significantdifference was observed between QDA and QDAnew.

The most relevant difference between LDA and QDAwas that the LDA was more robust against inter-daynonstationarities. For this reason, the subsequent resultsare presented for the LDA only.

The above results were presented for the adaptation ofthe initial classifier trained on the training day with thecalibration set of the current day. However, we need toclarify whether it is worthwhile for an ongoing classifieradaptation to extend the training data set by the calibra-tion data (LDADEA) or further adapt LDACMA insteadof the initial LDA (LDAFA). The results showed thatLDACMA significantly outperformed LDAFA (p < 10−4,Fig. 6 a)), whereas there was no significant differencebetween LDADEA and LDACMA (Fig. 6 b)).

In order to provide a direct comparison between our ap-proach and the generally accepted state-of-the art featureset [4], the Hudgin feature set [3], we further investigated




Mo Tu We Th Fr0.4

0.5

0.6

0.7

0.8

0.9

1

day

accura

cy

LDA

LDAMA

LDACMA

LDAnew

(a) LDA able-bodied subjects

Mo Tu We Th Fr0.4

0.5

0.6

0.7

0.8

0.9

1

day

accura

cy

LDA

LDAMA

LDACMA

LDAnew

(b) LDA amputees

Mo Tu We Th Fr0.4

0.5

0.6

0.7

0.8

0.9

1

day

accura

cy

QDA

QDAMA

QDACMA

QDAnew

(c) QDA able-bodied subjects

Mo Tu We Th Fr0.4

0.5

0.6

0.7

0.8

0.9

1

day

accura

cy

QDA

QDAMA

QDACMA

QDAnew

(d) QDA amputees

Fig. 5: Illustration of LDA and QDA adaptation performance for the able-bodied subjects in a) and c) as well as forthe amputees in b) and d). In all cases, a remarkable improvement of classification accuracy across days is obtainedby the mean adaptation. The additional adaptation of the covariance matrices only results in a slight improvementin the case of QDA. For comparison, the classifier trained exclusively on the new calibration data slightly improvesthe classification accuracy but performs worse than the adapted classifiers. Note that the variation between subjectsis plotted as standard error.

0 0.5 10

0.5

163.6%

11.4%

p=6.04e−05**

accuracy (LDAFA)

accu

racy (

LD

AC

MA

)

(a) LDACMA vs LDAFA

0 0.5 10

0.5

136.4%

38.6%

p=0.406

accuracy (LDADEA)

accu

racy (

LD

AC

MA

)

(b) LDACMA vs LDADEA

Fig. 6: Comparison between the performance of threeadaptation strategies LDACMA, LDAFA, and LDADEAfor all subjects and all days but the first day, which wasused for training.

the influence of the feature choice and found an averageimprovement of 25% for LDA and of 11% for LDACMAwhen using logvar instead of the Hudgin feature set.

D. Influence of force levelWe repeated the analyses described above, when adap-

ting with the different contraction strengths (30, 60 and90%). Additionally, we used each day once for training andall other days for testing.

Mo Tu We Th Fr0.7

0.75

0.8

0.85

0.9

day

accu

racy

LDACMA 30

LDACMA 60

LDACMA 90

LDA

Fig. 7: The daily comparison of the LDACMA performan-ce when adapting with the single contraction strengths of30, 60 and 90%, respectively. Using a calibration set with60% contraction strength for adapting LDA indicated thebest performance.

In Fig. 7 the mean LDACMA performance of the testdays are shown and compared to the baseline LDA per-formance. We performed a two-way repeated ANOVA testover all subjects to make a general statement about thestatistical differences between the methods. We observe




0 5 10 15 200

5

10

15

20

96.3%

0%

p=8.17e−06

**

correct movements with LDA

corr

ect m

ovem

ents

with L

DA

CM

A**

(a) LDACMA vs LDA

0 5 10 15 200

5

10

15

20

92.6%

3.7%

p=1.24e−05

**

correct movements with LDA

corr

ect m

ovem

ents

with L

DA

new

**

(b) LDAnew vs LDA

0 5 10 15 200

5

10

15

20

70.4%

14.8%

p=0.00902

**

correct movements with LDAnew

corr

ect m

ovem

ents

with L

DA

CM

A

*

(c) LDACMA vs LDAnew

Fig. 8: Illustration of the 1-min online test results, where the subjects (8 able-bodied and 1 amputee) were asked toimitate certain movements for one second. In total, two tests (runs) were recorded on 3 different days (sessions). Thescatter plots illustrate the number of correct classified movements by LDA and LDACMA against each other. Eachcircle corresponds to the mean results of one session for one subject, where the amputee data is marked by green circles.The percentage on each side of the partition line represents the amount of results of the respective method.

that an adaptation with 60% contraction strength signi-ficantly outperformed the baseline LDA (p = 0.019),whereas no significance difference was observed for anadaptation with 30 or 90% contraction strength (p = 0.603and p = 0.082). Among the adaptation models, a 60%adaptation only outperformed the 30% adaptation (p =0.012) but not the 90% one.

E. Online ExperimentDuring the 1-min online tests, we counted the number

of correctly imitated movements that should be held by asubject for 1 s. On three test days, each algorithm (LDA,LDACMA, and LDAnew) was tested twice.

First, we plotted the performance of the methodsagainst each other as shown in Fig. 8. From Fig. 8 a) and b)we observe that LDACMA as well as LDAnew significantlyoutperformed LDA for all able-bodied subjects (blue dots)as well as for the amputee (green dots). When comparingLDACMA and LDAnew, we observe that for 70,4% of thesubjects LDACMA performed better than LDAnew, whereonly 14,8% of the subjects got a performance gain (Fig.8 c)). The test run, which benefited the most from ourapproach (LDACMA) is highlighted in red, once for theable-bodied subjects and once for the amputee.

For further analysis, we additionally incorporated theperformance of LDAnew as well as the user controllabilityratings of each method. In order to be able to show si-multaneously those joint effects, we standardized the data.This was done subject-wise by z-scoring, i.e. first removingthe mean and then dividing by the standard deviation.Each line in the polar plot shown in Fig. 9 indicatesone run of one subject. The algorithm performance isreported on the x-axis and the user controllability ratingon the y-axis. The majority of the subjects valued thecontrollability using LDA worse than using LDACMA orLDAnew. More precisely, 96% of the LDA results pointedin the direction of decreased user-rating, whereof 94% alsopointed in the direction of decreased performance. On thecontrary we record for LDACMA an increased user rating

1

2

3

4

LDA

LDACMA

LDAnew

Fig. 9: The classifier performance (x-axis) is plottedagainst the user controllability rating (y-axis) for LDA,LDACMA, and LDAnew. To show simultaneously thosejoint effects, we standardized the data by subject-wise z-scoring. Each line in the polar plot indicates one sessionrun of one subject.

of 96% and 87% for LDAnew. Even when the LDACMAperformance decreased, the user still perceived a pleasantcontrollability unlike LDAnew, where the user rated thealgorithm lower although an increased performance wasrecorded.

In order to visualize the joint effects of performance anduser-rating for each subject individually, we reported bothagainst each other as shown in Fig. 10. In Fig. 10 a) thedifference between achieved user-hits with LDACMA andLDA is reported on the x-axis and the difference betweenuser controllability ratings of LDACMA and LDA on they-axis. The test results of each subject were highlighted bydifferent colors, where subject 2, colored in orange indica-ted the test results of the amputee. The user performancewas positively associated to the user controllability acrossthe subjects. Additionally, Fig. 10 c) shows that for all test




0 2 4 6 8 101

2

3

4

5

6

7

user−hits of LDACMA−LDA

us

er−

rati

ng

s o

f L

DA

CM

A−

LD

A

subj 1

subj 2

subj 3

subj 4

subj 5

subj 6

subj 7

subj 8

subj 9

(a) LDACMA vs LDA

0 2 4 6 8 100

1

2

3

4

5

6

user−hits of LDAnew−LDA

us

er−

rati

ng

s o

f L

DA

ne

w−

LD

A

subj 1

subj 2

subj 3

subj 4

subj 5

subj 6

subj 7

subj 8

subj 9

(b) LDAnew vs LDA

−1 1 2 3 4

−0.5

0.5

1

1.5

2

2.5

user−hits of LDACMA−LDAnew

user−

rati

ng

s o

f L

DA

CM

A−

LD

An

ew

subj 1

subj 2

subj 3

subj 4

subj 5

subj 6

subj 7

subj 8

subj 9

(c) LDACMA vs LDAnew

Fig. 10: Subject specific behavior when comparing the classifier performance with the controllability rating. Theresults are shown as differences between the two methods, respectively, where the latter is subtracted from the first.Examplary, in a), the x-axis show the hit difference between LDACMA and LDA, hence: hits of LDACMA minus hitsof LDA (same for the user controllability ratings). Each circle indicates the average result of all session runs for onesubject, where each subject is colored differently.

iterations except one, LDACMA outperformed LDA (sameas shown in Fig. 8). We observed a similarity in resultsbetween LDAnew and LDA as shown in Fig. 10 b). Whencomparing LDACMA with LDAnew 7 of 9 subjects hadboth higher performance and higher user controllabilityrating with LDACMA. Although one able-bodied subjectperformed better with LDAnew, there was no differencebetween LDACMA and LDAnew in the user controllabilityrating. For one subject, there was no significant differencebetween the performance with LDACMA or LDAnew, butthe subject preferred the controllability with LDACMA.

IV. DISCUSSIONLDA was more robust than QDA for both able-bodied

subjects and amputees. This may be caused by the factthat QDA is highly dependent on the class-wise covariancematrices, which were estimated with a relatively smallnumber of samples and therefore less stable and proneto overfitting. Also little changes in EMG signals maydetermine large changes in the data distributions, with theconsequence that the QDA quadratic boundaries mightfail in movement classification. Conversely, LDA uses thepooled covariance matrix, where the nonstationarities ofthe EMG signals have little influence. This was shownby the fact that the classification accuracy over days ofLDA decreased less than for QDA on the one hand (Fig.5). On the other hand, the additional covariance matrixadaptation had less improvement for LDA than for QDA,since LDACMA performed similar to LDAMA. Our resultsare conform with the findings in [7], which representedthe mean and the covariance matrix adaptation, firstseparated from each other and then together. Also inthat study, the class mean adaptation gave the largesteffect in performance compared to the covariance matrixadaptation for LDA. The results suggest that for real-world EMG control applications LDA might be a betterchoice than QDA since it is more stable over time.

The presented adaptation method, where we adaptedthe trained classifier towards a short re-calibration set

showed a high performance gain, both in offline and onlineanalysis.

The LDACMA adaptation with 60% force level perfor-med best since in general the overall mean, consistingof data with 30, 60, and 90% contraction strengths, ismost closely to the data with 60% contraction strength.In the offline analysis LDAnew outperformed LDA butperformed significantly worse than LDACMA. In the on-line analysis, although the performance of LDAnew wascomparatively similar to LDACMA (in average LDACMAachieved one hit more than LDAnew), the user rated thecontrollability with LDACMA higher, which is shown inFig. 10 c). Although, one able-bodied subject performedbetter with LDAnew, there was no difference for him/herin the controllability between LDACMA and LDAnew.For one able-bodied subject there was no difference in theperformance between LDACMA and LDAnew, however,he/she rated the controllability with LDACMA highercompared to LDAnew. Furthermore, the results from Fig.8 c) indicate that LDACMA significantly outperformedLDAnew (p < 0.01). The interesting observation thatLDAnew performed better in the online than in the offlineanalysis can be explained by an adaptation of the user,and by the fact that only one specific contraction levelwas used for training and testing the classifier in the onlineexperiment.

The main difference in performance between able-bodied and amputee users was a lower classification accu-racy for the amputees, which is also reported in [15], [22],[23] and emphasized in Fig. 2 by the false predicted classlabels. Moreover, the variability of results was significantlygreater for amputees. This is due to the fact that able-bodied subjects performed actual movements of the handwhile amputees could only attempt it without a visual andsensory feedback from the hand. In amputees, the absenceof visual feedback on the actual hand movement may alsocause the impossibility of generating well distinguishedmuscle activations for the different tasks. It should benoted, that on average no absolute recognition accuracy




difference was found between the experienced and inexpe-rienced subjects, since the data set we used incorporatedan extensive training (2 hours for 5 days) [15].

Unsupervised adaptation is sensitive to wrong adapta-tion, as observed, e.g., in [8] where all the unsupervisedadaptation variations tested showed an increased errorrate over time. Thus, fully unsupervised adaptation isnot a robust approach at least not until the influenceof misclassification is minimized by a high confidence injudging the accuracy of the test data.

Our approach thus provided a reasonable trade-off bet-ween high classification accuracy over days and a minimaleffort by the user in re-training; we therefore consider it apractical method in real world EMG control applications.In this paper we considered the same adaptation for allclasses. However, there are movements that are classifiedwith lower accuracy than others as shown in Fig. 2. Notethat the nonstationarities influence the class distributionsdifferently. In future work, it will be interesting to adaptparameters differently for each class and to determinesubject-specific optimal shrinkage parameters (Fig. 4).Future work will also extend of our approach also toregression based myoelectric control [23], [24].

V. CONCLUSIONSPattern recognition is of high benefit for controlling

myoelectric prosthetic devices but can lack robustnesswhen tested in daily-life conditions. Offline and onlineresults of our proposed adaptation of the initial classifiertowards a short, less than 1 min newly recorded data setdemonstrated the significant gain in classification accuracyfor both, able-bodied subjects and amputees over days.Furthermore, the relative improvement trends achievedwith our proposed methodology were the same for allsubjects, which underlines the relevance of our methodfor both experienced and novel users. In conclusion, theproposed adaptation approach can be used as a practicallyfeasible method for improving the robustness of patternrecognition for myocontrol.

Literatur[1] E. Scheme and K. Englehart, “Electromyogram pattern recogni-

tion for control of powered upper-limb prostheses: state of theart and challenges for clinical use.” Journal of RehabilitationResearch & Development, vol. 48, no. 6, 2011.

[2] L. J. Hargrove, E. J. Scheme, K. B. Englehart, and B. S.Hudgins, “Multiple binary classifications via linear discriminantanalysis for improved controllability of a powered prosthesis,”Neural Systems and Rehabilitation Engineering, IEEE Transac-tions on, vol. 18, no. 1, pp. 49–57, 2010.

[3] K. Englehart and B. Hudgins, “A robust, real-time controlscheme for multifunction myoelectric control,” IEEE Trans.Biomed. Eng., vol. 50, no. 7, pp. 848–854, 2003.

[4] B. Hudgins, P. Parker, and R. N. Scott, “A new strategyfor multifunction myoelectric control,” Biomedical Engineering,IEEE Transactions on, vol. 40, no. 1, pp. 82–94, 1993.

[5] N. Jiang, S. Dosen, K.-R. Muller, and D. Farina, “Myoelectriccontrol of artificial limbs-is there a need to change focus,” IEEESignal Processing Mag., vol. 29, no. 5, pp. 149–152, 2012.

[6] J. W. Sensinger, B. A. Lock, and T. A. Kuiken, “Adaptive pat-tern recognition of myoelectric signals: exploration of conceptualframework and practical algorithms,” IEEE Trans. Neural Syst.Rehab. Eng., vol. 17, no. 3, pp. 270–278, 2009.

[7] X. Chen, D. Zhang, and X. Zhu, “Application of a self-enhancingclassification method to electromyography pattern recognitionfor multifunctional prosthesis control,” Journal of neuroengi-neering and rehabilitation, vol. 10, no. 1, p. 44, 2013.

[8] J. He, D. Zhang, and X. Zhu, “Adaptive pattern recognition ofmyoelectric signal towards practical multifunctional prosthesiscontrol,” in Intelligent Robotics and Applications. Springer,2012, pp. 518–525.

[9] N. Jiang, I. Vujaklija, H. Rehbaum, B. Graimann, and D. Farina,“Is accurate mapping of emg signals on kinematics needed forprecise online myoelectric control?” Neural Systems and Reha-bilitation Engineering, IEEE Transactions on, vol. 22, no. 3, pp.549–558, 2014.

[10] J. Hahne, S. Dahne, H. Hwang, K.-R. Muller, and L. Parra,“Concurrent adaptation of human and machine improves simul-taneous and proportional myoelectric control,” Neural Systemsand Rehabilitation Engineering, IEEE Transactions on, 2015.

[11] M. Sugiyama, M. Krauledat, and K.-R. Muller, “Covariateshift adaptation by importance weighted cross validation,” TheJournal of Machine Learning Research, vol. 8, pp. 985–1005,2007.

[12] M. Sugiyama, T. Suzuki, S. Nakajima, H. Kashima, P. Bunau,and M. Kawanabe, “Direct importance estimation for covariateshift adaptation,” Annals of the Institute of Statistical Mathe-matics, vol. 60, no. 4, pp. 699–746, 2008.

[13] C. Vidaurre, M. Kawanabe, P. Von Bunau, B. Blankertz, andK. Muller, “Toward unsupervised adaptation of lda for brain–computer interfaces,” Biomedical Engineering, IEEE Transac-tions on, vol. 58, no. 3, pp. 587–597, 2011.

[14] M. M.-C. Vidovic, L. P. Paredes, H.-J. Hwang, S. Amsuss,J. Pahl, J. M. Hahne, B. Graimann, D. Farina, and K.-R. Muller,“Covariate shift adaptation in emg pattern recognition for pros-thetic device control,” in Engineering in Medicine and BiologySociety (EMBC), 2014 36th Annual International Conferenceof the IEEE. IEEE, 2014, pp. 4370–4373.

[15] S. Amsuss, P. M. Gobel, N. Jiang, B. Graimann, L. Paredes, andD. Farina, “Self-correcting pattern recognition system of surfaceemg signals for upper limb prosthesis control,” Biomedical En-gineering, IEEE Transactions on, vol. 61, no. 4, pp. 1167–1176,2014.

[16] A. Al-Timemy, R. Khushaba, G. Bugmann, and J. Escudero,“Improving the performance against force variation of emgcontrolled multifunctional upper-limb prostheses for transradialamputees,” 2015.

[17] S. Lemm, B. Blankertz, T. Dickhaus, and K.-R. Muller, “Intro-duction to machine learning for brain imaging,” Neuroimage,vol. 56, no. 2, pp. 387–399, 2011.

[18] J. Hahne, B. Graimann, and K.-R. Muller, “Spatial filtering forrobust myoelectric control,” IEEE Trans. Biomed. Eng., vol. 59,no. 5, pp. 1436–1443, 2012.

[19] T. Hastie, R. Tibshirani, and J. Friedman, The elements ofstatistical learning. Springer, 2009, vol. 2, no. 1.

[20] B. Blankertz, S. Lemm, M. Treder, . Haufe, and K.-R. Muller,“Single-trial analysis and classification of ERP components - atutorial,” NeuroImage, vol. 56, no. 2, pp. 814–825, 2011.

[21] T. A. Kuiken, G. Li, B. A. Lock, R. D. Lipschutz, L. A. Miller,K. A. Stubblefield, and K. B. Englehart, “Targeted musclereinnervation for real-time myoelectric control of multifunctionartificial arms,” Jama, vol. 301, no. 6, pp. 619–628, 2009.

[22] Y. Huang, K. B. Englehart, B. Hudgins, and A. D. Chan, “Agaussian mixture model based classification scheme for myo-electric control of powered upper limb prostheses,” BiomedicalEngineering, IEEE Transactions on, vol. 52, no. 11, pp. 1801–1811, 2005.

[23] H.-J. Hwang, J. M. Hahne, and K.-R. Muller, “Channel selectionfor simultaneous and proportional myoelectric prosthesis controlof multiple degrees-of-freedom,” Journal of neural engineering,vol. 11, no. 5, p. 056008, 2014.

[24] J. Hahne, F. Biessmann, N. Jiang, H. Rehbaum, D. Farina,F. Meinecke, K.-R. Muller, and L. Parra, “Linear and nonlinearregression techniques for simultaneous and proportional myo-electric control,” IEEE Trans. Neural Syst. Rehab. Eng., vol. 22,no. 2, pp. 269–279, 2014.

[elearnica.ir]-improving the robustness of myoelectric pattern recognition for upper limb

Documents

user performance

sebastian amsuss

neural systems

janne hahne

dario farina

emg signals

tu berlin

oine analysis