fine muscular activity recognition using electromyographyrzheng/course/cas765fa15/lectures/... ·...
TRANSCRIPT
![Page 1: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/1.jpg)
FinemuscularactivityrecognitionusingElectromyography
AlaShaabana
![Page 2: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/2.jpg)
Outline• WhatisElectromyography?
• AlittleBiology.
• SurfaceEMG(sEMG)Signal
• ClassifyingsEMG.
• MyoelectricControlstrategies.
• Ourapproachinfinemuscularactivityrecognition.
![Page 3: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/3.jpg)
Electromyography
• Medicinalelectro-diagnostictechniqueformeasuringandevaluatingtheelectrical output(actionpotential)ofskeletalmuscles.
• Signalscanbeanalyzedtodetectmedicalabnormalities,activationlevels,recruitmentorder,ortoanalyzethethebiomechanicsofhumanoranimalenvironment.
• Twotypes:1. IntramuscularEMG2. SurfaceEMG
![Page 4: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/4.jpg)
IntramuscularEMG
• Typicallyperformedusingmonopolarneedleelectrode– A finewireinserted intoamusclewithasurfaceelectrode forreference,or
– Twofinewires inserted intothemusclereferencedtoeachother.
• Needpre-injectionsandskinpreparations.
• Lessnoise
Invasive,painful.
![Page 5: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/5.jpg)
SurfaceEMG(sEMG)• Surfaceelectrodesonlyprovidealimitedassessmentofthemuscleactivity.– Arrayofelectrodes typicallyused
• Signalisacomposite ofallthemusclefiberactionpotentialsoccurring inthemusclesunderlyingtheskin.
• Pronetonoise,artifacts• Non-invasive
![Page 6: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/6.jpg)
Muscles
• Composedofbundlesofspecializedcells thatareresponsibleforcontractionandrelaxation.– Generateforces,movements,&abilityforexpression.
• 4mainfunctionsofmuscles:1. Producemotion.2. Movingsubstancew/i body.3. Stabilization.4. Generateheat.
![Page 7: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/7.jpg)
Muscles• 3typesofmuscletissue:
1. Skeletalmuscle.2. Smoothmuscle.3. Cardiacmuscle.
• Skeletalmuscles areaformofmuscletissue thatisunderthecontrolofthesomaticnervoussystem.– Voluntarilycontrolled.
• Attachedtothebonebybundles ofcollagen fibers,knownastendons.
• Whenmusclescontractandrelax,theyreleaseatinybioelectricpulsecalledtheActionPotential
![Page 8: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/8.jpg)
sEMG Electrodes
• ElectrodesmeasuringthesEMG signalformanemg channel.• 3maintypes:
– Bipolarconfiguration– Monopolar configuration– Laplacin configuration
• Onechannelbipolarderivationtypical.
• Recentinteresthasalsoincreasedtowardhigh-density sEMG (HD-sEMG).
![Page 9: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/9.jpg)
sEMG Signal
• RawEMGsignalstypicallycomeinasomewhatuselessform.
• Beforebeginningpatternrecognition,severalstepsaretypically followed:1. Cleanup2. Segmentation3. Analysis&preprocessing
![Page 10: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/10.jpg)
sEMG SignalCleanup
• Consistsofseveralsub-steps:
1. Filtration&De-noising.• LPFmosttypical• HPForBandpass alsoused.• WaveletTransforms
2. Rectification.• Halfwave.• Fullwave.
![Page 11: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/11.jpg)
sEMG SignalSegmentation
• Twowindowingtechniques:1. Overlappedwindowing2. Adjacentwindowing
![Page 12: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/12.jpg)
Features
• Featuresselectedcanhavehigher effectonclassificationaccuracythanclassifiertype.
• 3qualitiesdeterminequalityoffeaturespace:1. Maximum classseparability.2. Robustness3. Computational complexity
![Page 13: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/13.jpg)
EMGFeatures
• RawsEMG signalstypicallymappedintosmaller-dimensionfeaturevectors.
• Classifiersperformfaster.– Improvesreal-time properties ofthesystem.
• Canbegroupedinto4categories:– Timedomain(TD)features– Frequencydomain(FD)features– Time-frequency domain(TFD)features– Spatialdomain(SD)features.
![Page 14: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/14.jpg)
OptimalFeatures
• LittleconsensusonstudiestryingtofindtheoptimalfeaturesforclassificationofsEMG signals
• ComparativestudiesshowTDfeaturesachievehigheraccuracyforLDAclassifier.– TFDfeaturesoutperformthemforSVM.– TDfeaturesclassifiedwithLDAhavebeensuggested asoptimal forsEMGclassification.
• Conclusionsnotreliable.– Madeinlow-noise labenvironment.
![Page 15: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/15.jpg)
Timedomain(TD)features
• MostcommonfeaturegroupinsEMG signalclassification.
• Nomathematicaltransformationsneeded.– Fastcalculations, suitable forreal-time– Sensitive tonoiseandartifacts.
• Phinyomark etal.dividedTDfeaturesinto4maintypes1. Energyandcomplexity informationmethods.2. Frequencyinformationmethods.3. Predictionmodelmethods.4. Time-dependence methods.
![Page 16: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/16.jpg)
Timedomain(TD)Features
• SeveralattemptstodeterminetheoptimalTDfeaturevectorforseveralclassifiers.
• Mostcommoncombination:LDAclassifierwithHudgin’s featurevector– Meanabsolute value(MAV)– Waveformlength(WL)– Zerocrossing(ZC)– Signalslopechanges(SSC)
• Lowcomputationalcomplexity,highaccuracy.
![Page 17: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/17.jpg)
Frequencydomain(FD)features
• Calculatedfrompowerspectraldensity(PSD).– Canbedetermined byfiringrateofrecruitedmotorunits ormorphologyofMUAPstravelingalongrespectivemuscle fibers.• Dependsonmusclebeingmeasured.
• Phinyomark studiedthepropertiesof37TDandFDfeatures– TDfeaturessuperiortoFDfeatures.– FDfeaturescomputationally morecomplex, lessaccurateclassification.– WhataboutTD+FDfeaturevector?
• BetterthansingleTDfeaturevector.
![Page 18: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/18.jpg)
Time-frequencyDomain(TFD)Features
• Describesignalinbothtimeandfrequencydomainssimultaneously.
• ComputationallymorecomplexthanTDfeatures.– Implemented withfastalgorithms.– Showntobecapable ofmeeting real-time requirements insEMG classificationusingappropriatedimensional reductionandsegmentation techniques.
• Yieldhighdimensionalfeaturevectorthatrequiresdimensionalityreductiontoincreasespeedandaccuracyofclassification.
![Page 19: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/19.jpg)
Time-frequencyDomain(TFD)Features
• TFDfeaturestypicallyusedinsEMG classificationinclude:– ShorttimeFouriertransforms(STFT)– Discretewavelettransform(DWT)– Continuouswavelettransform(CWT)– Wavelet packettransform(WPT)
• STFTcannotincreasebothtimeandfrequencyresolutionsimultaneously.
• CWT,DWT,WPTovercomethissomewhat.– Goodfrequencyresolution andpoortimeresolution inlowfrequencyband.– Badfrequencyresolution andgoodtimeresolution inhighfrequencyband.– DWTmostpopularduetobeingcomputationally moreefficient
![Page 20: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/20.jpg)
Spatialdomain(SD)features
• Regionsofmuscleactivateddifferentially.– Dependsonthepositionofthejointanddurationandstrengthofthecontraction.
• HD-sEMG measurementshavemadeitpossibletoextractspatialinformationfromsEMG recordings.
• Improvedifferentiationbetweenposturesandforcelevels.– ProvideinformationaboutMUAPsandload-sharingbetweenmuscles.
• Recentlyadopted,fewstudiesinvestigatedanddesignedSDfeatures forthispurpose.
![Page 21: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/21.jpg)
MyoelectricControlStrategiesinsEMG
• 2controlstrategies:1. Patternrecognition based.2. Non-patternrecognition based.
• Patternrecognitionbasedcontrolusesclassifiers.– Mapfeaturevectorstodesiredcommands– Allowsmoreversatilecontrolscheme thannon-patternrecognitionbased
control.
• Nonpatternrecognitionbasedcontrolcomparesvalueofsinglefeaturetopredeterminedthreshold.
![Page 22: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/22.jpg)
Patternrecognition-basedcontrol• Reliesonassumption:
– Classifieriscapableofrecognizinginputvaluesintroducedintrainingsessionandassigneachinputvaluetooneofagivensetofclasses.
• Input:featurevectorscalculatedofthesEMG signal.• Classes:
– Differentcontrolcommandssenttothedevice.– Different“poses”madebymuscles.
• Comparativestudiesshowmostclassifiershavesimilarclassificationaccuracy.– Usingappropriatefeaturesetsandsufficientnumberofchannels.– Bestclassifiersarefasttotrain,simpletoimplement,meetreal-timecontractions.
• LDA,SVM,HMM• LDAmostcommonlyused.
![Page 23: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/23.jpg)
Non-patternrecognition-basedcontrol
• Simplestructure.– Limitation: onlysomanycommandscanberecognized.
• Showntoprovideintuitive interfacefornavigationmenus,wheelchairs,andassistiverobotics.– Allofwhichrequirefewercommandsthanotherapplications, likeprosthesis.
• Methodsincludedinnon-patternrecognition-basedcontrol:– Proportionalcontrol– Onsetanalysis– Finitestatemachines (FSM)
![Page 24: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/24.jpg)
Finemuscularactivityrecognition
• Nowweknowthe“magic”behindmuscularsignalsandEMG.– Wecanbegintotalkaboutwhatweareabletoexplorewiththisknowledge.
• Fineactivityrecognitionisnotyeta“hottopic” likeEMGcontrolsandregularactivityrecognition.– Easytotell ifsomeone isstandingorsitting.– Wedomanyactivities withourfingers.
• Typing,writing,eating,etc.• Manyneuromusculardiseasesshowsymptomsatthefingerlevel
– Parkinson’s, Muscular Dystrophy, MS, Carpel Tunnel
![Page 25: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/25.jpg)
Overallstructure
• Weneedtobeableto:
Collectdata
ExtractFeatures Classify Predict
finger
![Page 26: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/26.jpg)
Myo DataRecognition
• Whiletypingina“naturalisticsetting”, isitpossibletoextractwhatfingersusersareutilizing?– Allowsforfingermovementanalysis– Unobstructive, uninvasive.
• UsingstandardQWERTYkeyboard– Standardfingerpositioning
• Usingtwo MYOarmbands– Collecting EMGdataat200Hz– Collecting IMUdataat50Hz
![Page 27: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/27.jpg)
Myo DataRecognition
• EMGFeatures– TDFeatures:
• MeanAbsoluteValue(MAV)• WaveformLength(WL)• SlopeSignChanges(SSC)• ZeroCrossings(ZC)
– TFDFeatures:• EMGShort TimeFourierTransform• IMUShortTimeFourierTransform
![Page 28: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/28.jpg)
Myo DataRecognition
• IMUFeatures– Averageofacceleration forx,y,zaxes.– Standarddeviation ofeachx,y,zaxes.– AverageAbsoluteDifference.– AverageResultant Acceleration.
• Thesefeaturesdon’ttellusmuchaboutaccelerationchangeswhentyping,sincethemovementsaremostlyalongoneaxis,andareverysmallinthegrandscheme.– Otherfeaturesstillunderconsideration.
![Page 29: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/29.jpg)
Conditions
• WearingaMyo armbandoneacharm,usertypesoutanarticle.
• Typingspeedhastobeslow,inordertogetlargerwindowsizes.– Approx.20words/minute orless
• Userholdsthekeydownwhenpressedforapprox.1second.
![Page 30: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/30.jpg)
“Traditional”classification
Results:• 50%-60%accuracyinidentifyingwhichfingerwasusedatnormal typingspeed(~60wpm).
• ~76%- 80%accuracyatslow typingspeed(~20wpm).• Why?
![Page 31: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/31.jpg)
Hierarchicalclassification
Windowdata
• “chop”dataupintowindows
Typingactivity
• Classifywhethertheuseristypingornot.
Fingersused
•Onceweknowanactivityisoccurring,wecanextrapolatefingers
Collectdata
ExtractFeatures Classify
![Page 32: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/32.jpg)
Hierarchicalclassification• Firstcheckiftypingactivity ishappening.• Ifyes,classify finger.Results:• Typingornottyping:
– Majorityvote.• Windowchoppedintosubwindows.• Subwindows areclassified.
– ~100%accuracyusingLDA,Perceptron,andSVM regardlessoftypingspeed.• Fingerclassification
– 55%-60%accuracyinidentifyingwhichfingerwasusedatnormal typingspeed(~60wpm).– ~78%- 82%accuracyatslow typingspeed(~20wpm).
• WHY?!?!?!
![Page 33: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/33.jpg)
Culprit
• MuscleActionPotentialatsupinators/pronatorsrateis~500Hz
• Myo claimssamplingrateis~200Hz, actuallycloserto~150Hz.
• Windowsizesmaystillbetoosmall.Classificationaccuracysuffers.
![Page 34: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/34.jpg)
Thankyou
![Page 35: Fine muscular activity recognition using Electromyographyrzheng/course/CAS765fa15/lectures/... · 2015. 11. 18. · Ala Shaabana. Outline • What is Electromyography? • A little](https://reader036.vdocuments.net/reader036/viewer/2022071406/60fb3ddaf1ed2112124734bc/html5/thumbnails/35.jpg)
TDfeaturevectorsusedinsEMG interfaces