![Page 1: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/1.jpg)
Enabling Always-Available Input with Muscle-Computer Interfaces
T. Scott SaponasUniversity of Washington
Desney S. Tan Microsoft Research
Dan Morris Microsoft Research
Ravin Balakrishnan University of Toronto
Jim Turner Microsoft Corporation
James A. Landay University of Washington
![Page 2: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/2.jpg)
Mobile Computing Enables…
![Page 3: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/3.jpg)
“How the computer sees us.”
Igoe & O'Sullivan
![Page 4: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/4.jpg)
Hands Busy
Physically Active
![Page 5: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/5.jpg)
![Page 6: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/6.jpg)
![Page 7: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/7.jpg)
![Page 8: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/8.jpg)
![Page 9: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/9.jpg)
Muscle-Computer Interfaces
![Page 10: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/10.jpg)
Muscles Activate via Electrical Signal
![Page 11: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/11.jpg)
Muscles Activate via Electrical Signal
Electrical Signal can be sensed by Electromyography (EMG)
![Page 12: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/12.jpg)
EMG for Diagnostics, Prosthetics & HCI
Jacobsen, et al. “Utah Arm”
![Page 13: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/13.jpg)
Jacobsen, et al. “Utah Arm”
Costanza, et al. “Intimate interfaces in action”
EMG for Diagnostics, Prosthetics & HCI
![Page 14: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/14.jpg)
Jacobsen, et al. “Utah Arm”
Costanza, et al. “Intimate interfaces in action”
Naik, et al. “Hand gestures”
EMG for Diagnostics, Prosthetics & HCI
![Page 15: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/15.jpg)
Jacobsen, et al. “Utah Arm”
Costanza, et al. “Intimate interfaces in action” Wheeler & Jorgensen “Neuroelectric joysticks”
Naik, et al. “Hand gestures”
EMG for Diagnostics, Prosthetics & HCI
![Page 16: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/16.jpg)
Finger Gestures Detected from Upper Forearm
![Page 17: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/17.jpg)
Detecting Finger Gestures Challenging
![Page 18: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/18.jpg)
Offline Classification of Finger Gestures on a Surface
Saponas, et al. CHI 2008
![Page 19: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/19.jpg)
Real-Time Classification ofFree Space & Hands Busy Gestures
Pinch
Mug Bag
![Page 20: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/20.jpg)
Bimanual Gesture
+
dominant handgesture
non-dominant hand squeeze
![Page 21: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/21.jpg)
Sensor Placed on Upper Forearm
![Page 22: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/22.jpg)
![Page 23: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/23.jpg)
Stimulus / Response Training
![Page 24: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/24.jpg)
Gesture Classification Technique
30 millisecond sample
X 6 Sensors
Support VectorMachine
labeledtraining data
user specific model
machine learning
![Page 25: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/25.jpg)
Gesture Classification Technique
30 millisecond sample
Root Mean Square (RMS) ratios between channels
Frequency Energy10 Hz bands
Phase Coherence ratios between channels
X 6 Sensors
Features Support VectorMachine
labeledtraining data
user specific model
machine learning
![Page 26: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/26.jpg)
Gesture Classification Technique
30 millisecond sample
Root Mean Square (RMS) ratios between channels
Frequency Energy10 Hz bands
Phase Coherence ratios between channels
X 6 Sensors
Features
Support VectorMachine
user specific model
machine learning
gesture classification
![Page 27: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/27.jpg)
![Page 28: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/28.jpg)
12 Person Experiment
Pinch
Mug Bag
![Page 29: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/29.jpg)
Training vs Testing in Several Postures
TrainTest
Left Center Right
Left 78% 72% 57%
Center 70% 79% 74%
Right 68% 73% 74%
![Page 30: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/30.jpg)
4 Finger 3 Finger0%
10%20%30%40%50%60%70%80%90%
100%
Hands-Free Gesture Accuracy
Posture Independent Pinching
![Page 31: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/31.jpg)
Bag in Hand Better Recognized
Mug Bags0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Hands-Busy Gesture Accuracy
Four FingersThree Fingers
![Page 32: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/32.jpg)
Worked Well for Those Who “got it”
all bottom 50% top 50%0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Bags in Hands Gestures
Four Fingers
Three Fingers
![Page 33: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/33.jpg)
80% Accurate with 70 Seconds Training
0 25 50 75 100 125 150 1750%
10%20%30%40%50%60%70%80%90%
100%
Quantity of Training Data vs. Classification Ac-curacy
Seconds of Training Data
Clas
sific
ation
Acc
urac
y
![Page 34: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/34.jpg)
Portable Music Player Menus
• Some participants navigated menus easily• Other participants found interaction difficult
![Page 35: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/35.jpg)
Limitations of Current Technique
• Works best for SINGLE user SINGLE session• Wired Sensors with Gel and Adhesive• Sitting or Standing at a Desk in the Lab
![Page 36: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/36.jpg)
Ongoing & Future WorkWireless Armband, Dry Electrodes, Cross-Session Models
![Page 37: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/37.jpg)
Ongoing & Future Work
Walking & Jogging
Wireless Armband, Dry Electrodes, Cross-Session Models
![Page 38: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/38.jpg)
Ongoing & Future Work
Walking & Jogging
Interactive Tabletops
Wireless Armband, Dry Electrodes, Cross-Session Models
![Page 39: Enabling Always-Available Input with Muscle-Computer Interfaces](https://reader036.vdocuments.net/reader036/viewer/2022081520/568166ec550346895ddb3820/html5/thumbnails/39.jpg)
Enabling Always-Available Input with Muscle-Computer Interfaces
T. Scott SaponasUniversity of Washington
Desney S. Tan Microsoft Research
Dan Morris Microsoft Research
Ravin Balakrishnan University of Toronto
Jim Turner Microsoft Corporation
James A. Landay University of Washington
Thanks for Listening