enabling always-available input with muscle-computer interfaces
DESCRIPTION
Enabling Always-Available Input with Muscle-Computer Interfaces. T. Scott Saponas University of Washington Desney S. Tan Microsoft Research Dan Morris Microsoft Research Ravin Balakrishnan University of Toronto Jim Turner Microsoft Corporation James A. Landay University of Washington. - PowerPoint PPT PresentationTRANSCRIPT
Enabling Always-Available Input with Muscle-Computer Interfaces
T. Scott SaponasUniversity of Washington
Desney S. Tan Microsoft Research
Dan Morris Microsoft Research
Ravin Balakrishnan University of Toronto
Jim Turner Microsoft Corporation
James A. Landay University of Washington
Mobile Computing Enables…
“How the computer sees us.”
Igoe & O'Sullivan
Hands Busy
Physically Active
Muscle-Computer Interfaces
Muscles Activate via Electrical Signal
Muscles Activate via Electrical Signal
Electrical Signal can be sensed by Electromyography (EMG)
EMG for Diagnostics, Prosthetics & HCI
Jacobsen, et al. “Utah Arm”
Jacobsen, et al. “Utah Arm”
Costanza, et al. “Intimate interfaces in action”
EMG for Diagnostics, Prosthetics & HCI
Jacobsen, et al. “Utah Arm”
Costanza, et al. “Intimate interfaces in action”
Naik, et al. “Hand gestures”
EMG for Diagnostics, Prosthetics & HCI
Jacobsen, et al. “Utah Arm”
Costanza, et al. “Intimate interfaces in action” Wheeler & Jorgensen “Neuroelectric joysticks”
Naik, et al. “Hand gestures”
EMG for Diagnostics, Prosthetics & HCI
Finger Gestures Detected from Upper Forearm
Detecting Finger Gestures Challenging
Offline Classification of Finger Gestures on a Surface
Saponas, et al. CHI 2008
Real-Time Classification ofFree Space & Hands Busy Gestures
Pinch
Mug Bag
Bimanual Gesture
+
dominant handgesture
non-dominant hand squeeze
Sensor Placed on Upper Forearm
Stimulus / Response Training
Gesture Classification Technique
30 millisecond sample
X 6 Sensors
Support VectorMachine
labeledtraining data
user specific model
machine learning
Gesture Classification Technique
30 millisecond sample
Root Mean Square (RMS) ratios between channels
Frequency Energy10 Hz bands
Phase Coherence ratios between channels
X 6 Sensors
Features Support VectorMachine
labeledtraining data
user specific model
machine learning
Gesture Classification Technique
30 millisecond sample
Root Mean Square (RMS) ratios between channels
Frequency Energy10 Hz bands
Phase Coherence ratios between channels
X 6 Sensors
Features
Support VectorMachine
user specific model
machine learning
gesture classification
12 Person Experiment
Pinch
Mug Bag
Training vs Testing in Several Postures
TrainTest
Left Center Right
Left 78% 72% 57%
Center 70% 79% 74%
Right 68% 73% 74%
4 Finger 3 Finger0%
10%20%30%40%50%60%70%80%90%
100%
Hands-Free Gesture Accuracy
Posture Independent Pinching
Bag in Hand Better Recognized
Mug Bags0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Hands-Busy Gesture Accuracy
Four FingersThree Fingers
Worked Well for Those Who “got it”
all bottom 50% top 50%0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Bags in Hands Gestures
Four Fingers
Three Fingers
80% Accurate with 70 Seconds Training
0 25 50 75 100 125 150 1750%
10%20%30%40%50%60%70%80%90%
100%
Quantity of Training Data vs. Classification Ac-curacy
Seconds of Training Data
Clas
sific
ation
Acc
urac
y
Portable Music Player Menus
• Some participants navigated menus easily• Other participants found interaction difficult
Limitations of Current Technique
• Works best for SINGLE user SINGLE session• Wired Sensors with Gel and Adhesive• Sitting or Standing at a Desk in the Lab
Ongoing & Future WorkWireless Armband, Dry Electrodes, Cross-Session Models
Ongoing & Future Work
Walking & Jogging
Wireless Armband, Dry Electrodes, Cross-Session Models
Ongoing & Future Work
Walking & Jogging
Interactive Tabletops
Wireless Armband, Dry Electrodes, Cross-Session Models
Enabling Always-Available Input with Muscle-Computer Interfaces
T. Scott SaponasUniversity of Washington
Desney S. Tan Microsoft Research
Dan Morris Microsoft Research
Ravin Balakrishnan University of Toronto
Jim Turner Microsoft Corporation
James A. Landay University of Washington
Thanks for Listening