[mobilehci2013] vibpress: estimating pressure input using vibration absorption on mobile devices
DESCRIPTION
This paper introduces VibPress, a software technique that enables pressure input interaction on mobile devices by measuring the level of vibration absorption with a built-in accelerometer when the device is in contact with a damping surface (e.g., user’s hands). This is achieved using a real-time estimation algorithm running on the device. Through a user evaluation, we provide evidence that this system is faster than previous software-based approaches, and as accurate as hardware-augmented approaches (up to 99.7% accuracy). We also provide insight about the maximum number of pressure levels that users can reliably distinguish, reporting usability metrics (time, errors, and cognitive loads) for different pressure levels and types of gripping gestures (press and squeeze).TRANSCRIPT
VibPress
Estimating Pressure Input
Using Vibration Absorption on Mobile Devices
Presenter: S.J. Hwang
Ph.D. Candidate | 6th Semester
BACKGROUND VibPress
BACKGROUND
Human interacts with environmental objects by grasping,
holding, twisting, pressing with pressure.
BACKGROUND
However, current touch screen
does not support pressure-sensitive input.
BACKGROUND
1. It can free the user from spatial restrictions and repetitive movement (Miyaki, & Rekimoto, 2009).
2. It is suitable for one-handed interaction, adds a degree of
freedom to touch locations (Boring et al., 2012).
3. Pressure input allows relatively stable and accurate interactions
when user is in mobile context (Wilson et al., 2011).
4. It can be used to alleviate occlusion problems and provide
ways for rich contextual selections (Ramos et al., 2004).
Advantages of pressure input for mobile device
PREVIOUS WORK VibPress
PREVIOUS WORK
Hardware augmentation approach Pressure estimation in software
Pressure input techniques for mobile devices
PREVIOUS WORK
GraspZoom (Miyaki et al., 2009)
Pressure-based Text Entry (Brewster et al., 2009)
Squeezing the Sandwich (Essl, 2009)
Pressure-based Menu Selection (Wilson et al., 2010)
ForceGestures (Heo et al., 2011)
Indirect Shear Force (Heo et al, 2013)
Multi-digit Pressure Input (Wilson et al., 2012)
ForceDrag (Heo et al., 2012)
Hardware Augmentation Approach
PREVIOUS WORK
Hardware Augmentation Approach to Enable Pressure Interaction
PREVIOUS WORK
Hardware Augmentation Approach
Specialize Hardware offers reliable and fast input pressure
readings.
However,
- It is still not widely supported in current mobile devices.
- It represents additional costs (production cost for vendors and
maintenance costs for users).
c
t
a
PREVIOUS WORK
Software Approach to Enable Pressure Interaction
A tangible controller (Kato et al., 2009)
Vision-based Force Sensor (Sato et al., 2012)
The Fat Thumb (Boring et al., 2012)
Muscle Tremors (Strachan et al., 2004)
Use the Force or something (Essl et al., 2010)
ForceTap (Heo et al., 2011)
Expressive typing (Iwasaki et al., 2009)
camera
touchscreen
accelerometer
c t
a
Technique (Author) Sensor
Repurposed P levels Modality emulated (type) Property Used A/P
A tangible controller
(Kato et al., 2009) Camera (cont) Pressure (S) Marker position and size P
Vision-based Force Sensor
(Sato et al., 2012) Camera (cont) Pressure (S) Marker position and size P
Muscle Tremors
(Strachan et al., 2004) accelerometer 2 Pressure (S) Vibration P
Expressive typing
(Iwasaki et al., 2009)
Accelerometer
Keyboard 2 Pressure (E) physical movement P
ForceTap
(Heo et al., 2011)
Accelerometer
Touchscreen 2 Pressure (E) physical movement P
Use the Force or something
(Essl et al., 2010) Accelerometer N/A Pressure (E) physical movement P
The Fat Thumb
(Boring et al., 2012) Touchscreen 2 Pressure (E) Contact size P
A, Active; P, Passive; P/A, Passive/Active; S, Streaming based; E, Event-based; R, Recognition-based;
PREVIOUS WORK
Software Approach to Enable Pressure Interaction
PREVIOUS WORK
Software Approach to Enable Pressure Interaction
- They do not continuously measure pressure. (e.g., ForceTap, MicPen, and Expressive typing)
- The interaction area is limited by hardware settings.
(e.g., PseudoButton)
- An additional calibration process is needed.
(e.g, The Fat Thumb, and Muscle tremor)
These approaches successfully estimated the input pressure using only inertial sensors on mobile devices.
However,
OUR METHOD VibPress
OUR METHOD
Vibration Absorption As Proxy For Pressure Input
Amount of pressure on a mobile device can be approximated by using an
accelerometer to measure the spatial displacement generated when the internal
vibration motor vibrations.
OUR METHOD
Implementation
- Implemented on the Samsung Galaxy S3, running Android OS 4.0.4.
- Activated internal linear vibration motor with a continuous pulse at
maximum amplitude (setting on OS).
- Sampled the data from the internal three-axis accelerometer at 100Hz.
- Passed through a low-pass filter to reduce sampling noise and
Euclidean distance between last reading and its predecessor is
computed.
- Finally, we used the sum of these values to reach a good estimation of
the amount of pressure exerted on the device.
OUR METHOD
VibPress
10-2012-0110731, Method and apparatus for sending touch strength on user terminal and method, Korea Patent, Pended (2012.10.05) 10-2012-0066225, Apparatus and method for touch Sensing, Korea Patent, Pended (2012.06.19)
Finally, we built software that reliably and quickly estimate the input pressure by
measuring the spatial displacement absorbed. And we filed some patents based
on the technique.
PREVIOUS WORK
GripSense (Goel et al., Oct, 2012)
Muscle tremor (Gyroscope) + + Finger size (Touchscreen)
Machine learning algorithm
Damping effect (Gyroscope)
PREVIOUS WORK
Main differences from the previous work
GripSense (Goel et al., 2012)
Technique P levels Sensor Used Estimation
Method A/P
Postures
Explored
GripSense
(Goel et al., 2012) <= 3
Gyroscope
Touchscreen Machine-learning Active Pressure
VibPress
(Hwang et al., 2013) <= 6 Accelerometer Real-time readings Active
Pressure
Squeeze
Our approach is fast and lightweight, supporting two types of gestures.
EVALUATION VibPress
EVALUATION
Pilot study
The goal of the pilot study was to establish the average minimum
and maximum pressure that user could exert on the mobile device in
order to derive an appropriate estimation of the maximum number
of distinguishable pressure levels.
Usability study
The goal of the usability study was to test the input performance
(time, errors, and cognitive load) for different pressure levels and
hand gripping gestures derived from the pilot.
Pilot Test
- 4 participants (2 women) aged between 26 and 33 (average
29.5 , SD 3.1)
- Asked to sequentially apply maximum and minimum
pressure with their dominant hands using two postures
(touch and grip).
- For touch gesture, participants asked to press the center of
the screen with their thumbs.
- For the grip gesture, participants asked to hold the device
with the dominant hand and activated the vibration motor by
clicking a button with the other hand.
- Every press event took 1.5 seconds and repeated ten times.
- We discarded the first 500ms to allow for pressure
stabilization.
Touch
Grip
Pilot Test : Result
Ranges of average minimum and maximum pressure according to hand-gripping type.
Usability Test
The graphical interface used for testing pressure input with 2
levels (L2), 4 levels (L4) and 6 levels (L6).
Continuous visual feedback + Dwell
Usability Test
- 12 volunteers (6 women) with ages between 24 and 33 (average 27.5, SD 2.5).
- Participants were asked to familiarize themselves with the device for a maximum of five
minutes.
- We asked users to perform 36 successful targeting trials (the first 12 discarded as
practice) for each pressure level (L2, L4, L6) with two hand gestures (press and
squeeze).
- NATA TLX questionnaire for each pressure level.
- Every target level was evenly represented and balanced, collecting data for a total of
288 trials per hand gesture type.
- We recorded measures of performance (e.g., the selection time of successful trials,
number of wrong selections, and the variations of the pressure within the targets).
- The experiment took approximately one hour.
Usability Test : Result
Metrics for selection times and error rate (low is better).
(F(2,11)=126.6, p<0.01) *
* (F(1,11)=6.3, p<0.05)
Overall error rates for levels L2, L4, and L6 are, respectively, 0.3%, 7.6%
and 12.5% for the press gesture, and 0.7%, 11.5%, and 33% for the
squeeze posture.
(F(2,11)=25.1, p<0.01) *
* (F(1,11)=5.1, p<0.05)
Usability Test : Result
Users’ cognitive workload with a TLX.
The error rate is also reflected in the results from the TLX (cognitive
load is directly proportional to the number of pressure levels).
(F(2,11)=84.5, p<0.01) *
Usability Test : Result
% of error distribution among the selection levels for L6 (low is better).
The distribution of errors was uneven and most errors
happened in the central targets (especially in the box5)
rather than at the extremes.
Box 6
Box 1 268 (MAX PRESSURE)
0 (MIN PRESSURE)
coarse
smooth
Usability Test : Result
- Most participants expressed that VibPress is enjoyment and quick adoption.
- However, some reported that “continuous vibrations are somehow disturbing.”
- Overall, participants agreed that for L2 and L4, the performance and the usability level were satisfactory
- Despite the accuracy level, users preferred the squeeze gesture to the press one, as they felt it took less physical effort.
Interview
Usability Test : Result
VibPress shows …
- Selection times similar to those obtained with specialized
hardware (Cechanowicz et al., 2007; Ramos et al., 2004).
- But faster (Goel et al., 2012).
- With fewer errors than software pressure techniques (Goel et al.,
2012; Heo et al., 2011; Hwang et al., 2012).
When comparing these results with those from previous
work for analogous dwell-based targeting tasks,
VIDEO VibPress
CONCLUSION VibPress
LIMITATION
Compatibility across devices
(different motors or device cases propagate vibrations
differently).
Battery consumption due to the continuous activation
of the vibration motor. (However, these issues could be
alleviated by using frequent but short vibration patterns
at lower intensities).
CONCLUSION
- We showed evidence that, with continuous visual feedback,
users can reliably and quickly input with accuracy ranging from
99.7% to 67% and from ~1 to ~4 seconds interaction time,
depending on the different pressure levels and hand gripping
used.
- We also described how different users favorably judged the
usability and potential applications of the system.
- Future work will aim to identify hand postures and specific
pressure for selective portions of the screen. Finally, we also
plan to measure pressure inputs using tangible objects (e.g.,
styli, tokens) rather than fingers.
END
Contact : www.UXInventor.com
Demo Here !
Thank you so much !