an empirical characterization of touch-gesture input-force on mobile devices

An Empirical Characterization of Touch-Gesture Input-Force on

Mobile Devices

Faisal TaherJason Alexander

John HardyEduardo Velloso

GHOST project, funded by the EU

ACM ITS 2014Interactive Tabletops and Surfaces

Research Aim

• Characterization of gesture input-force – Touch-screen devices.– Common touch gestures.– Various contexts.

• Aid the design of force-sensitive input devices– Realize force ranges and tolerances.– Effects of context.– What does gesture force look like?

Research Aim

• User evaluations– 2 controlled studies: 14 and 24 participants.– 1st study – baseline characterization.– 2nd study – effect of various factors.

• A large number of factors to examine!

Interactive displays

Form factor

Mobile phone

Tablet

Tabletop

Situated/On-the-wall

Situational

Urgency Ambient lighting Attention distribution

Intent

Gaming Communicat-ionBrowsing

Interactions

Panning/Dragging

Pinching (two-finger)

Tapping/Selecting

Tilting

Typing

Rotating

Multi-touch pan

Long pressContent

Button size Visibility Device lag

Walking Sitting

User position

Standing

Interactive displays

Form factor

Mobile phone

Tablet

Tabletop

Situated/On-the-wall

Situational

Urgency Ambient lighting Attention distribution

Intent

Gaming Communicat-ionBrowsing

Interactions

Panning/Dragging

Pinching (two-finger)

Tapping/Selecting

Tilting

Typing

Rotating

Multi-touch pan

Long pressContent

Button size Visibility Device lag

Walking Sitting Standing

User position

Studies Set-up

• Gesture application– Pinch-in, pinch-out, panning,

tapping, rotating, and typing.

Studies Set-up



• FingerTPS force sensing equipment.

Glove with force sensor

Studies Set-up



• FingerTPS force sensing equipment.

• Control application to processsensor readings.

Studies Set-up• Procedure:

• Conditions:– Study 1: Seated, Tablet, Untimed.– Study 2:

• Form factor: Mobile phone vs. Tablet.• Position: Sitting vs. Walking.• Urgency: Timed tasks vs. Untimed tasks.• 2x2x2 factor design

• We recorded force and time readings of individual gestures.

Results – Profiles

• Force profiles – Force and time normalized.– High-level comparison.

• STUDY 1 – baseline profiles– Typing and Panning: similar shapes.– Tapping: more variation.


Forc

e

Time0

1

1

TypingFo

rce

Time0

1

1

Panning

Forc

e

Time0

1

1

Tapping


• Force profiles – Force and time normalized.– High-level comparison.

• STUDY 1 – baseline profiles– Typing and Panning: similar shapes.– Tapping: more variation.– Two finger gesture were more interesting.

Results – Profiles Rotate

Forc

e

Time

Index finger Thumb

Forc

e

Time0

1

10

1

1

0

1

1

Index finger

Forc

e

Time

Thumb

Forc

e

Time0

1

1

Zoom-in

Forc

e

Time

Index finger

Forc

e

Thumb

Time0

1

1 0

1

1

Zoom-out

Results – Profiles • Study 2 – effect of form factor, position, urgency

– Typing, tapping and panning:• Generally the same shape.• Slight variations for typing and panning.• Tapping remained the same.


Forc

e

Time0

1

1

TappingFo

rce

Time0

1

1

TypingFo

rce

Time0

1

1

PanningStudy 1 force profile

Study 2 force profile

Results – Profiles • Study 2 – effect of form factor, position, urgency

– Typing, tapping and panning:• Generally the same shape.• Slight variations for typing and panning.• Tapping remained the same.

– Rotation, Zoom-in and Zoom-out:• Varied thumb force profiles. • Index finger was the same (except Zoom-out).• Thumb profiles harder to predict.• More force when fingers are further apart.



0

1

1

Index finger

Forc

e

Time

Thumb

Forc

e

Time0

1

1

Zoom-in

Index finger

Forc

e

Time

Forc

e

Time

Thumb

0 1

1

0

1

1

Zoom-out

RotateIndex finger

Forc

e

Time

Forc

e

Time0

1

10

1

1

Thumb


Results – Factors• Form factor: Mobile vs. Tablet

– Participants pressed harder on a tablet.– Gestures affected: Typing, panning, zooming, rotating.

– Individual gestures took longer on the mobile phone.– Gestures affected: Typing, panning, zooming.

• Position: Sitting vs. Walking– Participants pressed harder whilst walking.– Gestures affected: Tapping, panning, zooming, rotating.

• Urgency: Timed vs. Untimed– Participants pressed harder whilst typing during the timed condition.

Results – Force Averages

Typing Tapping Panning Rotating (index)

Rotating (thumb)

Zoom-in (index)

Zoom-in thumb

Zoom-out (index)

Zoom-out (thumb)

0

10

20

30

40

50

60

70

80

Average Force (grams) by gesture / finger

Series1

Gesture

Aver

age

forc

e (g

ram

s)

Results – Time Averages

Typing Tapping Panning Rotating (index)

Rotating (thumb)

Zoom-in (index)

Zoom-in thumb

Zoom-out (index)

Zoom-out (thumb)

0

50

100

150

200

250

300

350

Average time (milliseconds) by gesture / finger

Series1

Gesture

Aver

age

time

(mill

isec

onds

)

Limitations

• Controlled setting.

• Specific set of tasks.

• Wearing force-sensing equipment can affect natural user interaction.

Further Explorations

• Factors: – Device lag.– Target size.– Situated devices.– Complex gestures: multi-finger panning,

bi-manual interactions.

• Develop empirical models to predict gestures based on input-force.

Summary• More force applied on tablet.

• More force applied whilst walking.

• Force profile identification.

• More force applied when index and thumb are further apart.

• Thumb force profiles are harder to predict.

• Index finger is more dominant / consistent during index-thumb gestures.

Thank You

EXTRA MATERIAL

Results – Force Ranges

Results – Time Ranges

Key points

• High-level profiles of common touch gestures.

• Aid gesture recognition.

• Aid device calibration.

• Effects of context.

an empirical characterization of touch-gesture input-force on mobile devices

Technology