haptics for mobile devices

Haptics for Mobile Communication Devices

Steve Dai, Device Technology, Motorola

Steve Dai, Haptics Purdue October 21, 2008

Outline

Haptics Overview

Haptics for Mobile Devices

Localized Piezo Haptics

Haptics Characterization

Summary


Haptics Overview

Definition: tactile feedback, more specifically active feedback

Dominant Types of Active Feedback

VibrotactileVibration sensed by nerves in the skin

KinestheticSensations associated with body position, movement, or weight sensed by the nerves

in muscles, tendons, or joints

current focus


Haptics Overview

Sample applications

mice

gaming controllers

BMW I-drive

vibrating call alert

medical training

3D Design & Interaction

Braille displays


Haptics Overview

Comparison with other senses

Neurons “Data” rate(bits/s)

Temporal Acuity

Touch ~106 102 5 ms

Hearing ~104-105 104 0.01 ms

Sight ~106 106-109 25 ms

Lynette Jones, MIT


Haptics Overview

Frequency Dependence of Perception

Fletcher-Munson curves, 1933 Minimum audible=threshold

SOUND: 20~20,000 HzSIGHT: 540 THz

Verrillo-RT. Subjective Magnitude Functions for Vibrotaction. IEEE Transactions on Man-Machine Systems MMS-11(1): 19-24. (1970

TOUCH: 20~1000 Hz


Haptics Overview

ReceptorsSOUNDLight TOUCH



Why haptics?

“Data” rate(bits/s)

Temporal Acuity

Human to device Input

Touch –fingertip

102 5 ms

Audio – voice 104 0.01 ms

Sight – eye 106-109 25 ms


Haptics for Mobile Devices Value to users

FUNCTION SAMPLE USE CASES HAPTICS REQUIREMENTS

Notification Vibe alert Simple vibration or patterns

Confirma- tion

Key pressstatus change

Short click-like responseLow latencyLocalized preferred over global?

AmusementEntertain- ment

Base boostTouchscreen UI eventGaming

Rich variety of haptic contents Visual/audio synchronization“Essential but unnoticed”

haptics

Communi- cation

Sensorial communication

Standards and infrastructure“Hapton”

capable phones

Incr

easi

ng C

ompl

exity



Global vs. Localized hapticsGlobal: vibrates the entire device

(phone)

Localized: vibration is localized to an input surface (keypad, display)

ROKR E8 (2008)

Samsung F700 & SCH-W559(2007)

Samsung Anycall

2008Mot A1000(2004)

RAZR2(2007)

LG Voyager(2008)Mot

Krave(2008)

http://images.google.com/imgres?imgurl=http://www.tech2.com/media/images/2008/Mar/img_55021_e8_faces.jpg&imgrefurl=http://www.tech2.com/india/topstuff/mobile-phones/touch-screen-mobiles-part-iii-haptics/32221/0&h=768&w=1024&sz=74&hl=en&start=13&um=1&tbnid=3XZaQYMqVLVRqM:&tbnh=113&tbnw=150&prev=/images%3Fq%3Dmotorola%2Be8%26um%3D1%26hl%3Den%26rlz%3D1T4GFRC_en___US202%26sa%3DN

http://images.google.com/imgres?imgurl=http://www.3g.co.uk/PR/October2004/MotorolaA1000.jpg&imgrefurl=http://www.3g.co.uk/PR/October2004/8536.htm&h=607&w=300&sz=40&hl=en&start=11&um=1&tbnid=z6UYZ4R40eWDXM:&tbnh=136&tbnw=67&prev=/images%3Fq%3Dmotorola%2Ba1000%26um%3D1%26hl%3Den%26rlz%3D1T4GFRC_en___US202%26sa%3DG

http://images.google.com/imgres?imgurl=http://skattertech.com/media/2007/10/verizon-lg-voyager.jpg&imgrefurl=http://skattertech.com/2007/10/verizon-announces-4-new-phones/&h=1144&w=750&sz=379&hl=en&start=3&um=1&tbnid=zrag-6gjKgvtZM:&tbnh=150&tbnw=98&prev=/images%3Fq%3Dlg%2Bvoyager%26um%3D1%26hl%3Den%26rlz%3D1T4GFRC_en___US202%26sa%3DN

http://images.google.com/imgres?imgurl=http://www.techshout.com/images/samsung-sch-w420-phone.jpg&imgrefurl=http://www.techshout.com/mobile-phones/2008/26/samsung-sch-w420-touch-screen-haptic-mobile-phone-unveiled/&h=450&w=450&sz=36&hl=en&start=2&um=1&tbnid=2D9qudzbLUFCDM:&tbnh=127&tbnw=127&prev=/images%3Fq%3Dsamsung%2Bhaptics%2Bphone%26um%3D1%26hl%3Den%26rlz%3D1T4GFRC_en___US202%26sa%3DN

http://images.google.com/imgres?imgurl=http://www.prepaidreviews.com/blog/wp-content/uploads/2007/08/motorola-razr2-v8-1.jpg&imgrefurl=http://www.prepaidreviews.com/blog/%3Fpaged%3D2%26s%3Drazr&h=503&w=350&sz=33&hl=en&start=8&um=1&tbnid=Jw6SWx9pHz-0AM:&tbnh=130&tbnw=90&prev=/images%3Fq%3Drazr2%26um%3D1%26hl%3Den%26rlz%3D1T4GFRC_en___US202



Actuation Technologies

USE CASE & IMPLEMENTATION

VIBE ALERT

ADVANCED HAPTICS

KEYPRESS

YES YES NO NO

Global Localized

NO

OK OK GOOD EXLTGOOD

LMTD LMTD LMTD EXLTOKINTEGRATIONCOMPLEXITY

NO NO LO MODLO

Rotary Linear Linear F Reactor Piezo

http://images.google.com/imgres?imgurl=http://www.switchesplus.com.au/products/alps/forcereactor/alps_forcereactor_af.jpg&imgrefurl=http://www.switchesplus.com.au/products/alps/forcereactor/forcereactor.htm&h=150&w=200&sz=19&hl=en&start=1&um=1&tbnid=n33HgDrwo_vA9M:&tbnh=78&tbnw=104&prev=/images%3Fq%3DforceReactor%26ndsp%3D20%26um%3D1%26hl%3Den%26rlz%3D1T4GFRC_en___US202%26sa%3DN


POKR E8 Haptics The review

“This new type of haptics really works, and quite well. The effect is best described as "spooky". It works so well that if Motorola had told us it had real keys under the surface -

and wasn't a touch keypad at all -

we would have believed them and never doubted it…. ”

Eric Lin, Eric M. Zeman and Rich BromeJanuary 7, 2008 www.phonescoop.com


Localized Piezo Haptics The challenge

“Morphing” keypad= Display?

Lack of keyclick feedback for key press

Sample photo placement

ROKR E8 ModeShiftTM

keypad


Localized Piezo Haptics The solution

Direct bonding of piezo element on phone chassis behind keypad

Shrinkage/expansion of piezo under electrical field translates to “buckling” motion of keypad

haptics to fingertip

Proof-of-concept

Force sensing resistor


Localized Piezo Haptics Why piezo?

Typical Popple Click (Press Only)

-15

-10

-5

0

5

10

15

0 10 20 30 40 50 60

Time (ms)

Acc

eler

atio

n (g

)

2.0 4.0

Time (ms)

Popple switch Optimized DC rotary motor

Piezo Actuator vs. Click

5

5

5

5

0 10 20 30 40 50 60Time (ms)

Popple Click

Piezo Actuator

Optimized linear motor

Linear Motor with Reverse Drive vs. Popple Click

-15

-10

-5

0

5

10

15

0 10 20 30 40 50 60Time (ms)

Acc

eler

atio

n (g

) - P

oppl

e Popple ClickLinear (Rev. Drive)

Immersion VibeTonz vs. Click

0 10 20 30 40 50

Time (ms)

Popple Click

VibeTonz

MARGINAL GOOD BEST

Piezoelectric actuator

Global

Localized

http://images.google.com/imgres?imgurl=http://www.sparepart.dk/Graphics/Products/153_m.jpg&imgrefurl=http://www.sparepart.dk/product.asp%3Fproduct%3D153%26page%3D1&h=250&w=200&sz=9&tbnid=VN6RKxVWUVSC1M:&tbnh=106&tbnw=84&hl=en&start=32&prev=/images%3Fq%3Dkeypad%2Bmembrane%2Bswitch%26start%3D20%26svnum%3D10%26hl%3Den%26lr%3D%26sa%3DN


Haptics Characterization What to measure for a keyclick?

Virtual tapping study (J Feine, Stanford Univ, Haptics2006)

–

Haptic simulations of tapping on a hard object feels most real when hand acceleration produced by virtual contact are matched to those of contact with real object

Key click profile–

Push and release acceleration pulses–

Each pulse: <5 ms, 40~100 g Accel_pp

Impact other than acceleration needs to be further studied

2.0 4.0

Time (ms)


Haptics Characterization Passive vibrotactile

0

50

100

150

200

250

300

0 100 200 300 400 500

Frequency (Hz)

Disp

lace

men

t (μm

)

Max_dMin_d

0

2

4

6

8

10

12

0 100 200 300 400 500Frequency (Hz)

Acce

l_pp

(g)

Max_g

Min_g

Acceleration Comparizon

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0 100 200 300 400

Frequency (Hz)

Acc

eler

atio

n (g

)

SSRCAverageMountcastle1972

Displacement Comparizon

0

2

4

6

8

10

0 100 200 300 400

Frequency (Hz)

Dsi

plac

emen

t (μ

m)

SSRCAverageMountcastle1972

Thre

shol

dC

omfo

rt ra

nge


Haptics Characterization Click threshold and comfort level

Test setup

Subjects control test processesUp/down key to adjust voltage levelSpace bar to register the desired voltage level

6 data points for each wave setting, 3 runs starts from low and other 3 runs from high

Alternated test sequence on threshold and comfort level for counter balance

Total 12 test subjects

12 levels at 2 dB a step


Haptics CharacterizationWaveforms and resultant acceleration profiles:

Step, sin300_1, sin300_2, sing150_1 and sing 150_2 generated in Audition

Audio signal (Vpp = 0 ~ 2 V) to 20x piezo amplifier for threshold (max Vpp ~ 40 V)100x Kepco voltage amplifier for comfort level (max Vpp ~ 200 V)Wave forms played at both press and release of FSR

step w ave

-50

0

50

100

150

200

250

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Volta

ge (v

)

step w ave p-p acceleratoin

-15

-10

-5

0

5

10

15

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Acc

eler

atio

n (g

)

Sin300_1 w ave

-50

0

50

100

150

200

250

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Volta

ge (v

)

sin300_1 w ave p-p acceleratoin

-15

-10

-5

0

5

10

15

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Acc

eler

atio

n (g

)Sin300_2 w ave

-50

0

50

100

150

200

250

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Volta

ge (v

)


-15

-10

-5

0

5

10

15

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Acc

eler

atio

n (g

)

Sin150_1 w ave

-50

0

50

100

150

200

250

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Volta

ge (v

)


-15

-10

-5

0

5

10

15

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Acc

eler

atio

n (g

)

Sin150_2 w ave

-50

0

50

100

150

200

250

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Volta

ge (v

)


-15

-10

-5

0

5

10

15

-0.005 0 0.005 0.01 0.015 0.02

Time (s)

Acc

eler

atio

n (g

)


Haptics Characterization Click threshold

Step waveAccel_pp ~ 2 g Single pulse?

Frequency effectAccel_pp ~ 0.39 g at 150_1 Hz, Accel_pp ~ 0.71 g at 300_1 Hz

Number of wave Accel_pp ~ decreases at double waves

Drive voltageVoltage lower in double waves driveVoltage lower as frequency goes up

Threshold Acceleration

0

1

2

3

Step Sin150_1 Sin150_2 Sin300_1 Sin300_2

Haptics

Thre

shol

d A

ccel

e_pp

(g)

95% CI High

95% CI Low

Mean

Vpp

0

5

10

15

20

Step Sin150_1 Sin150_2 Sin300_1 Sin300_2

Haptics

Thre

shol

d Vo

ltage

(V)

95% CI High

95% CI Low

Mean


Haptics Characterization Click “Comfort”

Level

Similar acceleration and voltage profiles as threshold

Frequency effectAccel_pp ~ 3.2 g at 150_1 Hz, Accel_pp ~ 6.7 g at 300_1 Hz

Number of wave Accel_pp ~ decreases at double

waves

Drive voltageVoltage lower in double waves driveVoltage lower as frequency goes up

Comfort Acceleration Level

0

5

10

15

20

25

Step Sin150_1 Sin150_2 Sin300_1 Sin300_2Haptics

Com

fort

Acc

ele_

pp (g

) 95% CI High

95% CI Low

Mean

Comfort Voltage Level

0

50

100

150

Step Sin150_1 Sin150_2 Sin300_1 Sin300_2Haptics

Com

fort

Lev

el V

olta

ge (V

) 95% CI High

95% CI Low

Mean


Haptics Characterization single, double waves and vibrotactile

Lower acceleration and displacement for multiple waves

1 wave

2 waves

…. Vibrotactile, 250 ms, 37~75 wavesPiezo Thunder Mini shaker

Threshold Accel_pp

(g)Sin150 0.39 0.25 0.03 0.03Sin300 0.71 0.51 N/A 0.12

Threshold displ* (μm)

Sin150 4.3 2.8 0.33 0.32Sin300 2.0 1.4 N/A 0.32

1 wave

2 waves

…. Vibrotactile, 250 ms, 37~75 wavesPiezo Thunder Mini shaker

Comfort level Accel_pp

(g)Sin150 3.21 2.04 0.12 ~0.9 0.92 ~ 5.37Sin300 6.71 4.48 N/A 3.2 ~ 10.3

Comfort level displ* (μm)

Sin150 35.4 22.5 4.6 ~ 34.5 10.2 ~ 60.4Sin300 18.5 12.4 N/A 8.8 ~ 28.5


Haptics Characterization “Comfort”

vs threshold levels

For current interactive click feedback test, the comfort level is approximately 18~19 dB over threshold. The ratio is appx a constant, and is independent of the wave forms, frequency and number of pulses

Accel_Cmf/Accel_Thr In dB (=20*log(Cmf/Thr))Step 8.79 18.9Sin150_1 8.29 18.4Sin150_2 8.02 18.1Sin300_1 9.44 19.5Sin300_2 8.76 18.8


Haptics Characterization Issues of existing in-line test fixture

200 g wt on linear sliderGood tool for production line with good R&R

Discrepancy of fixture vs handMagnitude of pk-pk accelerationTrend of frequency response

Fixture Hand

Frequency response


Haptics Characterization Literature --

human finger study

Multiple joints, muscles/tendons to actuate fingerBiomimetic finger: use SMA for actuation* Vishalini

Bundhoo

and Edward J. Park, “Design of an Artificial Muscle Actuated Finger towards Biomimetic

Prosthetic

Hands”, IEEE, 2005


Haptics Characterization Literature –

fingertip

Findings•

Mass m: ~ 6 g•

Stiffness k: up linearly with force•

Damping b: large zero-f value, up linearly with force

Rapid transient measurement

Model)()()( tkxtxbxmtF ++= &&&

* A. Haijun

and R. Howe, “Identification of the mechanical impedance at the human finger tip”, J Biomechanical Eng, Vol

119, P 109, Feb 1997


Haptics Characterization Internal Study --

finger impedance vs frequency

Model

2-spring system?•

Both response to lower frequency•

One dominates at higher frequency* C Fu and M Oliver, “Direct Measurement of Index Finger Mechanical

Impedance at Low Force”, World Haptics 2005

02468

10121416

0 100 200 300 400 500 600Frequency, Hz

|Z|,

N-s

/m

Z(ω

) = F(ω

) x& (ω

) = (mω − k ω

)j + b

Two springs?

1 DOF?


Haptics Characterization Improved test fixture –

proof of concept

Schematic•

2-spring system–

Arm–

probe•

1 DOF at the joint•

Weight and position to reach 150~200 g at the probe

“Primitive” fixture•

stainless steel = spring 1•

200 g weigh: F @ contact ~ g•

Accelerometer: MS ACH-01•

Poron: Rogers 4790-92-15125-04 cellular urethane foam, 3 mm, = spring 2

•

Probe: Al block + screw with round tip

wtPoron

A-meterProbe

Phone

Springs

1 DOF


Haptics Characterization Results –

Finger Press vs Fixture

Fairly good tracking at 200, 250 and 300 Hz, nearly 1:1

Good tracking at all voltage levels

Slightly off at 150 Hz

Overall the tracking is very much improved over the existing inline test fixture


Summary

Haptics is an essential functionality of mobile devices

Proper developed haptics could greatly enhance device usability

Piezoelectric enabled localized haptics can provide a nearly true keyclick experience

Haptics characterization is critical for technology development


Challenges

Haptics characterization–

Understanding of the physical parameter space responsible for a wide range of tactile sensations that can be communicated from a mobile device to the human hand and/or the wrist

–

Provide psychophysical evidence for the design of tactile patterns to be used with mobile devices

Haptics implementation–

Touchscreen solutions–

Novel UI enabled or enhanced by haptics in conjunction with visual and audio effects


Backup slides


Haptics Characterization Voltage –

Acceleration Calibration

•

Observation–

Good linearity between Vpp to piezo and Accel_pp in all driving waveforms•

For sinusoidal wave, Accel ~ ω2*Displ ~ ω2*V –

Consistent slopes in both low and high voltage ranges–

Parameters from linear curve fitting are used to calculate the actual Accel_pp

Threshold Calibrationy = 0.216x - 0.033

R2 = 0.9996

0

2

4

6

8

10

0 10 20 30 40 50Voltage (v)

Acce

lera

tion

p-p

(g)

Step

sin150_1

Sin150_2

sin300_1

Sin300_2

Linear (Step)

Comfort Level Calibration

y = 0.212x - 0.7498R2 = 0.9986

0

10

20

30

40

50

0 50 100 150 200 250Voltage (v)

Acce

lera

tion

p-p

(g)

Step

sin150_1

Sin150_2

sin300_1

Sin300_2

Linear (Step)

Microelectronics & Physical Sciences Research

Multi-Function Transducer (MFT)

DIAPHRAGM

POLE PIECE

FRAME

MAGNET

TOP PLATE COIL PROTECTOR1

SUSPENSION

SUSPENSION

FOR SOUND

FOR VIBRATIONPROTECTOR2 SPACER

ACTUATORS

haptics for mobile devices

Documents