improving access to mobile technologies using tactile feedback

Improving access to mobile

technologies using tactile feedback

Huimin Qian

Information Systems Department

UMBC

October 3rd, 2012

2

Motivations

Proposed solution

Projects

Summary

Mobile devices

Interacting with desktop computers Desktop computers

Interacting with mobile devices

In mobile contexts

3

Motivations

Proposed solution

Projects

Summary

Mobile devices

Interacting with desktop computers Desktop computers

Interacting with mobile devices

In mobile contexts

4

Inaccessible for

blind users

[Shaun, 2011]

Motivations

Proposed solution

Projects

Summary

5

Inaccessible for

sighted users in a number of situations

[Sears et al., 2003; Brown et al., 2005; Wobbrock, 2006]

Walking Driving

In a class In social situations

Motivations

Proposed solution

Projects

Summary

http://www.verizonwirelessphone.biz/cell-phone-safety-drive/using-cell-phone-while-driving/

6

[Sears et al., 2003; Brown et al., 2005; Wobbrock, 2006]

Walking Driving

In a class In social situations

Motivations

Proposed solution

Projects

Summary

“Situationally-induced impairments and disabilities”

SIID [Sears & Young, 2003]

Inaccessible for

sighted users in a number of situations

http://www.verizonwirelessphone.biz/cell-phone-safety-drive/using-cell-phone-while-driving/

7

• Sight

• Smell

• Taste

• Hearing

• Touch

Restricted

Impractical

5 communication channels

Motivations

Proposed solution

Projects

Summary

8

5 communication channels

In loud & noisy environment

In social situations

In situations like those,

users’ hearing capability

is restricted.

• Sight

• Smell

• Taste

• Hearing

• Touch

Restricted

Impractical

Motivations

Proposed solution

Projects

Summary

9

Proposed solution

• Develop touch-based interaction techniques,

using vibrations/tactile icons (tactons) to

convey peripheral messages via mobile

devices in the absence of vision and audio.

• Note:

• Vibrations, Tactile icons, Tactons, Haptic

cues

Motivations

Proposed solution

Projects

Summary

10

Advantages of proposed solution

Reduces visual demands associated with

using mobile applications

Avoids disruption on users’ main tasks

Enables eyes-free and hands-free interactions

Provides a private communication channel

Motivations

Proposed solution

Projects

Summary

11

Design, evaluate & use tactile icons

• [Design]

• [Evaluation]

• [Use]

Motivations

Proposed solution

Projects

Summary

Phase 1: Develop tactile icons for use with mobile devices

Phase 2: Examine the effectiveness of tactile icons with

auditory distracters

Phase 3: A haptic personal training application

12

Phase 1: Design

• Goals:

– To build a low-cost & configurable tactile

display system

– To broaden design space of tactile icons by

exploring new parameters of touch

– To understand human tactile perceptual

limitations

Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


13

Arduino BT Board

Vibration actuators

Phase 1: Tactile display system

Nokia N95

Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


14

Phase 1: Parameters of touch

Parameters Value One Value Two

Duration Long (0.8Sec.) Short (0.2Sec.)

Interval Long (1Sec.) Short (0.2Sec.)

Intensity Strong (255Hz) Weak (153Hz)

Location Volar Wrist Dorsal Wrist

Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary

16 types of tactile icons


15

Phase 1: Examine parameters of

touch

• 20 participants ,within-subject study

– Training (10 minutes)

– Trials (40 minutes)

• Each type of tactile icons repeated 3 times

• Participants can replay each icon as many times as

they feel need to

• Participants submit answer in an online form after

recognizing a tactile icon

– Post-study interview (10 minutes)


Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary

16

Phase 1: Results

• The tactile icons are easy to identify:

– Recognition rate - > 87%

– Recognition time - <14 seconds

– Cognitive workload - below 3 in 1-5 Likert scale (1:

extremely low workload; 5: extremely high workload)

Users can recognize 4 tactile parameters with

moderately high recognition rate and medium level

of cognitive workload

Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


17

Phase 2: Evaluation

• Goal:

– To examine the effects of auditory distracters

on the use of tactile icons in mobile devices

[Qian, Kuber & Sears, IJHCS ’11] Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary

Phase 2: Examine tactons in lab with auditory distracters

18

Phase 2: Experiment design

• Three types of auditory distracters:

1. White noise

2. Street noise

3. Rock music

• Presented via headphones and speakers

• Two types of tacton pairs:

– A distinct pair

– A non-distinct pair

• 18 participants (8F, 10M, Age 19-27)

[Qian, Kuber & Sears, IJHCS ’11]

Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


19

Phase 2: Results & implications

• Different audio affects tactile perception

differently

– Higher recognition accuracy and less recognition time

in quiet lab with no external sound presented

Tactile perceptual tests should be performed under

conditions which include background audio.

– Longer recognition time and higher cognitive workload

in music, compared with street noise

– White noise resulted in increased cognitive workload,

but did not impact the tactile recognition

White noise is not an acceptable substitute for more

realistic sound.


Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


20


• The more distinct tacton pair can better resist

auditory distracters

– Less recognition time to identify distinct

tacton pair

– Higher recognition accuracy to identify

distinct tacton pair

Careful design to maximize distinctiveness of

tactons is even more important for mobile

devices.



Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary

21

Phase 3: Use

• A mobile exertion application integrated with

tactile feedback to help senior adults monitor and

adjust walking speed

• Step rate detected every 4 seconds using built-in

accelerometers

• Two tactile/auditory icons to notify users to walk

faster/slower

[Qian, Kuber & Sears, CHI ’10, INTERACT ‘11; Qian, Kuber, Sears & Murphy, INTCOM ’11]


Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary

A pedometer & a phone holder

22

Phase 3: Use

• Goals:

– To assess the effectiveness of tactile icons in

conveying commands in mobile applications

– To examine the impact of motion on

perception of tactile icons

– To study the change of users’ behaviors along

with the use of the mobile application


Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


23

• 3 healthy 65+ senior adults over 2-week

period

• Training in lab

– 3 types of feedback with the same rhythm

(Audio icons, Tactile icons, Audio+Tactile icons)

• Data collection methods

– Automated logging on the phone

– Participant-maintained diaries

– Interviews

Phase 3: A field study


Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


24


• Multimodal icons work better than unimodal

icons

In compromising environment or demanding

tasks, multimodal icons should be used

• Tactile icons work better than auditory icons

• Users’ preferences change depending on the

situational context

The design of interaction system for senior

adults needs to consider the age-related

declines in hearing and touch


Motivations

Proposed solution

Projects

Phase 1

Phase 2

Phase 3

Summary


25

Summary

• [Design]

• [Evaluation]

• [Use]

A set of identifiable tactile parameters are identified

A low-cost tactile display system is developed

The impact of various auditory distracters is examined

Implications for mobile interface designers are provided

A haptic personal training application is developed

The use of tactile icons are evaluated in field

Guidance to developers and researchers is provided

Motivations

Proposed solution

Projects

Summary

26

Questions

Huimin Qian

Ph.D. Candidate

Information Systems Department

UMBC

Email: [email protected]

Motivation

Related work

Proposed solution

Research Qs

Finished studies

Upcoming studies

Timeline

27

Publications

• Towards Identifying distinguishable tactons for use with mobile

devices (ASSETS, 2009)

• Towards Developing Perceivable Tactile Feedback for Mobile

Devices. (International Journal of Human Computer Studies,

2011)

• Maintaining levels of activity using a haptic personal training

application. (CHI ,2010)

• Maintaining and Modifying Pace Through Tactile and Multimodal

Feedback (INTCOM, 2011)

• A Longitudinal Pilot Study to Evaluate Non-Visual Icons in a

Mobile Exertion Application. (Proceedings of INTERACT, 2011)

Motivations

Proposed solution

Projects

Summary

improving access to mobile technologies using tactile feedback

Documents