Improving user experience by leveraging the contextThe rise of the mobile sensors

Florent StroppaProduct Director

About me• studied electronics, optics, computer science, network

architecture and MEMS

• joined the mobile scene 10 years ago and still passionate about it

• not a graphic designer but a UX fanatic

• work with UX designers, software engineers, product managers on projects ranging from Voice, SMS services to iPhone and Web apps

Which elements define the device user experience?

An engineer perspective

What form it is?Mobile phones come in very different form factors.

While manufacturers are continually coming up with new designs, there are several common categories which can be identified.

http://www.phonescoop.com/glossary/term.php?gid=4

How does it feel?

• Is it heavy or light?• Is it cold or warm?• Does it have curves or straight lines?• Does it look expensive or cheap?• How does it sound?

Which input?• Does it have a keypad?• Does it have a keyboard?• Does it have a touch screen?• Does it have a trackball?

What is the UI philosophy?

How fast it is?• How long does it take to start an app?• How long does it take to make a call, take a picture? • How does it respond to user’s input? • How fast does it change screens?• How fast does it connect to the network? • What about network latency?

How smart it is?

• Does it predict my input?• Does it predict my intent?• Does it know where I am and act upon it?• Does it do what I want to do?

While there have been strong improvements in device design, operating systems, user interfaces, application development and networksit seems that...

Phones are still stupid

They are always with us, in the palm of our hands, in our pockets or purses

but...

while I am speaking

while she’s driving

while he’s dancing

while he’s sleeping

They don’t know about it

This is improving...

Phones begin to behave differently

based on...

orientation

location and directions

acceleration

This is not enough

Phones need to be aware of the context

date, time, location, ambiance, our agenda and our relationship with others.

What is the context?

Context

Situation Date and time

Location Ambiance Proximity

Personal Data ContactsCalendar

User ProfilePhotos

Social contextSocial networks

Local newsGlobal news

PhysicalPositionActivityState

PatternsShort-termNew contact

Search New locationsCrowd pattern

Long-term Call logs

Messaging Previous app uses

Crowd pattern

date and time

• Date : day of week, week-end, holidays, seasons• Time : day or night, alarm clock• Mash-up with Calendar data

it’s not okay to be called

it’s okay to be called

Location

• Current user location• Usual user location : home, office, home country• Contact location : time zone, weather, price of calls• Roaming: time zone, calling scheme

work or holidays?

doesn’t locate like work

Situation : ambiance

• Sounds : noise level, noise pattern• Lights : light level, main colours• Motion : still, slow motion, running

sounds like a meeting

doesn’t sound like a meeting

contacts

• Gender for contextual messages and contact picture • Mother tongue for predictive text and spell checking• Group: family, friends, colleagues, customers

events

• Day events : meeting, business trip, conference call, reminders

• Evening : home, outside• Inconsistencies between current location and expected location

• Holidays

The enablers behindcontext-aware applications

The rise of the mobile sensors

The enablers• Processing power• Mobile sensors• Apps running in background• Personal data synchronisation• Communication patterns• Artificial Intelligence• The cloud

Mobile sensors• Location sensors• Accelerometers• Gyroscopes• Microphones• Camera• Compass• Pressure sensors

Future is bright for sensors

• In 2010, about 1 in 3 smart-phones will have accelerometers.

• Pressure sensors and gyroscope will follow.• The market for MEMS will top $1 Billion.

Background process

• To understand environment and context, apps need to run in the background

• No smart context-aware apps without background process

• Android, the best platform for background process management. iPhone does not provide support for developers.

“The mobile platforms, Android and the others, are so powerful now that you can build client apps that do magical things that are connected with the cloud”Eric Schmidt, Google

http://www.techcrunch.com/2009/10/28/googles-eric-schmidt-on-magical-potential-of-mobile-cloud/

The cloud

Leveraging them

Context

Situation

Personal Data Behavior

Expected reaction

Recommendation

Short-termLong-term

Patterns

AI +

Cloud

Learning

Perception

Pattern recognition

Data analysis

Crowd

Some samples of innovative concepts

Eye tracking concept

tat - 3D Eye Tracking

Reference : http://www.youtube.com/watch?v=7SImOIMcMlk&feature=player_embedded

Schizophrenic device

MIT Media LabHandheld devices showing different personalitiesReference: http://www.newscientist.com/article/dn16617-gadget-reads-users-minds-from-their-grip.html

Some Apple patents...Motion sensitive UIThe filing, entitled Variable Device Graphical User Interface, describes a self-adjusting user interface

"For example, if the user wishes to view the contact information for 'John Adams,' the user touches the display over the area of the row for the contact 'John Adams,'" Apple says. "While the device is moving, the motion of the device can be detected. The device can change the size of the rows of the contacts in the contact list application to give the user a larger target area for each contact. For example, the height of a row can be increased. This gives the user a larger touch area with which to select a contact. In some implementations, the height of the toolbar can be increased as well."

What does thismean for user experience

team?

UX work will be harder

• We are going from digital to analog, from binary logic to fuzzy logic

• The input is not only the user. There are also the environment, the context and the user’s data

• New tools will need to be found to formalise those new aspects

Agile methodology

• Agile development • Cross-functional team (ux, ui, tech, marketing)• Prototyping• Rapid and iterative process

Our experience withfuture mobile phones will

be radically different

Phones will be

Beautiful Intelligent&

LocationCalendarClockSimple AI

Won’t interrupt me while I am speaking thanks to

LocationAccelerometerGyroscope

Will use the speakerwhile she’s driving thanks to

MicrophoneCameraGyroscope

Will vibrate in his pocketwhile he’s dancing thanks to

LocationAccelerometerClockMicrophone :)

Won’t ringwhile he’s sleeping thanks to

Thank you