Use of Context Information for User Experience

Design

Mikhail Matveev,National Technology Officer,

Microsoft Russia

Daniel Kornev,Program Manager,

Microsoft Russia

Alexander Popov,MSTLab,Moscow State University

Agenda1. Existing examples of context

information2. Context definition3. Sensors and their applications4. Applying context information5. Sensor and Location Platform6. Market opportunities7. Summary

Apple iPhone• Natural User Interface:

• Gestures• Zoom in/out• Physical inertia applied to

digital objects

• Context-Aware Interfaces:

• UI Orientation (changes when physical orientation changes)

• Light-aware UI • Screen uses presence

sensor to automatically turn off while on call

• Automatic switching between tasks with applying priorities (media player pauses while on call and after call resumes playing)

Contextual tabs in Office 2007• Natural User

Experience:

• Ribbons for editing special objects like images, charts, tables etc appear only when these objects are selected by user

GPS and location-based applications• Context-Aware User

Experiences:

• Detection of user’s location

• Contextual information – POIs near user

• Directions• Traffic and auto-correction

of trip• Show user’s friends

location on map in real mode

Sound profiles in HTC Touch Pro• Sound Profiles:

• Alarm can wake you even if you turned off system sound off

• Sound profiles – normal/vibration/without sound

• Special profile “Automatic” – switches app behavior to “Vibration” while on meetings (data is populated from Calendar)

Existing examples• Ambient Light Sensors in cars,• GPS and location-aware applications,• IM Presence (Skype/Messenger/ICQ)• Changing screen layout depending on

device’s physical orientation (iPhone)• Contextual Tabs in Microsoft Office 2007• Sound Profiles in HTC Touch Pro• …

Context is…• Situation, in which the product is

used and situation changes

• Let’s take a look on definitions of Context given by different companies and organizations, Intel and ISO

What is Context? (Intel)

© Intel 2008, “Context Aware User Interface”, Intel Developer Forum

• Context-aware system can be aware of:

• Environmental Contexts– Physical environment– Devices– Services

• User context– Identity– Physical state– Physiological state– Activities– Contacts– Preferences

• Platform Context– Location– Motion– Network– Batter Status– Platform Features– Running Apps– Memory usage

What is Context? (ISO)User• Goals• Tasks

SocialPhysical

Infrastructure• Hardware• Software• Materials

Context combines users, their goals and tasks, infrastructure (hardware, software, materials), physical and social environment in which the product is used.*

*as defined by ISO 9421-11

What are context-aware systems?

• Context-aware systems are those changing their behavior when the situation changes

Sensors• Sensor is a device that measures a

physical quantity and converts it into a signal which can be read by an observer

• Sensors make possible for computer devices to detect elements of contexts that are hard to impossible to discover by using traditional methods.

Sensors and context-aware solutions are on the market

alreadyResearch

organizations Industry Sensors Vendors

+Universities

Research Institutes

+PartnersSoftware

Developers

Ideas to think about1. The computer volume automatically adjusts based on the noise level of

the environment. For example, when I’m in a noisy location, my media player sets the volume level up.

2. The display adjusts brightness and contrast based on current lighting conditions.

3. Text, links, and buttons get bigger on my touch-enabled mobile computer when I’m on-the-go, so they are easier to use. For example, when I’m holding my device in hands, the text and images on the screen automatically get larger.

4. The computer knows when it has moved from work to home and changes the desktop background. It can also hide work-related environment and show game and entertainment applications shortcuts.

5. When the display is rotated, layout changes from landscape to portrait and vice versa.

6. The computer can monitor the health status of me or a person I care about. For example, it can monitor their vital signs, including blood pressure, heart rate, and blood sugar levels.

Limitations of sensors today• Location devices are exposed

as virtual COM ports– Exclusive application access– Not secure– Proprietary data formats (NMEA, others)

• GPS doesn’t work indoors– Hard to support multiple technologies at once

• Sensors are integrated as vertical solutions– Applications need to know sensor

hardware specifics– Limited adoption and scope

Sensor and Location Platform

• New platform for Windows 7• Provides unified driver model for

all types of sensor devices– Physical sensors (e.g., GPS devices, Light Sensors)– Logical sensor (e.g., Wi-Fi triangulation resolver)

• Provides standard APIs for accessing sensors– Sensor API: C++/COM

• Raw access to any sensor– Location API: C++/COM, IDispatch (scriptable)

• Abstracted API for location data• Puts user in control of information disclosure

Sensor and Location Platform

UMDF Sensor Driver

Sensor Class Extension

Sensor API

Gadget or Script Application

Location and Other

Sensors Control Panel

SensorDevice

Location APILocation IDispatch Interface

Application

UMDF Sensor Driver

Sensor Class Extension

Logical Location Sensor(Triangulation)

UserSystem

Location Platform benefits• Single API call to answer “Where am I?”– Independent of provider used to

determine location– Synchronous and Asynchronous models– Script/automation compatible

• Automatic transition between providers– Most accurate providers have priority

• Concurrent access for multiple applications• Default Location– Provided by user as fallback when

no other sources are available

Types of location data• Geographic Data (ILatLongReport)– Latitude, longitude, altitude,

associated error required– Most common format– Best format for precise location– Can reverse geo-code later

• Civic Address (ICivicAddressReport)– Zip, Country required– Most human readable– Best for ‘rough’ location estimates,

street directions

Demo

Privacy and Access Control• Location data is considered PII

– User consent is required to share data• All sensors are disabled by default• Admin rights required

to enable a sensor• Sensors can be

configured on a per-user basis

• “Enable Dialog” invoked by applications

Resources and materialsSites and resources:

• Sensor and Location Platform web site:• http://www.microsoft.com/whdc/sensors/

• Dan Polivy (Lead PM, Sensor & Location Platform) presenting at Microsoft PDC 2008:

• http://channel9.msdn.com/pdc2008/PC25/• http://www.aeroxp.org/2008/11/windows-

7-sensors-framework-interview-from-pdc/

• MSDN Forum for developers:• MSDN Forum

• Reverse Code (LatLong ↔ CivicAddress)with Virtual Earth:

• http://msdn.microsoft.com/en-us/library/cc966828.aspx

Blogs:

• Gavin Gear (Sensors PM):• http://blogs.msdn.com/gavingear/

• Jim Travis (SDK Lead):• http://blogs.msdn.com/jimtravis/

• Daniel Kornev:• http://blogs.msdn.com/semantics/

• Jensen Harris (Office):• http://blogs.msdn.com/jensenh/ar

chive/tags/Contextual+UI/default.aspx

Maslow's hierarchy of needsSelf-actualization

Esteem

Social

Safety

Physiological

Microsoft’s positionApplications

Sensor Platform

Operating System

Computer hardware

Sensors

Business Model: from Innovation to Platform

Innovative idea generation and

prototype development

Standardization and

integration into product

Use of the scale effect

Transformation

into platform

ROI measuremen

t

Customers satisfaction

Investors satisfaction

Ecosystem satisfaction

Trends

• Sensor and actuator markets are experiencing a rapid growth• Industry shifts towards active use of sensors and context information in

user interface and consumer product development

Summary

User

SocialPhysical

Infra-structu

re

ContextSensors

Context-aware systemsPlatform

Call to action• Utilize the Windows Sensor and Location

platform to build location-aware applications, gadgets and services

• Implement light-aware UI in your applications to support betterreadability and UX under various environmental conditions

• Build environmentally-aware applications using additional sensors with the sensor and location platform

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

The End