from context-awareness to human behavior patterns

22
From Context-Awareness to Human Behavior Patterns Detection of Daily Routines using Smartphones Ville Antila (Research Scientist, M.Sc.) VTT Technical Research Centre of Finland, Oulu, Finland Philips Research, Eindhoven, The Netherlands (Visiting researcher)

Upload: ville-antila

Post on 13-Jan-2015

542 views

Category:

Technology


3 download

DESCRIPTION

Invited talk at Human Media Interaction unit at University of Twente

TRANSCRIPT

Page 1: From Context-awareness to Human Behavior Patterns

From Context-Awareness to Human Behavior Patterns Detection of Daily Routines using Smartphones

Ville Antila (Research Scientist, M.Sc.)

VTT Technical Research Centre of Finland, Oulu, FinlandPhilips Research, Eindhoven, The Netherlands (Visiting researcher)

Page 2: From Context-awareness to Human Behavior Patterns

Background – Smarcos project

• Smarcos creates solutions to allow devices and services to exchange context information, user actions, and semantic data

• One important part of the work has been to investigate the practical usage of context information and to develop models that can be dynamic and adaptive as well as applicable to different applications

• www.smarcos-project.eu

Page 3: From Context-awareness to Human Behavior Patterns

Outline of the talk

• Introduction

•From context logging to routine detection• Continuos, low-power “life-logging”

• Interpreting the data (what’s the meaning of it)

• Using domain knowledge to reason about what we don’t know

•Example applications

•Discussion and summary

•Video?

Page 4: From Context-awareness to Human Behavior Patterns

Introduction - context awareness

• Idea that computers can both sense and react to their environment• “any information that can be used to characterize the situation of an entity” [Dey et al., 2001]

• Human factors: information about the user, social environment, user’s task

• Physical factors: Location (absolute, relative, co-location), infrastructure, physical conditions

• Context aware systems should be able to gather context (sense the situation), abstract and understand it and adapt application behavior based on the context

• Some classical use cases:

• adapt user interfaces

• tailor application-relevant data (e.g. filtering information)

• increase precision of information retrieval

• make the user interaction implicit

• discover services & build smart environments

Page 5: From Context-awareness to Human Behavior Patterns

Introduction - context awareness

• Smartphone is the epitome of a sensing platform for context-awareness

• Personal and mobile (almost always with the user)

• Comes with a lot of in-built sensors and communication capabilities

• Used everywhere, for multiple tasks (on-the-go)

Page 6: From Context-awareness to Human Behavior Patterns

Introduction - challenges & opportunities

• The notion of ‘context’ can not be objectively defined (a prior) by settings, actions and actors

• Rather, context is the meaning that the actions and actors acquire at any given time from the subjective perspective [Mancini et al., 2009]

• Context information can have multiple purposes (from user’s perspective)• “Declaring one’s position is perhaps as much about deixis (pointing at and referencing features of the

environment) as it is about telling someone exactly where you are” [Benford et al., 2004]

• When going from individuals to larger groups of people it’s possible to extract patterns from the data [Eagle and Pentland, 2006]

• social patterns (e.g. urban heat maps)

• identification significant locations

• model organizational rhythms

Page 7: From Context-awareness to Human Behavior Patterns

From Context Logging to Routine Detection

• Use the gathered knowledge to form decisions about system behavior in different contexts/situations

Context Interpreter

Support for decision making

Reasoning Domain knowledge

Data Layer

Applications

• Infer new knowledge from information

• Capture raw data from device sensors

• Transform raw data into information about the user’s context

• Example applications...

Page 8: From Context-awareness to Human Behavior Patterns

Continuous Life Logging

• Context-awareness

• Using smartphones as sensors for human activities (e.g. important locations, mobility patterns)

• Low-power context logging software

• Semantic location detection using cell-id (low power, always available)

• Device usage detection (algorithms for mining location relative to smartphone application usage)

• Lo-fi physical activity detection (e.g. is the user moving currently)

• Scanning Bluetooth snapshots to determine indoor environment (e.g. is the user at his office desk)

Data Layer

Page 9: From Context-awareness to Human Behavior Patterns

Context Components For real-time user behavior detection

• Collection of software components for enabling

• continuous context logging

• development of context-based adaptation for variety of applications

• Implementations available for several platforms (some of which are becoming obsolete)

• Android, Symbian, Maemo/Meego/Linux, BlackBerry

Page 10: From Context-awareness to Human Behavior Patterns

Interpreting the data...

• Estimation of life patterns such as the semantic location of the user (e.g. “home”, “office”)

• Detection of device usage in different locations

• Detection of physical activity in different situations

• Detection of changes in routines

Context Interpreter

Page 11: From Context-awareness to Human Behavior Patterns

Context CaptureContext-based awareness cues in information sharing

• We explored the usage of contextual information cues in informal information sharing

• The study focused on practices of ‘abstraction’ when publicly sharing contextual information

• Field test for our backend

Page 12: From Context-awareness to Human Behavior Patterns

Smartphones are used almost everywhere...Moreover there is an “app” for almost anything...

An opportunity (to use smartphone apps as sensors for situations...)?

Image sources: http://adage.com/article/digital/placing-ads-underestimate-mobile/230853/

http://www.resultrix.com/blog/index.php/tag/tablet-marketinghttp://www.google.com/googleblogs/pdfs/mobile_understanding_smartphone_users.pdf

Page 13: From Context-awareness to Human Behavior Patterns

• An approach to detect day-to-day ‘routines’ by logging the smartphone application usage and locations where they are used

• Analysis of logged usage data into identifiable patterns (clustering based on location and time of use)

• Implemented an experimental smartphone application with a functionality to create automated ‘tasks’ out of the identified patterns

• Conducted a two-week user study to analyze the approach and to gather user feedback

Routine Maker End-user automation of smartphone routines

Page 14: From Context-awareness to Human Behavior Patterns

Extending the knowledge...

• By modeling domain knowledge we can reason about the consequences of what the derived context information means in the particular application scenario

• For example, if know (to a certain degree of accuracy) that the user is cycling, then we can reason that:

• available devices are mobile devices (phone & activity monitor)

• availability for receiving messages is low

Domain knowledge Reasoning

Page 15: From Context-awareness to Human Behavior Patterns

Where would this information be useful?

• Determine the devices that surround the user

• e.g. at work, the user has access to his personal computer

• Time and adapt system feedback based on the situation

• e.g. time-shift notification to where user is more receptive

• Log important events and use those to automate tasks

• e.g. migrating task sessions automatically between personal devices

Page 16: From Context-awareness to Human Behavior Patterns

Example application:

Context-adaptive Feedback

• Goal: increase effectiveness and decrease interruptions

• How: adaptive selection of device, modality and timing of feedback

• Example of adaptive feedback delivery:

• IF the situation is suitable, THEN send the message (as it is)

• IF the situation is not suitable AND the message is not urgent, it should be time-shifted

• IF the situation is not suitable AND the message is urgent, then the content should be adapted to the situation

Applications

Page 17: From Context-awareness to Human Behavior Patterns

Example application:

Context-based User Interface Migration

• Migration concept• Interface moving from source device to target device

• Interaction state persistence

• Interaction continuity

Applications

Page 18: From Context-awareness to Human Behavior Patterns

Example application:

Context-based User Interface Migration

• Taking advantage of the known properties of the environment in any given time (e.g. Bluetooth, GPS, device stability and orientation) we can automate tasks such as UI migration triggering

Applications

Page 19: From Context-awareness to Human Behavior Patterns

Discussion

• Just like the smartphone sensor APIs have matured (GPS, acc, gyro, proximity), the basic context abstractions will also be served as OS-level services in the future (e.g. walking/running/still, home/office/school/on-the-go)

• Better optimizations regarding battery, CPU and memory usage etc. (resolves the iOS background processing challenge)

• Cross application usage

• Fusion with other available data on the device and “in the cloud”

• Going further we can also foresee taking inputs from the environment (e.g. WSNs) as well as “negotiating with” other smart devices while trying to reach a better approximation of the situation

Page 20: From Context-awareness to Human Behavior Patterns

Discussion - challenges

• Quality/accuracy of detection (have to be started off with simple cases), provenance of quality measures into the application level

• User interaction/awareness of application behavior

• Designing for applications that adapt to situations... Patterns, guidelines, best practices?

• Prototyping context-adaptive applications (from early interactive prototypes to functional prototypes)

• Testing

• Functional testing, performance/quality testing

• User testing (‘in the wild’)

Page 21: From Context-awareness to Human Behavior Patterns
Page 22: From Context-awareness to Human Behavior Patterns

Thank you!Questions?

Ville Antila [email protected]

From Context-Awareness to Human Behavior PatternsDetection of Daily Routines Using Smartphones