het collectief centrum van de Belgische technologische industrie

Tom Tourwé, Elena Tsiporkova (Sirris, BE)

Ville Antila (VTT, FI)

Monique Hendriks (Philips, NL)

Context-aware Personal Attentive Systems

Smarcos – Goal & motivation

• Smarcos aims to improve the interusability of interconnected embedded devices

• Smarcos relies on the solid technological basis for system-level interoperability resulting from other EU R&D projects

• Eg. ARTEMIS-Sofia and FP7-SmartProducts

• Smarcos addresses challenges arising from user-levelinteraction of interconnected systems

• Cross-platform interaction• Multi-user and multi-device collaboration• Synchronisation of system actions to maintain fluent task flow• Application and service adaptation to changing situational

contexts

Smarcos - Research topics

Design methods for embedded UIs

Interface, interaction & context models

Runtime UI & context composition

UI migration

(Distributed) Context processing

Distributed functionality

Model-based UI generation

Personal attentive systems

• People have difficulty incorporating healthy habits into their daily life

• Office workers: working on a computer, prolonged sitting, deadlines, meetings, travel by car

• Diabetes type II patients: monitoring of glucose levels, effects of food, medicine adherence, physical activity, learning to cope with the disease

• Doctors and lifestyle coaches can provide advice, but they cannot

• Intervene at the right moment

• Provide feedback on behaviour directly afterwards

• Have intricate knowledge of people’s daily routine

PAS in Smarcos

Context Interpreter

Decision Engine

Context Reasoner

Context Models

Data Layer Capture raw data from device sensors

Transform raw data into infor-mation about the user context

Infer new knowledge from information

Conceptual architecture

Determine content, timing &modality of feedback

Life-pattern detection through life logging

• Low-power context logging software

• Semantic location detection using cell-id (low power, always available)

• Device usage detection (algorithms for mining location relative to smartphone application usage)

Data Layer

Detecting important places and daily routines

• Estimation of life patterns such as the semantic location of the user (e.g. “home”, “office”)

• Detection of device usage in different locations

• Detection of physical activity in different situations

Context Interpreter

Can be used to …

• Determine the devices that surround the user• e.g. at work, the user has access to his personal computer

• Time and adapt system feedback based on the situation• e.g. time-shift notification to where user is more receptive

• Log context changes and react appropriately• e.g. user entering an interesting location/situation such as ‘office’

Context Interpreter

Domain knowledge for PAS

• Such domain knowledge is any factual information about persons, activities, locations and devices which influences the decision about at what time, on which device and through which modality feedback should be delivered to the user

• Such domain knowledge is captured in a context model

Context Models

Example domain knowledge

• If the user is cycling then• available devices are mobile devices (phone & activity monitor)

• availability for receiving messages is low

• A pill reminder message is• an urgent message

• best delivered through the audio modality

Context Models

Infer new knowledge from information

• The goal of the reasoner is to classify devices as being ACCESSIBLE, AVAILABLE and/or USED

• using the OWL reasoner and SWRL rules

• ACCESSIBLE > AVAILABLE > USED • A device is an ACCESSIBLE_DEVICE if the user has access to it

• A device is an AVAILABLE_DEVICE if it is an ACCESSIBLE_DEVICE which is ON

• A device is a USED_DEVICE if it is being used by a user

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

THING

elenas_televisionTHING

elenas_livingroom

is_performed_by

is_performed_on

is_accessible_from

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

THING

elenas_televisionTHING

elenas_livingroom

is_performed_by

is_performed_on

is_accessible_from

performs

provides_access_to

Inverse properties

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_televisionTHING

elenas_livingroom

is_performed_by

is_performed_on

is_accessible_from

performs

provides_access_to

WATHINGTELEVISION is_performed_on only TELEVISION

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

TELEVISION is_accessible_from only HOME

is_performed_by

performs

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

performs o is_performed_on -> uses

uses

performs

is_performed_by

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

Inverse properties

uses used_by

performs

is_performed_by

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

uses o is_accessible_from -> is_located_in

uses used_by

is_located_in

performs

is_performed_by

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

TELEVISION

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

Inverse properties

uses used_by

is_located_in

is_location_of

performs

is_performed_by

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

USED_DEVICE

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

USED_DEVICE used_by min 1 USER

uses used_by

is_located_in

is_location_of

performs

is_performed_by

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

AVAILABLE_DEVICE

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

AVAILABLE_DEVICE is superclass of USED_DEVICE

uses used_by

is_located_in

is_location_of

performs

is_performed_by

Context Reasoner

USER

elena

WATCHINGTELEVISION

watching_television

ACCESSIBLE_DEVICE

elenas_televisionHOME

elenas_livingroom

is_performed_on

is_accessible_from

provides_access_to

ACCESSIBLE_DEVICE is superclass of AVAILABLE_DEVICE

uses used_by

is_located_in

is_location_of

performs

is_performed_by

Context Reasoner

Determine content, timing& modality of feedback

• IF the situation is suitable, THEN send the message (as it is)

• IF the situation is not suitable AND the message is not urgent, it should be time-shifted

• IF the situation is not suitable AND the message is urgent, then the content should be adapted to the situation

Decision Engine

as_is(Message, User, Device) :-send_message(User, Message),user_availability(User, high), has_access_to(User, Device),supports(Device, Modality),requires_modality(Message, Modality).

adapt(Message, User, Device, Alternate_Modality) :-send_message(User, Message),urgent(Message),has_access_to(User, Device),requires_modality(Message, Preferred_Modality),not(supports(Device, Preferred_Modality)),supports(Device, Alternate_Modality).

Conclusion

• Presentation mainly reports on research results obtained by VTT and Sirris within the PAS use case defined by the Dutch consortium

• Future research• Implementation of the decision engine in a Java-based logic

engine

• Integration of all individual components in a common prototype

• Explore distributed context processing application

• Carry out field studies to validate the effectiveness of timing and device choice

• Deriving general statements regarding use case specific reasoning about context