application development for mobile and ubiquitous ...ts2/admuc/lecture1415/2. context...

Department of Computer Science Institute for System Architecture, Chair for Computer Networks

Application Development for Mobile and Ubiquitous Computing

2. Context Awareness

Dr. Ing. Thomas Springer Technische Universität Dresden

Chair of Computer Networks

Lecture Structure

Dr. Thomas Springer Slide 2 Application Development - 2. Context Awareness

Structure

§  Basics •  Context definition •  Properties of context

§  Gathering §  Abstraction §  Provisioning

•  Context modelling •  Middleware for context-awareness

Dr. Thomas Springer 3 Application Development - 2. Context Awareness

Motivation

Mobility and location

Computing environment, device and

network resources

Interaction capabilities and user attention

Situation/location dependent information

Personalized and simplified user interactions

Adaptive system behaviour

Proactive, autonomous systems

Context-aware Applications Understanding of users current situation

Control of adaptation processes


Social environment and activity

Definition of Context

§  “… all but the explicit input and output of an application” [LiS00] §  “… that which surrounds, and gives meaning to, something

else” [o.V.00]

§  “Context is a subjective concept that is defined by the entity that perceives it" and „contextual states [...] are inherently associated with specific objects” [Pascoe]

§  „Context is an operational term: Something is context because of the way it is used in interpretation, not due to its inherent properties. [...] Features of the world become context through their use.“ [Winograd]

§  “Context is any information that can be used to characterise the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves” [Anind K. Dey]


Types of Contextual Information


C o n t e x t u a l dimension

Respective contextual information

P h y s i c a l context

Location, time, temperature, light and noise intensity, nearby persons

T e c h n i c a l context

Network (bandwidth, latency, error rate), Device (input and output capabilities, memory, software support), available services, service preferences

P e r s o n a l context

Address, phone number, payment, preferences, schedule, service preferences

Social context Nearby persons, groups (teams) to which the user belongs

O p e r a t i o n a l Context

Roles, activities, to-do-items, content of the inbox of the user

System Example: Technology for Enabled Awareness (TEA)

§  Sensors on mobile device §  Goal: Sensing the situation of a

mobile device and its user §  Location and usage of

mobile phone §  Automatic selection of

phone profile

§  Used Sensors §  two photodiodes §  two microphones §  dual axis accelerometer §  digital temperature sensor §  touch sensor §  board extensible with further sensors (slots available on the

board)


TEA – Computed higher-level Contexts

§  4-layered model: 1. Sensors (physical, logical) 2. Cue (processing of one sensor output)

o  contains array of values, operations on these values, e.g. average, standard deviation

3. Context (describes disjunct situations together with probability) o  Description of the current situation on an abstract level o  based on logic rules operating on cues

4. Application level (Situation and application specific) o  definition of actions based on scripts, triggered if a certain situation is

entered or left with a given probability

Application Development - 2. Context Awareness 8 Dr. Thomas Springer

TEA – Computed higher-level Contexts

§  Cues •  Computing method: statistical functions

•  Average value (for values of single sensor over about 1 min., applied for light, acceleration, temperature, and pressure)

•  Standard derivation (for values over about 1 min., applied for light, passive IR, acceleration)

•  Base frequency (applied for light and acceleration, types of light (flickering), acceleration pattern for moving)

•  First derivative (applied for passive IR (Infra Red) and acceleration)


TEA - Example

§  Exclusive contexts •  Rule set for recognizing the situations that device is situated in

Hand, Table and Suitcase based on the sensors light, and acceleration in two directions (X and Y)

•  Constants Dx, Dy, L, Xnormal, Ynormal, D, and Q defined based on observations

•  Location of mobile phone: hand/table/suitcase •  Device usage: inside/outside, car/bus/train, stationary/moving car


Comprehensive View on Context

temperature (12°) cold cloud formation, recommend user humidity (90%) high humidity high probably of to use the car air pressure (1005 hPa) high air pressure rain and storm instead of bicycle

11

Abstraction

Environmental information

Ci

Cm

Application

A1

Aj

An

C = ContextValue A = Action

C1'Interpretation

InterpretationFusion C2'

Interpretation

C4'

Context Capturing Context Abstraction Context Usage

History

Interpretation C1'

C1

Dr. Thomas Springer Application Development - 2. Context Awareness

Context Capturing Possible Context Sources

§  Context Sources are: •  sensor devices (e.g. GPS, temperature, light intensity, noise) •  databases (e.g. extracting context from structured data in DB) •  application data (e.g. scheduling app) •  user monitoring and input (e.g. location or task from current

user activity)


Context Capturing Sensor Access in Android

§  Access via the Android SDK •  android.hardware.SensorManager •  sensorManager = (SensorManager) getSystemService(SENSOR_SERVICE);

§  Sensor supported? •  sensorManager.getSensorList(SensorManager.SENSOR_ALL)

§  EventListener observes changes in sensor data class CompassListener implements SensorEventListener {

public void onAccuracyChanged(Sensor sensor, int accuracy) {…} public void onSensorChanged(SensorEvent event) { myCompass = Math.round(event.values[0]); printDirection(); }

§  Register for the updates when Activity is in foreground protected void onResume() { super.onResume(); sensorManager.registerListener(compassListener, sensorManager.getDefaultSensor(Sensor.TYPE_ORIENTATION), SensorManager.SENSOR_DELAY_NORMAL);

}


Context Capturing Properties of Context Attributes

§  Time dependent §  highest relevance at capturing time

-> decreases constantly §  History: represents values at

different points in time §  Location dependant

§  highest relevance at capturing place -> decreases with distance

§  Imperfect §  Unknown – sensor failure or

unavailability §  Ambiguous – due to conflicting

alternative values §  Imprecise – due to uncertainty in

measurements, application of heuristics

§  Erreneous – due to measurement failures, wrong assumptions for derivation, wrong user input


Context Abstraction

§  Goal: create higher level context from raw data or low-level information

§  Large set of methods possible

•  2 Examples

Dr. Thomas Springer Application Development - 2. Context Awareness 15

Managing Context Information in Mobile Devices (Korpipää et. al)

§  Application Scenario •  Environmental sound intensity controls volume of

operating tones •  Font size, screen brightness and service content are

adapted according to user activity and ambient light level §  Used Sensors

•  microphone, accelerometers, two channels for light, sensors for temperature, humidity, touch

§  Computed higher-level context •  Location {indoor, outdoor} •  SoundType {Car, Elevator, RockMusic, ClassicalMusic,

TapWater, Speech, OtherSound}

16 Dr. Thomas Springer Application Development - 2. Context Awareness


§  1. Phase: abstraction of raw sensor data (comparable to cues) •  Crisp limits: true-false labeling of sensed data (e.g. Silent,

Moderate, Loud for environment sound intensity) •  Fuzzy sets: overlapping ranges (e.g. 0.7/Silent,

+ 0.3/Moderate+0/Loud)

17 Dr. Thomas Springer Application Development - 2. Context Awareness


2. Phase: computing higher-level context using naive Bayesian networks


Context Abstraction Rule-based Reasoning

§  Use Case: Smart Home

§  Reasoning about users current situation at home

§  Modeling: •  CONON Ontology (CONtext

ONtolotgy) •  OWL, RDF Triples

§  Reasoning: •  based on OWL light and

RDF triples •  Rule-based Reasoner

(simple pattern matching algorithms)


Context Abstraction Methods

§  Interpretation •  Assigning raw data with semantics •  e.g. raw sensor data to temperature in K •  Mapping of numeric to symbolic values •  e.g. temperature in K to [cold|warm|…]

§  Alternatives •  e.g. choice between alternative values for

one context value •  e.g. dependent on different data qualities

or availability

§  Merging •  e.g. location information based on GSM

and signal strength of WLAN access point •  combined to more exact location

information

§  Derivation •  e.g. user logged in on local device and

device is connected -> user online

KV Interpretation KV’

Metainformation

KVtype A

Chose One KVtypeA

Metainformation

KVtypeA

KVtype A

Merging KVtypeA

Metainformation

KVtypeA

KVtype A

Derivation KVtypeC

Metainformation

KVtypeB


Context Abstraction Levels of Abstraction

ContextService

Abstraction

Source Domain Application Personal

Abstraction

Abstraction

Abstraction

Abstraction

Abstraction

Abstraction


Provisioning

§  Goal: Provide Context information to applications

§  Two aspects: •  Context middleware •  Context modelling

Dr. Thomas Springer Application Development - 2. Context Awareness 22

Characteristics of Context Sources

§  Source type – sensors, data bases, application data •  Access – SQL, sensor driver, extraction code •  Ganularity – user db contains comprehensive user information,

GPS device provides GPS coordinates only •  Level of abstraction – Sensor provides raw data, scheduling

application provides high level date information •  Distribution – mobile device, infrastructure

§  Semantics – homonyms (same word for different things), synonyms (different words for same thing)

§  Owner – user, network provider, building owner §  Relevance – user, application, domain, generic

Ø  Users have their own configuration of context sources


Context Provisioning Why a Context Service?

§  Decoupling of applications and context sources §  Development support §  Reuse of generic functionality

•  Gathering, abstraction, storage , transport, application access •  Mediates contextual information between context sources and

context users

24

context source

context-awareapplication

context source



context source


context source



context service


Context Provisioning Basic Abstractions – Context Toolkit

§  Widgets: •  Sensor abstraction •  Represent single context value •  Encapsulate details of sensors

and other sources •  Current value, history,

subscriptions §  Aggregator

•  Assign several context data (widgets, interpreters) to entities

•  Easier management §  Interpreter

•  Processing of sensor data •  Generation of higher-level

context •  One or several input values •  Can be used by Widgets,

Aggregators, Interpreters and Applications

Context Architecture

Application Application

Interpreter Aggregator

Sensor Sensor

Interpreter

Widget Widget


Context Provisioning Sentient Computing

§  System Building blocks are Sentient Objects (SO)

§  Each SO represents context-aware funtionality of gathering, processing or applying of contextual information

§  Different types of objects •  sensors •  transformer •  actuator

§  Sentient objects contains internal represenation of contexts as well as transformation code (e.g. inference mechanisms)

§  Combination of sentient objects to context-aware applications


Mobile Device Mobile Device

Mobile Device Mobile Device

Context Provisioning Architecture – Central Context Server

+  Server can provide high storage capacity and performance to process context

+  Easy to discover and manage +  Central access control

-  Single point of failure -  Limited scalability due

centralized resource for high amount of clients

-  Network connection and infrastructure required between mobile device and server for context access


Context Server



Context Provisioning Architecture - Decentralized

+  Independent from network connection to certain server

+  Support of ad-hoc scenarios +  High scalability due to

distribution of context access

-  Availability of information depends on device availability, limited availability of user profiles

-  Limited performance and storage capacity on mobile devices for context processing

-  No central control to context access


Mobile Device

Application

Mobile Device

Application

Mobile Device

Application

Context Middleware

Context Middleware

Context Middleware

Context Provisioning Two Categories of Information

§  State Information •  Application actively requests required context (pull) •  Access to actual and historical data •  E.g. current location, device, etc.

§  Change Events •  Application registers for particular change events, Waits

passively for the events •  Context service notifies registered applications about changes of

state (push) •  E.g. location changes, access network changes


state event information current location of user, list of

printers nearby person X enters the room

action print a document using the closest printer

start communication with person x after she has entered the room

Context Provisioning Context Middleware API



Consumer API

request response

Provider API

Internal Context Representation

Context Source

Context Source

subscribe publish

Shared understanding about context information required

Context Modelling

§  Common understanding between context sources and context middleware

§  Sharing of context information between context middleware and application

§  Model for application development -> base for decision making

§  Approaches •  Context Profiles •  Object Role Models •  Ontology-based Models


Context Modelling Context Profiles

§  Name/Value Pairs: •  username = „Thomas Springer“, temperature = 21 C

§  Profiles •  triples – subject, predicate, object •  <http://www.examplepage.de> - author – „Peter Müller“ •  RDF (Resource Description Framework) •  CC/PP (Composite Capabilities/Preference Profiles) •  CSCP (Comprehensive Structured Context Profiles)


Context Modelling Object Role Model - (Henricksen, Indulska)

§  Goal: extend context modelling to handle imperfection §  Context is modelled based on facts, objects and roles §  Model can be mapped to relational data model


Fact type Object type

Context Modelling Ontology-based Context Models

§  describes real-world as formalized concepts

§  consists of concepts and roles (T-Box)

§  individuals as instances of concepts (A-Box)

§  Web Ontology Language (OWL) •  defined by W3C •  based on RDF •  3 variants: OWL lite, OWL

DL, OWL Full


terminological box (T-Box)

assertional box (A-Box)

Thing

Person Location

locatedAt

livesAt

Resident

Building

Home

Resident = Person and (livesAt some Home)

Alice AliceHomelivesAt

ResidentAtHome = Resident and (locatedAt some (Home or (isPartOf some Home)))

ResidentAtHome

nearby

§  “An ontology is an explicit specification of a conceptualisation” [Gru93]

Context Modelling CONON – Domain-Specific Ontology


Ref: Infocom Research, Singapore (Zhang)

CONON – CONtext ONtology

Ref: Infocom Research, Singapore (Zhang)


Context Models - Comparision

§  Key/Value Pairs +  Most simple approach, easy to manage -  Lack of structure and semantics

§  Context Profiles +  Hierarchically structured, Simple semantics -  Mainly for static and rarely changing context ->

Does not support frequent updates -  Goarse granularity for context exchange -> whole profile or

profile segments §  Object Role Model

+  Rich structure based on facts, objects and roles +  Can be mapped to relational data models -  Mix up modelling and capturing

§  Ontologies +  Can represent complex knowledge +  Clear structure and reuse based on upper ontologies +  Inference based on description logics reasoners -  High complexity, performance might be weak


System Example Distributed Context Service

Slide 38 Application Development - 2. Context Awareness

Terminal Device

Application (Client)

ContextManager

ContextBroker

NetworkLinkSource

DeviceSource

RemoteSource

Server

Application (Server)

ContextManager

ContextBroker

UserServicePrefSource

PersonSource

Terminal Device

Application (Client)

ContextManager

ContextBroker

NetworkLinkSource

DeviceSource

XML XM

L

RemoteSource

XML

PersonSource

UserServicePrefsSource

LocalSource

RemoteSource XML Remote Context Exchange

Main principles are: •  Decentralization •  Local handling of context sources •  Local access to context sources •  Abstract view on context sources (API)

Dr. Thomas Springer

Context Meta-Model: Syntactic Model

§  Meta model is the basis for context model and service implementation

§  Enables the representation of arbitrary contextual information

entity

entity

entity

attribute group

attribute value 1

entity

attribute value 1

attribute value n

attribute group

attribute group

attribute value 1

attribute value n

attribute group

attribute value 1

attribute value 1

attribute value n

attribute group

attribute group

attribute value 1

attribute value nassociation

association

association


Publish-Subscribe Access

Slide 40

1

0..*

1 1..*

1

0..*

0..*1

0..*1

ContextAttribute

-Type:String

+ContextAttribute+subscribeToEvent:boolean+unsubscribeFromEvent:void

Entity

-Type:String-AssociationList:Vector

+Entity+getAttribute:ContextAttribute+addAssociation:void+getAssociation:Association[]+subscribeToEvent:boolean+unsubscribeFromEvent:void

ValueGroup

+ValueGroup+getAttribute:ContextAttribute+subscribeToEvent:boolean+unsubscribeFromEvent:void

ContextValue

-Unit:String-Constraint:String-Value:Object-Dvalue:Object

Association

-Type:String

+Association+getAssociationEnd:Entity+subscribeToEvent:boolean+unsubscribeFromEvent:void

event type

relevant to class

description

change ValueGroup, ContextValue, Entity, Association

an event is created if the value or at least one of the values of a ValueGroup or Entity change in any way

enter ContextValue an event is created if the value changes from any value different to that given value

leave ContextValue an event is created if a value changes from a given value to any other value

buildup Association an event is created if a new association is built up

destroy Association an event is generated if an existing association is destroyed


Subscription Handling


consumer 1

Context Service A

consumer 2 consumer 3

Context Service B

consumer 4

local context source

consumer 5

consumer 6 subscriptions:- Context Service A- Context Service B- consumer 5- consumer 6

subscriptions:- consumer 3- consumer 4

subscriptions:- consumer 1- consumer 2

Context Service C

Context Ontology: Semantic Model

Person

NetworkLink

Device

Codec

uses

hasLink

supportsCodecUserServicePrefs

hasPrefs

Service

prefsFor

Schedule

hasSchedule

GlobalServicePrefsglobalPrefsFor

isBuddy

isUsedBy

AudioCodec

VideoCodec

AMM-Service

VideoOnDemand-Service

AMM-GlobalServicePrefs

VideoOnDemand-GlobalServicePrefs

AMM-UserServicePrefs

VideoOnDemand-UserServicePrefs

scheduleOf

codecOf

linkOfBuddy

entity subtype directedassociation

symmetricassociation

inverseassociations mapping view

<entity name="PersonEntity"> <attribute name="name" type="String" /> <attribute name="email" type="String" /> <assoc name=“usedDevice" type=“uses" /> </entity> <entity name="DeviceEntity"> <attribute name="processor" type="String" /> <attribute name="display.dim" type="Dimension"/> <assoc name=“usedBy" type=“uses" /> </entity> <association name=“uses"> <member role=“User" type="PersonEntity" /> <member role="Device" type="DeviceEntity" /> </association>


Model Integration

§  Integrated approach combining the aspects of syntactic and semantic models §  Context model is shared between applications, context service and sources

Domain A

Structure Definitionmobile communication domain

Structure Definitiondomain B

Domain B

Top-Level Ontology forMobile Networking and Communication

Lower Ontologymobile communication domain

Lower Ontologydomain B


Summary

§  Context information enables awareness of excecution environment

§  Pre-requisite to control adaptation processes •  Usually mapping between context information and

control parameters/information necessary §  The major phases of context processing

•  Gathering •  Abstraction •  Decision making

§  Context models allow sharing of context §  Context middleware provides general functionality to

gather, process, store, distribute and access context information


References

§  Loke, S.: Context-Aware Pervasive Systems – Architectures for a New Breed of Applications. Auerbach Publications, ISBN 978-0-8493-7255-1, 2006

§  Dey, A.K.: Understanding and Using Context. Personal Ubiquitous Computing, 2001, 5, 4-7

§  Schilit, B.N.; Adams, N. & Want, R. Context-Aware Computing Applications Proc. of the 1st Int. Workshop on Mobile Computing Systems and Applications, IEEE, 1999, 5-90

§  Winograd, T.: Architectures for Context. Human-Computer Interaction, 2001, 16, 401-419

§  Gu, T.; Pung, H. & Zhang, D.: Toward an OSGi-based infrastructure for context-aware applications. Pervasive Computing, IEEE, 2004, 3, 66-74

§  Strang, T. & Linnhoff-popien, C.: A Context Modeling Survey In: Workshop on Advanced Context Modelling, Reasoning and Management, UbiComp 2004 - The Sixth International Conference on Ubiquitous Computing, Nottingham/England, 2004

§  Lieberman, H. & Selker, T.: Out of context: Computer systems that adapt to, and learn from, context. IBM Systems Journal, 2000, 39, 617

§  Schmidt, A.; Aidoo, K.A.; Takaluoma, A.; Tuomela, U.; Laerhoven, K.V. & de Velde, W.V.: Advanced interaction in context. Proceedings of First International Symposium on Handheld and Ubiquitous Computing, HUC'99, Springer Verlag, 1999, 89-101


References

§  T. Springer, K. Kadner, F. Steuer and M. Yin: Middleware Support for Context-Awareness in 4G Environments. World of Wireless, Mobile and Multimedia Networks, 2006. WoWMoM 2006. International Symposium on a, , 203-211 , 2006

§  Korpipää, P.; Mäntyjärvi, J.; Kela, J.; Kernen, H. & Malm, E. Managing Context Information in Mobile Devices. IEEE Pervasive Computing, 2003, 2, 42-51

§  LUTHER, M.; FUKAZAWA, Y.; WAGNER, M. & KURAKAKE, S.: Situational reasoning for task-oriented mobile service recommendation. The Knowledge Engineering Review, 2008, 23, 7-19

§  E. Miluzzo, N. D. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X.Zheng, and A. T. Campbell: Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application. In Proceedings of the 6th ACM conference on Embedded network sensor systems (SenSys '08), 2008.

§  T. Springer and A.-Y. Turhan: Employing Description Logics in Ambient Intelligence for Modeling and Reasoning about Complex Situations. Journal of Ambient Intelligence and Smart Environments (JAISE), 235 - 259 , 2009


application development for mobile and ubiquitous ...ts2/admuc/lecture1415/2. context...

Documents