managing context information in mobile devices

Managing Context Information in Mobile Devices

2011. 3. 28.

Panu Korpipää, Jani Mäntyjärvi,Juha Kela, Heikki Keränen, and Esko-Juhani Malm

VTT Technical Research Centre of Finland

1

Outline

• A context management framework– Information sources– Ontology for sensor-based context information– Inference mechanism concept

• Bayesian reasoning for higher level contexts

• Application programming interface– Adding context– Requests and responses– Subscriptions and indications– Example application

• Discussion

2

A context management framework

•Motivation– Mobile device users’ desire

• Access and manipulation of the users’ information• Services specific to their location, time, and environment

– Gathering context information from various sources• Sensors, networks, device status, user profiles, and other sources

– Enhancing mobile applications’ usability• Letting the applications adapt to the users’ conditions

•Problems to achieve true context awareness– Uncertain, rapidly changing, partially true data from multiple

heterogeneous sources– Extracting relevant context information by fusing data from

several sensors– Many factors such as noise, faulty connections, drift,

miscalibration, wear and tear, humidity to degrade data acquisition

3


•Problems to achieve true context awareness– Overlapping extracted contexts, changing contexts with time,

only partially reliable approximations

•Mobile devices’ dynamic environment– Learning from multidimensional data

• Impossible easy generalization beyond training data

– Occurrence of problems using even sufficiently reliable contexts

• Users with different ideas of “context”

•Proposed approach to address the challenges– A uniform mobile terminal software framework

• Providing systematic methods for acquiring and processing useful context information from a user’s surroundings

• Giving the context to applications

4


•Framework recognizing semantic contexts in real time– Dealing with uncertain, noisy, and rapidly changing

information– Delivering contexts for the terminal applications in an event-

based manner– Application programming interface (API) for using semantic

context information– Using an expandable context ontology to define contexts

•Framework properties– Blackboard-based approach underlying communication

paradigm between framework entities– Focusing on mobile terminal capabilities rather than

infrastructure– Framework for the Symbian platform to achieve true device

mobility, high performance, and a broad user base

5


•Four main functional entities of the context framework– Context manager– Resource server– Context recognition service– Application

•Context manager– A central server for providing services to other entities– Running on the mobile device itself

•Black board-based context manager– Central node storing context information from any source– Serving context to clients in three ways

• Direct querying the manager to gain context data• Subscription to various context change notification services• Using higher-level (composite) contexts transparently

6


•Resource server– Connecting to any context data source– Posting context information to the context manager’s blackboard– Delivering context to the clients according to their subscriptions

• Delivered data’s abstraction level → high enough• Delivered data’s frequency → low enough (not overwhelming)

– Performing low-level recognition before delivering the data• Low-level context recognition process flow in the resource server

• Measurement phase – Reading the sensors and output raw data

•The preprocessing phase builds measurement data arrays that contain a certain number of samples (quantization of time dimension) and calculates generic features for each time interval.

7


•Resource server– Performing low-level recognition before delivering the data

• Preprocessing phase – Building measurement data arrays that contain a certain

number of samples (quantization of time dimension)– Calculation of generic features for each time interval

• Feature extraction– Calculation of more specific context features

• Quantization phase – Binding the feature values to the real-world context which

has a meaning for a person or an application according to a predefined ontology

– Using fuzzy sets or crisp limits for quantizing extracted features

– Outputs context atoms (base context units) that applications can use or recognition services can refine

8


•Resource server– Two methods for quantization

• Set crisp limits in a true-false labeling with fuzzy membership functions denoted as μ(x)

• Quantizing environment sound intensity– Three quantities (Silent, Moderate, and Loud)

corresponding to the three membership functions• If one of these is true, the others are false.

9


•Resource server– Two methods for quantization

• Apply a fuzzy set for features, resulting in continuous valued fuzzy labeling

• μL(x) = 0.7 / Silent + 0.3 / Moderate + 0 / Loud

10


•Context recognition services– Sharing recognized higher-level contexts– Addition and removal of recognition services by developers or

processes– Using a set at a certain time instant or a time series of context

atoms as input– Returning single higher-level contexts for the context manager

•Summary of framework– Resource server and recognition service

• Converting an unstructured raw measurement data flow into a representation defined in the context ontology (human-interpretable context information) for the applications in an event-based manner

– Framework providing a semantic interface that enables more systematic and rapid application development

– Efficient reuse of context information compared to using raw measurement data

11


•Blocks for change detection and security– Not focusing on the functionalities– Any change detection services to detect context change– Security module for checking the trustworthiness of incoming

contexts

12

Information sources

•Acquiring context information from many possible sources– Using a microphone, three accelerometers, two channels for light,

and sensors for temperature, humidity, and touch– Sensor data provided the starting point

• Device processes that represent mobile devices’ internal information, such as applications currently running

• The Internet and other networked resources• Global Positioning System, among the most important sources of

location information• Internal device processes, which provide explicit information on

device application use and users’ scheduled tasks, preferences, and social networks

• Time information, used to associate certain events with others and form event sequences that might relate to a current higher-level context or predict a future event

– Providing continuous and ad hoc connectivity to local and remote data

• Supporting various wireless networks, such as GSM and GPRS, and short-range ad hoc networks, such as Bluetooth

13

Ontology for sensor-based context information

•A common structure for representing information– Managing the context information systematically– Design of an ontology for representing sensor-based context

information

•A schema for ontology – A client-usable, extendable vocabulary that presents the terms

for describing context information– Using the Resource Description Framework (www.w3c.org) as

the description syntax14


•Description of each context using six properties– Containing at least Context type and Context value– Context type

• Referring to the context category• Primary parameter of all subscriptions and queries

– Context value • Referring to the semantic or absolute “value” of context type• Usually being used together with context type• Containing an absolute numerical value or feature describing

context in some cases

– Confidence• An optional property which describes the context’s

uncertainty• A probability or a fuzzy membership of context depending on

the source

– Source• Description of the context’s (semantic) source

15


•Description of each context using six properties– Timestamp

• Denoting the latest time the context occurred

– Attributes • Specifying the context expression freely• Containing any additional details not included in the other

properties

•Higher-level context and context atoms links to a recognition service

– Containing the Input context type set for recognizing the higher-level context

•Three properties for describing recognition services:– Context type

• Using the category of the higher-level context as a normal context atom to request the recognition for the type

16


•Three properties for describing recognition services:– Context recognition service

• Receiving input defined in Input context type set• Performing the classification, and returning the higher-level

context

– Input context type set• Subscription of the input context type set to be notified of the

changes• Inferring the higher level context upon change in input type

set

•Seven context values in higher-level contexts from a set of 47 audio-based context atoms

17


•Management of uncertainty of sensor-based information– Confidence property

• Description of context uncertainty: a context derived from multiple sensory input only holds true with a certain probability

– Partly true context • Boundary between two contexts is not clear and discrete

18

Inference mechanism concept

•The conceptual blackboard-based system– Use of any inference framework for reasoning new contexts

or actions

•Context uncertainty described in the confidence property– Useful in such inference frameworks as probabilistic

networks, clustering, or case based reasoning

•Recognition services – Input : information from the common data space– Output : higher-level contexts from either a set of contexts

at a certain moment or a context history

19

Bayesian reasoning for higher level contexts

•Using a naïve Bayes classifier to recognize higher-level contexts

•Naïve Bayes classifier for online context inference and sensor fusion

– Robust method with missing, uncertain, and incomplete information

• Possible to use context data described by the ontology, a vector of context atom confidence values

• Treating fuzzy membership values as virtual evidence

– Computationally efficient• Training and inference both have a linear complexity in input

data size

– No requirement of background information for modeling • Exception for choosing the relevant network inputs

•Examples of contexts to be recognized by a naive Bayes classifier

20


21


•Higher-level context Outdoors from the context atoms

• Indoor/Outdoor Bayesian network – Classification of the current situation’s context atom confidence

values (vector)– Conditional probability indicating each input’s probability given

the output– Learning conditional probabilities from the training data

•Using naïve Bayesian networks to classify 9 contexts of a mobile device user in their normal daily activities

– Nine contexts measured from a continuous real-life scenario containing different activities such as driving a car, running, walking, using an elevator, listening to different kinds of music, and speaking

– The measurement system hardware • A small sensor box attached to the shoulder strap of a backpack

containing a laptop22


•The classification results – Indication that naïve Bayes classifier can extract situations

fairly well– Valid only in a restricted scenario

•Experiments under controlled scenarios– Nine different contexts, and measured by four people– Car, Elevator, and Tapwater with nearly 100 percent accuracy– Supervised learning approach

• which defines the target classes and their inputs before training

23

Application programming interface

•Context manager API– Implemented on the Symbian platform

24

Adding context

•Processing and collecting the context properties by resource servers

•Method for adding context:

ContextAdd( ContextObject )

25

Requests and responses

•Request of context information from the context manager database

– Exposition of context ontology to application developer– Request context information by the context ontology tree

(sub branch)– Response containing all contexts under the sub branch– Denoting responses to requests with an arrow (→)

26

ContextRequest( Environment:Light:Intensity )→BrightContextRequest( Environment:Light )→{Bright, Natural, NotAvailable}ContextSetRequest( {Environment:Humidity,Device:Activity:Position}→{Dry, AntennaUp}ContextSetRecognitionRequest(Environment:Sound:Type )→Car

Subscriptions and indications

•Giving applications the required context information in an event-based manner

– Subscription to context change notifications

•Examples illustrate how applications use this service

27

ContextChangeSubscribe(Device:Activity:Placement)→AtHandContextChangeSubscribe(Environment:Temperature:Absolute )→ 21ContextStartSubscribe( Location:Facility, MovieTheatre)→MovieTheatreContextChangeSubscribe(Environment:Sound:Type )→Elevator

Example application

•Using the context ontology to demonstrate the use of sensor-based context information in mobile terminal applications

– Such as browsing and information presentation in the user interface

– Environment:Light:Intensity, Environment:Sound:Intensity, and User:Activity:PeriodicMovement

•Environment sound intensity– Control of the volume of operating tones

•User activity and ambient light level – Adaptation of font size, screen brightness, and service

content

•Context-based information representations of a bus timetable service

28

Example application

•Using fuzzy rules to help guide the application adaptation

•Fuzzy rule definition for adapting font size– Using the logical operation AND

• Intersection between two fuzzy sets• Environment:Light:Intensity and

User:Activity:PeriodicMovement:

– Using the logical operations OR and NOT

29

μFONTSIZE (x) =min(μUSER:ACTIVITY:PERIODICMOVEMENT(x),μENVIRONMENT:LIGHT:INTENSITY (x)).

Discussion

•Framework and context ontology – Providing a semantic application programming interface for

handling imprecise information from multiple sources– Many remaining obstacles to the rapid use of many contexts

derived from sensor data– Need for more sensors and a large data collection to gain

enough information

•Several advantages of high-level context– Event based communication of high-abstraction-level data– Communicating low-level data up to a client continuously

• Consumption of more processing power and the limited battery resources.

– Optimal use of the context manager storage

30

managing context information in mobile devices

Documents

context datasubscription

sourceserving context

useful context information

expandable context ontology

true data

semantic contexts

terminal applications

composite contexts