debs 2010 context based computing tutorial

IBM Haifa Research Lab – Event Processing

© 2010 IBM Corporation

Context Aware Computing andits Utilization in Event-based Systems

DEBS 2010 Tutorial

Opher Etzion ([email protected])Ella Rabinovich ([email protected])


© 2010 IBM Corporation2

Our entire culture is context sensitive

In the play “The Tea house of the August Moon” one of the characters says: Pornography question of geography

•This says that in different geographical contexts people view things differently

•Furthermore, the syntax of the language (no verbs) is typical to the way that the people of Okinawa are talking

When hearing concert people are not talking,

eating, and keep their mobile phone on “silent”.



Our IT systems are catching up

Context is becoming a first class citizen in computing

Gartner has designated

Context enriched services and

Context delivery architecture as

Part of its application integration

Hype Cycle of 2009



This tutorial’s agenda

Part I: Context in generalIntroduction – context in pragmatics and languageRoles of contexts in computingIntroduction contexts in event processing

Part II: Context in depth Introduction to Context categoriesTemporal contextSegmentation contextBREAK Spatial contextState contextsContext composition

Part III: Context in practice Operational semantics of contextContext implementation in practice in event processingContext implementation in other computing areasConclusion



Context in language and pragmatics



Context in language – why are we laughing from jokes?

A man walks in the street with a duck, his friend approaches him

• Friend: why are you walking with a duck?

•Man: I went out of home and this duck followed with me and doesn’t leave me alone

•Friend: take him to the zoo

Next day the same man still walks in the street with a duck, his friend approaches him again

• Friend: why haven’t you taken the duck to the zoo?

•Man: I took him to the zoo, cinema, pub– he still doesn’t leave me alone

Where is the context switch?



Another joke example

Four researchers are exploring

the jungle, and are captured by a

tribe of cannibals

The chief of the tribe said: we shall eat

this one today, this one tomorrow, and

this one the day after

His deputy asks: what about the blond

Guy?

The chief answers: we shall release him, since he helped me preparing to the PhD qualification exam in MIT



The context switch principle

The cannibals are wild

The cannibal chief has PhD from MIT



Meaning disambiguation by context

The table?

“The” is always within context



Some parts of the language affected by context

Addressing people

Request form

Code switching for multi-lingual people

Discourse context – e.g. shared experience



Context as “behavior selection”

In some countries I carry money in my wallet, in other countries I hide the money under my clothes

When the radio tells me about a traffic jam in the highway I take an alternative route

If I am getting a request from an important person I re-arrange my priorities to handle it first (otherwise I put it in a queue)

A Jewish person who observes the religious laws does not eat diary food within six hours from the time of eating meat

I need to get to a certain address in Sidney using public transportation, I am looking for fastest way (train, bus, boat)

Spatial Context

State-oriented Context

Segmentation

Context

Temporal + segmentation

Context

Spatial + temporal + state-oriented context



Context in pragmatics – travelling in Sydney

Request Determine Context

SelectService

Take me to address X

Requester is now in location Y; no car; has daily ticket for all public transport; it is rush hour

Selected alternative



Roles of context in computing



Roles Of Context In Computing

Context in the world of software: A major step towards modeling the real world functional behavior

Most of the software considers only direct information

Context-aware software acts also upon (indirect) relevant information, which represents the conditions under which the processing is done

Examples Automatic tow service ordering system

Calls routing application

“Context-aware computing is the concept of leveraging information about the end user to improve the quality of the interaction”

(Gartner 2007)



Software Architecture in the Context of History

Era of the Mainframe

OLTP

Batch

Agility of Enterprise IT

1960s 1980s1970s 1990s 2000s 2010s

Era of the Server

Two-tier

Era of the Web

SOA

Three-tier

CoDA

Era of Context

PresenceMobilityWeb 2.0Social computing

Time2020s

Advanced SOA = event-driven SOACoDA = Context-driven Architecture

WebWeb ServicesMulti-channel

PCLAN

ServerInternet

Advanced SOA

XTPCEP

S/360SNAMainframe

(Gartner 2007)



Context-aware Computing

Context (indirect relevant

information)

ServiceService

Input(direct imperative

information)

Data stores

Web feedsServices

Events

Sensors

QueuesLogs



Context Applications

Telecommunication GeoVector applications

Smart phones users can find out when the museum opens, find an ATM on the way home, ask for a coupon, make a purchase by simply pointing the mobile device at the desired direction or object

Smart call routing systems

Routing phone calls based on the user’s location andpre-defined preferences

At home – wire line phone At office – IP phone or instant messaging application At the movie theater – SMS only



Context Applications – cont.

Social computing Google search and advertising

Improving search relevance, and placement of advertisements according to context

Amazon B2C offers the user the most-relevant experience in e-commerce by using:

History of a user's activity and his location, current activities by other relevant users, political and social news environment

Facebook bases its service offering on user’s context

Content a user creates, seeks and finds depends on his social context: identity, memberships, and the power of his connections to other members



Introduction to context in event processing



Context has three distinct roles (which may be combined)

Partition the incoming events

The events that relate to each customer are processed separately

Grouping events together

Different processing forDifferent context partitions

Determining the processing

Grouping together events that happened in the same hour at the same location



Contextual representation of event

Spatial coordinates

System entry

point

Detection

time

Occurrence

time

Definition Element describingevent type

Header Payload Event Relations

Event Identifier Attribute list Event composition true/false Chronon millisecond/second/minute…

Occurrence Time Event Certainty Event Annotation Event Identity Detection Time Event Source

Header attribute indicators

Type description attributes

Attribute Name Data Type Semantic Role

Event Type Relationship type Generalization/

Specialization/ Retraction Member



Introduction to context categories



Context Definition

A context is a named specification of conditions that groups eventinstances so that they can be processed in a related way. It assignseach event instance to one or more context partitions.

A context may have one or more context dimensions.

A context is a named specification of conditions that groups eventinstances so that they can be processed in a related way. It assignseach event instance to one or more context partitions.

A context may have one or more context dimensions.

TemporalTemporal

SpatialSpatial

State OrientedState Oriented

Segmentation OrientedSegmentation Oriented



Context Types Examples

SpatialSpatial


TemporalTemporal

ContextContext

“Every day between 08:00and 10:00 AM”

“A week after borrowing a disk”

“A time window bounded byTradingDayStart andTradingDayEnd events”

“Every day between 08:00and 10:00 AM”

“A week after borrowing a disk”

“A time window bounded byTradingDayStart andTradingDayEnd events”

“3 miles from the trafficaccident location”

“Within an authorized zone ina manufactory”

“3 miles from the trafficaccident location”

“Within an authorized zone ina manufactory”

“All Children 2-5 years old”“All platinum customers”“All Children 2-5 years old”“All platinum customers”

“Airport security level is red”“Weather is stormy”“Airport security level is red”“Weather is stormy”




Context Types

Fixed locationFixed location

Entity distance locationEntity distance location

Event distance locationEvent distance location

SpatialSpatial


Fixed intervalFixed interval

Event intervalEvent interval

Sliding fixed intervalSliding fixed interval

Sliding event intervalSliding event interval

TemporalTemporal


ContextContext



Context Building Block

Context identifierContext dimensionContext typeContext parametersContext initiator policies

Context identifierContext dimensionContext typeContext parametersContext initiator policies

Building Block Describing ContextBuilding Block Describing Context

Context DetailsContext Details Explicit PartitionsExplicit Partitions

Partition identifierPartition parametersPartition identifierPartition parameters

Composition DetailsComposition Details RelationshipsRelationships

Member contextsPriority OrderMember contextsPriority Order

Location serviceglobal stateLocation serviceglobal state



Temporal context



Temporal Context

Events grouping - group events to process them togetherbased on the fact they have occurred during the same interval Example: count all the bids within the auction period

EPA applicability - indicate that a certain event processing agent is applicable within certain intervals and is not applicable in other intervals. Example: detect if all printers at the same floor are off-line

during working hours.

Temporal interval is a half open interval [Ts, Te)

Ts is start-point, included in the interval Te is end-point, not included in the interval

Temporal context is aimed to partition the temporal space intotime intervals.Temporal context is aimed to partition the temporal space intotime intervals.



Fixed Interval

Fixed interval

Interval start

Interval end

Recurrence

Temporal ordering

Fixed interval

Interval start

Interval end

Recurrence

Temporal ordering

In a fixed interval context each window is an interval that has a fixedtime length; there may be just one single window or a periodicallyrepeating sequence of windows.

In a fixed interval context each window is an interval that has a fixedtime length; there may be just one single window or a periodicallyrepeating sequence of windows.

July 12, 2010,2:30 PM

+ 3 hours

08:00 10:00 08:00 10:00 08:00 10:00



Event IntervalIn an event interval context each window is an interval that startswhen the associated EPA receives an event that satisfies a specifiedpredicate.

It ends when it receives an event that satisfies a second predicate, orwhen a given period has elapsed.

In an event interval context each window is an interval that startswhen the associated EPA receives an event that satisfies a specifiedpredicate.

It ends when it receives an event that satisfies a second predicate, orwhen a given period has elapsed.

Event interval

Initiator event list

Terminator event list

Expiration time offset

Expiration event

count

Initiator policy

Terminator policy

Temporal ordering

Event interval

Initiator event list

Terminator event list

Expiration time offset

Expiration event

count

Initiator policy

Terminator policy

Temporal ordering

Patient’s admittance

Patient’s release

Earthquake

From patient’s admittance to patient’s release

+ 3 days

Within 3 days from an earthquake



Sliding Fixed Interval

In a sliding fixed interval context each window is an interval with fixedtemporal size. New windows are opened at regular intervals relative toone another.

In a sliding fixed interval context each window is an interval with fixedtemporal size. New windows are opened at regular intervals relative toone another.

Sliding fixed interval

Interval period

Interval duration

Interval size (events)

Temporal ordering


Interval period

Interval duration


Temporal ordering

2 hours2 hours

2 hours

1 hour 1 hour 1 hour



Sliding Event Interval

A sliding event interval is an interval of fixed size (events number)that continuously slides on the time axis.A sliding event interval is an interval of fixed size (events number)that continuously slides on the time axis.

Sliding event interval

Event list


Event period

Temporal ordering


Event list


Event period

Temporal ordering

Every 3 blood pressure measurements



Context Initiator Policies

Within three days of an volcano eruption:initiator event = volcano eruption, expiration time offset = + 3 days

March 5, 2010 10:00

March 7, 2010 06:00



Context Initiator Policies – cont.

A context initiator policy is a semantic abstraction that defines thebehavior required when a window has been opened and a subsequentinitiator event is detected.

A context initiator policy is a semantic abstraction that defines thebehavior required when a window has been opened and a subsequentinitiator event is detected.

add

ignore

refresh

extend



Context Terminator Policies

Starting with Island volcano eruption till full flight resumptioninitiator event = volcano eruption, terminator event = flights resumption



Context Terminator Policies – cont.

A context terminator policy is a semantic abstraction that defines thebehavior required when several windows have been opened by subsequent initiators.

A context terminator policy is a semantic abstraction that defines thebehavior required when several windows have been opened by subsequent initiators.

first

last

all



Temporal Ordering Policy

Determine whether a specific event falls within the interval:

By the detection time - the timestamp created by the system when the event enters the system

By the occurrence time - the time in which the event source specifies as the time in which the event occurred in reality

May 1, 2010,11:45:00 AM

May 1, 2010,11:45:00 AM



Segmentation context



Segmentation Oriented Context

A segmentation-oriented context assigns events to context partitionsbased on the values of one or more event attributes, either using thevalue of these attributes to pick a partition directly, or using predicateexpressions to define context partition membership.

A segmentation-oriented context assigns events to context partitionsbased on the values of one or more event attributes, either using thevalue of these attributes to pick a partition directly, or using predicateexpressions to define context partition membership.

Segmentation-oriented

Attribute list

Partition identifier

Partition expression

Segmentation-oriented

Attribute list

Partition identifier

Partition expression

Unrestricted number of partitions Segmentation by CustomerId

Fixed number of partitions Segmentation by different age ranges

age < 21 age 30 and age < 50 age 50 and age < 67 age 67



Segmentation Oriented Context – cont.

Unrestricted number of partitions: Average of customer’s deposits over last month

John

Tim

Helen

David



Segmentation Oriented Context – cont.

Fixed number of partitions Distribution of alcohol consumption by age

18-25

26-50

50-



Spatial Context



Spatial context

F ixed Loc at ion

Enti ty di stance locat ion

Event d istance locat ion

W ith in th e hous e

Withi n 2 KM from the motel

W ith in 10 K M f ro m the acc id ent



Spatial properties of events

Point

Car CCar F

Car G Car ACar E

Car H

Area

Line

Road 62

The green neighborhood

Point (X1, Y1)



Fixed location A fixed location context has predefined context partitions based on specific

spatial entities. An event is included in a partition if its location attribute indicates that it is correlated with the partition’s spatial entity.

entityev ent

entity

event

C ontains

overlaps

entity

evententity

evententity

event

entity

event

entityevent

disjo in t

entity event entity ev ent entity ev ent entity

ev ent

entity

ev ent

equals

entity , event

e1entity, event entity , ev ent

touchesentity event entity

event

entity ev ent entity ev ent

entity

ev ent

entity

ev ent

evententity

event

entity

C ontained Inevent

entity

evententity

event

entity

ev ent entity

Relations between the event’s location and the context entity’s location

An event is classifiedto a context partitionif satisfy a spatial relationshipwith fixed entity



Fixed location examples

A person enters the house:

The house location is an area, the person’s location is a point; the relation is touches

A car parks across two lanes

The car parked location is an area, the lane is an area; the relation is overlaps

An Alzheimer patient has gone outside of the safe zone defined by his family.

His location is a point, the zone is an area; the relation is disjoint



Entity Distance Location

An entity distance location context assigns events to context partitions based on their distance from a given entity. This entity is either specified by an event attribute or is

given as part of the context specification

Entity

d1

d2



Entity distance location examples

A driver enters the restaurant’s interest context when arriving to a certain radius from the restaurant

A bus is within the context of a certain bus station if it is within a certain distance from the station



Event Distance Location

An event distance location context assigns events to context partitions based on their distance from the location of the event that triggered the creation of the partition.

d1d2



Event distant location example

A fire breaks within a certain radius from the earthquake epicenter

The distance space is partitioned, different actions for different partitions

A suspect fits the criminal’s profile

is arrested within a certain radius

from the crime scene



State oriented context



Where states are coming from?

Explicit state machines

Observation in

Workflow/BPM

State of some entity

Typically stored in DB/ global variable



State examples

When the CPU is overloaded

When a backup is running

When a person is presenting

When the mood in Wall Street is bullish.



The temporal aspects of state

Occurs while a state is valid?

The “while” may be determined by detection time or by occurrence time

If the state is – raining, Then it makes sense thatthe association of events is determined according tooccurrence time

If the state is – CPU is overloadedThen it makes sense that events that result in further processing are associated according to their detection time.



Context composition



Composite context

C ustom ers

H ours5:00 6:00 7: 00 8:00 9 :00 10:0 0

E ach square in thi s illust rat ion desig nates a separate group in th isc om posite contex t

A composite context is a context that is composed from two or more contexts. Example: the set of context partitions for the composite context is the Cartesian product of the partition sets of its constituent contexts.

A composite context is a context that is composed from two or more contexts. Example: the set of context partitions for the composite context is the Cartesian product of the partition sets of its constituent contexts.



Composition of context – some observations:

The most common combination is: segmentation and temporal

The relations between the composed contexts can be – union or intersection (intersection is the more common)

There may be multiple composition participants

In some cases a priority is needed to disambiguate the context affiliation

State: rainy unionTemporal: every day between midnight and 6am

Segment: customerTemporal: Every 10 orders

Segment: driverTemporal: Within 1 hour from an accidentSpatial: within 5KM from the accident




Priorities in event composition

Temporal: Every 10 ordersSegment: customer

First create group of 10 orders and this group by customer

The temporal context has higher priority



Priorities in event composition


First group of by customer and then count 10 orders for each customer

The segmentation context has higher priority



Operational semantics of context



Context in Applications

Currently in most applications the context logic is embedded in the traditional code. No separation of concerns resulting in: No reuse of context logic

Change management is troublesome and non-efficient

Increased application maintenance cost

Reduced business agilityApplication

Context



Context Decoupling Externalizing the rapidly-changing context logic from the

traditional application code leads to: Increased flexibility and adaptability

Better reflection of business policies on business logic

All building blocks are explicitly defined Reduced application development and maintenance time,

as well as costs

Application

Context



Introducing Context Service

Context is a logical component in EPN

Context service is a runtime component implementing and managing the context



The Role of Context Service

Initiation of new context’s partitions Terminating of existing context’s partitions

Deciding on affiliation of an objectinstance to a certain partition Based on the partition attributes

Based on the object attributes

Context service

context 2

EPA 1EPA 1EPA 2EPA 2Partitions Partitions

EPA 1EPA 1PartitionsPartitions

Context 1Context 1

Context 2Context 2

Context 3Context 3EPA 1EPA 1

EPA 2EPA 2Partitions Partitions

context 1

Context service

Context service manages the active contexts lifecycle



Context Service – Points for Consideration

Context partitions and event instance management Architecture for the context service

Centralized vs. distributed architecture



Partition and Event Instance Management

The role of managing the content in each of the partitions can either fall on the context service the agent (EPA) associated with the specific context

In case the context service is managing the partition’s content:

In case the agent is managing the partition’s content:

E1 ,E2, E3 E1 ,E2, E3 EPA 1EPA 1

EPA 2EPA 2Partitions Partitions

Context 1Context 1

E1,E2

E3

Context service Agent

Pattern evaluation

E1,E2

E1 ,E2, E3 E1 ,E2, E3

Agent

Pattern evaluation

EPA 1EPA 1EPA 2EPA 2Partitions Partitions

Context 1Context 1

Context service

E1,E2E1,E2

E3 E3



Centralized vs. Distributed Architecture

Should take into consideration some functional andnon-functional requirements Functional requirements

Correctness of initiator/terminator/participant semantics Non functional requirements

Performance, scalability, reliability



Centralized Context Service

Performance bottleneck Non scalable in terms of throughput and number of agents

Agent

Context service

Agent AgentAgent



Distributed Context Service on Agent’s Level

Managing shared context in a completely distributed environment Requires devising control mechanisms to communicate the

changes in shared context state to all interested parties

Agent

Context service

Agent

Context service

Agent

Context service

Agent

Context service

Agent

Context service



Hybrid Approach

Hybrid approach – distributed context service on the system’s level Context service – one per one runtime engine

The context state is shared between distributed instancesof the context service

Agent

Context service

Agent

Context service Agent

Context service

Context Context statestate



Context Implementation in Practice:Event Processing Implementations



Context in Practice - Streambase

CREATE WINDOW tenSecondsInterval(

SIZE 10 ADVANCE 1 {TIME});

CREATE WINDOW eventsInterval (

SIZE 10 ADVANCE 1 {TUPLE} TIMEOUT 20);





Context in Practice – Oracle EPL

SELECT * FROM Withdrawal RETAIN 5 EVENTS

SELECT * FROM Withdrawal RETAIN 4 SECONDS



Context in Practice –Sybase (Aleri/Coral8) CCL

CREATE WINDOW Book_w SCHEMA Book_t KEEP ALL;

INSERT INTO Book_w

SELECT * FROM Book_s;

The KEEP policy specifies the kind of window.

Here are some examples:

KEEP LAST PER Id

KEEP 3 MINUTES

KEEP EVERY 3 MINUTES

KEEP UNTIL (”MON 17:00:00”)

KEEP 10 ROWS

KEEP LAST ROW

KEEP 10 ROWS PER Symbol



Context in Practice – Progress Apama



Context in Practice – AMiT

A key is a collection of event attributes that are used tomatch different events.

These attributes must have syntactic compatibility, and are usually semantically equivalent.

OrderId KeyString

OrderId KeyString

CustomerOrderOrderId

CustomerOrderOrderId

CustomerOrderCancelOrderId

CustomerOrderCancelOrderId

OrderDeliveryId

OrderDeliveryId

OrderShipmentReportNumber

OrderShipmentReportNumber




A lifespan is a time interval during which pattern detection is relevant – temporal context.

A lifespan is bounded by an initiator and a terminator. Initiator – beings the lifespan. The initiator can be an event,

absolute time, startup (CEP engine startup). A lifespan can use more than one initiator

Terminator – ends the lifespan. The terminator can be event, expiration time, absolute time, or no-terminator.

A lifespan can use more than one terminator



Context in Practice – WBE



ruleCore Reakt



Contexts implementations in other types of computing



Data in context

Contextual data:

Temporal database: transaction time, valid time

Spatial database: geographic abstraction

Spatio-temporal database: intersection of these two



ContextPhone

http://www.cs.helsinki.fi/group/context/



MIT Media Lab – context in kitchen appliances

The Context-Aware Computing group is focused on demonstrating the possibilities for controlling systems with interpreted human intention. The goal is to demonstrate how “context” such as who we are, what we are doing, where we are doing it, why we might be doing it, and when it should be done can simplify our ability to control systems.

http://context.media.mit.edu/press/index.php/about/



MIThril – context aware wearable computing

http://www.media.mit.edu/wearables/mithril/

Based on sensing and machine learning – context engine and wearable sensors and processors



Sixth sense

http://www.media.mit.edu/research/highlights/sixthsense-wearable-gestural-interface-augment-our-world



Context and websites

Contextualization of websites: making them appear in searches of certain category (reverse engineering of the search engine algorithms)

Using the user’s context to generate results

Contextualization of websites: making them appear in searches of certain category (reverse engineering of the search engine algorithms)



IBM Entity Analytics – context acquisition by dependencies

Good Guys Subjects of Interest

Hotel guests

Loyalty club enrollment

Employees

Vendors

Victims

Specially designated nationals

Excluded persons

Gaming license revocations

Known cheaters

Interpol

FBI Most Wanted



Summary



Context

Separation of contextfrom the logic

Next phase of abstraction independence

BPM and rules provided separation between programming in the large and programming in the small

Databases provided data independence



Context is:

One of the key building blocks in event processing modeling

A language construct that starts to appear in event processing languages

An emerging concept in enterprise computing

debs 2010 context based computing tutorial

Technology