er 2013 tutorial: modeling the event driven world

95
Knowledge Partners International LLC ●www.kpiusa.com ● www.thedecisionmodel.com 1 Modeling the event driven world ER 2013 – TUTORIAL Opher Etzion and Barbara von H

Upload: opher-etzion

Post on 09-May-2015

2.030 views

Category:

Technology


0 download

DESCRIPTION

This tutorial is a first public exposure of the event model

TRANSCRIPT

Page 1: ER 2013 tutorial:  modeling the event driven world

© 2012 Knowledge Partners International LLC ●www.kpiusa.com ● www.thedecisionmodel.com 1

(TEM)

Modeling the event driven world

ER 2013 – TUTORIAL

Opher Etzion and Barbara von Halle

Page 2: ER 2013 tutorial:  modeling the event driven world

2

Prologue:The Internet of everything for situation awareness

None of the authorized drivers location is near the car’s location

theft is concluded

Whom to notify;

whether to activate stopper

Use a built-in car stopper to slow the intruder and dispatch the security company

A person enters a car and the car starts moving;

the person does not look like one of the authorized drivers

Such applicationsbecome possible

since everything isconnected

Page 3: ER 2013 tutorial:  modeling the event driven world

3

Prologue:Does the “Internet of everything” really exhibit the benefits of the Internet?

The success of the Internet is attributed to its relative simplicity to connect to create content to search

Imagine that any search in the Internet would have been done using SQL queries…

How pervasive do you think the Internet would have been?

Page 4: ER 2013 tutorial:  modeling the event driven world

4

Prologue: For situational awareness….we are not even in the SQL era

EventPatterns

Listing 1.1 Example of the MonitorScript event programming language from Apama

if isAuto then { DeliveryBid db; on DeliveryBid(store=dr.store):db within(ASSIGNMENT_TIME){ #1 assignmentTimer.quit(); route Assignment(dr.requestId, #2 dr.store, db.driver, dr.addresseeLocationPointX, dr.addresseeLocationPointY, db.committedPickUpTime, dr.requiredDeliveryTime); watchForPickUp(dr, db.driver, db.committedPickUpTime); watchForDelivery(dr, db.driver); } }

Most of the event-based programming today is still ad-hoc and hand-coded; existing languages are rather low level

Page 5: ER 2013 tutorial:  modeling the event driven world

5

Agenda I

Topic I: Introduction

– Brief history of

event processing

II

Topic II: The major

differentiation factors of event-based thinking vs. traditional computing

III

Topic III: The

computational

independent model

IV

Topic IV: Drill down to

the model components

V

Topic V: Epilogue

5

Page 6: ER 2013 tutorial:  modeling the event driven world

6

How did we get here?

First start-upsDescendents

of academic projects

Apama acquisition

By Progress

Around 2000

2007

TIBCO andOracle announce

products

StreambaseCoral8

2005

EPTS Established

Hittingthe analysts

hype …

2008

IBMJoins 2012

New players:SAS. Yahoo ,

Twitter

2013

EP at the heightof BIG DATA hype

Cycle

M&A:TIBCO/Streambase

Software AG/Apama

6/30/2013 6

Page 7: ER 2013 tutorial:  modeling the event driven world

7

Where are events used? Virtually everywhere

Events, as “data in motion” is one of the fundamental ingredients in big data:event -driven analytics

Event -driven services / making events part of SOA

Event-based decision making and event driven Optimization

Event -driven processing as a backboneof next generation systems: event-based robotic, autonomic vehicles, human enhancement technology…

6/30/2013 7

Page 8: ER 2013 tutorial:  modeling the event driven world

8

Gartner’s Big Data hype cycle 2012Source: Gartner publication G00235042, July 31, 2012

8

Event processingAgain in the hype Cycle – in different context

Page 9: ER 2013 tutorial:  modeling the event driven world

9

The event processing market today

Source:

Event processing Manifesto

Page 10: ER 2013 tutorial:  modeling the event driven world

10

The reality

A year ago, Roy Schulte from Gartner published a personal blog entitled “does anybody care about event processing”

He admitted that his predictions about the actual size of the event processing market is smaller than predicted

His observation is that 95% of the event processing market is not visible since it is home-built and not labeled as EP

In this tutorial we will discuss what is event driven thinking and how smart modeling is able to help the world to exploit the power of events

Page 11: ER 2013 tutorial:  modeling the event driven world

11

Barriers to wider adoption

Lack of standards: SOA took off only when WS standards were accepted

Lack of sufficient awareness and good ROI understanding: Need entry points and methodology about benefits to individuals, enterprises, packaged applications providers.

Lack of understanding of what is event-based thinking and how to translate it to implementation

Lack of skills – current tools requirehighly skilled developers to do tricky programming

6/30/2013 11

Page 12: ER 2013 tutorial:  modeling the event driven world

12

Major gap -- skills required

A comprehensive user survey showsthat 84% of the users wish that event rules could be defined by business users

There is a gap

Current models:Implementationoriented

Business analysts orientedModeling

6/30/2013 12

Page 13: ER 2013 tutorial:  modeling the event driven world

13

Summary of topic I

In the business level – it is not well understood how tothink in events and how to utilize events

In the application level – the life-cycle of event-based systems require skilled IT developers

Next - we explain what is different about event-driven thinking

6/30/2013 13

Page 14: ER 2013 tutorial:  modeling the event driven world

14

Agenda I

Topic I: Introduction

– Brief history of

event processing

II

Topic II: The major

differentiation factors of event-based thinking vs. traditional computing

III

Topic III: The

computational

independent model

IV

Topic IV: Drill down to

the model components

V

Topic V: Epilogue

14

Page 15: ER 2013 tutorial:  modeling the event driven world

15

Event driven applications follow the 4D paradigm

I want to know about it immediately and react in the best possible way

Detect Derive DoDecide

Did SomethingHappen?

What should we do about it?

Yes!

Awareness ReactionSituation

6/30/2013 15

Page 16: ER 2013 tutorial:  modeling the event driven world

16

I want to know about it immediately and react in the best possible way

Detect Derive DoDecide

A suspicious account is detected whenever there are at least three large cash deposits within 10 days

A suspicious account Open investigationAssign risk score and determine course of

action

Cash depositTransfer abroad

Let’s take a very simple example of money laundering

Page 17: ER 2013 tutorial:  modeling the event driven world

17

Some features of this scenario

Events trigger action

Events influence logic for the results There may be multiple

events whose combined content influences the results

Temporal contexts (10 days) influence the results

I want to know about it immediately and react in the best possible way

A suspicious account is detected whenever there are at least three large cash deposits within 10 days

6/30/2013 17

Page 18: ER 2013 tutorial:  modeling the event driven world

18

The way that most people would approach it

Insert the event into a database; use periodic or on-demand queries to process the events

The processing may notbe efficient – many of therequests will not yield results

The processing may notbe effective – the time to react may be missed

6/30/2013 18

Page 19: ER 2013 tutorial:  modeling the event driven world

19

A suspicious account design using traditional BPMN…

Page 20: ER 2013 tutorial:  modeling the event driven world

20

A suspicious account using BPMN…

Event logic hidden in the business process

Difficult to define and maintain

Cumbersome to introduce changes

Page 21: ER 2013 tutorial:  modeling the event driven world

21

A suspicious account using BPMN…

Data changing processes are spread throughout

Most of these resulting changes get put into

data warehouses or transactional data stores

BUT

! The data loses its contextual meaning

! The data loses its value as a function of time

! The opportunity to deal with the issue may only be

available for short period of time

Page 22: ER 2013 tutorial:  modeling the event driven world

22

Difficulties in the way that most people would approach it

The event-driven vs. request-driven nature

Effectiveness andEfficiency issues

The temporal oriented behavior

The hidden statehandling

6/30/2013 22

Page 23: ER 2013 tutorial:  modeling the event driven world

23

Efficiency and effectiveness issues

The processing may notbe efficient – many of therequests will not yield results

The processing may notbe effective – the time to react may be missed

6/30/2013 23

Page 24: ER 2013 tutorial:  modeling the event driven world

24

While people typically are event-driven we tend to think about computerized systems in request driven way

Searching the web, database queries, use of web services, use of mobile applications

6/30/2013 24

Page 25: ER 2013 tutorial:  modeling the event driven world

25

What are the differences in thinking?

6/30/2013 25

Question Response Driven Event Drive

Why is an action being taken? As a response to a specific request

Triggered by the fact of a specific situation

When is an action being taken? When the request is being processed

Determined based on the context of the situation

What happens when the request / event occurs?

A response is always produced

The event can be ignored, increment the state, trigger an internal derive event, or trigger a situation

Page 26: ER 2013 tutorial:  modeling the event driven world

26

Temporal consideration changes everything

The logic is sensitiveto timing of events

A delivery should be confirmed by the deadline

The logic is sensitiveto the order of events

The winner in the bid is the first one who made the highest bid

The logic is determined by timing considerations

What? Why? When?

In traditional models temporal functions are hand-coded, adding complexity

6/30/2013 26

Page 27: ER 2013 tutorial:  modeling the event driven world

27

The logic is sensitive to the timing of events’ occurrences

Events during rush hour are of interest, events outside rush hour are not

Events that occur or don’t occur relative to a deadline

Is the reported problem already solved, or is it still open?

6/30/2013 27

Page 28: ER 2013 tutorial:  modeling the event driven world

28

The logic is sensitive to the order of events’ occurences

Who arrived first?

Has the bid arrived while the auction was still open?

6/30/2013 28

Page 29: ER 2013 tutorial:  modeling the event driven world

29

The logic of situation is determined by timing considerations

Determine the status of a patient based on blood pressure measurements:

Every 8 measurements

Every 5 hours

6/30/2013 29

Page 30: ER 2013 tutorial:  modeling the event driven world

30

Handling the state

EventPatterns

Pattern “event1 occurs after event2” requires keeping state of all unmatched instances of event1

6/30/2013 30

Page 31: ER 2013 tutorial:  modeling the event driven world

31

Event processing applications are a step further …but is this enough? (1/3)

I want to know about it immediately and react in the best possible way

A suspicious account is detected whenever there are at least three large cash deposits within 10 days

Let’s see how this situation is implemented using a popular SQL like open source Event Processing language

Page 32: ER 2013 tutorial:  modeling the event driven world

32

Event processing applications are a step further… but is this enough? (2/3)

// Large cash deposit insert into LargeCashDepositselect * from Cash deposit where amount > 100,000

// Frequent (At least three) large cash deposits create context AccountID partition by accountId on Cash deposit;Context AccountIDInsert into FrequentLargeCashDeposits select count(*) from LargeCashDeposit having count(*)>3;

// Frequent cash deposits followed by transfer abroad Context AccountIDinsert into SuspiciousAccount select * from pattern [ every f=FrequentCashDeposit -> t=TransferAbroad where timer.within(10 days)]

Page 33: ER 2013 tutorial:  modeling the event driven world

33

Event processing applications are a step further… but is this enough? (3/3)

// Large cash deposit insert into LargeCashDepositselect * from Cash deposit where amount > 100,000

// Frequent (At least three) large cash deposits create context AccountID partition by accountId on Cash deposit;Context AccountIDInsert into FrequentLargeCashDeposits select count(*) from LargeCashDeposit having count(*)>3;

// Frequent cash deposits followed by transfer abroad Context AccountIDinsert into SuspiciousAccount select * from pattern [ every f=FrequentCashDeposit -> t=TransferAbroad where timer.within(10 days)]

Non-intuitive code to business users

Business logic embedded in the code

Difficult to define and maintain

Cumbersome to introduce changes

Page 34: ER 2013 tutorial:  modeling the event driven world

34

Summary of topic II

In many cases – event driven functionality is expressed using the traditional request-response fashion

Fundamental differences exist between the two paradigms, and benefits exist in using event-driven modeling and implementation for certain applications

Next – drilling down to the essence of event driven model

6/30/2013 34

Page 35: ER 2013 tutorial:  modeling the event driven world

35

Agenda I

Topic I: Introduction

– Brief history of

event processing

II

Topic II: The major

differentiation factors of event-based thinking vs. traditional computing

III

Topic III: The

computational

independent model

IV

Topic IV: Drill down to

the model components

V

Topic V: Epilogue

35

Page 36: ER 2013 tutorial:  modeling the event driven world

36

The vision for event based systems: Shift governance from the programmer to the knowledge worker

Governance occurs through developmentand maintenance of

program code

Governance occurs through developmentand maintenance of

event models

TODAY TOMORROW

CODE LEVEL

Page 37: ER 2013 tutorial:  modeling the event driven world

37

The basic requirements of event modeling

4. Automatic translation to code in regular or specific engine language

3. Free of implementation assumptions

1. Rigorous verifiable structure

2. Represented as a collection of tables

Page 38: ER 2013 tutorial:  modeling the event driven world

38

The model driven engineering approach

The event model is CIM --- translation to PIM and to PSM is automated

Page 39: ER 2013 tutorial:  modeling the event driven world

39

Eliminating noise from the model

Current models are close to the implementation models – and from pure logic view contain “noise”.

Bringing data from current state

Query Enrichment Inclusion in events

ExamplesDetermine if a customer is a platinum customer;Fetch the credit limit of a customer

Other noise : workarounds

Page 40: ER 2013 tutorial:  modeling the event driven world

40

TEM Concepts

Facts

Actors

EventsStates

Event DerivationLogic Transitions

Goals IT elements

Glossary Logic

ComputationLogic

Page 41: ER 2013 tutorial:  modeling the event driven world

41

Simple example:Top down design of event model for suspicious account derivation

Bank transaction system

Compliance officerSuspicious Account

Frequent large cash deposits

Frequent large cash deposits

Large cash deposit

Large cash deposit

cash amount <Cash deposit>

customer threshold

Page 42: ER 2013 tutorial:  modeling the event driven world

42

Simple example: TEM Logic Specification for deriving Suspicious Account

Suspicious account Logic

Row # When Expression

When Start

When End

Partition by Filter on event Pattern Filter on pattern

Account ID Frequent large cash deposits

1always same is Detected

Frequent large cash deposits Logic

Row # When Expression

When Start

When End

Partition by Filter on event Pattern Filter on pattern

Account ID Count(Large cash deposit)

1every 10

days same > 3

Large cash deposit Logic

Row # When Expression

When Start

When End

Partition by Filter on event Pattern Filter on pattern

Customer ID cash amount <Cash deposit>

1always same >= customer

threshold

Page 43: ER 2013 tutorial:  modeling the event driven world

43

Let’s complicate the Suspicious Account story

A situation of a Suspicious account is derived when any of the following three derived events is detected

Suspicious account

Lack of account activityno transaction of any kind, of the same account, takes place within 20 days

Frequent large cash deposits At least 3 cash deposits of the same account, within 10 days

Frequent cash deposits followed by transfers abroadAt least 10 occurrences of cash deposits followed by transfers abroad of the same account within 30 days30 days

Page 44: ER 2013 tutorial:  modeling the event driven world

44

TEM in Action in Six Steps

Step 1: Draw the situationStep 2: Add the situation’s skeletonStep 3: Go one step furtherStep 4: Complete the picture Step 5: Create the Logic Specification (TEM Tables)Step 6: Complete the Glossary

Page 45: ER 2013 tutorial:  modeling the event driven world

45

TEM in ActionStep 1: Draw the situation

Compliance officerSuspicious Account

Page 46: ER 2013 tutorial:  modeling the event driven world

46

The notion of situation

FRUSTRATED CUSTOMER

Sometimes the situation is determined by detecting thatsome pattern occurred in theflowing events.

Toll violation Frustrated customer

Sometimes the events can approximate or indicate withsome certainty that the situation has occurred

A raw or derived event that requires reaction

Page 47: ER 2013 tutorial:  modeling the event driven world

47

TEM in ActionStep 2: Add the situation’s skeleton

Participants in deriving the conclusion which are also conclusions

Situation

Context

Compliance officerSuspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Consumer

Page 48: ER 2013 tutorial:  modeling the event driven world

48

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

TEM in ActionStep 3: Go one step further

Add structures for the three required derived events

Connect event producers

Page 49: ER 2013 tutorial:  modeling the event driven world

49

TEM in ActionStep 4: Complete the Diagram

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Page 50: ER 2013 tutorial:  modeling the event driven world

50

TEM in ActionStep 4: Complete the Diagram

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Cash deposit or Transfer abroad Account ID

Page 51: ER 2013 tutorial:  modeling the event driven world

51

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Cash deposit or Transfer abroad Account ID

TEM in ActionStep 5: Create the Logic Specification (TEM Tables)

Large cash deposit Logic

Row # When Expression

When Start

When End

Partition by Filter on event Pattern Filter on pattern

Customer ID cash amount <Cash deposit>

1always same >= customer

threshold

Page 52: ER 2013 tutorial:  modeling the event driven world

52

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Cash deposit or Transfer abroad Account ID

TEM in ActionStep 5: Create the Logic Specification (TEM Tables)

Cash deposit followed by transfer abroad Logic

Row #When

Expression When Start

When End

Partition by Filter on event Pattern Filter on pattern

Account ID Cash deposit cash amount

<Cash deposit>

1

Cash deposit

+3 days same Occurs before

Transfer abroad

<= transfer amount

<Transfer abroad>

Page 53: ER 2013 tutorial:  modeling the event driven world

53

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Cash deposit or Transfer abroad Account ID

TEM in ActionStep 5: Create the Logic Specification (TEM Tables)

Frequent large cash deposits Logic

Row # When Expression

When Start

When End

Partition by Filter on event Pattern Filter on pattern

Account ID Count(Large cash deposit)

1every 10

days same > 3

Page 54: ER 2013 tutorial:  modeling the event driven world

54

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Cash deposit or Transfer abroad Account ID

TEM in ActionStep 5: Create the Logic Specification (TEM Tables)

Frequent cash deposits followed by transfers abroad Logic

Row # When Expression

When Start

When End

Partition by Filter on event Pattern Filter on pattern

Account ID Count(Cash deposit followed

by transfer abroad)

1every 30

days same > 10

Page 55: ER 2013 tutorial:  modeling the event driven world

55

TEM in ActionStep 5: Create the Logic Specification (TEM Tables)

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Cash deposit or Transfer abroad Account ID

Lack of account activity Logic

Row #

When Expression

When Start

When End

Partition by Filter on event Pattern Filter on pattern

Account ID Cash deposit Transfer abroad

1

Cash deposit , Transfer abroad

+20 days

same is Absent is Absent

Page 56: ER 2013 tutorial:  modeling the event driven world

56

TEM in ActionStep 5: Create the Logic Specification (TEM Tables)

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Cash deposit or Transfer abroad Account ID

Suspicious Account Logic

Row #When

Expression When Start

When End

Partition by Filter on event Pattern Filter on pattern

Account ID Frequent large cash deposits

Frequent cash deposits followed

by transfers abroad

Lack of account activity

1 always same is Detected

2 always same is Detected

3 always same is Detected

Page 57: ER 2013 tutorial:  modeling the event driven world

57

Step 6: Complete the Glossary

Concept name Concept type Concept sub-type Description

Account ID Fact Universal bank account numberCustomer ID Fact Universal customer IDTransfer abroad Event Raw Details of a transfer abroad transactionlarge cash deposit Event Derived A cash deposit above a given thresholdcash amount Fact Concrete The amount of the cash deposit

Concepts Lexicon

Fact Type name Fact Type sub-type

Aliases Contained in Event Contained in Actor

Data Type

Domain valid values

Universal Fact Type reference

Default value

cash amount Regular Cash deposit

Actor Name Role Event

Compliance officer Consumer Suspicious accountBank transaction system Producer Cash depositBank transaction system Producer Transfer abroad

Fact Types Details

Actors Details

All concepts should eventually be defined in

the knowledge model

Page 58: ER 2013 tutorial:  modeling the event driven world

58

Summary of topic III

Event model as a computational independent concept

Diagrams and table – top-down design

Next – Drill down to the model ‘s components

6/30/2013 58

Compliance officer

Frequent large cash deposits

Large cash deposit

Frequent cash deposits followed by transfers abroad

Cash deposit followed by transfer abroad

Suspicious Account

Frequent large cash deposits

Frequent cash deposits followed by transfers abroad

Lack of account activity

Cash deposit followed by transfer abroad

Cash deposit

Transfer abroad

Large cash deposit

cash amount <Cash deposit>

customer threshold

Lack of account activity

Cash deposit is Absent

Transfer abroad is Absent

Bank transaction system

Page 59: ER 2013 tutorial:  modeling the event driven world

59

Agenda I

Topic I: Introduction

– Brief history of

event processing

II

Topic II: The major

differentiation factors of event-based thinking vs. traditional computing

III

Topic III: The

computational

independent model

IV

Topic IV: Drill down to

the model components

V

Topic V: Epilogue

59

Page 60: ER 2013 tutorial:  modeling the event driven world

60

Drilling down to the concept model – back to the car theft example

None of the locations of authorized drivers for this car are in the car: theft is concluded

Either notify police or chase the car by private agency

Stop the car by police

A specific car is moving

Page 61: ER 2013 tutorial:  modeling the event driven world

61

Players in this story

Sensors:Car GPS sensorCar cameraPerson’s location sensor

Events:Car moving Person changed location Person enters car

Situation:Person enters car and then Car moving Person location for all eligibledrivers is not near car locationEntering person does not look like any eligible driver

Actuators:Car stopperSecurity enforcers

Page 62: ER 2013 tutorial:  modeling the event driven world

62

Concepts

Concepts

Event

Actor

Domain

Fact

ProcessingElement

Context

Situation

Compute derivation

Page 63: ER 2013 tutorial:  modeling the event driven world

63

Fact – a first class citizen in the model

Piece of information about actor or event What?

How?A function of the actor/event

to a domain

Fact Actor

Provides information about

M 1

Fact Event M 1

DomainFact1M

Examples:Car-id of Car Car-id of Moving CarAuthorized-driver of Car

Car-id of Car

Car

Car-id of Moving Car

Moving Car

Car-idCar-id of Car

Provides information about

Is a classified to

Page 64: ER 2013 tutorial:  modeling the event driven world

64

Domain

Describes an entity type in the real world What?

How?An abstract term, associated

with data type and

Examples:Car-idDriver-id

Name: Car Synonyms:Vehicle

Data type:2 char + 8 digits

Page 65: ER 2013 tutorial:  modeling the event driven world

65

Event

Something that happens What?

How?Represented as a aggregationof characteristics and associated facts

Examples:Moving CarPerson enters carPerson changes location

Name: Person enters car

Characteristics:Time: 22/10/2013 22:24Location: Barcelona, Balmes 132Certainty: 1Source : car camera 453430

Car id:14321313

Picture:

Page 66: ER 2013 tutorial:  modeling the event driven world

66

Actor Any entity, person, or organization

that has a relevant roleWhat?

How?Represented through associated

Fact-types and event related roles

Actor EventRole

M n

Examples:CarDriver Car GPSMobile phone

Car GPS Moving Car

Page 67: ER 2013 tutorial:  modeling the event driven world

67

Roles of actor with respect to events

Event

Actor

Event producer

Event consumer

Actuator

Event subject

Event descriptor

Car GPSSecurity officer

Car stopper

Car

Eligible driver

Data provider Driving authorization store

Page 68: ER 2013 tutorial:  modeling the event driven world

68

Compute derivation

Computes the value of facts associated with derived events

Participating event

Participating event

Function

CopyAggregation Arithmetic function

Derived computation

Page 69: ER 2013 tutorial:  modeling the event driven world

69

The notion of context is pervasive in the human culture

In the play “The Tea house of the August Moon” one of the characters says: Pornography question of geography

• This says that in different geographical contexts people view things differently

• Furthermore, the syntax of the language (no verbs) is typical to the way that the people of Okinawa are talking

When listening to a concert people are not talking,

eating, and keep their mobile phone on “silent”.

Page 70: ER 2013 tutorial:  modeling the event driven world

70

Context has three distinct roles (which may be combined)

Partition the incoming events

The events that relate to each customer are processed separately

Grouping events together

Different processing forDifferent context partitions

Determining the processing

Grouping together events that happened in the same hour at the same location

Page 71: ER 2013 tutorial:  modeling the event driven world

71

Context types

Fixed location

Entity distance location

Event distance location

Spatial

State Oriented

Fixed interval

Event interval

Sliding fixed interval

Sliding event interval

Temporal

Segmentation Oriented

Context

Page 72: ER 2013 tutorial:  modeling the event driven world

72

Context type examples

Spatial

State Oriented

Temporal

Context

“Every day between 08:00and 10:00 AM”

“A week after borrowing a disk”

“A time window bounded byTradingDayStart andTradingDayEnd events”

“3 miles from the trafficaccident location”

“Within an authorized zone ina manufactory”

“All Children 2-5 years old”“All platinum customers”

“Airport security level is red”“Weather is stormy”

Segmentation Oriented

Page 73: ER 2013 tutorial:  modeling the event driven world

73

Fixed interval

Fixed interval

Interval start

Interval end

Recurrence

Temporal ordering

In a fixed interval context each window is an interval that has a fixedtime length; there may be just one single window or a periodicallyrepeating sequence of windows.

July 12, 2010,2:30 PM

+ 3 hours

08:00 10:00 08:00 10:00 08:00 10:00

Page 74: ER 2013 tutorial:  modeling the event driven world

74

Event interval

In an event interval context each window is an interval that startswhen the associated EPA receives an event that satisfies a specifiedpredicate.

It ends when it receives an event that satisfies a second predicate, orwhen a given period has elapsed.

Event interval

Initiator event list

Terminator event list

Expiration time offset

Expiration event

count

Initiator policy

Terminator policy

Temporal ordering

Patient’s admittance

Patient’s release

Earthquake

From patient’s admittance to patient’s release

+ 3 days

Within 3 days from an earthquake

Page 75: ER 2013 tutorial:  modeling the event driven world

75

Sliding fixed interval

In a sliding fixed interval context each window is an interval with fixedtemporal size. New windows are opened at regular intervals relative toone another.

Sliding fixed interval

Interval period

Interval duration

Interval size (events)

Temporal ordering

2 hours2 hours

2 hours

1 hour 1 hour 1 hour

Page 76: ER 2013 tutorial:  modeling the event driven world

76

Sliding event interval

A sliding event interval is an interval of fixed size (events number)that continuously slides on the time axis.

Sliding event interval

Event list

Interval size (events)

Event period

Temporal ordering

Every 3 blood pressure measurements

Page 77: ER 2013 tutorial:  modeling the event driven world

77

Segmentation oriented context .

Unrestricted number of partitions:Average of customer’s deposits over last month

John

Tim

Helen

David

Page 78: ER 2013 tutorial:  modeling the event driven world

78

Segmentation context – (II)

Fixed number of partitionsDistribution of alcohol consumption by age

18-25

26-50

50-

Page 79: ER 2013 tutorial:  modeling the event driven world

79

Segmentation context

F ixed Loc at ion

Enti ty di stance locat ion

Event d istance locat ion

W ithin th e hous e

Withi n 2 KM from the motel

W ithin 10 K M f ro m the acc id ent

Page 80: ER 2013 tutorial:  modeling the event driven world

80

The notion of situation

EventPatterns

Situation: an event type of interest that may require a decision or reaction

Events of various types are streaming in the system and we are looking at patterns – like “Lego templates” – of event combinations whose occurrence issue the conditions for deriving a conclusion. In our example the pattern is “OCCURS BEFORE”. After discussing the example we shall see more patterns.

Page 81: ER 2013 tutorial:  modeling the event driven world

81

Logic tables – the condition part

In words: We are looking at combination of two events – one of them of the type “Cash deposit”, the other of the type “Transfer abroad”, with four conditions:

Condition A provides filter for the Cash deposit event – at least 150KCondition B provides filter for the Transfer abroad event – at least 100K Condition C designates the pattern – relationship among the two events – Cash deposit should occur before Transfer abroadCondition D provides condition of such combination – the two events should belong to two different accounts (of the same customer)

All events need to belong to the same customer and occur at the same week (context)

Suspicious customer logic

Row #

Context Conditions

When Partition by Event filter Pattern on events Filter on patterned events

Expression

Start End Customer ID Amount <Cash deposit>

Amount <Transfer Abroad>

Cash deposit Account <Cash Deposit>

1 Every week

same >= 150K >= 100K OCCURS BEFORE

Transfer Arboad

IS NOT

Account <Transfer Abroad>

A B C D

Page 82: ER 2013 tutorial:  modeling the event driven world

82

Back to the example: Conditions A and B intend to select the event that participate in the pattern matching. There can be multiple conditions, each of them refers to a single event (typically to one or more of the fact-type within this event).

Event filter

Suspicious customer logic Row

# Context Conditions

When Partition by Event filter Pattern on events Filter on patterned events

Expression

Start End Customer ID Amount <Cash deposit>

Amount <Transfer Abroad>

Cash deposit Account <Cash Deposit>

1 Every week

same >= 150K >= 100K OCCURS BEFORE

Transfer Arboad

IS NOT

Account <Transfer Abroad>

A B C D

Page 83: ER 2013 tutorial:  modeling the event driven world

83

Pattern on events

Pattern on events designates what the relationship between events is. In this case conditions C states that an event should occur before another.

Suspicious customer logic Row # Context Conditions

When Partition by Event filter Pattern on events Filter on patterned events

Expression

Start End Customer ID Amount <Cash deposit>

Amount <Transfer Abroad>

Cash deposit Account <Cash Deposit>

1 Every week

same >= 150K >= 100K OCCURS BEFORE

Transfer Abroad

IS NOT

Account <Transfer Abroad>

A B C D

Page 84: ER 2013 tutorial:  modeling the event driven world

84

Filter on patterned events

Filter on patterned events are conditions among the members of the “matched set” of events and not on the individual events. In this case condition D is among the two events “Cash Deposit” and “Transfer Abroad” that participate in the pattern result.

Suspicious customer logic Row

# Context Conditions

When Partition by Filter on event Pattern on events Filter on patterned events

Expression

Start End Customer ID Amount <Cash deposit>

Amount <Transfer Abroad>

Cash deposit Account <Cash Deposit>

1 Every week

same >= 150K >= 100K OCCURS BEFORE

Transfer Arboad

IS NOT

Account <Transfer Abroad>

A B C D

Page 85: ER 2013 tutorial:  modeling the event driven world

85

Sample of pattern types

all pattern is satisfied when the relevant event set contains at least one instance of each event type in the participant set

any pattern is satisfied if the relevant event set contains an instance of any of the event types in the participant set

absence pattern is satisfied when there are no relevant events

relative N highest values pattern is satisfied by the events which have the N highest value of a specific attribute over all the relevant events, where N is an argument

value average pattern is satisfied when the value of a specific attribute, averaged over all the relevant events, satisfies the value average threshold assertion.

always pattern is satisfied when all the relevant events satisfy the always pattern assertion

sequence pattern is satisfied when the relevant event set contains at least one event instance for each event type in the participant set, and the order of the event instances is identical to the order of the event types in the participant set.

increasing pattern is satisfied by an attribute A if for all the relevant

events, e1 << e2 e1.A < e2.A

relative max distance pattern is satisfied when the maximal distance between any two relevant events satisfies the max threshold assertion

moving toward pattern is satisfied when for any pair of relevant events

e1, e2 we have e1 << e2 the location of e2 is closer to a certain object then the location of e1.

Page 86: ER 2013 tutorial:  modeling the event driven world

86

Summary of topic IV

Concept-based model

Logic tables

Next – epilogue and for the rest of the picture

6/30/2013 86

Page 87: ER 2013 tutorial:  modeling the event driven world

87

Agenda I

Topic I: Introduction

– Brief history of

event processing

II

Topic II: The major

differentiation factors of event-based thinking vs. traditional computing

III

Topic III: The

computational

independent model

IV

Topic IV: Drill down to

the model components

V

Topic V: Epilogue

87

Page 88: ER 2013 tutorial:  modeling the event driven world

88

From Computational Independent Model to Platform Independent Model

There is additional lexicon entity called “IT element”.Each concept can be mapped to IT element, which can be used to complete the missing links in the CIM model

The next step is to translate the CIM model to a canonic model that still does not conform with specific implementation, The PIM model follows the model introduced in the EPIA book

Page 89: ER 2013 tutorial:  modeling the event driven world

89

Event Driven Architecture

Event driven architecture: asynchronous, decoupled; each component is autonomic.

Page 90: ER 2013 tutorial:  modeling the event driven world

90

The model’s building blocks

Event Processing AgentContext

EventChannel

EventConsumer

EventType

EventProducer

GlobalState

Page 91: ER 2013 tutorial:  modeling the event driven world

91

The building blocks connected

Event Producer 1

Event Consumer 1

Event Consumer 2

Event Producer 2

Event Consumer 3

Agent 2

Channel

Agent 1

State

Agent 3

Page 92: ER 2013 tutorial:  modeling the event driven world

92

Event Processing Agents

Event Processing Agent

Filter Transform Detect Pattern

Translate Aggregate Split Compose

Enrich Project

Page 93: ER 2013 tutorial:  modeling the event driven world

93

More things to model

Instrumentation

Reaction decision

Non functional requirements

Page 94: ER 2013 tutorial:  modeling the event driven world

94

Other computational independent models

The decision model – a declarative model for decision management (BRMS)

Artifact centric BPM

Page 95: ER 2013 tutorial:  modeling the event driven world

95

One more thing

OUR DRIVING FORCE IS TO HELPEVERYBODY REALIZE THE POWER OF EVENTS TO CREATE A BETTER

WORLD