use case tutorial - lessons learned (7/7)

epts event processing technical society

eptsevent processing technical society

Pedro Bizarro on behalf of the group

Use Case TutorialLessons Learned


Lessons Learned

Architecture

Lesson 1: State-based event processing

Lesson 2: Incident objects

Lesson 3: Integration with other data management systems

Lesson 4: System of systems

Languages

Lesson 5: Query languages

Lesson 6: Classification and rules groups

Lesson 7: Customization

Glossary

Lesson 8: Changes to glossary

Use Cases

Lesson 9: Changes to questionnarie

Lesson 10: Better instructions on describing a use-case


3

Use Cases

• Retail Fraud • Thomas Paulus (CITT)

• Health Care • Pedro Bizarro, Diogo Guerra (U Coimbra), Dieter Gawlick (Oracle)

• Fleet Management • Matthew Cooper (EventZero)

• Very Large EPN – Bio-defense• Harvey G. Reed (MITRE) – Dieter on behalf of Harvey

• BPM/BAM • Hans-Arno Jacobsen (University of Toronto)

http://members.ep-ts.com/index.php?title=EPTS_UC_RF_Lessons

http://members.ep-ts.com/index.php?title=EPTS_UC_HC_Lessons

http://members.ep-ts.com/index.php?title=EPTS_UC_RF_Lessons

http://members.ep-ts.com/index.php?title=EPTS_UC_BD_Lessons

http://members.ep-ts.com/index.php?title=EPTS_UC_BB_Lessons


4

Expected next steps

• Other EPTS groups review these lessons

• Answer eventual questions and suggestions

• Wait for each group to integrate/reject suggestions

• Wait for other groups to provide initial recommendation/rework

• Develop improved questionnaire


STATE-BASED EVENT PROCESSING(ARCHITECTURE)

Lesson 1

5


6

State-based event processing

•Need to know – in timely fashion – which

–objects enter/exit specific states, overstay in a state

•Problems:

–“state” is undefined, interaction between events are state undef

–Starting from different architecture views: state in DBMS, cache?

Fraud in Retail

Health Care

Transport Management

Very Large EPN (Bio-Defense)

BPM/BAM


7

State-based event processing – Fraud in Retail

• Transaction data has to be captured in a database

• States have to be derived based on the processed events (e.g., item released from shelf --> state change)

• State (un)changes can trigger timers(e.g., item longer than x minutes released and not payed)

• Both state changes and timers can fire new events


8

State-based event processing – Health Care

• All data is captured in a (temporal) database

• Registered queries and Rules

– Registered queries produce internal events on state changes

– Those events used to fire rules, alarms, and new events

– Being too long in a state represents an event • “Critical Temperature” event: above 42º for more than 10 minutes• “Critical Heart Rate” event: above 150 bpm for more than 1

minute


9

State-based event processing – Transport Management

• Source events are used to build a model of the state of a system (e.g., state as in a business process)

• Model may be built up over long periods of time (e.g., weeks, months)

• Models need to be persistent

• Two engines: for events, for data/state

• Rules over state and time changes rather thanover the source events themselves

– if state is still in low voltage state in 5 minutes time then generate alarm

– If lights are off after current lighting up time then generate alarm


10

State-based event processing – Very Large EPN

• Determine state from events

– Associate level of severity to (derived) events

– Derive state from severity

• Different levels of bio-toxin / numbers of sick animals different states of emergency different actions


11

State-based event processing – BPM/BAM

• State changes should be detected by the CEP engine("events" are updates on the state in the form of attribute-value pairs)

• Databases used to support historic queries

– to include “past events” to correlate with future events


THE INCIDENT OBJECT(ARCHITECTURE)

Lesson 2

12


The Incident Object

Internal data structures created from complex events to facilitate complex queries in the future

Capture relevant (updatable) information

Health Care

Fraud in Retail


BPM/BAM

13


The Incident Object – Health Care

• Created, updated and deleted in response to events

• Can be updated by programs

• Contains a set of properties and references

• More important than events themselves

14


The Incident Object

• Fraud in Retail: Used to model fraud scenarios

–Suspicious Customer object created from suspicious activity

• BPM/BAM:

– Incident Objects resultant from complex event subscriptions

– Used to correlate with future events

15


The Incident Object – Very Large EPN (Bio-defense)

• “Interesting Translation” object

• Otherwise “we end-up in a sea of data”

• Needs to be acknowledged, verified (false-positive?)

• Logged, auditable, queriable (OLAP)

• Joins with raw data

16


INTEGRATION WITH OTHERDATA MANAGEMENT SYSTEMS(ARCHITECTURE)

Lesson 3

17


18

Integration with other data management systems(Not about input/output adapters)

• Core functionality and core data more naturallyhandled or stored in more than one system

• Integrating multiple distinct systems(e.g., CEP, DBMS, DW, Data Mining)

Fraud in Retail

Health Care

BPM/BAM

Transport Management


19

Integration – Fraud in Retail

• Database with business data

• Event Processing: fraud management as parallel working system which monitors the business data– Can notify / influence / control active running business processes

• Data Mining: used to identify fraud based on stored data


20

Integration – Health Care

• Event processing as part of data management

– Event processing provides timely awareness of information

• Needs to support operational characteristics(e.g., reliability, availability, security, auditing, tracking)

– Could also perceive event processing as a new operational characteristics: Timeliness (in acquiring information)

•Integration with data mining: e.g., predict cardiac arrest

•Integration with data warehousing: e.g., #of reintubations/day

•Integration with (temporal) databases:e.g., keep complete record


21

Integration – BPM/BAM

• Databases: to store past events/states

• Databases: to store past incident objects

Integration – Transport Management

• Databases: to store raw events

• Databases: to store summary historical data


SYSTEM OF SYSTEMS(ARCHITECTURE)

Lesson 4

22


23

System of systems

• Multitude of systems, people, message types, sources, sinks, CEP engines

• Different use cases reach different conclusions:

– Use case questionnaire, architecture, glossary do not have enterprise-wide, system-of-systems perspective

– System-of-systems no different than single system

Fraud in Retail


BPM/BAM


24

System of Systems – Fraud in Retail

• Retail infrastructure naturally decentralized

– Multiple event processing nodes for local data processing

– Centralized event processing or storage is necessary

• Still, no special problem in having system of systems


25

System of Systems – BPM/BAM

• Event processing carried out in multiple places

• But still provides point-processing black-box view

• “joining multiple systems is no different than creating a single system”

– Hundreds of nodes

– Assume buffer to re-order out-of-order events


26

System of Systems – Very Large EPN

• Very Large Event Processing Networks typical of large goverment activities

• Multitude of systems, people, message types, sources, sinks, CEP engines

• Peer-to-peer and hierarchical relationships

• Build for growth/extension: must have clear interfaces

• Parallel building and parallel extending

• Need to reconcile CEP with persistence

• E.g., filtering subscribers based on matching content with Identity, and corresponding fine-grained authorization

• E.g., filtering out subscribers who fail to meet geo-tagging/proximity tests

– Crosses system boundaries (on need-to-know basis), different units/timezones/coordinates

– Which police car(s) should go to some emergency?

• Should glossary include these differences?


QUERY LANGUAGES(LANGUAGE)

Lesson 5

27


28

Query Languages

Rules: to support ECA paradigm

Continuous queries:stream-based CEP

Registered queries: state-based CEP

Data mining models: predictions(non-hypothesis driven)

Health Care

Fraud in Retail

Very Large EPN – Bio-defense


29

Query Languages – Health Care

• Registered queries: check when state changes

– E.g., is heart rate different?

• Rules: check combination of conditions, fires alarms

– E.g., is heart rate above 150 bpm for more than 1 minute?

• Data mining models: focused on predictions

– E.g., predict cardiac arrest

– Should be embedded in any query language used

– Scoring

• Need alignment of query and event language


30

Query Languages – Fraud in Retail

• System listens to all events to detect fraud

– Detect fraud attempts based on pre-defined fraud scenarios

– Scenarios can be modeled with continuous queries, rules, data mining, or graph modeling tools.

Query Languages – Transport Management

• Needs language to manipulate/react/query state

– set low voltage state if below threshold

• Need alignment between event engine and database

– Retrieve a trucks current and past stat


CLASSIFICATION AND GROUPS(LANGUAGE)

Lesson 6

32


33

Classifications and Groups

• Classifications and groups used to climb-up abstraction level

• Classifications buried in application code

Health Care


34

Classifications and Groups – Health Care

150 bpm

Is it an emergency?

OK! Emergency!!


35

Classifications and Groups – Health Care

• Alert a doctor if:

• blood value is rapidly deteriorating

• Oxygen level in serious level or above

• What is rapidly?

• What is the serious level?

• What is above the serious level?

• Fire rule if one red-rule fires,

• or three orange-rules, or five black-rules

Think aboutConsumption and

Precedence


CUSTOMIZATION(LANGUAGE)

Lesson 7

36


37

Customization – Health Care

• A patient with a cardiac condition and a patient without a cardiac condition;

• A male baby with high heart rate and a female senior with high heart rate;

• A patient that started to have high fever and a patient that has had high fever more than 30 minutes.

• Precedence needed for:

– Universal Rules (for all patients)

– For a specific patient

– For a specific doctor

– For a specific patient-doctor


38

Customization – Fraud in Retail

• The customer should easily:

– add/remove/de-/activate fraud scenarios

• Stores should be able to override default rules


TALKING ABOUT EVENT PROCESSING(GLOSSARY)

Lesson 8

39


40

Glossary

• Definitions that need work

– Types of event processing

– Events / messages / alerts

– States / state-transitions

– Online scoring

– Classification

– Incident object

– Remove references to timeWhat time to use? Application? Import? Wall-clocks?

• Glossary has to be based on reference architecture


CHANGES TO QUESTIONNAIRE(USE CASES)

Lesson 9

41


42

Changes to Questionnaire

• Entice responders to provide measurable characteristics and evidence (e.g. exact/approximate event rates, event size, etc.)

• Responses should be precise enough to serve as basis for developing benchmarks.

• System of systems approach

• Is it too long?


DESCRIBING USE CASES (USE CASES)

Lesson 10

43


44

Describing Use Cases

• Group should define guidelines for structuring use case presentations


eptsevent processing technical society

If you want to participate in these discussions, contribute to this topic and/or others topics, and be part of the community effort to advance the state of the art and state of the practice:

JOIN EPTS

For details: http://www.ep-ts.com/

use case tutorial - lessons learned (7/7)

Technology