indusrty experience with the ibm active middleware technology (amit)
DESCRIPTION
Presented at DEBS2010.TRANSCRIPT
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation
Industry Experience with the IBM Active Middleware Technology (AMiT) Complex Event Processing Engine
Ella Rabinovich ([email protected])Joint work with Yonit Magid, Guy Sharon, Sarit Arcushin, Idan Ben-Harrush
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation2
Agenda
Introduction IBM Active Middleware Technology (AMiT) Overview Industry Experience – 3 use cases
Location Awareness
Continuous Control Monitoring (CCM)
Observation
Lessons learned
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation3
IBM Active Middleware Technology (AMiT) - History
1999
2005
2008
• Establishment of the group
Integrated with:• WebSphere Message Broker v6.1• WebSphere Premises Server v6.1• Entity Analytic Solutions v4.2.1• DB2 Health Analyzer• Continuous Control Monitor
Integrated with:• WBI-Message Broker v5• WebSphere Server Express
• AMiT v3• Customer engagements, POC,
prototypes, solutions, services and show cases
2006
2004
• AMiT v2.1• Integrated with MQSI v2.1• Services Insurance Asset• Claim processing• Underwriting
2001
• AMiT v1• IBM Global Technology Services offering
2003
• Part of the IBM Autonomic Computing ToolkitalphaWorks
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation4
From Single Events to Patterns
Events
Event ProcessingRules
SituationDetection
Definitions
Detected Situations
eventsources
Run Time
Build Time
AuthoringTool
ActionsRuntimeEngine
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation5
AMiT Model – the Situation Concept
Situation
Conditions
Lifespan
initiator terminator
Event SelectionEvent Selection ActionsActionsOperationOperation
Keys
Input events
OperatorJoiningCountingTemporalAbsenceAggregation
Notifications
Messages
Definition updates
User plug-ins
e1
e2
e3
e5 e8
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation6
IBM Active Middleware Technology (AMiT) –Engine Architecture and Building Blocks
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation7
Flexible Design
Core can be adapted fora new environment
Implementation ofeach component can be
changed without affectingthe whole engine
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation8
Industry Experience Through Use Cases
Experience through use cases from different domains Location Awareness Applications
Continuous Control Monitoring (CCM)
Observation with CEP
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation9
Location Awareness Applications
For each factory zone, a policy is defined regarding visitor presence: Visitor is not allowed to enter
this zone at all
Visitor in a zone should always be escorted
Visitor in a zone is allowed to stay unaccompanied
The event processing engine gets updates of a visitor’s location and detects unauthorized or dangerous situations, such as: A visitor enters unauthorized
zone alone
Visitor is apart from his escort in an unauthorized zone
Zone 1
Zone 2 Zone 3
Zone 4
Zone 5
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation10
Location Awareness Applications – cont.
Event Bus
RTLS Provider
(low-level event source)
FilteringSmoothing
Persistence
Zones Determination
Container Checking
Backend
AMiT
Input Adapter Output Listener
Backend Interface
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation11
Continuous Control Monitoring (CCM)
Continuous Control Monitoring (CCM) is about givingauditors an automated way to continuously monitor high risk transaction exceptions based on predefined criteria.
Risk Pattern to MonitorRisk Pattern to Monitor
Split transactions are processedin order to by-pass the requiredlevel of approvals
Executable CEP RuleExecutable CEP Rule
Alert when multiple transactions withwrite-off adjustments in aggregate aregreater than 25K within a 90 daysrolling window
CCM Application
Alert on exception
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation12
Continuous Control Monitoring (CCM) – Solution
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation13
Observation with CEP
A client of the bank has the option to purchase the service of being notified on bank related activities regarding their bank account Cashing of bank checks of amount exceeding a specified threshold
Account balance exceeds or sinks below a specified threshold (notification once every crossing of the threshold)
Completion of an approval process of various types of transactions
A client may choose from several notification options and set some individual parameters for each type of notification through web interface
A client may preference the method by which to notify Email or text message to a mobile phone
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation14
Observation with CEP - Solution
The AMiT engine integrated withthe ESB (used by the bank) as another processing node
Situations are detected asynchronously - not necessarily as a response to an incoming event
The broker node performs an observation task – the flow and content of the messages are not affected
Ctrl
In
EPObserver Node
Situation
CEPRun-Time
Engine
reformat
msg
event
Catch
Failure
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation15
Observation with CEP - Solution
WebSphere Message Broker
Batchsources
Retrieve customer
registration data from
DB
AlertPublish
DB2 Event Publisher
Data source 2
DashboardDashboard
AlertDetection
Activate rule logic using MB plug-ins
if transaction.type="cash_check“and transaction.amount>=transaction.parameter_check_threshold
AMiT
Type “cash_check”
Customer 111
Amount 500
customer 111threshold 400send by SMS
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation16
Lessons Learned
Application build time Raising the abstraction level – increases usability Expressiveness – ability to detect complex patterns Flexibility and change management (templates and hot-updates)
Application run time Scalable (both in number of rules and in volume of events) Supports transactions (is able to run as part of a global transaction) Satisfies the various performance requirements (throughput, latency)
encountered for different use cases
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation17
Lessons Learned – cont.
Event processing logic often “leaks” out of the centralized CEP engine to other parts of the solution
Event Processing Network (EPN)Raises the abstraction level for the whole event processing logic, not just the part
handled by the engine
Moxey C. et al: A Conceptual model for Event Processing Systems, an IBM RedguideTM publication.
IBM Haifa Research Lab – Event Processing
© 2010 IBM Corporation18
QuestionsQuestions