siddhi cep 2nd sideshow presentation
DESCRIPTION
The sideshow used at the 2nd presentation of the project Siddhi CEPTRANSCRIPT
SIDDHI-CEP
A HIGH PERFORMANCE COMPLEX EVENT PROCESSING ENGINE
QUICK RECAP
• What is CEP?
• Why Siddhi?
WHAT IS CEP
In abstract, the tasks of the CEP is to identify meaningful patterns, relationships and data abstractions among unrelated events and fire an immediate response such as an Alert message.
WHY SIDDHI?
Cons in current CEP solutions
• Proprietary• Not enough support for complex queries• Less efficient - High latency
and memory consumption
Advantages in Siddhi
PROGRESS SO FAR...
PROGRESS SO FAR...
• Initial research • System Design• 1st iteration• Web site• Improved Siddhi API • 2nd iteration
• All major functionalities are implemented
• Profiling and Performance Testing
SIDDHI FUNCTIONALITIES
• Filter• State Machine (Sequence Query)• Join Event Streams• Time Window• Length Window
API TO WRITE QUERIES
Sample query for Filter
Query query = qf.createQuery( "StockQuote", qf.output("price=CSEStream.price"), qf.inputStream(cseEventStream), qf.condition("CSEStream.symbol", EQUAL, "IBM") );
API TO WRITE QUERIES
Sample query for Sequence Query
Query query = qf.createQuery( "StockQuote", qf.output("action=$0.action", "priceA=$1.price", "priceA=$2.price"), qf.inputStreams(cseEventStream, infoStock), qf.sequence( qf.condition("infoStock.action", EQUAL, "buy"), qf.every( qf.condition("CSEStream.price", GREATERTHAN, "75"), qf.condition("CSEStream.price", GREATERTHAN, "$1.price") )
) );
API TO WRITE QUERIES
Aggregators • avg• count• max• min• sum
Query query = qf.createQuery( "StockQuote", qf.output( "symbol=CSEStream.symbol“,
"avgPrice=avg(CSEStream.price)", "count=count(CSEStream.symbol)"),
qf.inputStream(cseEventStream), qf.condition("CSEStream.symbol", EQUAL, "IBM") );
PERFORMANCE TESTING
Details of the machine we used
Hardware : Intel- Core 2 Duo 2.10 GHz Memory 1.9GB
OS : Ubuntu Release 9.10 Kernel Linux 2.6.31-14-generic GNOME - 2.28.1
PERFORMANCE COMPARISON WITH ESPER
Performance Comparison for a simple filter without a window
Events Siddhi(ms)
Esper(ms)
10 4 5
100 5.6 9.8
1000 34.5 55.2
10,000 252.6 491
100,000 961.4 1669.2
1000,000 3730.6 5546.8
PERFORMANCE COMPARISON WITH ESPER
Performance Comparison for a timed window query for average calculation for a given symbol
Events Siddhi(ms)
Esper(ms)
10 5 9
100 10.6 16.2
1000 52 109.2
10,000 419.2 1159.2
100,000 1302.8 2724
1000,000 7883.2 11589.8
PERFORMANCE COMPARISON WITH ESPER
Performance Comparison for a State machine query
Events Siddhi(ms)
Esper(ms)
10 5.8 30
100 11 58.4
1000 98.6 333
10,000 550.4 2285.6
100,000 1606 9663.2
1000,000 12563 77641
Questions?
Thank You