siddhi cep 2nd sideshow presentation

SIDDHI-CEP

A HIGH PERFORMANCE COMPLEX EVENT PROCESSING ENGINE

QUICK RECAP

• What is CEP?

• Why Siddhi?

WHAT IS CEP

In abstract, the tasks of the CEP is to identify meaningful patterns, relationships and data abstractions among unrelated events and fire an immediate response such as an Alert message.

WHY SIDDHI?

Cons in current CEP solutions

• Proprietary• Not enough support for complex queries• Less efficient - High latency

and memory consumption

Advantages in Siddhi

PROGRESS SO FAR...

PROGRESS SO FAR...

• Initial research • System Design• 1st iteration• Web site• Improved Siddhi API • 2nd iteration 

• All major functionalities are implemented  

• Profiling and Performance Testing

SIDDHI FUNCTIONALITIES

• Filter• State Machine (Sequence Query)• Join Event Streams• Time Window• Length Window

API TO WRITE QUERIES

Sample query for Filter

Query query = qf.createQuery(                    "StockQuote",                    qf.output("price=CSEStream.price"),                    qf.inputStream(cseEventStream),                    qf.condition("CSEStream.symbol", EQUAL, "IBM")            );

API TO WRITE QUERIES

Sample query for Sequence Query

Query query = qf.createQuery(                    "StockQuote",                    qf.output("action=$0.action", "priceA=$1.price", "priceA=$2.price"),                     qf.inputStreams(cseEventStream, infoStock),                    qf.sequence(                            qf.condition("infoStock.action", EQUAL, "buy"),                            qf.every(                                    qf.condition("CSEStream.price", GREATERTHAN, "75"),                                    qf.condition("CSEStream.price", GREATERTHAN, "$1.price")                            )

                    )            );

API TO WRITE QUERIES

Aggregators • avg• count• max• min• sum

Query query = qf.createQuery(                    "StockQuote",                    qf.output( "symbol=CSEStream.symbol“,

"avgPrice=avg(CSEStream.price)", "count=count(CSEStream.symbol)"),

                    qf.inputStream(cseEventStream),                    qf.condition("CSEStream.symbol", EQUAL, "IBM")            );

PERFORMANCE TESTING

 Details of the machine we used

Hardware : Intel- Core 2 Duo 2.10 GHz Memory 1.9GB

OS : Ubuntu Release 9.10 Kernel Linux 2.6.31-14-generic GNOME - 2.28.1

PERFORMANCE COMPARISON WITH ESPER

 Performance Comparison for a simple filter without a window

Events Siddhi(ms)

Esper(ms)

10 4 5

100 5.6 9.8

1000 34.5 55.2

10,000 252.6 491

100,000 961.4 1669.2

1000,000 3730.6 5546.8

PERFORMANCE COMPARISON WITH ESPER

 Performance Comparison for a timed window query for average calculation for a given symbol

Events Siddhi(ms)

Esper(ms)

10 5 9

100 10.6 16.2

1000 52 109.2

10,000 419.2 1159.2

100,000 1302.8 2724

1000,000 7883.2 11589.8

PERFORMANCE COMPARISON WITH ESPER

 Performance Comparison for a State machine query

Events Siddhi(ms)

Esper(ms)

10 5.8 30

100 11 58.4

1000 98.6 333

10,000 550.4 2285.6

100,000 1606 9663.2

1000,000 12563 77641

Questions?

Thank You