research article a novel complex event processing engine...

15
Research Article A Novel Complex Event Processing Engine for Intelligent Data Analysis in Integrated Information Systems Dong Wang, 1 Mingquan Zhou, 1,2 Sajid Ali, 3 Pengbo Zhou, 2 Yusong Liu, 4 and Xuesong Wang 2 1 School of Information Science and Technology, Northwest University, Xi’an 710127, China 2 College of Information Science and Technology, Beijing Normal University, Beijing 100875, China 3 Department of Computer Science, University of Education, Lahore 60000, Pakistan 4 School of Information, Renmin University of China, Beijing 100872, China Correspondence should be addressed to Mingquan Zhou; mqzhou [email protected] Received 19 November 2015; Revised 31 January 2016; Accepted 24 February 2016 Academic Editor: Paolo Bellavista Copyright © 2016 Dong Wang et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Novel and effective engines for data analysis in integrated information systems are urgently required by diverse applications, in which massive business data can be analyzed to enable the capturing of various situations in real time. e performance of existing engines has limited capacity of data processing in distributed computing. Although Complex Event Processing (CEP) has enhanced the capacity of data analysis in information systems, it is still a big challenging task since events are rapidly increasing in diverse applications. In this paper, a lightweight intelligent data analysis system with a novel CEP engine named LIDA-E is introduced, which employs the knowledge base with rules and an event processing algorithm for analysis. Event models as well as operators support the rules for event selection and aggregation. ese operators and rules have been utilized for constructing new CEP system architecture which combines expressiveness and efficiency in analysis. It adopts the agents and filter conception explicitly to provide the event transmission mechanism efficiently. Finally, the comparison between the proposed engine and the existing engine shows that LIDA-E has 48.65% averagely reduced time cost in different tests. e experimental results demonstrate that the developed architecture has better performance in both transmitting and analyzing a large number of events. 1. Introduction In recent years, the capacity and usage of information systems have been increasing. ese are integrated together primarily due to business processes [1]. e data volume generated by a variety of heterogeneous sources has increased around the real world [2, 3]. As a result, it is important that systems need to efficiently collect and analyze the huge amount of data and events in real time to discover meaningful results. Data analysis is a process of inspecting, cleaning, trans- forming, and modeling data with the goal of discovering useful information, suggesting conclusions, and supporting decision-making [4]. It is increasingly vital for the success of business information systems. erefore, there is a need of extremely efficient and flexible data analysis platforms to manage and process such data sets [5]. Traditional methods are not suitable for the massiveness and variety of data. As a result, a new efficient solution for data processing is imminently needed. Some reliable approaches are available for data processing [6, 7]. e well-known MapReduce paradigm has been widely used and experienced by both academia and enterprise. It is a programming model and soſtware framework developed by Google. It simplifies the processing of vast amounts of data in parallel on large clusters of commodity hardware in a reliable, fault-tolerant manner [6]. However, Hadoop and MapReduce are not suitable for processing event streams. ey were designed for offline batch processing of static data, in which all input data needs to be stored on a distributed file system in advance. Storm is another efficient solution [7]. It is a distributed, reliable, and fault-tolerant system, which is used particularly for processing continuous real-time data Hindawi Publishing Corporation International Journal of Distributed Sensor Networks Volume 2016, Article ID 6741401, 14 pages http://dx.doi.org/10.1155/2016/6741401

Upload: others

Post on 10-Oct-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

Research ArticleA Novel Complex Event Processing Engine forIntelligent Data Analysis in Integrated Information Systems

Dong Wang1 Mingquan Zhou12 Sajid Ali3 Pengbo Zhou2

Yusong Liu4 and Xuesong Wang2

1School of Information Science and Technology Northwest University Xirsquoan 710127 China2College of Information Science and Technology Beijing Normal University Beijing 100875 China3Department of Computer Science University of Education Lahore 60000 Pakistan4School of Information Renmin University of China Beijing 100872 China

Correspondence should be addressed to Mingquan Zhou mqzhou nwuyahoocom

Received 19 November 2015 Revised 31 January 2016 Accepted 24 February 2016

Academic Editor Paolo Bellavista

Copyright copy 2016 Dong Wang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Novel and effective engines for data analysis in integrated information systems are urgently required by diverse applications inwhich massive business data can be analyzed to enable the capturing of various situations in real timeThe performance of existingengines has limited capacity of data processing in distributed computing Although Complex Event Processing (CEP) has enhancedthe capacity of data analysis in information systems it is still a big challenging task since events are rapidly increasing in diverseapplications In this paper a lightweight intelligent data analysis system with a novel CEP engine named LIDA-E is introducedwhich employs the knowledge base with rules and an event processing algorithm for analysis Event models as well as operatorssupport the rules for event selection and aggregationThese operators and rules have been utilized for constructing newCEP systemarchitecture which combines expressiveness and efficiency in analysis It adopts the agents and filter conception explicitly to providethe event transmission mechanism efficiently Finally the comparison between the proposed engine and the existing engine showsthat LIDA-E has 4865 averagely reduced time cost in different tests The experimental results demonstrate that the developedarchitecture has better performance in both transmitting and analyzing a large number of events

1 Introduction

In recent years the capacity and usage of information systemshave been increasingThese are integrated together primarilydue to business processes [1] The data volume generated bya variety of heterogeneous sources has increased around thereal world [2 3] As a result it is important that systems needto efficiently collect and analyze the huge amount of data andevents in real time to discover meaningful results

Data analysis is a process of inspecting cleaning trans-forming and modeling data with the goal of discoveringuseful information suggesting conclusions and supportingdecision-making [4] It is increasingly vital for the successof business information systems Therefore there is a needof extremely efficient and flexible data analysis platforms tomanage and process such data sets [5] Traditional methods

are not suitable for the massiveness and variety of dataAs a result a new efficient solution for data processing isimminently needed

Some reliable approaches are available for data processing[6 7]Thewell-knownMapReduce paradigmhas beenwidelyused and experienced by both academia and enterprise It isa programming model and software framework developedby Google It simplifies the processing of vast amounts ofdata in parallel on large clusters of commodity hardware ina reliable fault-tolerant manner [6] However Hadoop andMapReduce are not suitable for processing event streamsThey were designed for offline batch processing of static datain which all input data needs to be stored on a distributedfile system in advance Storm is another efficient solution [7]It is a distributed reliable and fault-tolerant system whichis used particularly for processing continuous real-time data

Hindawi Publishing CorporationInternational Journal of Distributed Sensor NetworksVolume 2016 Article ID 6741401 14 pageshttpdxdoiorg10115520166741401

2 International Journal of Distributed Sensor Networks

CEPengine

Event source

Event source

Event source

Userinterface

Rules and patterns

Analysis result

Events

Systemaction

Build timeRun time

Figure 1 General architecture of CEP application

streams It adopts a default scheduler which employs a simpleround-robin algorithm based cluster without consideringthe internode and interprocess traffic and it may have asignificant impact on performance [8] Besides this thedefault scheduler in Storm often uses all available nodes ina cluster regardless of workload For the light workload ofinput stream the operational cost can be reduced by using asingle node in a cluster [9]

Therefore a solution for real-time processing of large-scale web data stream with less cost should satisfy fourrequirements They are as follows (1) The local agent isrequired to get real-time data from information systems anddeliver events to an engine with high throughput It mustbe easily integrated in information systems without majorchanges (2)A high-performance engine is a key point for thesolution An efficient event processing algorithm is capableof processing large volume of incoming events It could beadopted by engine to improve the performance in real-timeprocessing (3) Event transmission mechanism for deliveringevents should reflect the up-to-date state of the actionsand operations in information systems The mechanism isexpected to achieve high throughput for transporting eventsfrom multisources to CEP engine (4)Well designed systemarchitecture is flexible and scalable in development anddeployment It is a combination of thementioned agent eventtransmission mechanism and engine

However the volume of data created by integrated infor-mation systems is enormous which is beyond the processingcapabilities of the system A new solution is needed to processthe continuous data in real time as well as discover the excep-tions threats and actionable information behind all dataindicating that theComplex Event Processing (CEP) is a goodcandidate Applying CEP approach could provide an ade-quate solution enable access to real-time information whileproviding advanced query capabilities and minimal pertur-bation [10 11] Therefore CEP could be used in intelligentdata analysis for enabling real-time processing and situationdetection consequently leading to a new quality of system

In general CEP applications engine usually processesevents according to the rules and detects different situationsduring events Rules are defined by system administrator inbuild time It is a component of CEP engine for saving thecomposite constraints of complex eventThe general architec-ture of such CEP applications is shown in Figure 1 It is

working in three steps Firstly the streams of incoming eventsare captured by CEP engine (run time) Secondly eventpatterns and CEP rules defined by users (build time) areimplemented in CEP engine to analyze the event streams anddetectmeaningful data (run time) Finally immediate actionsare triggered in CEP engine (run time)

According to the traditional CEP technology we aim tobuild lightweightmodularized and extensible architecture toimprove the data analysis capability in integrated informationsystem by CEP approach In particular the need of a novelCEP engine to deal with a large number of input events fromintegrated systems is obvious Starting from these premisesa lightweight intelligent data analysis system (LIDAS) isproposed It has a CEP engine LIDA-E which is developed toprovide event processing services In order to represent therelationship and operation of events we propose event mod-els with operators to describe event rule through Event Proc-essing Language (EPL) CEP engine is an important compo-nent which adopts an efficient event processing algorithmIt translates rule statement into rule instance for process-ing input events Inside the system we introduce a dataknowledge base used for business data knowledge storage andpresentation as well as a rule base presented for rules storageA distributed event transmission mechanism is presented forevent delivery from information systems (agents) to CEPengine (filter)

As a solution we propose LIDAS so that the administra-tor (has no knowledge onCEP) can concentrate on the defini-tion of event rules without the need of handwriting any codeThe contributions of this paper can be summarized as follows

(i) Event models define the semantics of both atomicand complex events They allow easy adaptation toinformation system needs and are used to describeevent aggregation in event processing rules

(ii) The high-performance CEP engine LIDA-E employsthe event processing algorithm and knowledge base isintroduced for intelligent data analysis in real time

(iii) A distributed event transmission mechanism is pro-posed by agents (deployed in information systems)and filter (deployed in LIDA-E engine) explicitlyconceived to realize easy and efficient event delivery

International Journal of Distributed Sensor Networks 3

(iv) We discuss the design and implementation of LIDASarchitecture which is an integration of LIDA-Eengine and event transmission mechanism

The rest of the paper is organized as follows In Section 2we discuss related works of CEP technology Section 3describes the relevant concepts of LIDAS such as eventmodel operators and SQL-like Event Processing LanguageThe proposed architecture is presented in Section 4 Section 5shows the experimental results Finally in the last section wesummarize our research work

2 Related Work

CEP is considered as extracting complex situations and react-ing on them frommassive events [12] Meanwhile CEP is nota new terminology as Luckham first introduced [13] It hasbeen widely exploited during the past years mostly in big dataanalytics [5] RFID [14] Internet of Things (IoT) [15] failureprediction systems [16] real-time grid monitoring [11] andhealthcare [17] According to CEP different algorithms havebeen proposed to increase Complex Event Processing capa-bilityThey adoptCEP engine (like Storm [7] Drools [18] andEsper [10 19]) in their applications while others design newevent processing engines and system architectures [11 12 1516 20] with evaluation to show the usefulness and scalability

Storm is a free and open source distributed real-timecomputation system which makes it easy to reliably processunbounded streams of data in real-time processing by batchoperations [7] It includes spouts and bolts which could beexecuted with many tasks in parallel on multiple machines(worker nodes) in a cluster [8] Storm is designed to processunbounded streams of data in a storm cluster (master nodeand worker nodes) However as discussed by Xu et al [8]Storm weakly focuses on the performance and job assign-ment with different workload Moreover the deployment ofStorm requires more nodes

Differing from Storm Drools [18] and Esper [10 19]are CEP engines which include a module providing nativesupport for events evaluation and temporal logic analysis ina single work node Yao et al [18] presented a Drools basedCEP framework which was used to process surgical eventsand provided sense and response capability for hospitals Inthe framework the author mentions that the knowledge withCEP is quite complicated most of which comes from expertsand is fuzzy and hard to verify the accuracy To combineknowledge with CEP system Bruns et al [10] describe anovel event-driven architecture for a decision support systemwhich leads to a new quality of M2M systems which areintelligent and flexible The system adopts Esper as the coreevent processing component and employs a rules base toexplicitly represent the M2M knowledge of domain expertsTherefore CEP could be used as an appropriate candidatewith rule base for intelligent data analysis

However with the development of web technology theopen source engines become less efficient to deal with hugevolume of data In order to get higher performance somespecial CEP engines have been developed for particular

domains (LiSEP [12] DPCEP [15] and RTE-CEP [20])Zappia et al [12] described the design and implementationof a lightweight and extensible Complex Event Processingengine called LiSEP for sensing and responding applicationsThe author adopted the Staged Event-Driven Architecture(SEDA) principles to clearly separate core event processinglogic from lower level resourcemanagement issues Howeverevent model and operators are not designed and used forLISEP as well as event collection The event processingmethod of LISEP is not clearly described in their research

Combining CEP with distributed systems GEMINI2 [11]is proposed as a custom framework dedicated for CEP basedreal-time monitoring of grid infrastructures In GEMINI2100 sensors loaded 400 events in one second and theperformance of the CEP engine was not evaluated Theperformance of working nodes and servers could not supportmassive events transmission

In summary most CEP system architectures used inexisting distributed environment are not oriented towardsreal-time processing for integrated information systems Theprevailing CEP engines are not to explore the performanceof event processing but rather expose the event processingmethods while the engines are suitable for processing eventstreams and are not well suited for analyzing massive data inintegrated systems The work described here is a result of theprevious investigations in CEP and intelligent data analysis indistributed network

3 CEP for Intelligent Data Analysis

31 Application Scenario A Project Management Informa-tion System (PMIS) is one important type of Computer-Based Information Systems (CBIS) and is used to collectprocess analyze and publish the data for a particular purpose[21] Project management (PM) is a knowledge-centric andexperience-driven activity supported by an appropriate PMIS[22] PMIS is an integrated information system that helpsproject manager to carry out projects systematically andmonitor progress of projects closely

A project life cycle is divided into many states suchas proposal review implementation and acceptance statesStarting from project proposal to implementation Taghavi etal [23] proposed a web based projectmanagement functionalmodel that separates project work progress managementinto time fund quality contract bidding demand integritycomprehensiveness handover and accomplishment Eachproject state is supported by different subsystems which arecombined together as a PMIS According to project states wepropose a project state model which shows the state changesin a project life cycle In Figure 2 ldquominusrdquo is a start state and ldquo+rdquois an end state others are middle states (proposal reviewimplementation and acceptance) they represent the wholelife cycle of a project In different project states systemsverify (119864Verify) the reasonability and validity of massiveproject data If the project data passes the verification astate transition event (119864declare 119864review 119864implement 119864accept and119864complete) changes the state from one to another Otherwise afailed event happens and changes the state to start

4 International Journal of Distributed Sensor Networks

1 2 3 4minus +

Efailed_1

Efailed_2

Efailed_3

Efailed_4

EVerify_1

EVerify_2

EVerify_3

EVerify_4

Edeclaring

Ereview

Eimplement

Eaccept

Ecomplete

Figure 2 Project state model

According to the events attributes in PMIS constraintsare necessary for analyzing data availability and validity whenevents occurred by user activities Some rules for data analysisare elaborated in the following Rule 1 monitor the activestate of a subsystem Rule 2 examine the document exami-nation time of a project in one day Rule 3 examine whetherthe number of projects of a manager is above a defined value

32 Event Model Luckham and Schulte [24] defined anevent as there is a large amount of events occurring insideor outside the system during multiple phases such as makinga business transaction receiving an email creating a salesreport or uploading a file Events are extracted from webservices user activities system log database and so onThey are separated into atom and complex atom events areidentified through data collection and complex events aredetected by the analysis of atom event stream [25 26]

Definition 1 (atom event) An atom eventmeans that a specialactivity or data transition occurs in a point in time which isrepresented by a six-tuple The elements of the six-tuple areas follows

Atom Event ae = ⟨AEidPid 119860 119878 119879 119905⟩ (1)

AEid is the identification of an atom event Pid presentsthe identification of a project And 119860 is the set of projectattributes and data 119878 is a source from where the event isgenerated 119879 is event type and 119905 is timestamp which identifieswhen the atom event occurs In the project managementsystem example each user activity or operation in systemgenerates an atom event Atom events usually represent thestate of a business attribute or some changes

Definition 2 (complex event) It is a combination of atomevents or complex events consisting of the predefined rulessuch as constraints logical relationship and time sequencecombination It is also denoted by a six-tuple

Complex Event ce = ⟨CEid 119860 119862 119879 119905119887 119905119890⟩ (2)

CEid is the identification of a complex event119862 is a combi-nation of the atom events and complex events that trigger thisevent to happen where 119862 = ae1 ae2 ae119899 ce1 ce2 ce119898 119899+119898 gt 0119860 is a set of attributes119860 = ae1119860 ae2119860 ae119899119860 ce1119860 ce2119860 ce119898119860 119905119887 is the starting time and 119905119890is ending time of complex event where 119905119890 gt= 119905119887 A complexevent is a combination of atom and complex events by patternmatching rules that is a specific set of event operators Wewill introduce the event operators and rules in the followingsubsection

Definition 3 (event type) The set of all similar events is calledthe event type 119879 It shows the object which triggers the eventHence

119864119879= 119890119879

1 119890119879

2 119890

119879

119894 119890

119879

119899 =

119899

⋃1

119890119879

119894 (3)

Events in an event type have the same attributes of 119879 Forinstance the type may refer to events regarding informationverification however the other event properties may bedifferent (eg source or attribute) An example is the typeof all events that denote the submitted information 119864submit

shows that this event type indicates submitted informationand is constituted by 119890submit

1 119890submit2 119890submit

119899

Definition 4 (event space) The set of all possible eventsknown from a certain information system is called the eventspace Hence

= 1198641198791 1198641198792 119864

119879119909 119864

119879120591 =

120591

⋃1

119864119879119909

= ae1 ae2 ae119899 ce1 ce2 ce119898

=

119899

⋃1

ae119894119898

⋃1

ce119895

(4)

The event space is formed by all the sets of event types aswell as all the atomic events and complex events For exampleportal represents all the events that occurred in portal It can

be illustrated as portal = 119890 | forall119890 eS = ldquoportalrdquo

33 Operators This section describes the operators for aggre-gating events They are used to show the relationship amongdetected events We extend them from event constructors[18 27] which consider the aggregation demands on eventcomposition We present three types of operators logicaloperator mathematical operator and temporal operatorTable 1 shows the most frequently used operators for eventdetection Atom and complex events are both denoted by 119890

These operators are used to aggregate events and detectcomplex patterns that catch the meaningful informationfrom real-time data streams For example an event patternWITHIN(1198901 or 1198902 1min) means when either 1198901 or 1198902 occurswithin time period less than 1 minute the pattern SEQ(1198901 and1198902 1198903 1198904 1198905) and (AVE(score 1198903 1198904 1198905) gt 80) is matched whenboth events 1198901 and 1198902 occur and then 1198903 1198904 and 1198905 occurcontinuously meanwhile the average score in 1198903 1198904 and 1198905 isgreater than 80

34 Event Processing Language and Rules Event ProcessingLanguage (EPL) provides a set of patterns like filtering corre-lation applying constraints and aggregating These patternsdefine the availability of events querying and filtering EPLstatement is a description of event operators that are usedto derive and aggregate information from event streams Theauthors in [27] elaborated twenty EPLs to compare their

International Journal of Distributed Sensor Networks 5

Table 1 Event processing operators

Number Type Operator Expression Description1

Logical

CON(and) 1198901 and 1198902 Conjunction of 1198901 and 1198902 without occurrence order2 DIS(or) 1198901 or 1198902 Disjunction of 1198901 and 1198902 without occurrence order3 NEG(sim) sim119890

1Negation of 119890

1

4 ANY(exist) exist(1198901 1198902) Any event that occurs of 1198901 and 11989025 EVERY(forall) forall(1198901) Every occurrence of 11989016 SEQ SEQ(1198901 1198902) Select a given sequence of events from input events7 SEL SEL(1198901) Select an event from input events8

Mathematical

AVE AVE(1198861 1198901 1198902) Average value of 1198861 in 1198901 and 11989029 SUM SUM(1198861 1198901 1198902) Summation value of 1198861 in 1198901 and 119890210 MAX MAX(1198861 1198901 1198902) Maximal value of 1198861 in 1198901 and 119890211 MIN MIN(1198861 1198901 1198902) Minimum value of 1198861 in 1198901 and 119890212 COUNT COUNT(1198901) Occurrence number of 119890113 FIRST FIRST(1198901 1198902) First event of 1198901 and 119890214 LAST LAST(119890

1 1198902) Last event of 119890

1and 119890

2

15

Temporal

WITHIN WITHIN(1198901 1199051 1199052) 119890

1occurs within time intervals 119905

1and 1199052

16 WITHIN WITHIN(1198901 119905) 1198901 occurs within less than 11990517 DURING DURING(1198901 1198902) 1198901 occurs during 119890218 WINDOW WINDOW(119890

1 119905) 119890

1occurs for time period 119905

19 WINDOW WINDOW(1198901 119899) 1198901 occurs 119899 times (119899 gt 0)20 AT AT(1198901 119905) 1198901 occurs at time 119905

operators and consumption modes According to the pro-posed event operators and consumption modes we employEsper EPL syntax [28] for specifying event processing rulessupporting by event operators in this investigation The EPLsyntax is presented in the following

select select list from stream def [as name] [ streamdef [as name]] [ ][where search conditions] [group by grouping expres-sion list] [having grouping search conditions][output output specification] [order by order byexpression list] [limit num rows]

The above EPL syntax includes some query clauses thatare introduced by Ahmad et al [29] The select clauses areused to select all properties or to specify the list of eventproperties and expressionsThe ldquofrom clauserdquo indicates eventstream name The ldquowhere clauserdquo is an optional clause usedto join and correlate event streams The ldquogroup by clauserdquoseparates the output events into groups The ldquohaving clauserdquois used in combination with the ldquogroup by clauserdquo to restrictthe groups of returned rows to only those whose condition istrue Adopting EPL syntax rules declared in Section 31 canbe represented in a standard structure as follows

Rule 1 MissingEventAgent

SELECT COUNT(eSid) WHITIN(RequireTime)IF (COUNT(eSid) = 0) THENACTION createMissingEventAgentWarning

Rule 2 UserRepeatDocExamine

SELECT COUNT(ePid) WHERE eTYPE =lsquoDocExaminersquo WHITIN(1 day)IF ANY(eCOUNT(ePid)) gt LIMIT DOCEX-AMINE THEN ACTION create UserRepeat-DocExamineWarning

Rule 3 Project ManagerExamine

SELECT COUNT(eManager) WHEREeTYPE = lsquoSubmitProjectInforsquo and eManager =e1015840Manager ande1015840ProjectState = lsquoUncompletedrsquoorlsquoSubmitedrsquoIF COUNT(eManager) gt LIMIT MANAGERTHEN ACTION create ManagerRepeatedError

We assume that the above rules provide analysis enginewith rules for what to do in event streams For this aim theengine is intelligent to execute rules by adopting a knowledgebase Knowledge base provides intelligence for the engine asit contains rich parameters for both engine and rules

For example LIMIT DOCEXAMINE is defined byLIDAS administrator in Rule 2 Its value is according tothe document of examination server runtime state Rule 2is constructed to balance the capacity of the server to eachproject and detect vicious document examination request Itspecifies that the trustworthiness of a document examinationrequest should be continuously monitored The notificationis generated as soon as the value rises above a given threshold

6 International Journal of Distributed Sensor Networks

Filter

Analysis engine

Knowledgebase

Rule base

Event model

CEP component

Analysisresult

User dashboard

Lightweight intelligent data analysis system

Integrated systems

Datastorage

User admin board

System N

Agent 3

System 2

Agent 2

System 1

Agent 1 middot middot middot

Figure 3 LIDAS modularized architecture with distributed systems

Main process thread

Agent 1

Agent 2

Agent n

Listener Preprocess

QueueQueues

Analysis thread Analysis

result

Filter Analysis engine (LIDA-E)

KnowledgebaseRule base

ae1ae2ae3

ce1ce2ce3

Figure 4 The event transmission and process workflow of LIDAS

value Rule 3 is made for analyzing how many uncompletedor submitted projects does a manager have in system If thenumber of a managerrsquos projects is beyond the limitationanalysis engine will create an error action

4 The Architecture of LIDAS

41 Architecture Design of LIDAS We have to extend thecapability of integrated information systems towards theintelligent data analysis by adopting advantage of event pro-cessing technologies keeping clearly apart each peculiarityof the CEP components to perform even deep changeswithout affecting related integrated systems In designing theintelligent data analysis architecture we started by specifyingthe CEP components that interact with the technologicalarchitecture and traditional integrated systems

As shown in Figure 3 LIDAS is a lightweight andmodularized framework designed to connect the integratedsystems for intelligent data analysisThe bottom of the frame-work is agents that are deployed in the traditional integratedinformation systems to extract data in complicated businessprocesses The upper part is an intelligent data analysis with

CEP which deals with event processing and data analyzing Itis implemented as an intelligent processing center connectedto explicit event queues in accordance with predefined eventrules System administrators define event processing rulesand knowledge through user admin board in system buildtime The users get analysis results from user dashboard inreal time

In order to show the workflow of LIDAS framework weintroduced agent filter and analysis engine to illustrate eventtransmit and processing (see Figure 4) First agents sendatomevents to the listenermodule in filter And then preproc-ess module reads events from listener and stores them inqueue it sends a message of event arriving to analysis engineAnd the engine receives messages from filter it enables theprocessing thread to access new events from queue Thegraphical representation of the workflow is illustrated inFigure 4 and introduced in the following subsections in detail

42 Event Transmission Component In order to improvethe scalability in large-scale integrated systems and reducethe network delay caused by online event streams deliveryalmost all current event processing frameworks are based on

International Journal of Distributed Sensor Networks 7

Main thread

QueueQueues

Event process

KnowledgebaseRule base

Analysisresult

Analysis engine (LIDA-E)

Filter2 3 5

4 6

7Event access

1ae1ae2ae3

ce1ce2ce3

Figure 5 The working pipeline of LIDA-E

CEP However event receiving and dispatching problems arecommonly ignored Figure 3 illustrates the simplified viewover the LIDAS architecture in distributed systemsThe com-bination of filter and agent works as a bridge that transmitsevents from information systems to LIDA-E They are basiccomponents which act as an event provider to CEP engine

We use agent which is similar to sensor [11] and adapter[30] used in monitoring system to collect information fromdistributed environment Agent is responsible for extractingreal-time business data and generating atom event It is eventsource and is deployed in an integrated system to generatemassive events with business data Using the proposed eventmodel agent reads data froman interface of integrated systemand translates them into atom events According to the eventmodel (Definition 1) agent puts data as attributes in 119860 asa list It generates atom event immediately when data isdetected in systems

Collaborating with CEP technology the filter detectscomposite events from different information systems [27]Filter is an event receiving and preprocessing module thathandles subscription related to control coming messagesfrom analysis engine and processes high volumes of eventobjects which are received from agents According to theavailability of business data filter pushes each valid atomevent into a map that is held in memory while it notifies theanalysis engine that a new event has arrived and is storedin map We summarize the functions of filter as follows (1)receive events from distributed agents (2) preprocess eventsstreams according to event availability (3) push events intomemory and notify analysis engine

Somemethods for example TCP FTP SMTP HTTP andTELNET can be used in event transfer method between filterand agent However we use a New IO (NIO) client serverNetty framework in our event transmission mechanism tofind a way to achieve ease of development performancestability and flexibility with high throughput low latency lessresource consumption andminimized unnecessary memorycopy We have developed a socket client and server by Nettyclient is deployed in agent and server which is adopted byfilter By accepting socket communication protocol filter hasthe capability to listen on a system port and receive eventsfrom different agents Event transmission mechanism is thefirst step that collects real-time data from integrated systems

the CEP based event processingmethodwill be introduced inthe next subsection

43 LIDA-E Engine The analysis engine called LIDA-E isthe core component of LIDAS architecture it is responsiblefor processing the events queue and judging whether there isa complex event inferred [25] It provides effective queuingscheduling time and count-window support and fast in-memory processing of high-speed continuous unboundeddata streams In order to divide functions of engine we designthree modules in LIDA-E they are main thread event accessand event process Main thread module is used to monitorfilter and control event access module Event access moduleis dedicated for accessing events from the queue which iswritten by filter Event process module executes rules andprocesses all the events It is a linear task controller thatdeploys rules one by one and manages events to be processedthrough them

CEP component comprises a set of rules and knowledgeto detect a predefined group of anomalies on the basis of thereceiving event attribute which are the same in [31ndash33] Therule base specifies how to infer complex events from the eventstream Howver knowledge base provides the link betweenknown information (the antecedent) and the information tobe deduced (the consequent) or actions to be executed It isa useful supplement to rule base for processing events Bothrules and knowledge are defined in build time and loadedwhen LIDAS starts to work

LIDA-E workflow is shown in Figure 5 it is a runtimeworkflow in which the analysis engine waiting for eventsarriving has already executed rules with knowledge in eventprocessmoduleTheworkflow follows the substeps as follows(1) filter pushes events into queue as soon as it receives eventsfrom different agents (2) filter sends a message to analysisengine to notify it that a new event has come (3) main threadmodule receives themessage andnotifies event accessmodulethat new events have been pushed in queue (4) event accessmodule reads events fromqueue and prepares the submissionfor event process module (5) event access module sendsevent to event processmodule immediately (6) event processmodule accesses queues for event processing in event processmodule each event is orderly processed by rules for eachrule at least one queue is provided for runtime event storage

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 2: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

2 International Journal of Distributed Sensor Networks

CEPengine

Event source

Event source

Event source

Userinterface

Rules and patterns

Analysis result

Events

Systemaction

Build timeRun time

Figure 1 General architecture of CEP application

streams It adopts a default scheduler which employs a simpleround-robin algorithm based cluster without consideringthe internode and interprocess traffic and it may have asignificant impact on performance [8] Besides this thedefault scheduler in Storm often uses all available nodes ina cluster regardless of workload For the light workload ofinput stream the operational cost can be reduced by using asingle node in a cluster [9]

Therefore a solution for real-time processing of large-scale web data stream with less cost should satisfy fourrequirements They are as follows (1) The local agent isrequired to get real-time data from information systems anddeliver events to an engine with high throughput It mustbe easily integrated in information systems without majorchanges (2)A high-performance engine is a key point for thesolution An efficient event processing algorithm is capableof processing large volume of incoming events It could beadopted by engine to improve the performance in real-timeprocessing (3) Event transmission mechanism for deliveringevents should reflect the up-to-date state of the actionsand operations in information systems The mechanism isexpected to achieve high throughput for transporting eventsfrom multisources to CEP engine (4)Well designed systemarchitecture is flexible and scalable in development anddeployment It is a combination of thementioned agent eventtransmission mechanism and engine

However the volume of data created by integrated infor-mation systems is enormous which is beyond the processingcapabilities of the system A new solution is needed to processthe continuous data in real time as well as discover the excep-tions threats and actionable information behind all dataindicating that theComplex Event Processing (CEP) is a goodcandidate Applying CEP approach could provide an ade-quate solution enable access to real-time information whileproviding advanced query capabilities and minimal pertur-bation [10 11] Therefore CEP could be used in intelligentdata analysis for enabling real-time processing and situationdetection consequently leading to a new quality of system

In general CEP applications engine usually processesevents according to the rules and detects different situationsduring events Rules are defined by system administrator inbuild time It is a component of CEP engine for saving thecomposite constraints of complex eventThe general architec-ture of such CEP applications is shown in Figure 1 It is

working in three steps Firstly the streams of incoming eventsare captured by CEP engine (run time) Secondly eventpatterns and CEP rules defined by users (build time) areimplemented in CEP engine to analyze the event streams anddetectmeaningful data (run time) Finally immediate actionsare triggered in CEP engine (run time)

According to the traditional CEP technology we aim tobuild lightweightmodularized and extensible architecture toimprove the data analysis capability in integrated informationsystem by CEP approach In particular the need of a novelCEP engine to deal with a large number of input events fromintegrated systems is obvious Starting from these premisesa lightweight intelligent data analysis system (LIDAS) isproposed It has a CEP engine LIDA-E which is developed toprovide event processing services In order to represent therelationship and operation of events we propose event mod-els with operators to describe event rule through Event Proc-essing Language (EPL) CEP engine is an important compo-nent which adopts an efficient event processing algorithmIt translates rule statement into rule instance for process-ing input events Inside the system we introduce a dataknowledge base used for business data knowledge storage andpresentation as well as a rule base presented for rules storageA distributed event transmission mechanism is presented forevent delivery from information systems (agents) to CEPengine (filter)

As a solution we propose LIDAS so that the administra-tor (has no knowledge onCEP) can concentrate on the defini-tion of event rules without the need of handwriting any codeThe contributions of this paper can be summarized as follows

(i) Event models define the semantics of both atomicand complex events They allow easy adaptation toinformation system needs and are used to describeevent aggregation in event processing rules

(ii) The high-performance CEP engine LIDA-E employsthe event processing algorithm and knowledge base isintroduced for intelligent data analysis in real time

(iii) A distributed event transmission mechanism is pro-posed by agents (deployed in information systems)and filter (deployed in LIDA-E engine) explicitlyconceived to realize easy and efficient event delivery

International Journal of Distributed Sensor Networks 3

(iv) We discuss the design and implementation of LIDASarchitecture which is an integration of LIDA-Eengine and event transmission mechanism

The rest of the paper is organized as follows In Section 2we discuss related works of CEP technology Section 3describes the relevant concepts of LIDAS such as eventmodel operators and SQL-like Event Processing LanguageThe proposed architecture is presented in Section 4 Section 5shows the experimental results Finally in the last section wesummarize our research work

2 Related Work

CEP is considered as extracting complex situations and react-ing on them frommassive events [12] Meanwhile CEP is nota new terminology as Luckham first introduced [13] It hasbeen widely exploited during the past years mostly in big dataanalytics [5] RFID [14] Internet of Things (IoT) [15] failureprediction systems [16] real-time grid monitoring [11] andhealthcare [17] According to CEP different algorithms havebeen proposed to increase Complex Event Processing capa-bilityThey adoptCEP engine (like Storm [7] Drools [18] andEsper [10 19]) in their applications while others design newevent processing engines and system architectures [11 12 1516 20] with evaluation to show the usefulness and scalability

Storm is a free and open source distributed real-timecomputation system which makes it easy to reliably processunbounded streams of data in real-time processing by batchoperations [7] It includes spouts and bolts which could beexecuted with many tasks in parallel on multiple machines(worker nodes) in a cluster [8] Storm is designed to processunbounded streams of data in a storm cluster (master nodeand worker nodes) However as discussed by Xu et al [8]Storm weakly focuses on the performance and job assign-ment with different workload Moreover the deployment ofStorm requires more nodes

Differing from Storm Drools [18] and Esper [10 19]are CEP engines which include a module providing nativesupport for events evaluation and temporal logic analysis ina single work node Yao et al [18] presented a Drools basedCEP framework which was used to process surgical eventsand provided sense and response capability for hospitals Inthe framework the author mentions that the knowledge withCEP is quite complicated most of which comes from expertsand is fuzzy and hard to verify the accuracy To combineknowledge with CEP system Bruns et al [10] describe anovel event-driven architecture for a decision support systemwhich leads to a new quality of M2M systems which areintelligent and flexible The system adopts Esper as the coreevent processing component and employs a rules base toexplicitly represent the M2M knowledge of domain expertsTherefore CEP could be used as an appropriate candidatewith rule base for intelligent data analysis

However with the development of web technology theopen source engines become less efficient to deal with hugevolume of data In order to get higher performance somespecial CEP engines have been developed for particular

domains (LiSEP [12] DPCEP [15] and RTE-CEP [20])Zappia et al [12] described the design and implementationof a lightweight and extensible Complex Event Processingengine called LiSEP for sensing and responding applicationsThe author adopted the Staged Event-Driven Architecture(SEDA) principles to clearly separate core event processinglogic from lower level resourcemanagement issues Howeverevent model and operators are not designed and used forLISEP as well as event collection The event processingmethod of LISEP is not clearly described in their research

Combining CEP with distributed systems GEMINI2 [11]is proposed as a custom framework dedicated for CEP basedreal-time monitoring of grid infrastructures In GEMINI2100 sensors loaded 400 events in one second and theperformance of the CEP engine was not evaluated Theperformance of working nodes and servers could not supportmassive events transmission

In summary most CEP system architectures used inexisting distributed environment are not oriented towardsreal-time processing for integrated information systems Theprevailing CEP engines are not to explore the performanceof event processing but rather expose the event processingmethods while the engines are suitable for processing eventstreams and are not well suited for analyzing massive data inintegrated systems The work described here is a result of theprevious investigations in CEP and intelligent data analysis indistributed network

3 CEP for Intelligent Data Analysis

31 Application Scenario A Project Management Informa-tion System (PMIS) is one important type of Computer-Based Information Systems (CBIS) and is used to collectprocess analyze and publish the data for a particular purpose[21] Project management (PM) is a knowledge-centric andexperience-driven activity supported by an appropriate PMIS[22] PMIS is an integrated information system that helpsproject manager to carry out projects systematically andmonitor progress of projects closely

A project life cycle is divided into many states suchas proposal review implementation and acceptance statesStarting from project proposal to implementation Taghavi etal [23] proposed a web based projectmanagement functionalmodel that separates project work progress managementinto time fund quality contract bidding demand integritycomprehensiveness handover and accomplishment Eachproject state is supported by different subsystems which arecombined together as a PMIS According to project states wepropose a project state model which shows the state changesin a project life cycle In Figure 2 ldquominusrdquo is a start state and ldquo+rdquois an end state others are middle states (proposal reviewimplementation and acceptance) they represent the wholelife cycle of a project In different project states systemsverify (119864Verify) the reasonability and validity of massiveproject data If the project data passes the verification astate transition event (119864declare 119864review 119864implement 119864accept and119864complete) changes the state from one to another Otherwise afailed event happens and changes the state to start

4 International Journal of Distributed Sensor Networks

1 2 3 4minus +

Efailed_1

Efailed_2

Efailed_3

Efailed_4

EVerify_1

EVerify_2

EVerify_3

EVerify_4

Edeclaring

Ereview

Eimplement

Eaccept

Ecomplete

Figure 2 Project state model

According to the events attributes in PMIS constraintsare necessary for analyzing data availability and validity whenevents occurred by user activities Some rules for data analysisare elaborated in the following Rule 1 monitor the activestate of a subsystem Rule 2 examine the document exami-nation time of a project in one day Rule 3 examine whetherthe number of projects of a manager is above a defined value

32 Event Model Luckham and Schulte [24] defined anevent as there is a large amount of events occurring insideor outside the system during multiple phases such as makinga business transaction receiving an email creating a salesreport or uploading a file Events are extracted from webservices user activities system log database and so onThey are separated into atom and complex atom events areidentified through data collection and complex events aredetected by the analysis of atom event stream [25 26]

Definition 1 (atom event) An atom eventmeans that a specialactivity or data transition occurs in a point in time which isrepresented by a six-tuple The elements of the six-tuple areas follows

Atom Event ae = ⟨AEidPid 119860 119878 119879 119905⟩ (1)

AEid is the identification of an atom event Pid presentsthe identification of a project And 119860 is the set of projectattributes and data 119878 is a source from where the event isgenerated 119879 is event type and 119905 is timestamp which identifieswhen the atom event occurs In the project managementsystem example each user activity or operation in systemgenerates an atom event Atom events usually represent thestate of a business attribute or some changes

Definition 2 (complex event) It is a combination of atomevents or complex events consisting of the predefined rulessuch as constraints logical relationship and time sequencecombination It is also denoted by a six-tuple

Complex Event ce = ⟨CEid 119860 119862 119879 119905119887 119905119890⟩ (2)

CEid is the identification of a complex event119862 is a combi-nation of the atom events and complex events that trigger thisevent to happen where 119862 = ae1 ae2 ae119899 ce1 ce2 ce119898 119899+119898 gt 0119860 is a set of attributes119860 = ae1119860 ae2119860 ae119899119860 ce1119860 ce2119860 ce119898119860 119905119887 is the starting time and 119905119890is ending time of complex event where 119905119890 gt= 119905119887 A complexevent is a combination of atom and complex events by patternmatching rules that is a specific set of event operators Wewill introduce the event operators and rules in the followingsubsection

Definition 3 (event type) The set of all similar events is calledthe event type 119879 It shows the object which triggers the eventHence

119864119879= 119890119879

1 119890119879

2 119890

119879

119894 119890

119879

119899 =

119899

⋃1

119890119879

119894 (3)

Events in an event type have the same attributes of 119879 Forinstance the type may refer to events regarding informationverification however the other event properties may bedifferent (eg source or attribute) An example is the typeof all events that denote the submitted information 119864submit

shows that this event type indicates submitted informationand is constituted by 119890submit

1 119890submit2 119890submit

119899

Definition 4 (event space) The set of all possible eventsknown from a certain information system is called the eventspace Hence

= 1198641198791 1198641198792 119864

119879119909 119864

119879120591 =

120591

⋃1

119864119879119909

= ae1 ae2 ae119899 ce1 ce2 ce119898

=

119899

⋃1

ae119894119898

⋃1

ce119895

(4)

The event space is formed by all the sets of event types aswell as all the atomic events and complex events For exampleportal represents all the events that occurred in portal It can

be illustrated as portal = 119890 | forall119890 eS = ldquoportalrdquo

33 Operators This section describes the operators for aggre-gating events They are used to show the relationship amongdetected events We extend them from event constructors[18 27] which consider the aggregation demands on eventcomposition We present three types of operators logicaloperator mathematical operator and temporal operatorTable 1 shows the most frequently used operators for eventdetection Atom and complex events are both denoted by 119890

These operators are used to aggregate events and detectcomplex patterns that catch the meaningful informationfrom real-time data streams For example an event patternWITHIN(1198901 or 1198902 1min) means when either 1198901 or 1198902 occurswithin time period less than 1 minute the pattern SEQ(1198901 and1198902 1198903 1198904 1198905) and (AVE(score 1198903 1198904 1198905) gt 80) is matched whenboth events 1198901 and 1198902 occur and then 1198903 1198904 and 1198905 occurcontinuously meanwhile the average score in 1198903 1198904 and 1198905 isgreater than 80

34 Event Processing Language and Rules Event ProcessingLanguage (EPL) provides a set of patterns like filtering corre-lation applying constraints and aggregating These patternsdefine the availability of events querying and filtering EPLstatement is a description of event operators that are usedto derive and aggregate information from event streams Theauthors in [27] elaborated twenty EPLs to compare their

International Journal of Distributed Sensor Networks 5

Table 1 Event processing operators

Number Type Operator Expression Description1

Logical

CON(and) 1198901 and 1198902 Conjunction of 1198901 and 1198902 without occurrence order2 DIS(or) 1198901 or 1198902 Disjunction of 1198901 and 1198902 without occurrence order3 NEG(sim) sim119890

1Negation of 119890

1

4 ANY(exist) exist(1198901 1198902) Any event that occurs of 1198901 and 11989025 EVERY(forall) forall(1198901) Every occurrence of 11989016 SEQ SEQ(1198901 1198902) Select a given sequence of events from input events7 SEL SEL(1198901) Select an event from input events8

Mathematical

AVE AVE(1198861 1198901 1198902) Average value of 1198861 in 1198901 and 11989029 SUM SUM(1198861 1198901 1198902) Summation value of 1198861 in 1198901 and 119890210 MAX MAX(1198861 1198901 1198902) Maximal value of 1198861 in 1198901 and 119890211 MIN MIN(1198861 1198901 1198902) Minimum value of 1198861 in 1198901 and 119890212 COUNT COUNT(1198901) Occurrence number of 119890113 FIRST FIRST(1198901 1198902) First event of 1198901 and 119890214 LAST LAST(119890

1 1198902) Last event of 119890

1and 119890

2

15

Temporal

WITHIN WITHIN(1198901 1199051 1199052) 119890

1occurs within time intervals 119905

1and 1199052

16 WITHIN WITHIN(1198901 119905) 1198901 occurs within less than 11990517 DURING DURING(1198901 1198902) 1198901 occurs during 119890218 WINDOW WINDOW(119890

1 119905) 119890

1occurs for time period 119905

19 WINDOW WINDOW(1198901 119899) 1198901 occurs 119899 times (119899 gt 0)20 AT AT(1198901 119905) 1198901 occurs at time 119905

operators and consumption modes According to the pro-posed event operators and consumption modes we employEsper EPL syntax [28] for specifying event processing rulessupporting by event operators in this investigation The EPLsyntax is presented in the following

select select list from stream def [as name] [ streamdef [as name]] [ ][where search conditions] [group by grouping expres-sion list] [having grouping search conditions][output output specification] [order by order byexpression list] [limit num rows]

The above EPL syntax includes some query clauses thatare introduced by Ahmad et al [29] The select clauses areused to select all properties or to specify the list of eventproperties and expressionsThe ldquofrom clauserdquo indicates eventstream name The ldquowhere clauserdquo is an optional clause usedto join and correlate event streams The ldquogroup by clauserdquoseparates the output events into groups The ldquohaving clauserdquois used in combination with the ldquogroup by clauserdquo to restrictthe groups of returned rows to only those whose condition istrue Adopting EPL syntax rules declared in Section 31 canbe represented in a standard structure as follows

Rule 1 MissingEventAgent

SELECT COUNT(eSid) WHITIN(RequireTime)IF (COUNT(eSid) = 0) THENACTION createMissingEventAgentWarning

Rule 2 UserRepeatDocExamine

SELECT COUNT(ePid) WHERE eTYPE =lsquoDocExaminersquo WHITIN(1 day)IF ANY(eCOUNT(ePid)) gt LIMIT DOCEX-AMINE THEN ACTION create UserRepeat-DocExamineWarning

Rule 3 Project ManagerExamine

SELECT COUNT(eManager) WHEREeTYPE = lsquoSubmitProjectInforsquo and eManager =e1015840Manager ande1015840ProjectState = lsquoUncompletedrsquoorlsquoSubmitedrsquoIF COUNT(eManager) gt LIMIT MANAGERTHEN ACTION create ManagerRepeatedError

We assume that the above rules provide analysis enginewith rules for what to do in event streams For this aim theengine is intelligent to execute rules by adopting a knowledgebase Knowledge base provides intelligence for the engine asit contains rich parameters for both engine and rules

For example LIMIT DOCEXAMINE is defined byLIDAS administrator in Rule 2 Its value is according tothe document of examination server runtime state Rule 2is constructed to balance the capacity of the server to eachproject and detect vicious document examination request Itspecifies that the trustworthiness of a document examinationrequest should be continuously monitored The notificationis generated as soon as the value rises above a given threshold

6 International Journal of Distributed Sensor Networks

Filter

Analysis engine

Knowledgebase

Rule base

Event model

CEP component

Analysisresult

User dashboard

Lightweight intelligent data analysis system

Integrated systems

Datastorage

User admin board

System N

Agent 3

System 2

Agent 2

System 1

Agent 1 middot middot middot

Figure 3 LIDAS modularized architecture with distributed systems

Main process thread

Agent 1

Agent 2

Agent n

Listener Preprocess

QueueQueues

Analysis thread Analysis

result

Filter Analysis engine (LIDA-E)

KnowledgebaseRule base

ae1ae2ae3

ce1ce2ce3

Figure 4 The event transmission and process workflow of LIDAS

value Rule 3 is made for analyzing how many uncompletedor submitted projects does a manager have in system If thenumber of a managerrsquos projects is beyond the limitationanalysis engine will create an error action

4 The Architecture of LIDAS

41 Architecture Design of LIDAS We have to extend thecapability of integrated information systems towards theintelligent data analysis by adopting advantage of event pro-cessing technologies keeping clearly apart each peculiarityof the CEP components to perform even deep changeswithout affecting related integrated systems In designing theintelligent data analysis architecture we started by specifyingthe CEP components that interact with the technologicalarchitecture and traditional integrated systems

As shown in Figure 3 LIDAS is a lightweight andmodularized framework designed to connect the integratedsystems for intelligent data analysisThe bottom of the frame-work is agents that are deployed in the traditional integratedinformation systems to extract data in complicated businessprocesses The upper part is an intelligent data analysis with

CEP which deals with event processing and data analyzing Itis implemented as an intelligent processing center connectedto explicit event queues in accordance with predefined eventrules System administrators define event processing rulesand knowledge through user admin board in system buildtime The users get analysis results from user dashboard inreal time

In order to show the workflow of LIDAS framework weintroduced agent filter and analysis engine to illustrate eventtransmit and processing (see Figure 4) First agents sendatomevents to the listenermodule in filter And then preproc-ess module reads events from listener and stores them inqueue it sends a message of event arriving to analysis engineAnd the engine receives messages from filter it enables theprocessing thread to access new events from queue Thegraphical representation of the workflow is illustrated inFigure 4 and introduced in the following subsections in detail

42 Event Transmission Component In order to improvethe scalability in large-scale integrated systems and reducethe network delay caused by online event streams deliveryalmost all current event processing frameworks are based on

International Journal of Distributed Sensor Networks 7

Main thread

QueueQueues

Event process

KnowledgebaseRule base

Analysisresult

Analysis engine (LIDA-E)

Filter2 3 5

4 6

7Event access

1ae1ae2ae3

ce1ce2ce3

Figure 5 The working pipeline of LIDA-E

CEP However event receiving and dispatching problems arecommonly ignored Figure 3 illustrates the simplified viewover the LIDAS architecture in distributed systemsThe com-bination of filter and agent works as a bridge that transmitsevents from information systems to LIDA-E They are basiccomponents which act as an event provider to CEP engine

We use agent which is similar to sensor [11] and adapter[30] used in monitoring system to collect information fromdistributed environment Agent is responsible for extractingreal-time business data and generating atom event It is eventsource and is deployed in an integrated system to generatemassive events with business data Using the proposed eventmodel agent reads data froman interface of integrated systemand translates them into atom events According to the eventmodel (Definition 1) agent puts data as attributes in 119860 asa list It generates atom event immediately when data isdetected in systems

Collaborating with CEP technology the filter detectscomposite events from different information systems [27]Filter is an event receiving and preprocessing module thathandles subscription related to control coming messagesfrom analysis engine and processes high volumes of eventobjects which are received from agents According to theavailability of business data filter pushes each valid atomevent into a map that is held in memory while it notifies theanalysis engine that a new event has arrived and is storedin map We summarize the functions of filter as follows (1)receive events from distributed agents (2) preprocess eventsstreams according to event availability (3) push events intomemory and notify analysis engine

Somemethods for example TCP FTP SMTP HTTP andTELNET can be used in event transfer method between filterand agent However we use a New IO (NIO) client serverNetty framework in our event transmission mechanism tofind a way to achieve ease of development performancestability and flexibility with high throughput low latency lessresource consumption andminimized unnecessary memorycopy We have developed a socket client and server by Nettyclient is deployed in agent and server which is adopted byfilter By accepting socket communication protocol filter hasthe capability to listen on a system port and receive eventsfrom different agents Event transmission mechanism is thefirst step that collects real-time data from integrated systems

the CEP based event processingmethodwill be introduced inthe next subsection

43 LIDA-E Engine The analysis engine called LIDA-E isthe core component of LIDAS architecture it is responsiblefor processing the events queue and judging whether there isa complex event inferred [25] It provides effective queuingscheduling time and count-window support and fast in-memory processing of high-speed continuous unboundeddata streams In order to divide functions of engine we designthree modules in LIDA-E they are main thread event accessand event process Main thread module is used to monitorfilter and control event access module Event access moduleis dedicated for accessing events from the queue which iswritten by filter Event process module executes rules andprocesses all the events It is a linear task controller thatdeploys rules one by one and manages events to be processedthrough them

CEP component comprises a set of rules and knowledgeto detect a predefined group of anomalies on the basis of thereceiving event attribute which are the same in [31ndash33] Therule base specifies how to infer complex events from the eventstream Howver knowledge base provides the link betweenknown information (the antecedent) and the information tobe deduced (the consequent) or actions to be executed It isa useful supplement to rule base for processing events Bothrules and knowledge are defined in build time and loadedwhen LIDAS starts to work

LIDA-E workflow is shown in Figure 5 it is a runtimeworkflow in which the analysis engine waiting for eventsarriving has already executed rules with knowledge in eventprocessmoduleTheworkflow follows the substeps as follows(1) filter pushes events into queue as soon as it receives eventsfrom different agents (2) filter sends a message to analysisengine to notify it that a new event has come (3) main threadmodule receives themessage andnotifies event accessmodulethat new events have been pushed in queue (4) event accessmodule reads events fromqueue and prepares the submissionfor event process module (5) event access module sendsevent to event processmodule immediately (6) event processmodule accesses queues for event processing in event processmodule each event is orderly processed by rules for eachrule at least one queue is provided for runtime event storage

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 3: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

International Journal of Distributed Sensor Networks 3

(iv) We discuss the design and implementation of LIDASarchitecture which is an integration of LIDA-Eengine and event transmission mechanism

The rest of the paper is organized as follows In Section 2we discuss related works of CEP technology Section 3describes the relevant concepts of LIDAS such as eventmodel operators and SQL-like Event Processing LanguageThe proposed architecture is presented in Section 4 Section 5shows the experimental results Finally in the last section wesummarize our research work

2 Related Work

CEP is considered as extracting complex situations and react-ing on them frommassive events [12] Meanwhile CEP is nota new terminology as Luckham first introduced [13] It hasbeen widely exploited during the past years mostly in big dataanalytics [5] RFID [14] Internet of Things (IoT) [15] failureprediction systems [16] real-time grid monitoring [11] andhealthcare [17] According to CEP different algorithms havebeen proposed to increase Complex Event Processing capa-bilityThey adoptCEP engine (like Storm [7] Drools [18] andEsper [10 19]) in their applications while others design newevent processing engines and system architectures [11 12 1516 20] with evaluation to show the usefulness and scalability

Storm is a free and open source distributed real-timecomputation system which makes it easy to reliably processunbounded streams of data in real-time processing by batchoperations [7] It includes spouts and bolts which could beexecuted with many tasks in parallel on multiple machines(worker nodes) in a cluster [8] Storm is designed to processunbounded streams of data in a storm cluster (master nodeand worker nodes) However as discussed by Xu et al [8]Storm weakly focuses on the performance and job assign-ment with different workload Moreover the deployment ofStorm requires more nodes

Differing from Storm Drools [18] and Esper [10 19]are CEP engines which include a module providing nativesupport for events evaluation and temporal logic analysis ina single work node Yao et al [18] presented a Drools basedCEP framework which was used to process surgical eventsand provided sense and response capability for hospitals Inthe framework the author mentions that the knowledge withCEP is quite complicated most of which comes from expertsand is fuzzy and hard to verify the accuracy To combineknowledge with CEP system Bruns et al [10] describe anovel event-driven architecture for a decision support systemwhich leads to a new quality of M2M systems which areintelligent and flexible The system adopts Esper as the coreevent processing component and employs a rules base toexplicitly represent the M2M knowledge of domain expertsTherefore CEP could be used as an appropriate candidatewith rule base for intelligent data analysis

However with the development of web technology theopen source engines become less efficient to deal with hugevolume of data In order to get higher performance somespecial CEP engines have been developed for particular

domains (LiSEP [12] DPCEP [15] and RTE-CEP [20])Zappia et al [12] described the design and implementationof a lightweight and extensible Complex Event Processingengine called LiSEP for sensing and responding applicationsThe author adopted the Staged Event-Driven Architecture(SEDA) principles to clearly separate core event processinglogic from lower level resourcemanagement issues Howeverevent model and operators are not designed and used forLISEP as well as event collection The event processingmethod of LISEP is not clearly described in their research

Combining CEP with distributed systems GEMINI2 [11]is proposed as a custom framework dedicated for CEP basedreal-time monitoring of grid infrastructures In GEMINI2100 sensors loaded 400 events in one second and theperformance of the CEP engine was not evaluated Theperformance of working nodes and servers could not supportmassive events transmission

In summary most CEP system architectures used inexisting distributed environment are not oriented towardsreal-time processing for integrated information systems Theprevailing CEP engines are not to explore the performanceof event processing but rather expose the event processingmethods while the engines are suitable for processing eventstreams and are not well suited for analyzing massive data inintegrated systems The work described here is a result of theprevious investigations in CEP and intelligent data analysis indistributed network

3 CEP for Intelligent Data Analysis

31 Application Scenario A Project Management Informa-tion System (PMIS) is one important type of Computer-Based Information Systems (CBIS) and is used to collectprocess analyze and publish the data for a particular purpose[21] Project management (PM) is a knowledge-centric andexperience-driven activity supported by an appropriate PMIS[22] PMIS is an integrated information system that helpsproject manager to carry out projects systematically andmonitor progress of projects closely

A project life cycle is divided into many states suchas proposal review implementation and acceptance statesStarting from project proposal to implementation Taghavi etal [23] proposed a web based projectmanagement functionalmodel that separates project work progress managementinto time fund quality contract bidding demand integritycomprehensiveness handover and accomplishment Eachproject state is supported by different subsystems which arecombined together as a PMIS According to project states wepropose a project state model which shows the state changesin a project life cycle In Figure 2 ldquominusrdquo is a start state and ldquo+rdquois an end state others are middle states (proposal reviewimplementation and acceptance) they represent the wholelife cycle of a project In different project states systemsverify (119864Verify) the reasonability and validity of massiveproject data If the project data passes the verification astate transition event (119864declare 119864review 119864implement 119864accept and119864complete) changes the state from one to another Otherwise afailed event happens and changes the state to start

4 International Journal of Distributed Sensor Networks

1 2 3 4minus +

Efailed_1

Efailed_2

Efailed_3

Efailed_4

EVerify_1

EVerify_2

EVerify_3

EVerify_4

Edeclaring

Ereview

Eimplement

Eaccept

Ecomplete

Figure 2 Project state model

According to the events attributes in PMIS constraintsare necessary for analyzing data availability and validity whenevents occurred by user activities Some rules for data analysisare elaborated in the following Rule 1 monitor the activestate of a subsystem Rule 2 examine the document exami-nation time of a project in one day Rule 3 examine whetherthe number of projects of a manager is above a defined value

32 Event Model Luckham and Schulte [24] defined anevent as there is a large amount of events occurring insideor outside the system during multiple phases such as makinga business transaction receiving an email creating a salesreport or uploading a file Events are extracted from webservices user activities system log database and so onThey are separated into atom and complex atom events areidentified through data collection and complex events aredetected by the analysis of atom event stream [25 26]

Definition 1 (atom event) An atom eventmeans that a specialactivity or data transition occurs in a point in time which isrepresented by a six-tuple The elements of the six-tuple areas follows

Atom Event ae = ⟨AEidPid 119860 119878 119879 119905⟩ (1)

AEid is the identification of an atom event Pid presentsthe identification of a project And 119860 is the set of projectattributes and data 119878 is a source from where the event isgenerated 119879 is event type and 119905 is timestamp which identifieswhen the atom event occurs In the project managementsystem example each user activity or operation in systemgenerates an atom event Atom events usually represent thestate of a business attribute or some changes

Definition 2 (complex event) It is a combination of atomevents or complex events consisting of the predefined rulessuch as constraints logical relationship and time sequencecombination It is also denoted by a six-tuple

Complex Event ce = ⟨CEid 119860 119862 119879 119905119887 119905119890⟩ (2)

CEid is the identification of a complex event119862 is a combi-nation of the atom events and complex events that trigger thisevent to happen where 119862 = ae1 ae2 ae119899 ce1 ce2 ce119898 119899+119898 gt 0119860 is a set of attributes119860 = ae1119860 ae2119860 ae119899119860 ce1119860 ce2119860 ce119898119860 119905119887 is the starting time and 119905119890is ending time of complex event where 119905119890 gt= 119905119887 A complexevent is a combination of atom and complex events by patternmatching rules that is a specific set of event operators Wewill introduce the event operators and rules in the followingsubsection

Definition 3 (event type) The set of all similar events is calledthe event type 119879 It shows the object which triggers the eventHence

119864119879= 119890119879

1 119890119879

2 119890

119879

119894 119890

119879

119899 =

119899

⋃1

119890119879

119894 (3)

Events in an event type have the same attributes of 119879 Forinstance the type may refer to events regarding informationverification however the other event properties may bedifferent (eg source or attribute) An example is the typeof all events that denote the submitted information 119864submit

shows that this event type indicates submitted informationand is constituted by 119890submit

1 119890submit2 119890submit

119899

Definition 4 (event space) The set of all possible eventsknown from a certain information system is called the eventspace Hence

= 1198641198791 1198641198792 119864

119879119909 119864

119879120591 =

120591

⋃1

119864119879119909

= ae1 ae2 ae119899 ce1 ce2 ce119898

=

119899

⋃1

ae119894119898

⋃1

ce119895

(4)

The event space is formed by all the sets of event types aswell as all the atomic events and complex events For exampleportal represents all the events that occurred in portal It can

be illustrated as portal = 119890 | forall119890 eS = ldquoportalrdquo

33 Operators This section describes the operators for aggre-gating events They are used to show the relationship amongdetected events We extend them from event constructors[18 27] which consider the aggregation demands on eventcomposition We present three types of operators logicaloperator mathematical operator and temporal operatorTable 1 shows the most frequently used operators for eventdetection Atom and complex events are both denoted by 119890

These operators are used to aggregate events and detectcomplex patterns that catch the meaningful informationfrom real-time data streams For example an event patternWITHIN(1198901 or 1198902 1min) means when either 1198901 or 1198902 occurswithin time period less than 1 minute the pattern SEQ(1198901 and1198902 1198903 1198904 1198905) and (AVE(score 1198903 1198904 1198905) gt 80) is matched whenboth events 1198901 and 1198902 occur and then 1198903 1198904 and 1198905 occurcontinuously meanwhile the average score in 1198903 1198904 and 1198905 isgreater than 80

34 Event Processing Language and Rules Event ProcessingLanguage (EPL) provides a set of patterns like filtering corre-lation applying constraints and aggregating These patternsdefine the availability of events querying and filtering EPLstatement is a description of event operators that are usedto derive and aggregate information from event streams Theauthors in [27] elaborated twenty EPLs to compare their

International Journal of Distributed Sensor Networks 5

Table 1 Event processing operators

Number Type Operator Expression Description1

Logical

CON(and) 1198901 and 1198902 Conjunction of 1198901 and 1198902 without occurrence order2 DIS(or) 1198901 or 1198902 Disjunction of 1198901 and 1198902 without occurrence order3 NEG(sim) sim119890

1Negation of 119890

1

4 ANY(exist) exist(1198901 1198902) Any event that occurs of 1198901 and 11989025 EVERY(forall) forall(1198901) Every occurrence of 11989016 SEQ SEQ(1198901 1198902) Select a given sequence of events from input events7 SEL SEL(1198901) Select an event from input events8

Mathematical

AVE AVE(1198861 1198901 1198902) Average value of 1198861 in 1198901 and 11989029 SUM SUM(1198861 1198901 1198902) Summation value of 1198861 in 1198901 and 119890210 MAX MAX(1198861 1198901 1198902) Maximal value of 1198861 in 1198901 and 119890211 MIN MIN(1198861 1198901 1198902) Minimum value of 1198861 in 1198901 and 119890212 COUNT COUNT(1198901) Occurrence number of 119890113 FIRST FIRST(1198901 1198902) First event of 1198901 and 119890214 LAST LAST(119890

1 1198902) Last event of 119890

1and 119890

2

15

Temporal

WITHIN WITHIN(1198901 1199051 1199052) 119890

1occurs within time intervals 119905

1and 1199052

16 WITHIN WITHIN(1198901 119905) 1198901 occurs within less than 11990517 DURING DURING(1198901 1198902) 1198901 occurs during 119890218 WINDOW WINDOW(119890

1 119905) 119890

1occurs for time period 119905

19 WINDOW WINDOW(1198901 119899) 1198901 occurs 119899 times (119899 gt 0)20 AT AT(1198901 119905) 1198901 occurs at time 119905

operators and consumption modes According to the pro-posed event operators and consumption modes we employEsper EPL syntax [28] for specifying event processing rulessupporting by event operators in this investigation The EPLsyntax is presented in the following

select select list from stream def [as name] [ streamdef [as name]] [ ][where search conditions] [group by grouping expres-sion list] [having grouping search conditions][output output specification] [order by order byexpression list] [limit num rows]

The above EPL syntax includes some query clauses thatare introduced by Ahmad et al [29] The select clauses areused to select all properties or to specify the list of eventproperties and expressionsThe ldquofrom clauserdquo indicates eventstream name The ldquowhere clauserdquo is an optional clause usedto join and correlate event streams The ldquogroup by clauserdquoseparates the output events into groups The ldquohaving clauserdquois used in combination with the ldquogroup by clauserdquo to restrictthe groups of returned rows to only those whose condition istrue Adopting EPL syntax rules declared in Section 31 canbe represented in a standard structure as follows

Rule 1 MissingEventAgent

SELECT COUNT(eSid) WHITIN(RequireTime)IF (COUNT(eSid) = 0) THENACTION createMissingEventAgentWarning

Rule 2 UserRepeatDocExamine

SELECT COUNT(ePid) WHERE eTYPE =lsquoDocExaminersquo WHITIN(1 day)IF ANY(eCOUNT(ePid)) gt LIMIT DOCEX-AMINE THEN ACTION create UserRepeat-DocExamineWarning

Rule 3 Project ManagerExamine

SELECT COUNT(eManager) WHEREeTYPE = lsquoSubmitProjectInforsquo and eManager =e1015840Manager ande1015840ProjectState = lsquoUncompletedrsquoorlsquoSubmitedrsquoIF COUNT(eManager) gt LIMIT MANAGERTHEN ACTION create ManagerRepeatedError

We assume that the above rules provide analysis enginewith rules for what to do in event streams For this aim theengine is intelligent to execute rules by adopting a knowledgebase Knowledge base provides intelligence for the engine asit contains rich parameters for both engine and rules

For example LIMIT DOCEXAMINE is defined byLIDAS administrator in Rule 2 Its value is according tothe document of examination server runtime state Rule 2is constructed to balance the capacity of the server to eachproject and detect vicious document examination request Itspecifies that the trustworthiness of a document examinationrequest should be continuously monitored The notificationis generated as soon as the value rises above a given threshold

6 International Journal of Distributed Sensor Networks

Filter

Analysis engine

Knowledgebase

Rule base

Event model

CEP component

Analysisresult

User dashboard

Lightweight intelligent data analysis system

Integrated systems

Datastorage

User admin board

System N

Agent 3

System 2

Agent 2

System 1

Agent 1 middot middot middot

Figure 3 LIDAS modularized architecture with distributed systems

Main process thread

Agent 1

Agent 2

Agent n

Listener Preprocess

QueueQueues

Analysis thread Analysis

result

Filter Analysis engine (LIDA-E)

KnowledgebaseRule base

ae1ae2ae3

ce1ce2ce3

Figure 4 The event transmission and process workflow of LIDAS

value Rule 3 is made for analyzing how many uncompletedor submitted projects does a manager have in system If thenumber of a managerrsquos projects is beyond the limitationanalysis engine will create an error action

4 The Architecture of LIDAS

41 Architecture Design of LIDAS We have to extend thecapability of integrated information systems towards theintelligent data analysis by adopting advantage of event pro-cessing technologies keeping clearly apart each peculiarityof the CEP components to perform even deep changeswithout affecting related integrated systems In designing theintelligent data analysis architecture we started by specifyingthe CEP components that interact with the technologicalarchitecture and traditional integrated systems

As shown in Figure 3 LIDAS is a lightweight andmodularized framework designed to connect the integratedsystems for intelligent data analysisThe bottom of the frame-work is agents that are deployed in the traditional integratedinformation systems to extract data in complicated businessprocesses The upper part is an intelligent data analysis with

CEP which deals with event processing and data analyzing Itis implemented as an intelligent processing center connectedto explicit event queues in accordance with predefined eventrules System administrators define event processing rulesand knowledge through user admin board in system buildtime The users get analysis results from user dashboard inreal time

In order to show the workflow of LIDAS framework weintroduced agent filter and analysis engine to illustrate eventtransmit and processing (see Figure 4) First agents sendatomevents to the listenermodule in filter And then preproc-ess module reads events from listener and stores them inqueue it sends a message of event arriving to analysis engineAnd the engine receives messages from filter it enables theprocessing thread to access new events from queue Thegraphical representation of the workflow is illustrated inFigure 4 and introduced in the following subsections in detail

42 Event Transmission Component In order to improvethe scalability in large-scale integrated systems and reducethe network delay caused by online event streams deliveryalmost all current event processing frameworks are based on

International Journal of Distributed Sensor Networks 7

Main thread

QueueQueues

Event process

KnowledgebaseRule base

Analysisresult

Analysis engine (LIDA-E)

Filter2 3 5

4 6

7Event access

1ae1ae2ae3

ce1ce2ce3

Figure 5 The working pipeline of LIDA-E

CEP However event receiving and dispatching problems arecommonly ignored Figure 3 illustrates the simplified viewover the LIDAS architecture in distributed systemsThe com-bination of filter and agent works as a bridge that transmitsevents from information systems to LIDA-E They are basiccomponents which act as an event provider to CEP engine

We use agent which is similar to sensor [11] and adapter[30] used in monitoring system to collect information fromdistributed environment Agent is responsible for extractingreal-time business data and generating atom event It is eventsource and is deployed in an integrated system to generatemassive events with business data Using the proposed eventmodel agent reads data froman interface of integrated systemand translates them into atom events According to the eventmodel (Definition 1) agent puts data as attributes in 119860 asa list It generates atom event immediately when data isdetected in systems

Collaborating with CEP technology the filter detectscomposite events from different information systems [27]Filter is an event receiving and preprocessing module thathandles subscription related to control coming messagesfrom analysis engine and processes high volumes of eventobjects which are received from agents According to theavailability of business data filter pushes each valid atomevent into a map that is held in memory while it notifies theanalysis engine that a new event has arrived and is storedin map We summarize the functions of filter as follows (1)receive events from distributed agents (2) preprocess eventsstreams according to event availability (3) push events intomemory and notify analysis engine

Somemethods for example TCP FTP SMTP HTTP andTELNET can be used in event transfer method between filterand agent However we use a New IO (NIO) client serverNetty framework in our event transmission mechanism tofind a way to achieve ease of development performancestability and flexibility with high throughput low latency lessresource consumption andminimized unnecessary memorycopy We have developed a socket client and server by Nettyclient is deployed in agent and server which is adopted byfilter By accepting socket communication protocol filter hasthe capability to listen on a system port and receive eventsfrom different agents Event transmission mechanism is thefirst step that collects real-time data from integrated systems

the CEP based event processingmethodwill be introduced inthe next subsection

43 LIDA-E Engine The analysis engine called LIDA-E isthe core component of LIDAS architecture it is responsiblefor processing the events queue and judging whether there isa complex event inferred [25] It provides effective queuingscheduling time and count-window support and fast in-memory processing of high-speed continuous unboundeddata streams In order to divide functions of engine we designthree modules in LIDA-E they are main thread event accessand event process Main thread module is used to monitorfilter and control event access module Event access moduleis dedicated for accessing events from the queue which iswritten by filter Event process module executes rules andprocesses all the events It is a linear task controller thatdeploys rules one by one and manages events to be processedthrough them

CEP component comprises a set of rules and knowledgeto detect a predefined group of anomalies on the basis of thereceiving event attribute which are the same in [31ndash33] Therule base specifies how to infer complex events from the eventstream Howver knowledge base provides the link betweenknown information (the antecedent) and the information tobe deduced (the consequent) or actions to be executed It isa useful supplement to rule base for processing events Bothrules and knowledge are defined in build time and loadedwhen LIDAS starts to work

LIDA-E workflow is shown in Figure 5 it is a runtimeworkflow in which the analysis engine waiting for eventsarriving has already executed rules with knowledge in eventprocessmoduleTheworkflow follows the substeps as follows(1) filter pushes events into queue as soon as it receives eventsfrom different agents (2) filter sends a message to analysisengine to notify it that a new event has come (3) main threadmodule receives themessage andnotifies event accessmodulethat new events have been pushed in queue (4) event accessmodule reads events fromqueue and prepares the submissionfor event process module (5) event access module sendsevent to event processmodule immediately (6) event processmodule accesses queues for event processing in event processmodule each event is orderly processed by rules for eachrule at least one queue is provided for runtime event storage

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 4: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

4 International Journal of Distributed Sensor Networks

1 2 3 4minus +

Efailed_1

Efailed_2

Efailed_3

Efailed_4

EVerify_1

EVerify_2

EVerify_3

EVerify_4

Edeclaring

Ereview

Eimplement

Eaccept

Ecomplete

Figure 2 Project state model

According to the events attributes in PMIS constraintsare necessary for analyzing data availability and validity whenevents occurred by user activities Some rules for data analysisare elaborated in the following Rule 1 monitor the activestate of a subsystem Rule 2 examine the document exami-nation time of a project in one day Rule 3 examine whetherthe number of projects of a manager is above a defined value

32 Event Model Luckham and Schulte [24] defined anevent as there is a large amount of events occurring insideor outside the system during multiple phases such as makinga business transaction receiving an email creating a salesreport or uploading a file Events are extracted from webservices user activities system log database and so onThey are separated into atom and complex atom events areidentified through data collection and complex events aredetected by the analysis of atom event stream [25 26]

Definition 1 (atom event) An atom eventmeans that a specialactivity or data transition occurs in a point in time which isrepresented by a six-tuple The elements of the six-tuple areas follows

Atom Event ae = ⟨AEidPid 119860 119878 119879 119905⟩ (1)

AEid is the identification of an atom event Pid presentsthe identification of a project And 119860 is the set of projectattributes and data 119878 is a source from where the event isgenerated 119879 is event type and 119905 is timestamp which identifieswhen the atom event occurs In the project managementsystem example each user activity or operation in systemgenerates an atom event Atom events usually represent thestate of a business attribute or some changes

Definition 2 (complex event) It is a combination of atomevents or complex events consisting of the predefined rulessuch as constraints logical relationship and time sequencecombination It is also denoted by a six-tuple

Complex Event ce = ⟨CEid 119860 119862 119879 119905119887 119905119890⟩ (2)

CEid is the identification of a complex event119862 is a combi-nation of the atom events and complex events that trigger thisevent to happen where 119862 = ae1 ae2 ae119899 ce1 ce2 ce119898 119899+119898 gt 0119860 is a set of attributes119860 = ae1119860 ae2119860 ae119899119860 ce1119860 ce2119860 ce119898119860 119905119887 is the starting time and 119905119890is ending time of complex event where 119905119890 gt= 119905119887 A complexevent is a combination of atom and complex events by patternmatching rules that is a specific set of event operators Wewill introduce the event operators and rules in the followingsubsection

Definition 3 (event type) The set of all similar events is calledthe event type 119879 It shows the object which triggers the eventHence

119864119879= 119890119879

1 119890119879

2 119890

119879

119894 119890

119879

119899 =

119899

⋃1

119890119879

119894 (3)

Events in an event type have the same attributes of 119879 Forinstance the type may refer to events regarding informationverification however the other event properties may bedifferent (eg source or attribute) An example is the typeof all events that denote the submitted information 119864submit

shows that this event type indicates submitted informationand is constituted by 119890submit

1 119890submit2 119890submit

119899

Definition 4 (event space) The set of all possible eventsknown from a certain information system is called the eventspace Hence

= 1198641198791 1198641198792 119864

119879119909 119864

119879120591 =

120591

⋃1

119864119879119909

= ae1 ae2 ae119899 ce1 ce2 ce119898

=

119899

⋃1

ae119894119898

⋃1

ce119895

(4)

The event space is formed by all the sets of event types aswell as all the atomic events and complex events For exampleportal represents all the events that occurred in portal It can

be illustrated as portal = 119890 | forall119890 eS = ldquoportalrdquo

33 Operators This section describes the operators for aggre-gating events They are used to show the relationship amongdetected events We extend them from event constructors[18 27] which consider the aggregation demands on eventcomposition We present three types of operators logicaloperator mathematical operator and temporal operatorTable 1 shows the most frequently used operators for eventdetection Atom and complex events are both denoted by 119890

These operators are used to aggregate events and detectcomplex patterns that catch the meaningful informationfrom real-time data streams For example an event patternWITHIN(1198901 or 1198902 1min) means when either 1198901 or 1198902 occurswithin time period less than 1 minute the pattern SEQ(1198901 and1198902 1198903 1198904 1198905) and (AVE(score 1198903 1198904 1198905) gt 80) is matched whenboth events 1198901 and 1198902 occur and then 1198903 1198904 and 1198905 occurcontinuously meanwhile the average score in 1198903 1198904 and 1198905 isgreater than 80

34 Event Processing Language and Rules Event ProcessingLanguage (EPL) provides a set of patterns like filtering corre-lation applying constraints and aggregating These patternsdefine the availability of events querying and filtering EPLstatement is a description of event operators that are usedto derive and aggregate information from event streams Theauthors in [27] elaborated twenty EPLs to compare their

International Journal of Distributed Sensor Networks 5

Table 1 Event processing operators

Number Type Operator Expression Description1

Logical

CON(and) 1198901 and 1198902 Conjunction of 1198901 and 1198902 without occurrence order2 DIS(or) 1198901 or 1198902 Disjunction of 1198901 and 1198902 without occurrence order3 NEG(sim) sim119890

1Negation of 119890

1

4 ANY(exist) exist(1198901 1198902) Any event that occurs of 1198901 and 11989025 EVERY(forall) forall(1198901) Every occurrence of 11989016 SEQ SEQ(1198901 1198902) Select a given sequence of events from input events7 SEL SEL(1198901) Select an event from input events8

Mathematical

AVE AVE(1198861 1198901 1198902) Average value of 1198861 in 1198901 and 11989029 SUM SUM(1198861 1198901 1198902) Summation value of 1198861 in 1198901 and 119890210 MAX MAX(1198861 1198901 1198902) Maximal value of 1198861 in 1198901 and 119890211 MIN MIN(1198861 1198901 1198902) Minimum value of 1198861 in 1198901 and 119890212 COUNT COUNT(1198901) Occurrence number of 119890113 FIRST FIRST(1198901 1198902) First event of 1198901 and 119890214 LAST LAST(119890

1 1198902) Last event of 119890

1and 119890

2

15

Temporal

WITHIN WITHIN(1198901 1199051 1199052) 119890

1occurs within time intervals 119905

1and 1199052

16 WITHIN WITHIN(1198901 119905) 1198901 occurs within less than 11990517 DURING DURING(1198901 1198902) 1198901 occurs during 119890218 WINDOW WINDOW(119890

1 119905) 119890

1occurs for time period 119905

19 WINDOW WINDOW(1198901 119899) 1198901 occurs 119899 times (119899 gt 0)20 AT AT(1198901 119905) 1198901 occurs at time 119905

operators and consumption modes According to the pro-posed event operators and consumption modes we employEsper EPL syntax [28] for specifying event processing rulessupporting by event operators in this investigation The EPLsyntax is presented in the following

select select list from stream def [as name] [ streamdef [as name]] [ ][where search conditions] [group by grouping expres-sion list] [having grouping search conditions][output output specification] [order by order byexpression list] [limit num rows]

The above EPL syntax includes some query clauses thatare introduced by Ahmad et al [29] The select clauses areused to select all properties or to specify the list of eventproperties and expressionsThe ldquofrom clauserdquo indicates eventstream name The ldquowhere clauserdquo is an optional clause usedto join and correlate event streams The ldquogroup by clauserdquoseparates the output events into groups The ldquohaving clauserdquois used in combination with the ldquogroup by clauserdquo to restrictthe groups of returned rows to only those whose condition istrue Adopting EPL syntax rules declared in Section 31 canbe represented in a standard structure as follows

Rule 1 MissingEventAgent

SELECT COUNT(eSid) WHITIN(RequireTime)IF (COUNT(eSid) = 0) THENACTION createMissingEventAgentWarning

Rule 2 UserRepeatDocExamine

SELECT COUNT(ePid) WHERE eTYPE =lsquoDocExaminersquo WHITIN(1 day)IF ANY(eCOUNT(ePid)) gt LIMIT DOCEX-AMINE THEN ACTION create UserRepeat-DocExamineWarning

Rule 3 Project ManagerExamine

SELECT COUNT(eManager) WHEREeTYPE = lsquoSubmitProjectInforsquo and eManager =e1015840Manager ande1015840ProjectState = lsquoUncompletedrsquoorlsquoSubmitedrsquoIF COUNT(eManager) gt LIMIT MANAGERTHEN ACTION create ManagerRepeatedError

We assume that the above rules provide analysis enginewith rules for what to do in event streams For this aim theengine is intelligent to execute rules by adopting a knowledgebase Knowledge base provides intelligence for the engine asit contains rich parameters for both engine and rules

For example LIMIT DOCEXAMINE is defined byLIDAS administrator in Rule 2 Its value is according tothe document of examination server runtime state Rule 2is constructed to balance the capacity of the server to eachproject and detect vicious document examination request Itspecifies that the trustworthiness of a document examinationrequest should be continuously monitored The notificationis generated as soon as the value rises above a given threshold

6 International Journal of Distributed Sensor Networks

Filter

Analysis engine

Knowledgebase

Rule base

Event model

CEP component

Analysisresult

User dashboard

Lightweight intelligent data analysis system

Integrated systems

Datastorage

User admin board

System N

Agent 3

System 2

Agent 2

System 1

Agent 1 middot middot middot

Figure 3 LIDAS modularized architecture with distributed systems

Main process thread

Agent 1

Agent 2

Agent n

Listener Preprocess

QueueQueues

Analysis thread Analysis

result

Filter Analysis engine (LIDA-E)

KnowledgebaseRule base

ae1ae2ae3

ce1ce2ce3

Figure 4 The event transmission and process workflow of LIDAS

value Rule 3 is made for analyzing how many uncompletedor submitted projects does a manager have in system If thenumber of a managerrsquos projects is beyond the limitationanalysis engine will create an error action

4 The Architecture of LIDAS

41 Architecture Design of LIDAS We have to extend thecapability of integrated information systems towards theintelligent data analysis by adopting advantage of event pro-cessing technologies keeping clearly apart each peculiarityof the CEP components to perform even deep changeswithout affecting related integrated systems In designing theintelligent data analysis architecture we started by specifyingthe CEP components that interact with the technologicalarchitecture and traditional integrated systems

As shown in Figure 3 LIDAS is a lightweight andmodularized framework designed to connect the integratedsystems for intelligent data analysisThe bottom of the frame-work is agents that are deployed in the traditional integratedinformation systems to extract data in complicated businessprocesses The upper part is an intelligent data analysis with

CEP which deals with event processing and data analyzing Itis implemented as an intelligent processing center connectedto explicit event queues in accordance with predefined eventrules System administrators define event processing rulesand knowledge through user admin board in system buildtime The users get analysis results from user dashboard inreal time

In order to show the workflow of LIDAS framework weintroduced agent filter and analysis engine to illustrate eventtransmit and processing (see Figure 4) First agents sendatomevents to the listenermodule in filter And then preproc-ess module reads events from listener and stores them inqueue it sends a message of event arriving to analysis engineAnd the engine receives messages from filter it enables theprocessing thread to access new events from queue Thegraphical representation of the workflow is illustrated inFigure 4 and introduced in the following subsections in detail

42 Event Transmission Component In order to improvethe scalability in large-scale integrated systems and reducethe network delay caused by online event streams deliveryalmost all current event processing frameworks are based on

International Journal of Distributed Sensor Networks 7

Main thread

QueueQueues

Event process

KnowledgebaseRule base

Analysisresult

Analysis engine (LIDA-E)

Filter2 3 5

4 6

7Event access

1ae1ae2ae3

ce1ce2ce3

Figure 5 The working pipeline of LIDA-E

CEP However event receiving and dispatching problems arecommonly ignored Figure 3 illustrates the simplified viewover the LIDAS architecture in distributed systemsThe com-bination of filter and agent works as a bridge that transmitsevents from information systems to LIDA-E They are basiccomponents which act as an event provider to CEP engine

We use agent which is similar to sensor [11] and adapter[30] used in monitoring system to collect information fromdistributed environment Agent is responsible for extractingreal-time business data and generating atom event It is eventsource and is deployed in an integrated system to generatemassive events with business data Using the proposed eventmodel agent reads data froman interface of integrated systemand translates them into atom events According to the eventmodel (Definition 1) agent puts data as attributes in 119860 asa list It generates atom event immediately when data isdetected in systems

Collaborating with CEP technology the filter detectscomposite events from different information systems [27]Filter is an event receiving and preprocessing module thathandles subscription related to control coming messagesfrom analysis engine and processes high volumes of eventobjects which are received from agents According to theavailability of business data filter pushes each valid atomevent into a map that is held in memory while it notifies theanalysis engine that a new event has arrived and is storedin map We summarize the functions of filter as follows (1)receive events from distributed agents (2) preprocess eventsstreams according to event availability (3) push events intomemory and notify analysis engine

Somemethods for example TCP FTP SMTP HTTP andTELNET can be used in event transfer method between filterand agent However we use a New IO (NIO) client serverNetty framework in our event transmission mechanism tofind a way to achieve ease of development performancestability and flexibility with high throughput low latency lessresource consumption andminimized unnecessary memorycopy We have developed a socket client and server by Nettyclient is deployed in agent and server which is adopted byfilter By accepting socket communication protocol filter hasthe capability to listen on a system port and receive eventsfrom different agents Event transmission mechanism is thefirst step that collects real-time data from integrated systems

the CEP based event processingmethodwill be introduced inthe next subsection

43 LIDA-E Engine The analysis engine called LIDA-E isthe core component of LIDAS architecture it is responsiblefor processing the events queue and judging whether there isa complex event inferred [25] It provides effective queuingscheduling time and count-window support and fast in-memory processing of high-speed continuous unboundeddata streams In order to divide functions of engine we designthree modules in LIDA-E they are main thread event accessand event process Main thread module is used to monitorfilter and control event access module Event access moduleis dedicated for accessing events from the queue which iswritten by filter Event process module executes rules andprocesses all the events It is a linear task controller thatdeploys rules one by one and manages events to be processedthrough them

CEP component comprises a set of rules and knowledgeto detect a predefined group of anomalies on the basis of thereceiving event attribute which are the same in [31ndash33] Therule base specifies how to infer complex events from the eventstream Howver knowledge base provides the link betweenknown information (the antecedent) and the information tobe deduced (the consequent) or actions to be executed It isa useful supplement to rule base for processing events Bothrules and knowledge are defined in build time and loadedwhen LIDAS starts to work

LIDA-E workflow is shown in Figure 5 it is a runtimeworkflow in which the analysis engine waiting for eventsarriving has already executed rules with knowledge in eventprocessmoduleTheworkflow follows the substeps as follows(1) filter pushes events into queue as soon as it receives eventsfrom different agents (2) filter sends a message to analysisengine to notify it that a new event has come (3) main threadmodule receives themessage andnotifies event accessmodulethat new events have been pushed in queue (4) event accessmodule reads events fromqueue and prepares the submissionfor event process module (5) event access module sendsevent to event processmodule immediately (6) event processmodule accesses queues for event processing in event processmodule each event is orderly processed by rules for eachrule at least one queue is provided for runtime event storage

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 5: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

International Journal of Distributed Sensor Networks 5

Table 1 Event processing operators

Number Type Operator Expression Description1

Logical

CON(and) 1198901 and 1198902 Conjunction of 1198901 and 1198902 without occurrence order2 DIS(or) 1198901 or 1198902 Disjunction of 1198901 and 1198902 without occurrence order3 NEG(sim) sim119890

1Negation of 119890

1

4 ANY(exist) exist(1198901 1198902) Any event that occurs of 1198901 and 11989025 EVERY(forall) forall(1198901) Every occurrence of 11989016 SEQ SEQ(1198901 1198902) Select a given sequence of events from input events7 SEL SEL(1198901) Select an event from input events8

Mathematical

AVE AVE(1198861 1198901 1198902) Average value of 1198861 in 1198901 and 11989029 SUM SUM(1198861 1198901 1198902) Summation value of 1198861 in 1198901 and 119890210 MAX MAX(1198861 1198901 1198902) Maximal value of 1198861 in 1198901 and 119890211 MIN MIN(1198861 1198901 1198902) Minimum value of 1198861 in 1198901 and 119890212 COUNT COUNT(1198901) Occurrence number of 119890113 FIRST FIRST(1198901 1198902) First event of 1198901 and 119890214 LAST LAST(119890

1 1198902) Last event of 119890

1and 119890

2

15

Temporal

WITHIN WITHIN(1198901 1199051 1199052) 119890

1occurs within time intervals 119905

1and 1199052

16 WITHIN WITHIN(1198901 119905) 1198901 occurs within less than 11990517 DURING DURING(1198901 1198902) 1198901 occurs during 119890218 WINDOW WINDOW(119890

1 119905) 119890

1occurs for time period 119905

19 WINDOW WINDOW(1198901 119899) 1198901 occurs 119899 times (119899 gt 0)20 AT AT(1198901 119905) 1198901 occurs at time 119905

operators and consumption modes According to the pro-posed event operators and consumption modes we employEsper EPL syntax [28] for specifying event processing rulessupporting by event operators in this investigation The EPLsyntax is presented in the following

select select list from stream def [as name] [ streamdef [as name]] [ ][where search conditions] [group by grouping expres-sion list] [having grouping search conditions][output output specification] [order by order byexpression list] [limit num rows]

The above EPL syntax includes some query clauses thatare introduced by Ahmad et al [29] The select clauses areused to select all properties or to specify the list of eventproperties and expressionsThe ldquofrom clauserdquo indicates eventstream name The ldquowhere clauserdquo is an optional clause usedto join and correlate event streams The ldquogroup by clauserdquoseparates the output events into groups The ldquohaving clauserdquois used in combination with the ldquogroup by clauserdquo to restrictthe groups of returned rows to only those whose condition istrue Adopting EPL syntax rules declared in Section 31 canbe represented in a standard structure as follows

Rule 1 MissingEventAgent

SELECT COUNT(eSid) WHITIN(RequireTime)IF (COUNT(eSid) = 0) THENACTION createMissingEventAgentWarning

Rule 2 UserRepeatDocExamine

SELECT COUNT(ePid) WHERE eTYPE =lsquoDocExaminersquo WHITIN(1 day)IF ANY(eCOUNT(ePid)) gt LIMIT DOCEX-AMINE THEN ACTION create UserRepeat-DocExamineWarning

Rule 3 Project ManagerExamine

SELECT COUNT(eManager) WHEREeTYPE = lsquoSubmitProjectInforsquo and eManager =e1015840Manager ande1015840ProjectState = lsquoUncompletedrsquoorlsquoSubmitedrsquoIF COUNT(eManager) gt LIMIT MANAGERTHEN ACTION create ManagerRepeatedError

We assume that the above rules provide analysis enginewith rules for what to do in event streams For this aim theengine is intelligent to execute rules by adopting a knowledgebase Knowledge base provides intelligence for the engine asit contains rich parameters for both engine and rules

For example LIMIT DOCEXAMINE is defined byLIDAS administrator in Rule 2 Its value is according tothe document of examination server runtime state Rule 2is constructed to balance the capacity of the server to eachproject and detect vicious document examination request Itspecifies that the trustworthiness of a document examinationrequest should be continuously monitored The notificationis generated as soon as the value rises above a given threshold

6 International Journal of Distributed Sensor Networks

Filter

Analysis engine

Knowledgebase

Rule base

Event model

CEP component

Analysisresult

User dashboard

Lightweight intelligent data analysis system

Integrated systems

Datastorage

User admin board

System N

Agent 3

System 2

Agent 2

System 1

Agent 1 middot middot middot

Figure 3 LIDAS modularized architecture with distributed systems

Main process thread

Agent 1

Agent 2

Agent n

Listener Preprocess

QueueQueues

Analysis thread Analysis

result

Filter Analysis engine (LIDA-E)

KnowledgebaseRule base

ae1ae2ae3

ce1ce2ce3

Figure 4 The event transmission and process workflow of LIDAS

value Rule 3 is made for analyzing how many uncompletedor submitted projects does a manager have in system If thenumber of a managerrsquos projects is beyond the limitationanalysis engine will create an error action

4 The Architecture of LIDAS

41 Architecture Design of LIDAS We have to extend thecapability of integrated information systems towards theintelligent data analysis by adopting advantage of event pro-cessing technologies keeping clearly apart each peculiarityof the CEP components to perform even deep changeswithout affecting related integrated systems In designing theintelligent data analysis architecture we started by specifyingthe CEP components that interact with the technologicalarchitecture and traditional integrated systems

As shown in Figure 3 LIDAS is a lightweight andmodularized framework designed to connect the integratedsystems for intelligent data analysisThe bottom of the frame-work is agents that are deployed in the traditional integratedinformation systems to extract data in complicated businessprocesses The upper part is an intelligent data analysis with

CEP which deals with event processing and data analyzing Itis implemented as an intelligent processing center connectedto explicit event queues in accordance with predefined eventrules System administrators define event processing rulesand knowledge through user admin board in system buildtime The users get analysis results from user dashboard inreal time

In order to show the workflow of LIDAS framework weintroduced agent filter and analysis engine to illustrate eventtransmit and processing (see Figure 4) First agents sendatomevents to the listenermodule in filter And then preproc-ess module reads events from listener and stores them inqueue it sends a message of event arriving to analysis engineAnd the engine receives messages from filter it enables theprocessing thread to access new events from queue Thegraphical representation of the workflow is illustrated inFigure 4 and introduced in the following subsections in detail

42 Event Transmission Component In order to improvethe scalability in large-scale integrated systems and reducethe network delay caused by online event streams deliveryalmost all current event processing frameworks are based on

International Journal of Distributed Sensor Networks 7

Main thread

QueueQueues

Event process

KnowledgebaseRule base

Analysisresult

Analysis engine (LIDA-E)

Filter2 3 5

4 6

7Event access

1ae1ae2ae3

ce1ce2ce3

Figure 5 The working pipeline of LIDA-E

CEP However event receiving and dispatching problems arecommonly ignored Figure 3 illustrates the simplified viewover the LIDAS architecture in distributed systemsThe com-bination of filter and agent works as a bridge that transmitsevents from information systems to LIDA-E They are basiccomponents which act as an event provider to CEP engine

We use agent which is similar to sensor [11] and adapter[30] used in monitoring system to collect information fromdistributed environment Agent is responsible for extractingreal-time business data and generating atom event It is eventsource and is deployed in an integrated system to generatemassive events with business data Using the proposed eventmodel agent reads data froman interface of integrated systemand translates them into atom events According to the eventmodel (Definition 1) agent puts data as attributes in 119860 asa list It generates atom event immediately when data isdetected in systems

Collaborating with CEP technology the filter detectscomposite events from different information systems [27]Filter is an event receiving and preprocessing module thathandles subscription related to control coming messagesfrom analysis engine and processes high volumes of eventobjects which are received from agents According to theavailability of business data filter pushes each valid atomevent into a map that is held in memory while it notifies theanalysis engine that a new event has arrived and is storedin map We summarize the functions of filter as follows (1)receive events from distributed agents (2) preprocess eventsstreams according to event availability (3) push events intomemory and notify analysis engine

Somemethods for example TCP FTP SMTP HTTP andTELNET can be used in event transfer method between filterand agent However we use a New IO (NIO) client serverNetty framework in our event transmission mechanism tofind a way to achieve ease of development performancestability and flexibility with high throughput low latency lessresource consumption andminimized unnecessary memorycopy We have developed a socket client and server by Nettyclient is deployed in agent and server which is adopted byfilter By accepting socket communication protocol filter hasthe capability to listen on a system port and receive eventsfrom different agents Event transmission mechanism is thefirst step that collects real-time data from integrated systems

the CEP based event processingmethodwill be introduced inthe next subsection

43 LIDA-E Engine The analysis engine called LIDA-E isthe core component of LIDAS architecture it is responsiblefor processing the events queue and judging whether there isa complex event inferred [25] It provides effective queuingscheduling time and count-window support and fast in-memory processing of high-speed continuous unboundeddata streams In order to divide functions of engine we designthree modules in LIDA-E they are main thread event accessand event process Main thread module is used to monitorfilter and control event access module Event access moduleis dedicated for accessing events from the queue which iswritten by filter Event process module executes rules andprocesses all the events It is a linear task controller thatdeploys rules one by one and manages events to be processedthrough them

CEP component comprises a set of rules and knowledgeto detect a predefined group of anomalies on the basis of thereceiving event attribute which are the same in [31ndash33] Therule base specifies how to infer complex events from the eventstream Howver knowledge base provides the link betweenknown information (the antecedent) and the information tobe deduced (the consequent) or actions to be executed It isa useful supplement to rule base for processing events Bothrules and knowledge are defined in build time and loadedwhen LIDAS starts to work

LIDA-E workflow is shown in Figure 5 it is a runtimeworkflow in which the analysis engine waiting for eventsarriving has already executed rules with knowledge in eventprocessmoduleTheworkflow follows the substeps as follows(1) filter pushes events into queue as soon as it receives eventsfrom different agents (2) filter sends a message to analysisengine to notify it that a new event has come (3) main threadmodule receives themessage andnotifies event accessmodulethat new events have been pushed in queue (4) event accessmodule reads events fromqueue and prepares the submissionfor event process module (5) event access module sendsevent to event processmodule immediately (6) event processmodule accesses queues for event processing in event processmodule each event is orderly processed by rules for eachrule at least one queue is provided for runtime event storage

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 6: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

6 International Journal of Distributed Sensor Networks

Filter

Analysis engine

Knowledgebase

Rule base

Event model

CEP component

Analysisresult

User dashboard

Lightweight intelligent data analysis system

Integrated systems

Datastorage

User admin board

System N

Agent 3

System 2

Agent 2

System 1

Agent 1 middot middot middot

Figure 3 LIDAS modularized architecture with distributed systems

Main process thread

Agent 1

Agent 2

Agent n

Listener Preprocess

QueueQueues

Analysis thread Analysis

result

Filter Analysis engine (LIDA-E)

KnowledgebaseRule base

ae1ae2ae3

ce1ce2ce3

Figure 4 The event transmission and process workflow of LIDAS

value Rule 3 is made for analyzing how many uncompletedor submitted projects does a manager have in system If thenumber of a managerrsquos projects is beyond the limitationanalysis engine will create an error action

4 The Architecture of LIDAS

41 Architecture Design of LIDAS We have to extend thecapability of integrated information systems towards theintelligent data analysis by adopting advantage of event pro-cessing technologies keeping clearly apart each peculiarityof the CEP components to perform even deep changeswithout affecting related integrated systems In designing theintelligent data analysis architecture we started by specifyingthe CEP components that interact with the technologicalarchitecture and traditional integrated systems

As shown in Figure 3 LIDAS is a lightweight andmodularized framework designed to connect the integratedsystems for intelligent data analysisThe bottom of the frame-work is agents that are deployed in the traditional integratedinformation systems to extract data in complicated businessprocesses The upper part is an intelligent data analysis with

CEP which deals with event processing and data analyzing Itis implemented as an intelligent processing center connectedto explicit event queues in accordance with predefined eventrules System administrators define event processing rulesand knowledge through user admin board in system buildtime The users get analysis results from user dashboard inreal time

In order to show the workflow of LIDAS framework weintroduced agent filter and analysis engine to illustrate eventtransmit and processing (see Figure 4) First agents sendatomevents to the listenermodule in filter And then preproc-ess module reads events from listener and stores them inqueue it sends a message of event arriving to analysis engineAnd the engine receives messages from filter it enables theprocessing thread to access new events from queue Thegraphical representation of the workflow is illustrated inFigure 4 and introduced in the following subsections in detail

42 Event Transmission Component In order to improvethe scalability in large-scale integrated systems and reducethe network delay caused by online event streams deliveryalmost all current event processing frameworks are based on

International Journal of Distributed Sensor Networks 7

Main thread

QueueQueues

Event process

KnowledgebaseRule base

Analysisresult

Analysis engine (LIDA-E)

Filter2 3 5

4 6

7Event access

1ae1ae2ae3

ce1ce2ce3

Figure 5 The working pipeline of LIDA-E

CEP However event receiving and dispatching problems arecommonly ignored Figure 3 illustrates the simplified viewover the LIDAS architecture in distributed systemsThe com-bination of filter and agent works as a bridge that transmitsevents from information systems to LIDA-E They are basiccomponents which act as an event provider to CEP engine

We use agent which is similar to sensor [11] and adapter[30] used in monitoring system to collect information fromdistributed environment Agent is responsible for extractingreal-time business data and generating atom event It is eventsource and is deployed in an integrated system to generatemassive events with business data Using the proposed eventmodel agent reads data froman interface of integrated systemand translates them into atom events According to the eventmodel (Definition 1) agent puts data as attributes in 119860 asa list It generates atom event immediately when data isdetected in systems

Collaborating with CEP technology the filter detectscomposite events from different information systems [27]Filter is an event receiving and preprocessing module thathandles subscription related to control coming messagesfrom analysis engine and processes high volumes of eventobjects which are received from agents According to theavailability of business data filter pushes each valid atomevent into a map that is held in memory while it notifies theanalysis engine that a new event has arrived and is storedin map We summarize the functions of filter as follows (1)receive events from distributed agents (2) preprocess eventsstreams according to event availability (3) push events intomemory and notify analysis engine

Somemethods for example TCP FTP SMTP HTTP andTELNET can be used in event transfer method between filterand agent However we use a New IO (NIO) client serverNetty framework in our event transmission mechanism tofind a way to achieve ease of development performancestability and flexibility with high throughput low latency lessresource consumption andminimized unnecessary memorycopy We have developed a socket client and server by Nettyclient is deployed in agent and server which is adopted byfilter By accepting socket communication protocol filter hasthe capability to listen on a system port and receive eventsfrom different agents Event transmission mechanism is thefirst step that collects real-time data from integrated systems

the CEP based event processingmethodwill be introduced inthe next subsection

43 LIDA-E Engine The analysis engine called LIDA-E isthe core component of LIDAS architecture it is responsiblefor processing the events queue and judging whether there isa complex event inferred [25] It provides effective queuingscheduling time and count-window support and fast in-memory processing of high-speed continuous unboundeddata streams In order to divide functions of engine we designthree modules in LIDA-E they are main thread event accessand event process Main thread module is used to monitorfilter and control event access module Event access moduleis dedicated for accessing events from the queue which iswritten by filter Event process module executes rules andprocesses all the events It is a linear task controller thatdeploys rules one by one and manages events to be processedthrough them

CEP component comprises a set of rules and knowledgeto detect a predefined group of anomalies on the basis of thereceiving event attribute which are the same in [31ndash33] Therule base specifies how to infer complex events from the eventstream Howver knowledge base provides the link betweenknown information (the antecedent) and the information tobe deduced (the consequent) or actions to be executed It isa useful supplement to rule base for processing events Bothrules and knowledge are defined in build time and loadedwhen LIDAS starts to work

LIDA-E workflow is shown in Figure 5 it is a runtimeworkflow in which the analysis engine waiting for eventsarriving has already executed rules with knowledge in eventprocessmoduleTheworkflow follows the substeps as follows(1) filter pushes events into queue as soon as it receives eventsfrom different agents (2) filter sends a message to analysisengine to notify it that a new event has come (3) main threadmodule receives themessage andnotifies event accessmodulethat new events have been pushed in queue (4) event accessmodule reads events fromqueue and prepares the submissionfor event process module (5) event access module sendsevent to event processmodule immediately (6) event processmodule accesses queues for event processing in event processmodule each event is orderly processed by rules for eachrule at least one queue is provided for runtime event storage

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 7: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

International Journal of Distributed Sensor Networks 7

Main thread

QueueQueues

Event process

KnowledgebaseRule base

Analysisresult

Analysis engine (LIDA-E)

Filter2 3 5

4 6

7Event access

1ae1ae2ae3

ce1ce2ce3

Figure 5 The working pipeline of LIDA-E

CEP However event receiving and dispatching problems arecommonly ignored Figure 3 illustrates the simplified viewover the LIDAS architecture in distributed systemsThe com-bination of filter and agent works as a bridge that transmitsevents from information systems to LIDA-E They are basiccomponents which act as an event provider to CEP engine

We use agent which is similar to sensor [11] and adapter[30] used in monitoring system to collect information fromdistributed environment Agent is responsible for extractingreal-time business data and generating atom event It is eventsource and is deployed in an integrated system to generatemassive events with business data Using the proposed eventmodel agent reads data froman interface of integrated systemand translates them into atom events According to the eventmodel (Definition 1) agent puts data as attributes in 119860 asa list It generates atom event immediately when data isdetected in systems

Collaborating with CEP technology the filter detectscomposite events from different information systems [27]Filter is an event receiving and preprocessing module thathandles subscription related to control coming messagesfrom analysis engine and processes high volumes of eventobjects which are received from agents According to theavailability of business data filter pushes each valid atomevent into a map that is held in memory while it notifies theanalysis engine that a new event has arrived and is storedin map We summarize the functions of filter as follows (1)receive events from distributed agents (2) preprocess eventsstreams according to event availability (3) push events intomemory and notify analysis engine

Somemethods for example TCP FTP SMTP HTTP andTELNET can be used in event transfer method between filterand agent However we use a New IO (NIO) client serverNetty framework in our event transmission mechanism tofind a way to achieve ease of development performancestability and flexibility with high throughput low latency lessresource consumption andminimized unnecessary memorycopy We have developed a socket client and server by Nettyclient is deployed in agent and server which is adopted byfilter By accepting socket communication protocol filter hasthe capability to listen on a system port and receive eventsfrom different agents Event transmission mechanism is thefirst step that collects real-time data from integrated systems

the CEP based event processingmethodwill be introduced inthe next subsection

43 LIDA-E Engine The analysis engine called LIDA-E isthe core component of LIDAS architecture it is responsiblefor processing the events queue and judging whether there isa complex event inferred [25] It provides effective queuingscheduling time and count-window support and fast in-memory processing of high-speed continuous unboundeddata streams In order to divide functions of engine we designthree modules in LIDA-E they are main thread event accessand event process Main thread module is used to monitorfilter and control event access module Event access moduleis dedicated for accessing events from the queue which iswritten by filter Event process module executes rules andprocesses all the events It is a linear task controller thatdeploys rules one by one and manages events to be processedthrough them

CEP component comprises a set of rules and knowledgeto detect a predefined group of anomalies on the basis of thereceiving event attribute which are the same in [31ndash33] Therule base specifies how to infer complex events from the eventstream Howver knowledge base provides the link betweenknown information (the antecedent) and the information tobe deduced (the consequent) or actions to be executed It isa useful supplement to rule base for processing events Bothrules and knowledge are defined in build time and loadedwhen LIDAS starts to work

LIDA-E workflow is shown in Figure 5 it is a runtimeworkflow in which the analysis engine waiting for eventsarriving has already executed rules with knowledge in eventprocessmoduleTheworkflow follows the substeps as follows(1) filter pushes events into queue as soon as it receives eventsfrom different agents (2) filter sends a message to analysisengine to notify it that a new event has come (3) main threadmodule receives themessage andnotifies event accessmodulethat new events have been pushed in queue (4) event accessmodule reads events fromqueue and prepares the submissionfor event process module (5) event access module sendsevent to event processmodule immediately (6) event processmodule accesses queues for event processing in event processmodule each event is orderly processed by rules for eachrule at least one queue is provided for runtime event storage

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 8: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

8 International Journal of Distributed Sensor Networks

Input Input eventOutput Complex event (ce) and action(1) Load Rules()(2) Load Knowledge()(3) WHILE (Message == lsquoEventArriversquo ampamp EventQueuesize() gt 0)(4) AtomEventItem ae = EventQueueshift()(5) if (ae == null)(6) Threadsleep(10)(7) continue(8) for (119894 = 1 119894 lt= CountRules 119894++)(9) if (Rule[119894]enable == True)(10) Rule[119894]getParament(11) ComplexEventItem ce = Rule[119894]excute(ae)(12) if (ce = null)(13) Action(ce)(14) AnalysisResultadd(ce)(15) EventStorageadd(ae ce)

Algorithm 1 Event processing algorithm

(7) after processing analysis results are submitted to LIDASusers

44The Proposed Event Processing Algorithm In this sectionwe present the new Complex Event Processing algorithmwhich is a simplified version of the algorithm It has beenimplemented in LIDA-E Each rule is translated into ruleinstance and applied in event processing module Additionalconstraints are applied by parameters which are stored inknowledge base such as LIMIT DOCEXAMINE in Rule 2 Itidentifies the max value of document examination time theinstance used to process the input events instead of automa-ton instance that was introduced in AIP Algorithm [34] Thedetailed steps for engine algorithm are shown in Algorithm 1

The key role in our approach is played by rule instanceswhich are used to detect complex events in LIDA-E Eventqueue is a queuewhich stores the undisposed events in LIDA-E At the beginning LIDA-E loads predefined rules andknowledge (lines 1 and 2) which run in the initial state forthe arrival of appropriate events First engine gets a messageof ldquoEventArriverdquo from filter and a new event arrives (line 3) Ifthe EventQueue is not null (line 3) a variable ae (atom event)is defined and assigned by an event in EventQueue (line 4)The function shift of EventQueue not only reads event fromEventQueue but also deletes the event which is read (line 4)If ae is null (line 5) the processing thread will sleep for 10milliseconds and then go on working (lines 6 and 7) If ae isnot null it will be processed by every rule (line 8) In eachrule instance once a complex event is detected a variablece (complex event) will be assigned by the detected event(line 11) After checking the validity of ce an action of ce isgenerated to system user (line 13) and the detected event issent to processing result (line 14) Finally all events would bestored in a database as history records (line 15)

The graphical representation of algorithm is shown inFigure 6 Input events are processed from rule 1 to119873 inwhicha complex event is sent to result once it is detectedOtherwisethe events are signed by pass Each rule instancemaintains its

Rule 1 Rule 2 Rule 3 Rule N

Complexevents type 1

Complexevents type 2

Complexevents type 3

Complexevents type N

Result

Pass

Detect Detect Detect Detect

Pass Pass Pass PassEvents

Knowledge base

middot middot middot

Figure 6 The graphical representation of the event processingalgorithm

own event processing status in a private queue Finally all thecomplex events results are collected from each rule instance

5 Experimental Evaluations

In order to perform functional validation and performanceevaluation of the proposed LIDAS architecture with LIDA-Ewe have developed a simple proof-of-concept prototype sys-tem supporting both the event transmission and data analy-sis capabilities with the use of currently available open sourcecomponents Evaluating the performance of the proposedLIDAS framework is not easy as it is strongly influenced bythe workloadThe number of rules and events to be processedis considered in each test case [34] It is also limited by theevent transmission capability in network environment Inexperiments we evaluated both the event transmissionmech-anism and the analysis engineThey show the performance ofLIDAS architecture All the test is demonstrated on notebookcomputer having Intel Core i5-3210M CPU 250GHz and4GB of RAM running 32-bit Windows 7 Professional

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 9: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

International Journal of Distributed Sensor Networks 9

Our evaluation had two main goals (1) studying theperformance of event transmissionmechanism in our systemarchitecture and (2) comparing engine LIDA-E with anothercommon processing engine that could handle some rulesdescribed in Section 3

51 Event Transmission Event transmission refers to the pro-cedure that an event is sent from an agent until it is receivedby filterWe started by usingHTTP request to send data agentto filter First we used the PHP to develop anApache server asfilter In experiment the Apache server (versions 22 and 24)could receive no more than 2000 HTTP requests in onesecond Moreover we built the filter by Servlet which isdeveloped by Java that extends the capabilities of a serverTheperformance of Servlet service was similar to PHP servicethat receives about 1500 events per second on Tomcat 60server Additionally we have used a special server technologyNodejs [35] to improve the performance of filter Howeverits performance was worse than the filter that is built by PHPand Servlet

Unfortunately themain technologies for acceptingHTTPrequests did not reach the requirement of massive eventstransmission in LIDAS architecture This issue was wellrecognized for the complex structure of HTTP request asdemonstrated by the investigation in HTTP request messageheaders [36] To address this issue we decided to use socket totransmit event data exploring socket client and server basedon Netty 4040 Final We implemented the proposed eventtransmission mechanism in a prototype based on a productcompliant with the JMS standard specifically Netty whichprovides a popular and powerful asynchronous event-drivennetwork application framework [37 38]

Netty client and server were developed in agent and filterwhich were performed on two notebook computers to checkthe performance of event transmission capability Duringour tests the number of events sent by agent was increasedfrom 10000 to 100000 and we recorded both time costs inagent and filter side Figure 7 shows the time cost of eventtransmission in agent and filter

In terms of the number of events to be transmitted thereis no apparent difference between agent and filter in eventtransmission time cost And event receiving time cost of filteris a little less than event sending time cost of agent Our exper-imental results show that the agent and filter have good per-formance to transmit huge volume of events in a short time

52 Event Processing Capability We evaluated the eventprocessing capability of our engine compared with theEsper [39] engine Esper has many features including highscalability memory efficiency in-memory computing SQLstandard minimal latency and historical event analysis It isa streaming-capable engine for processing real-time arrivingvariety data It was embedded in Java which makes thecomparison of our engine easier

Theprototype LIDA-Ewas implemented in Java languagewhich is the most popular programming language in 2015[40] The applicability and usefulness of our approach havebeen evaluated by sample implementation of LIDA-E thatprocesses events sent by filter (see Figure 5)

0

1

2

3

4

5

6

7

8

Tran

smiss

ion

time (

s)

0 40000 60000 80000 10000020000Input events

AgentFilter

Figure 7 The event transmission time cost of agent and filter

0

5

10

15

20Th

roug

hput

(k ev

ents

s)

10 20 30 40 500Input rate (k eventss)

EsperLIDA-E

Figure 8Theperformance comparison of filtering between LIDA-Eand Esper

521 Event Filtering The base operation of CEP engine isselecting input events as soon as the listener module receivesnew events For this reason the evaluation is started bythe comparison of filtering capability 100 different ruleswere deployed in both engines The rules selected inputevents according to their attributes and generated new eventscontaining the same attributes Therefore each event wasfiltered by 100 rules and this meant the event input rateshould be equal to the output rate The filtering results of thetwo CEP engines are shown in Figure 8

This evaluation highlights how LIDA-E processes inputevents faster than Esper with the same input events anddeployed rules As theworkloads of the two engines are identi-cal and are processed by a similar set of rules the throughput

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 10: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

10 International Journal of Distributed Sensor Networks

0 40000 60000 80000 10000020000Input events

0

2

4

8

6

Proc

essin

g tim

e (s)

EsperLIDA-E

Figure 9 The time cost comparison between LIDA-E and Esperusing Rule 2-1

should theoretically be the same We observe that bothengines process huge numbers of events in a short time par-ticularly when the input rate increased from 5000 to 50000events per second The throughput of LIDA-E is higher thanof Esper Esper handles up to about 9000 composite eventsper second as the maximum throughput However LIDA-E starts to drop the input events at rate of 15000 eventsper second The maximum throughput of LIDA-E is 18000composite eventss

522 Data Analysis Processing input events by ruleinstances is the main task of a CEP engine So we continueto perform the experiments by comparing event processingcapability of LDIA-E and Esper in complex operations Todemonstrate the performance of event processing withouttemporal operation we defined Rule 2-1 by deleting theWHITIN operator of Rule 2 Five rules (Rule 2-1) weredeployed in both engines in this test Figure 9 shows theresults in terms of processing time and number of inputevents in both engines

Thesemeasure highlights show that the LIDA-E processesevents faster than Esper when both engines deployed thesame rules Compared with Esper LIDA-E has 4865averagely reduced time cost in different tests It has apparentadvantage over Esper because of its event processing algo-rithm When the number of atom events was increased from10000 to 100000 the event processing time was increasedfrom 065 to 302 (seconds) in LDIA-E However if we usedEsper the processing time was increased from 086 to 718(seconds)

As a second one we have compared the performanceof LIDA-E on different number of rules This step used thedefinitions of Rule 2-1 We deployed 10 rules for performanceevaluation Figure 10 illustrates that more rules cost moretime in event processing the event processing time is linearlyincreased when rules increased When the number of atom

0 40000 60000 80000 10000020000Input events

0

1

2

3

4

5

6

Proc

essin

g tim

e (s)

LIDA-E 5 rulesLIDA-E 10 rules

Figure 10 LIDA-E time cost comparison between 5 and 10 rules

events increased from 10000 to 100000 the time cost isincreased from 065 to 302 seconds under 5 rules Howeverif we increased the number of rules to 10 the time cost alsois increased from 117 to 500 seconds Therefore the numberof rules has a significant impact on the performance of eventprocessing

523 Sliding Window The capability of the two engines inprocessing sliding window is compared here The third caseof LIDA-E is based on a simple testing rule computing thevalue of the selected events with two strict constraints Rule4-1 contains the timewindow andRule 4-2 includes the batchwindow

To evaluate the capability of event processing LIDA-Eand Esper were respectively deployed in Rule 4-1 and Rule4-2 20 rules with the same constraints were deployed in bothengines In order to simulate input events the event filter sentatom events with uniformly distributed attribute between 1and 100while all the input events had the same event type andsource In this test we have used three different workloadstasks to evaluate the engines transmitting respectively 2050 and 80 available events of input events (the remainingevents were not selected by the engine)

Rule 4-1 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE= lsquoCPU Usagersquo and eValue gt SelectivityWHITIN(01 sec)THEN ACTION create CPU Usage HighWarning

Rule 4-2 UserRepeatDocExamine

SELECT EVERY(eValue) WHERE eTYPE =lsquoCPU Usagersquo and eValue gt Selectivity BATCH(10)THEN ACTION create CPU Usage HighWarning

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 11: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

International Journal of Distributed Sensor Networks 11

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

0

1

2

3

4

5

6Pr

oces

sing

time (

s)

(a) 20 of input events are detected

0

2

4

6

8

10

12

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

3

6

9

12

15

18

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 11 Comparison between LAIPE and Esper using Rule 4-1

First of all Rule 4-1 is deployed in the two engines totest the capability of executing time window Figure 11 showshow processing time varies in relation to the selected rate ofavailable events We observed that both engines performedvery well even if the event provider sent a huge number ofinput events to them in a short time The tests also showprocessing time of both engines when the selected rate ofinput events is increased from 20 to 80 Even moreimportant from Figure 11 is that LIDA-E outperforms Esperin the three workloads Particularly the processing time ofLIDA-E is about half of Esperrsquos to deal with the same numberof input events which means LIDA-E processes events fasterthan Esper with the same deployed rules and workloads

Second Rule 4-2 is used to evaluate the performanceof the two engines in batch window Figure 12 shows theprocessing time of the two engines adopting batch operatorto deal with input events (batch size is 10) It can be observed

that the time cost of LIDA-Ewas less than of Esper in differentselected rate Therefore the performance of LIDA-E is morestabilized than Esperrsquos with different selected rate especiallywhen more input events are selected in CEP engine

In general our experimental results confirmed theadvance in real-time event processingThe implementation ofLIDAS with LIDA-E has power of dealing with large volumeof events during different experiments Therefore it efficientlyand reliably provides a real-time delivery and analysis serviceto high-frequency business data streams

6 Conclusion

In this paper a novel and effective architecture with thecombination of a lightweight intelligent data analytics sys-tem (LIDAS) and CEP engine (LIDA-E) is presented InLIDAS event models (atomic and complex) and operators

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 12: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

12 International Journal of Distributed Sensor Networks

0

2

4

6

8

10

12

14

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(a) 20 of input events are detected

0

3

6

9

12

15

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(b) 50 of input events are detected

0

2

4

6

8

10

Proc

essin

g tim

e (s)

0 200000 300000 400000 500000100000Input events

EsperLIDA-E

(c) 80 of input events are detected

Figure 12 Comparison between LIDA-E and Esper using Rule 4-2

(logical mathematical and temporal) are formulated forevent detection and analysis LIDAS is based on a modulararchitecture which clearly separates the core logic and isdevoted to event transmission from agent to filter and eventprocessing handled by LIDA-E engine Its knowledge basewith rules is responsible for intelligent data analysis with theproposed event processing algorithm An event transmissioncomponent explicitly designed to target the needs of appli-cations that have to cope with event generation and theirtransmission utilizes an efficient engine LIDA-E capable ofanalyzing large volume of input data in an efficient way

A comparison of LIDA-E and Esper which is the mostwidely used commercial solution for CEP and is known forits expressiveness and efficiency shows that the performanceof LIDA-E is better than of Esper in different tests In filtering

events the throughput of LIDA-E is almost twice that of Esperwhen input rate is above 20k eventss Comparing with EsperLIDA-E decreases the processing time in different selectedrates on dealing with sliding windowThe better performanceof LIDA-E can be clearly summarized through the results ofcomparative tests

Competing Interests

The authors declare that they have no competing interests

Acknowledgments

This work was supported by the Research on Key Tech-nology of Virtual Restoration Mosaic in Terracotta Army

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 13: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

International Journal of Distributed Sensor Networks 13

(20136101110019) Research on theMethod of Virtual Restora-tion of Damaged Terracotta Army Based on Global Opti-mization (61373117) and Web Services Monitoring Technol-ogy in Distributed Network Environment Based on CEP(YZZ14119)

References

[1] M P Papazoglou P Traverso S Dustdar and F LeymannldquoService-oriented computing state of the art and researchchallengesrdquo Computer vol 40 no 11 pp 38ndash45 2007

[2] S Tsuchiya Y Sakamoto Y Tsuchimoto and V Lee ldquoBigdata processing in cloud environmentsrdquo Fujitsu Scientific andTechnical Journal vol 48 no 2 pp 159ndash168 2012

[3] Dain Hansen Oracle Fast Data Real-Time Strategies for BigData and Business Analytics [J] Big Data Gets Real-time OracleFast Data Real-Time Strategies for Big Data and BusinessAnalytics [J] 2013

[4] Data analysis httpsenwikipediaorgwikiData analysis[5] C Esposito M Ficco F Palmieri and A Castiglione ldquoA

knowledge-based platform for Big Data analytics based onpublishsubscribe services and stream processingrdquo Knowledge-Based Systems vol 79 pp 3ndash17 2015

[6] httpwwwcscoloradoedusimkenaclasses5448s11presenta-tionshadooppdf

[7] Storm httpstorm-projectnet[8] J Xu Z Chen J Tang and S Su ldquoT-storm traffic-aware

online scheduling in stormrdquo in Proceedings of the IEEE 34thInternational Conference on Distributed Computing Systems(ICDCS rsquo14) pp 535ndash544 Madrid Spain July 2014

[9] A Verma G Dasgupta T K Nayak P De and R KotharildquoServer workload analysis for power minimization using con-solidationrdquo in Proceedings of the USENIX Annual TechnicalConference (USENIX rsquo09) p 28 USENIX Association SanDiego Calif USA June 2009

[10] R Bruns J Dunkel H Masbruch and S Stipkovic ldquoIntelligentM2M complex event processing formachine-to-machine com-municationrdquo Expert Systems with Applications vol 42 no 3 pp1235ndash1246 2015

[11] B Balis B Kowalewski and M Bubak ldquoReal-time Grid mon-itoring based on complex event processingrdquo Future GenerationComputer Systems vol 27 no 8 pp 1103ndash1112 2011

[12] I Zappia F Paganelli and D Parlanti ldquoA lightweight andextensible Complex Event Processing system for sense andrespond applicationsrdquo Expert Systems with Applications vol 39no 12 pp 10408ndash10419 2012

[13] D C Luckham and B Frasca ldquoComplex Event Processing inDistributed Systemsrdquo Tech Rep 1998

[14] Y-Z Liu and F-W Han ldquoRFID complex event processing forRTLSrdquo in Proceedings of the 4th International Conference onMultimedia and Security (MINES rsquo12) pp 392ndash395 NanjingChina November 2012

[15] Y HWang K Cao and X M Zhang ldquoComplex event process-ing over distributed probabilistic event streamsrdquoComputers andMathematics with Applications vol 66 no 10 pp 1808ndash18212013

[16] R Baldoni L Montanari and M Rizzuto ldquoOn-line failure pre-diction in safety-critical systemsrdquo Future Generation ComputerSystems vol 45 pp 123ndash132 2015

[17] D Wang E A Rundensteiner H Wang and R T EllisonIII ldquoActive complex event processing applications in real-time

health carerdquo Proceedings of the VLDB Endowment vol 3 no 1-2pp 1545ndash1548 2010

[18] W Yao C-H Chu and Z Li ldquoLeveraging complex eventprocessing for smart hospitals using RFIDrdquo Journal of Networkand Computer Applications vol 34 no 3 pp 799ndash810 2011

[19] J Boubeta-Puig G Ortiz and IMedina-Bulo ldquoAmodel-drivenapproach for facilitating user-friendly design of complex eventpatternsrdquo Expert Systems with Applications vol 41 no 2 pp445ndash456 2014

[20] C Zang and Y Fan ldquoComplex event processing in enterpriseinformation systems based on RFIDrdquo Enterprise InformationSystems vol 1 no 1 pp 3ndash23 2007

[21] M Ahmadzadeh Ghasemabadi and P D Shamsabadi ldquoApplica-tion of five processes of projectmanagement based on PMBOK-2008 standard to run EPM-2010 project management system acase study of Arya Hamrah Samaneh Cordquo in Proceedings of the2nd IEEE International Conference on Emergency Managementand Management Sciences pp 792ndash795 Beijing China August2011

[22] S Berzisa and J Grabis ldquoKnowledge reuse in configurationof project management information systems a change man-agement case studyrdquo in Proceedings of the 15th InternationalConference on Intelligent Engineering Systems (INES rsquo11) pp 51ndash56 IEEE Poprad Slovakia June 2011

[23] M Taghavi A Patel and H Taghavi ldquoWeb base projectmanagement system for development of ICTproject outsourcedby Iranian governmentrdquo in Proceedings of the 2nd IEEE Inter-national Conference on Open Systems (ICOS rsquo11) pp 273ndash278Langkawi Malaysia September 2011

[24] D Luckham and R Schulte ldquoEvent processing glossarymdashversion 11rdquo Tech Rep Event Processing Technical Society2008

[25] Q Li Y Jin T He and H Xu ldquoSmart home services basedon event matchingrdquo in Proceedings of the 10th InternationalConference on Fuzzy Systems and Knowledge Discovery (FSKDrsquo13) pp 762ndash766 Shenyang China July 2013

[26] S Jayasekara S Kannangara T Dahanayakage I RanawakaS Perera and V Nanayakkara ldquoWihidum distributed complexevent processingrdquo Journal of Parallel and Distributed Comput-ing vol 79-80 pp 42ndash51 2015

[27] A Hinze and A Voisard ldquoEVA an event algebra supportingcomplex event specificationrdquo Information Systems vol 48 pp1ndash25 2015

[28] EsperReference httpwwwespertechcomesperrelease-520esper-referencepdfesper referencepdf

[29] W Ahmad A Lobov and J L M Lastra ldquoFormal modellingof complex event processing a generic algorithm and itsapplication to a manufacturing linerdquo in Proceedings of the IEEE10th International Conference on Industrial Informatics (INDINrsquo12) pp 380ndash385 Beijing China July 2012

[30] F Meyer R Kroeger R Heidger and M Milekovic ldquoAnapproach for knowledge-based IT management of air trafficcontrol systemsrdquo in Proceedings of the 9th International Confer-ence on Network and Service Management (CNSM rsquo13) pp 345ndash349 IEEE Zurich Switzerland October 2013

[31] I Ari E Olmezogullari and O F Celebi ldquoData stream analyticsand mining in the cloudrdquo in Proceedings of the 4th IEEEInternational Conference on Cloud Computing Technology andScience (CloudCom rsquo12) pp 857ndash862 Taipei Taiwan December2012

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 14: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

14 International Journal of Distributed Sensor Networks

[32] F Terroso-Saenz M Valdes-Vela E den Breejen P Hanck-mann RDekker andA Skarmeta-Gomez ldquoCEP-traj an event-based solution to process trajectory datardquo Information Systemsvol 52 pp 34ndash54 2015

[33] B Idiri and A Napoli ldquoThe automatic identification systemof maritime accident risk using rule-based reasoningrdquo inProceedings of the 7th International Conference on System ofSystems Engineering (SoSE rsquo12) pp 125ndash130 IEEE Genoa ItalyJuly 2012

[34] G Cugola and A Margara ldquoLow latency complex event pro-cessing on parallel hardwarerdquo Journal of Parallel andDistributedComputing vol 72 no 2 pp 205ndash218 2012

[35] NodeJS httpsnodejsorg[36] M C Calzarossa and L Massari ldquoAnalysis of header usage

patterns of HTTP request messagesrdquo in Proceedings of theIEEE International Conference on High Performance Computingand Communications IEEE 6th International Symposium onCyberspace Safety and Security IEEE 11th International Confer-ence on Embedded Software and Systems (HPCC rsquo14 CSS rsquo14ICESS rsquo14) pp 847ndash853 Paris France August 2014

[37] S Zhang and S Zhu ldquoServer structure based on netty frame-work for internet-based laboratoryrdquo in Proceedings of the 10thIEEE International Conference on Control and Automation(ICCA rsquo13) pp 538ndash541 IEEE Hangzhou China June 2013

[38] Netty Project httpnettyioindexhtml[39] EsperTech httpwwwespertechcomesperindexphp[40] TIOBE Index for October 2015 httpwwwtiobecomindex

phpcontentpaperinfotpciindexhtml

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of

Page 15: Research Article A Novel Complex Event Processing Engine ...downloads.hindawi.com/journals/ijdsn/2016/6741401.pdf · A Novel Complex Event Processing Engine for Intelligent Data Analysis

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Active and Passive Electronic Components

Control Scienceand Engineering

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

RotatingMachinery

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Shock and Vibration

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation httpwwwhindawicom

Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Navigation and Observation

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

DistributedSensor Networks

International Journal of