fiware cep ge introduction, ict 2015
TRANSCRIPT
![Page 1: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/1.jpg)
CEP GEInna Skarbovsky
![Page 2: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/2.jpg)
Outline
1. Introduction to CEP
2. Working with the CEP GE
3. Available Documentation
4. Example
![Page 3: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/3.jpg)
3
CEP Runtime
CEP
![Page 4: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/4.jpg)
4
CEP Technology – expanding the ECA paradigm
From Event-Condition-Action to Pattern-Condition-Action In certain scenarios, single events are insignificant, a CEP engine can detect
combinations of events, and generate derived events, called situations, which are meaningful
CEPAuthoring
Tool
CEPEngine
Definitions
DetectedSituations
EventSources
Run Time
Build Time
Input Events
Event Processing Rules / Patterns
Detected Situations
![Page 5: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/5.jpg)
5
CEP Technology – expanding the ECA paradigm
CEPAuthoring
Tool
CEPEngine
Definitions
DetectedSituations
EventSources
Run Time
Build Time
From Event-Condition-Action to Pattern-Condition-Action Use CEP when there is a need to detect patterns over incoming events Other tools can be used when there is a need to respond to single events
with some condition (e.g., Context Broker GE)
Input Events
Event Processing Rules / Patterns
Detected Situations
![Page 6: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/6.jpg)
The Event Processing Network
An event processing network (EPN) describes the event processing flow execution. It consists of a collection of event processing agents (EPAs), producers, and consumers linked by channels.
![Page 7: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/7.jpg)
Event Processing Agent (EPA) - Pattern Examples
Basic type: A lamp battery charge is below 15 percent
Aggregate type:The percentage of failed measurements is higher than 10 percent in the last 5 minutes
Absent type: no sensor events arrived in the last 10 minutes
![Page 8: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/8.jpg)
Event Processing Agent (EPA) - Pattern Examples
Sequence type: The same customer bought and later sold the same stock within 24 hours
All type: Alert if customer performs a withdrawal and a deposit, irrelevant in which order, within 10 mins
![Page 9: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/9.jpg)
Processing Context Definition
Every EPA (event processing agent) is associated with a processing context.
A processing context groups event instances so that they can be processed in a related way.
![Page 10: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/10.jpg)
Processing Context Examples
Temporal Context
“5 seconds after sensor event with high value”
“A time window bounded by Order-Placed and Order-Delivered events”
“From each sensor event for 10 mins”
“Sensor’s ID”“Shipment’s ID”“Building’s ID”
Segmentation Context
“All events of the same “Sensor’s ID” arrived in the last hour”“All sensor events that arrived 5 seconds after sensor event with high value, all arriving from the same building”
Composite context – group of several contexts +
![Page 11: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/11.jpg)
2. Working with the Proton (Proactive Technology Online) CEP GE Instance
![Page 12: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/12.jpg)
12
Build Time Web User Interface
Events Context
Agents
Validation Layer
JSON Object
![Page 13: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/13.jpg)
13
Proton (CEP GE Instance) – Build Time Web User Interface
Definition types:• Event types• EPAs (Event Processing Agents) - From various types: Basic,
All, Sequence, Absent, Aggregation• Contexts (Temporal, Segmentation, Composite)• Producers – From various types: File, REST, Custom• Consumers – From various types: File, REST, Custom
![Page 14: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/14.jpg)
14
Creating Definitions
• Can be generated through the WEB User Interface
• Can be generated by external systems– In the expected JSON format
![Page 15: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/15.jpg)
15
Runtime Main Interactions
ProtonRuntime
Input
Events
Input Adapters
Files
REST Client
Custom
Output Events
ProtonRuntimeProtonRuntime
Rest Admin API
State
Output Adapters
Files
REST Client
Custom
Authoring Tool Web UIRest API
Push EventsREST Service
Engine StateDefinitions Repository
http://forge.fi-ware.org/plugins/mediawiki/wiki/fiware/index.php/Complex_Event_Processing_Open_RESTful_API_Specification
![Page 16: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/16.jpg)
16
Runtime Main Interactions (1) – Input & Output Adapters
ProtonRuntime
Input
Events
Input Adapters
Files
REST Client
Custom
Output Events
ProtonRuntimeProtonRuntime
Output Adapters
Files
REST Client
Custom
Producers Consumers
![Page 17: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/17.jpg)
17
Proton (CEP GE Instance) – Rest Input Adapter
• Rest Input Adapter Client– Has a Producer definition in the Web UI
• Activates a REST service as a client – Access the REST web service declared by the event producer
and pull events using the GET method (pull approach)
Supported formats:– Tag delimited:Name=ShipPosition;ShipID=RTX33;Long=46;Lat=55;Speed=4.0;Time=13330330;
– Json:{"Name":"TrafficReport", "volume":"1000"}
![Page 18: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/18.jpg)
18
Proton (CEP GE Instance) – Rest Output Adapter• Rest Output Adapter Client
– Has a Consumer definition in the Web UI• Activates a REST service as a client
– Access the REST web service declared by the event consumer and push events to the consumer using the POST method
Supported formats:– Tag delimited:Name=TrafficReport;Certainty=0.0;Cost=0.0;EventSource=;OccurrenceTime=null;
Annotation=;Duration=0.0;volume=1000; EventId=40f68052-3c7c-4245-ae5a-6e20def2e618;ExpirationTime=null;Chronon=null;DetectionTime=1349181899221;
– JSON (flat):{"Cost":"0.0","Certainty":"0.0","Name":"TrafficReport","EventSource":"","Duration":
"0.0","Annotation":"", "volume":"1000","EventId":"e206b5e8-9f3a-4711-9f46-d0e9431fe215","DetectionTime":"1350311378034"}
– NGSI XML – For sending output events to the Context Broker
![Page 19: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/19.jpg)
19
Runtime Main Interactions (2) - Push events REST API
ProtonRuntime
Input
Events
Input Adapters
Files
REST Client
Custom
Output Events
ProtonRuntimeProtonRuntime
Output Adapters
Files
REST Client
Custom
Rest API
Push EventsREST Service
![Page 20: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/20.jpg)
REST API for event push
POST http://{host}:{port}/ProtonOnWebServer/rest/events
Header: Content-Type: application/json
Data: {"Name":"TrafficReport", "volume":"1000"}
Supported event formats:•Tag delimited•JSON•NGSI-XML – allows to get events from the Context Broker
![Page 21: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/21.jpg)
21
Runtime Interactions(3) – REST admin API
ProtonRuntimeProtonRuntimeProtonRuntime
Rest Admin API
State
Authoring Tool Web UI
Engine StateDefinitions Repository
![Page 22: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/22.jpg)
REST Admin APIs
• Managing the Definitions Repository
– GET all the existing definitions in the repository – GET specific definition in JSON format– POST a new definition– PUT update for a definition– DELETE definition
![Page 23: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/23.jpg)
REST Admin APIs
Administrating runtime Instances
GET instance status
PUT instance status
Status:– The instance’s definition URI– The instance’s state (stopped or started)
![Page 24: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/24.jpg)
CEP GE Documentation• Start from the FI-WARE Catalogue
• Under the Complex Event Processing (CEP) - IBM Proactive Technology Online GE you can find links to:
– Open specification - Restful API– The CEP user guide and programmer guide– The CEP Installation and administration guide– CEP open source code– CEP image– CEP tutorial
• The Context Broker- CEP connection instructions can be found in the CEP User guide (Appendix – page 40 and on) - https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/CEP_GE_-_IBM_Proactive_Technology_Online_User_and_Programmer_Guide
![Page 26: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/26.jpg)
BACKUP
![Page 27: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/27.jpg)
Pattern: Expected CrashWhen there is a sequence of 3 TrafficReport events, with more than 50% volume increase between each two events,while all arrived within 3 minutes after startup,derive an Expected Crash output event
Example
3 minutes after startup window
TrafficReport
TrafficReport
TrafficReport
Expected Crash
![Page 28: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/28.jpg)
ExampleEPA
Context
![Page 29: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/29.jpg)
Example
• Experimental Authoring tool URL:http://130.206.81.23:8080/AuthoringTool/Main.html
Example Project: DoS AttackEvent: Traffic Report (input), Expected Crash(output)EPA: Increasing Traffic (see previous slide)Temporal Processing Context – 3 minutes After Startup Consumer: Report to file (DoSAttack_PredictedCrash.txt)
• Export definition to definition repository
![Page 30: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/30.jpg)
Example
• Check engine instance state (Run status, Used definition file)
Expected Result: Status:200 OK
• Activate engine
Expected Result: Status:200 OK
GET http://{host}:{port}/ProtonOnWebServerAdmin/resources/instances/ProtonOnWebServer
{"state":"stopped ","definitions-url":"\/ProtonOnWebServerAdmin\/resources\/definitions\/DoSAttack2"}
PUT http://{host}:{port}/ProtonOnWebServerAdmin/resources/instances/ProtonOnWebServer{"action":"ChangeState","state":"start"}
![Page 31: FIWARE CEP GE introduction, ICT 2015](https://reader036.vdocuments.net/reader036/viewer/2022062823/5876e9341a28ab046d8b6c11/html5/thumbnails/31.jpg)
Example• Post 3 input events
Expected Result: Status:200 OK
• Examine output file:– Tomact-Installation-dir/sample/DoSAttack_PredictedCrash.txt See
log file:– Tomact-Installation-dir/logs/catalina.out
POST http://{host}:{port}/ProtonOnWebServer/rest/eventsHeader: Content-Type: application/json
Data: {"Name":"TrafficReport", "volume":"1000"}Data: {"Name":"TrafficReport", "volume":"1600"}Data: {"Name":"TrafficReport", "volume":“2500"}