[wso2con eu 2017] open interoperability of wso2 analytics platform

Director - Solutions Architecture, WSO2

Open Interoperability of WSO2’sAnalytics Platform

Shiroshica Kulatilake

Let’s Meet a Few Everyday Characters...

● Senior Operations Manager at a sports equipment manufacturing factory

● Should○ Make sure floor operations are

complete○ Be aware of any outages in stocks ○ Project raw material needs○ Decide on improving efficiencies

● Uses○ Dashboards for monitoring

operations○ Email and SMS alerts for urgent

notifications

Agnes

Informed & Proactive

Connected & Taken Care Of

Patrick● Retired School Master who’s had a fall apart from usual health concerns

● Requires to○ Know when his next appointment

will be in advance ○ Monitor daily blood sugar and

notify his doctor○ Get monthly prescription of

medicine

● Uses○ SMS alerts for appointment

reminders○ A connected device to measure

and view updates

Agnes

David● Senior Business Executive who travels

frequently to meet business counterparts

● Needs information on○ Possible delays at the airport○ Average predicted waiting time before

boarding○ Changes to latest boarding gates

● Uses○ A travel application that provides a

personalized set of data ○ SMS and email notifications

Updated & Punctual

What’s Common Among All of Them?

They are Exposed to Relevant and Contextual data

How has this been made possible ?

They are Connected to a “System” that’s Driven by Analytics

What Does this Mean?

Analytics is part and parcel of your Life

Analytics is part and parcel of your Business

Is this Aligned with what Analysts say?

2018: 50% of large organizations will compete using Advanced Analytics

causing disruption in entire industries- - Gartner

2021: Insight Driven Businesses will earn $1.8 trillion - Forrester

The Big Data Tech Market will grow 3 times more than Overall Tech Market

- Forrester

How is this relevant to YOU ?

Your Business Provides such ‘Systems’ which are Driven by

Analytics

What are the Prerequisites to do so?

Your Business should have a State-of-the-art Analytics

Component

1

This Analytics Component Should be Easy to Integrate and be

Interoperable

2

• Key expectations from an analytics platform• Meeting those expectations with WSO2

technology• Interoperability in real-world use cases

Discussion Outline

Identifying Your Expectations

• To make better decisions– Operational and business monitoring

• Effectively communicate information– Visualization – Alerting mechanisms

• Offer new value propositions– Insight driven products and services

Business Perspective

• Non-intrusive integration– Pull data from heterogeneous data sources and

analyze– Push results to existing flows with no

interruptions

• Intrusive integration– Receive input from multiple flows, systems and

sources of data– Push results to existing flows and systems

Solution Perspective

• Interoperability in receiving data to analyze– Multiple protocols and formats– Pre-process before analyzing

• Interoperability in communicating results– Multiple protocols and formats

• Interoperability in data analysis– Integration with existing metadata and models– Extend with new analysis capabilities

Product Perspective

Meeting Those Expectations

Steps of an Analytics SolutionThe Typical Components

- Define data model- Specify Schema- Pick a protocol- Transform as needed- Ingest events

Analyze Data

- Select type of analysis needed

- Configure relevant analytics components

- Analyze received Events

- Define result schema definition

- Pick communication mode

- Communicate Results

Collect Data Communicate Results

To Be “Interoperable,” Each Step Should Be Easy to Integrate

Interoperability in Data ReceivingSupporting Multiple Transports and Formats

XMLJSONTextMap

Binary

Custom WSO2 Data Agent

JVM Based Systems

JMX Agent

File Systems

Log Agent

WSO2 Analytics

Event Receivers

WSO2 products

WSO2 Event Data Agent

WSO2 Enterprise Integrator

Publish Event Mediator

Connectors

...

Other Systems

Standard Protocols

Collect

Interoperability in Data ReceivingPre-Process Incoming Data Before Analysis

JVM Based Systems

JMX Agent

File Systems

Log Agent

WSO2 Analytics

Receive and Transform

WSO2 products

WSO2 Event Data Agent

Other Systems

Standard Protocols

Custom WSO2 Data Agent

WSO2 Enterprise Integrator

Publish Event Mediator

Connectors

...

Collect

File and Database

Interoperability in Data ReceivingPre-Process Incoming Data Before Analysis

WSO2 Analytics

Receive

WSO2 products

WSO2EventConnector

WSO2 Integration

Transform

Legacy SaaS products

Connectors

Other Systems

Standard Protocols

Publish Event Mediator

Connectors

...

Collect

• Lookup metadata stores as Event Tables• Join event tables while processing• Update data stores while processing

Interoperability in Data AnalysisIntegration with Existing Data Stores

Analyze

Interoperability in Data AnalysisIntegration with Existing Models

• Import prediction models in PMML form• Load prediction models and use these while

processing• Import Spark scripts for batch processing

Analyze

Interoperability in Data AnalysisExtend Additional Analysis Capabilities

• Add new functional capabilities as extensions to the underlying processing engine

• Add the ability to run external scripts through integration extensions

Analyze

Interoperability in Result CommunicationMultiple Transports & Formats, Alerting and Storage

Custom WSO2 Data publisher

RDBMS / Cassandra

Data store

JMS / Kafka message bus

JMS / Kafka

WSO2 Analytics

Event Publishers

Email Alerts / Notifications

SMTP server

WSO2 Enterprise Integrator

Proxy or http endpoint

Connectors

...

Communicate

• Ability to build and deploy custom gadgets for existing dashboards

• Ability to integrate with third-party visualization tools

• Ability to integrate with third-party reporting tools

Interoperability in Result VisualizationIntegration with Existing Systems and Dashboards

Communicate

A Peek into The Real World

Use Case: “How Are My Patients Doing Today?”

• Interoperability in receiving data

• Interoperability through adapting to existing deployment pattern

Use Case: “Would You Also Like to Order This?”

• Interoperability in receiving and communicating

• Interoperability through integrating with existing data stores

Use Case: “Will I Be On Time For My Flight?”

• Interoperability through receiving data

• Interoperability through integrating with existing data stores

Use Case: “Am I Subjected to Fraud?”

• Interoperability in the real-time engine to be used as an embedded library

Use Case: “Are We Lagging Behind in Ops?”

• Interoperability in data receiving through multiple formats and pre-processing

• Interoperability in building visualization extensions

wso2.com