wso2con eu 2015: reference architecture for eda
TRANSCRIPT
Event Driven Reference Architecture
John Mathon VP, Enterprise Evangelism
and Product Strategy WSO2 Inc.
“How do you share information between users and applications in a networked
world?”
Platform 1 Platform 2 and 3
Publish/Subscribe Dream
Joe Mary Frank Exchange Clearing
Event Driven Computing was born
Subject Filter Subject Filter Subject Filter Subject Filter Subject Filter
Real-Time Simultaneous Delivery to all Subscribers
Auto-Discovery and Inclusion
Self-Describing Data
What Point-to-Point looks like after a while Extremely Brittle Extremely NON-AGILE
Alternative EDA – Google - Polling has delays
Web Site A
The Cloud
Search
Indexed Site A
For Google it can still be a month before you are indexed
My dream: What if the cloud was Publish/Subscribe?
Web Site A
The Cloud
Search A
Traffic Data GPS info
Mapping Yelp
The Common EDA Pattern - Hub and Spoke
Message Broker
ServerA
AppC
ServerB ServerC
AppB AppA
Technology is in Rapid Evolution
100 Billion API calls/minute
20,000 Public/80,000 Private APIs
Projects doubling Every 18 months
1.5 Billion Smartphones
$100 Billion IaaS, PaaS, SaaS, iPaaS, BaaS, DaaS, …
Billions of Users
& IoT
Advantages of Publish/Subscribe Event Driven Architecture
• Reduced Complexity – Easily mediated to many purposes – Easier Programming – Reduced Complexity for Publisher and Subscriber
• Multiple Levels of failure recovery
– Local distribution even if network slow or disconnected – Action Locally when possible – Increased Redundancy
• Reuse of Events – Things become current – Things become smart
• Shared Intelligence – Reduced complexity of individual service
• Reduced Load – Only responsible to deliver once
• Higher Agility – Easy to replace services – Easy to provide multiple versions of services
EDA spurred a set of standard tools Platform 2 Distributed SOA EDA
The Network
Mediation/ESB Activity
Monitor
Message Broker
Business Process
Data Services Registry Discovery
Pattern Detection Complex Event Processing
Logical Completeness – Use the Best Tool
Dimensionality X: Stateful or not Y: batch or real time Z: low change or high change X’: Long or short running
Platform 3 The Connected World
• Mobile – 1.5 Billion Smartphones – 600,000 Mobile Apps
• IoT – Billions of devices • Social – Billions of users • APIs – 100 Billion API Calls/Minute • Devops / PaaS – deliver new software daily • Cloud – IaaS, PaaS, SaaS, iPaaS, CaaS, BaaS, DaaS - $130 billion and growing fast
What is needed is a glue for NEW and OLD technology
Cassandra/Hbase/Mongo
Social API/App Store
API Management
IoT
Platform 3 +API Management, BIG Data, Cloud, Social, Mobile,
Internet of Things, Device Management, Security everywhere
Business Process
Pattern Detection Complex Event Processing The Cloud
Activity Monitor + SOCIAL
BIGDATA
Data Services RDB +
BigData
Mediation/ESB + APIs + Cloud
Adapters Social
Message Broker + BigData
Gov Registry + Social Enterprise Store + Mobile Mgt
IOT Paas / SaaS / DevOps Cloud
Integration
Identity Management
API Management Mobile Device Management
Non-Unitary Evolution
• Cloudification - Virtual Enterprise Evolution
• API Evolution
• Mobile / IoT Evolution
• BigData – Digitization Evolution
Cloudification – Virtual Enterprise Evolution
• Mostly this is about cost reduction – SaaS Applications – Personal Cloud Services – IaaS – PaaS – iPaaS – Security and Identity Management – Cloud Management – Best Practices
Complete Cloud Platform
Identity and Access Management
Enterprise App Management
It’s like a catalogue!
API Evolution
• Mostly this is about top line growth and agility – Re-factor the Enterprise – Existing Services made into “API’s” – Build Mobile Apps – Offer Services to external parties – Integration – Reuse
APIs Taking Over 100 Billion API Calls/Minute
Uninteresting App if it doesn’t use the Cloud
Smartphone1 In a car
Smartphone2 In a restaurant
Waze Uber Yelp
Medical Monitor Service
Smartphone4 On the street
Smartphone3 In a taxi
Tesla Glucose Monitor
Drone GPS
Emergency Vehicle
Search
United Airlines
API Management – A must
• External API Delivery – You can hack this yourself but will ultimately want a real API Management solution
• Internal API socialization and reuse – Enterprise Refactoring – Key to Enterprise Renewal, increasing innovation
• Management of External APIs – What are allowed services to use? Governance
• Management of all services – Quality of Service, Load, accounting, insight into users
Reuse/Re-Factoring EnterpriseAPI Management
Publish and Socialize to facilitate Reuse
BigData – Digitization Evolution
• Increase Top Line – Increased sensitivity to customer – Understanding the customer – Reaching the customer – Maximum efficiency in marketing and sales efforts – Better serving the customer
BigData Lambda Architecture Best Prac>ce / Best of both worlds
http://srinathsview.blogspot.co.uk/2014/03/implementing-bigdata-lambda.html
Mobile / IoT Evolution – 3 Use Cases!
• For the Factory / Enterprise – Cost Reduction / Improved Efficiency / Automation
• For the Employees – Productivity of Employees
• For the Customer – Connected Business – New Interaction Paradigms – Increase Stickiness
Event Driven Computing New Era
Platform 3
“How do you share information between users and applications in a connected Social IoT Mobile world?”
Social IoT Mobile Apps
IoT and Mobile Mgt
API Mgt and Reuse
BigData
Mediation / Broker
Micro Services
Analytics
PaaS/DevOps
Security
Perf Mgt Scaling
EDA Publish / Subscribe Mesh Network
IoT
IoT
IoT
IoT
IoT
IoT
IoT
IoT
IoT
Router1
Router2
No Single Point of Failure / Like the Internet Functionality = Combination of all Services Mesh1
Mesh2 Mesh3
Cloud Control and Aggregation
Mesh1
The Cloud Mesh2
Customers
Employees
UI
IOT Reference Architecture
Device
API Mgt Dev Mgt
Service Bus BPM BigData
Long Term Analysis
Activity Mgt
Cloud
CEP
Cloud Orchestration/Integration
Communication
Prov ider
OS
Application
ESB, Security
OS
ESB, IM, PEP
Data Infrastructure
Hub Orchestration Integration
Rules Engine
Data
Dev Mgt
Rules Engine
User Dashboard
IoT Mgt Dashboard
IoT Reference Architecture
EDA Example Architectures
IoT, Mobile
Trading Floor
Trader Market Data Visualization
Risk Analytics Market Data Source
Trader Order Placement Trader Analytics Visualization
Order Processing
Message Broker / Information Bus
What’s EDA on a trading floor?
Message Broker Topics
Mediation/ESB
DSS to RDB
Data Feed Data
Feed Data Feed Data
Feed Data Feed Data
Feed
Trader Order Entry
Trader Market View
Trader Analytics
Message Broker Queues
Activity Monitor
Activity Logs
Complex Event Processor
Risk Desk
Operations Desk
Business Process
EDA Architecture: Ufer Taxis
Uber User App
Uber Driver App
Meetup Mobile App
United Airline Mobile App
Mediation/ESB
Business Processes
DSS to RDB
Message Broker
Payments Square / Zuora … Activity
Monitor
DSS Big Data Activity Log
API and App Store
Analytics Peak Demand Pricing Dispute Handling City Configuration Reserve Service Monitor Service
Driver Onboard Driver Rating Customer Onboard Driver Interaction Driver Selection Driver Payments Customer Payments Avail Query
Online Sales
Mediation/ESB
DSS to RDB Inventory
SKU / UPCs Pricing
Consumer Web App
Partner / OEM
Message Broker Queues
Activity Monitor
Activity Logs
Complex Event Processor
Consumer Mobile App
Business Process
App Server
Shipping
API Management
Health Enrollment –Insurance State or HMO
Mediation/ESB
DSS to RDB Patient Data
Patient Web Application
Patient Data Sources
Message Broker Queues
Activity Monitor
Activity Logs
Complex Event Processor
Patient Mobile Application Business
Process
App Server API Management
Patient Medical Records
Patient Data Sources
Patient Data Sources
Connected Car
API Mgt Dev Mgt Upgrade
Cust Svc Service Bus
Design
BPM
BigData Long Term Analysis
Activity Mgt
Service Bus
Polling Logging Events
Brakes Brakes Brakes Brakes
Console
Phone
Watch
3rd Party App / Device 3rd Party App /
Device 3rd Party App / Device
Brakes
Hi Priority
Planning
Lo Priority Apps
Cell Ven Cloud
CEP - Service
Connected Construction
API Mgt Dev Mgt
Cust Svc
Service Bus
Design
BPM
BigData Long Term Analysis
Activity Mgt
Phone
Vehicles
Tools
Hi Priority Planning Lo Priority
Cloud
CEP
Building Sites
Vehicles Vehicles
Phone Phone
Tools Tools
Tools
Data Gather Device Data Gather
Devices
Wharehouse Wharehouses
Tools
Data Gather Device
Contractors
Design Contractor Planning
Tool
Connected Construction
WSO2 “Platform 3.0” Connected Enterprise Architecture Gives you:
• Event Driven everywhere respond instantly • Legacy Integration and renewal • APIs instantly, managed and scalable • Cloud Architecture and Deployment Flexibility and Speed • Big Data Everywhere • Easy Integration to Social • Export APIs • Easy Integration to payments, social, gps
WSO2 Advantage: 75 % of your service is
already written and integrated with the latest
new technology from BigData, Cloud, Social,
APIs is scalable and secure
Connected-Health Reference Architecture
About the Author
• 10 patents, publish / subscribe, multipath low latency high reliability messaging, Content management in file systems, peer to peer messaging over WAN… others
• founder and CTO of TIBCO • WSO2 Enterprise Product Strategy and Evangelism
• Twitter: @john_mathon • Blog: CloudRamblings johnmathon.wordpress.com
Thank You!