Download - Realtime Bigdata Use Cases for Retail
Catch Them in the Act: CEP for Realtime Ecommerce
InfluenceSrinath Perera
Director, ResearchWSO2 Inc.
Retail• World 7B People
– 6.5B have a phone– 2.5B online
• Global E-commerce itself was 1.5 Trillion market (215$/year for everyone in the world).
Usecases
Recommendations • One of the main retail
scenarios, can be extended to targeted marketing and deals
• Search let us make sense of unstructured Web
• In retail, there are too many choices, Recommendations help us make a decision.
A Main driver of traffic (e.g. 70% amazon site is devoted for recommendations, 70% Netflix sales though recommendations)
Finding Recommendation
• Problem: based on customer activities (purchases, ratings, views) etc., find out products that are of interest to him– Done by finding similar items or similar users (common
case)
• There are often Millions of customers and even millions of items
Recommendations (Collaborative Filtering)
What if you can freeze time!!• Most solutions are overnight • Think how you would buy something!
Research a bit and buy, often overnight is too late.
• But not all trends takes time – People change their mind – Trends move fast – React to what customer is doing (do not let him
move away)
• At CEP speed 400k/sec, if each event takes a second, it takes 4 days to pass a second in real world!!
Big data Processing Technologies Landscape
Real-time Analytics
• Idea is to process data as they are received in streaming fashion
• Used when we need – Very fast output – Lots of events (few 100k to millions)– Processing without storing (e.g. too much data)
• Two main technologies– Stream Processing (e.g. Strom,
http://storm-project.net/ )– Complex Event Processing (CEP)http://wso2.com/products/complex-event-processor/
Complex Event Processing (CEP)• Sees inputs as Event streams SQL like Queries language • Supports Filters, Windows, Join, Patterns and Sequences • Fast, 300K+/sec over network per node!
from p=PINChangeEvents#win.time(3600) join t=TransactionEvents[p.custid=custid][amount>10000]
#win.time(3600)return t.custid, t.amount;
Lambda Architecture with WSO2 Products
WSO2 Big Data Platform
Demo Usecases
• Operations Dashboard • Automatic Offers• Recommendations
based on personal trends
• Recommendations based on overall trends
Architecture
Demo
Usecase 1: Dashboard
• CEP will do realtime analytics – Most queries are look very SQL like
• Then push updates to Dashboard.
from transactionStream[metadata== 'purchase' or metadata== 'select']#window.time(1 min)
select tag, count(*) as count group by tag
Recommendations (Collaborative Filtering)
Realtime Recommendations
• As part of Collaborative filtering, data is broken into similar groups– E.g. Via collaborative filtering – This is done batch style, e.g. once a day
• For realtime matching, we will search only within that group
Usecase 2 & 3: Realtime Recommendations with CEP
Usecase 4: Automatic Offers
• Often pretty straight forward CEP scenario! Track and trigger when conditions match.
define partition transactionStream.customerID{from transactionStream#window.time(1 week)
having sum(amount) > 20 or count (*) > 3
insert into OfferStream}
Usecases
Thank You