strata presentation
DESCRIPTION
Strata Presentation. Tim Garnto - SVP Engineering, edo Interactive Rob Rosen – Big Data Field Lead, Pentaho. Hello. About . Full-Spectrum Analytics Platform Open Source Heritage: Founded in 2004 Over 10,000 Production Customers Spanning 185 Countries - PowerPoint PPT PresentationTRANSCRIPT
SM STRATA PRESENTATIONTim Garnto - SVP Engineering, edo Interactive
Rob Rosen – Big Data Field Lead, Pentaho
HELLO.
3Proprietary and Confidential, edō Interactive, Inc.
ABOUT • Full-Spectrum Analytics Platform
• Open Source Heritage:• Founded in 2004
• Over 10,000 Production Customers Spanning 185 Countries
• Certified Interoperability with Widest Variety of Data Sources:• Hadoop
• NoSQL
• Traditional Structured Datasets
4Proprietary and Confidential, edō Interactive, Inc.
SIMPLIFIED ANALYTICS FOR ALL USERS
ANY Analytics• Reports• Dashboards• Visualizations• Discovery• Predictive
Analytics
ANY Environment• Data warehouses• Data marts• Stack vendors• Cloud• Embedded
Existing & New Data
Infrastructure & Processes
ANY Data• Relational• Operational• Big Data• Data sources not yet
anticipated…
Billing
Location
Social Media
Customer
Web
Network
5Proprietary and Confidential, edō Interactive, Inc.
EDO MISSIONWe’re transforming shopping and saving with the simplest solution to find merchants new customers with personalized offers available on consumer debit and credit cards
6Proprietary and Confidential, edō Interactive, Inc.
OUR STORY
• Founded in 2007 & Venture-backed
• Defining the intersection of payments & advertising
• Co-headquartered in Nashville and Chicago
• 2014 Brill Street Top 50 Employer
MAKING SHOPPING & SAVING SIMPLE
7Proprietary and Confidential, edō Interactive, Inc.
ECOSYSTEM
3 of the top 6 financial institutions
200+ Enrolled Banks
Total reach of 200 million cards
1000+ Retailers and Restaurants
Leading National Brands
Local Community Businesses
Banks and Issuers Advertisers
BUSINESS INTELLIGENCE
9Proprietary and Confidential, edō Interactive, Inc.
ISSUE• First generation BI platform was built on traditional data
warehouse star schema
• Business growth was constrained by platform scalability
• Warehouse build was not always completing resulting in stale data
• Data volumes quickly caused analytics and offer processing to extend beyond 24 hours to launch offers
• Total data volumes increasing with new partners and increased adoption
• Slow development cycle to generate new reports/insights/etc…
10Proprietary and Confidential, edō Interactive, Inc.
SOLUTION APPROACH• Implement Cloudera Hadoop
• Migrate data processing, analytic processing and offer assignment to use tools
• Leverage Pentaho Big Data capabilities
• Accelerate adoption with external help - OpenBI
11Proprietary and Confidential, edō Interactive, Inc.
IMPLEMENTATION PROCESS• Step 1 - Transitioned from Traditional Warehouse to Hadoop
using same processes and queries
• Step 2 – Optimize queries and processes for Hadoop in Pentaho
• Step 3 – Update processes to use Map/Reduce and Pig from within Pentaho
12Proprietary and Confidential, edō Interactive, Inc.
OUR BI PLATFORM
13Proprietary and Confidential, edō Interactive, Inc.
OUR DATA PIPELINE
14Proprietary and Confidential, edō Interactive, Inc.
ANALYTICS AND OFFERS
15Proprietary and Confidential, edō Interactive, Inc.
CONFIGURATIONTotal Size
Nodes 20
Cores 480
Memory 1.9 TB
Storage 255 TB
Node ProfileCores 24
Memory 96 GB
Storage 12 TB24 TB
VersionsCloudera Manager 4.8.0
CDH 4.5.0
Impala 1.2.3
Hadoop-LZO 0.3.15
Pentaho 5.0.3
RESULTS
17Proprietary and Confidential, edō Interactive, Inc.
DATA GROWTH
Jan 2013 July 2013 Jan 20140%
200%
400%
600%
800%
1000%
100%146%
256%
100%
697%
974%
100%
178%
361%
CustomerDataOffers
18Proprietary and Confidential, edō Interactive, Inc.
PROCESSING RESULTSDaily Warehouse Loads Analysis and Offer Process
Jan 2013 July 2013 Jan 20140.0
5.0
10.0
15.0
20.0
25.0
30.0 29.5
4.3 3.8
Hours Total Records
Jan 2013 July 2013 Jan 20140.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
29.5
4.5 5.0
Hours Weekly Offers
19Proprietary and Confidential, edō Interactive, Inc.
FINAL THOUGHTS• Pentaho Big Data Capabilities allowed edo to
quickly move to Hadoop
• Pentaho Map/Reduce and Pig Steps allow deeper integration through familiar workflow
• Pentaho Analyzer with Impala is fast reporting against billions of rows
Jan 2013 July 2013 Jan 2014
THANK YOU
edointeractiveEdoInteractiveEdo Interactive