big data for product managers
TRANSCRIPT
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 1
Ben Hopkins Product Marketing Manager - Pentaho
Jim Stascavage Vice President of Engineering - ESRG
Big Data for Product Managers
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 2
① Data Trends & Data Types
② Big Data Challenges & Technology Solutions
③ Big Data & Product Innovation
④ Case Study: ESRG
Quick Agenda
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 3
Pentaho We Enable & Empower Data-Driven Businesses
Customer momentum • Over 1,500 commercial customers
• Over 10,000 production deployments
Innovation through open source • Open, pluggable, purpose-built for the future
• Sustained leadership in Big Data ecosystem with technology innovation
Modern, cohesive business analytics & data integration platform • Full spectrum of analytics for all key roles
• Embeddable, cloud-ready analytics
• Broadest and deepest Big Data integration
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 4
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
40000 30000 20000 10000
Exa
byte
s of
Dat
a
Source: IDC’s Digital Universe Study, sponsored by EMC, April 2014
We are ONLY here!
77% of data relevant to enterprises will be unstructured
At the Beginning of a Data Revolution
40% Machine Data
50X Growth
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 5
The Most Compelling Insights Come from Blending Data
38% 38%
16% 4% 4%
Existing Underutilized “Dark Data”
More Customer & Supplier Detail
Social Media Content
Commercially Available Data
Publicly Available Data
+ + + +
Big ROI Opportunity
“Which source of data represents the most immediate opportunity to transform your business?”
Summary of Analyst Surveys on Big Data: Gartner, Forrester and Ventana Research
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 6
Traditional ERP, CRM, and transactional data
Web logfile, clickstream and social media post data
Human language, text, email, audio, video, and image data
Sensor, machine-to-machine, network, and geospatial data
More Data Types of Interest
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 7
Addressing New Challenges
Flexibility to land a wide variety of data types in the data store thanks
to schema-on-read
Highly varied data structures are difficult to bring into relational DBs
and blend for analysis
‘Divide and conquer’ with distributed storage and processing on
affordable hardware
Explosive growth in data volume is straining existing data warehouse
infrastructure
Leverage high performance random read/write access, streamline
analytic queries for speed
Time-sensitive data is being rapidly generated, and there is pressure to
deliver faster analytics
Big Data Challenges Technology Solutions
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 8
Big Data Technologies Not an Exhaustive List
Hadoop • Distributed file
system and MapReduce framework
• Ideal for high volume, diverse data processing
NoSQL • Broad group of
DBs with flexible data models: graph, key/value, document, etc
• Often ideal for rapid ingestion, random read/write access
Analytic DB • Relational DB
designed for high performance BI
• Ideal for complex analytic/OLAP queries
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 9
Why Does it Matter for Product Management?
New Opportunities for Valuable Products
Potential for Built-in Intelligence
Future-Proof for Scale and Growth
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 10
Big Data Products
Source: Aaron Kimball, “The secrets of designing and building big data apps,” venturebeat.com, 12/24/2013
• Data structured in pre-defined
(relational) way
• Designed for specific problem; data access, interfaces & protocols reflect that purpose
• Application data often isolated from other relevant data
• Example: CRM system for storing customer info, prioritizes calls based on purchase data
Traditional Application
• Accommodates various data
structures (and speeds) • Framework that can potentially
solve multiple problems
• Includes process for ingesting new data sources and building & iterating on predictive models
• Example: Adds in prioritization of calls from predictive model on customer behavior & purchase data
Big Data Application
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 11
To create completely individualized apps • Understand all relevant
user intents, actions
• Leverage mobile, behavioral, & profile data
• Incorporate more user data for a more complete view
• Predictive modeling to anticipate and respond
Big Data + Predictive Analytics
Source: “Predictive Apps are the Next Big Thing in Customer Engagement,” Forrester, 6/25/2013
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 12
Architecture Patterns We See
Weblog & social media data
Machine, sensor, & device data
Customer profile data
Existing application data
Unstructured & Semi-structured
Structured & Relational
Hadoop Cluster
Relational Database
NoSQL Store
Analytic Database
Client-side User Interface
Web-based
experience including embedded visual
analytics
NoSQL
‘Massive Archive’ ‘Operational Speed Layer’
‘Existing System of Record’
‘Powerful BI Performance’
© 2015, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 13
In Brief - Use Cases & Products
“Personalization Engines” • Using predictive analytics on web-
sized data sets to individualize customer relationships
• Example: RichRelevance delivers personalized content to customers online and in-store, based on Big Data predictive analytics on over 50 mln shopping sessions per day. • Tech: Hadoop, Hbase, Hive,
others
“Machine Data Analysis” • Monitoring and analyzing sensor
and network data to understand equipment/device performance
• Example: Ruckus Wireless leverages Big Data to provide decade-long analysis of semi-structured WiFi network data for telecom carriers and enterprises • Tech: Hadoop, Vertica, others
ESRG Customer Case Example Industrial Predictive Analytics
Jim Stascavage – VP, Engineering
Presentation overview
• Introduc)on
• Os)aEdge industrial predic)ve analy)cs and Pentaho integra)on
• Marine case example: Saving fuel and avoiding failure
• Liquid packaging case example: Predic)ng )me to failure
• Conclusion
15
Founded 2000 10+ years product development
History
Exper6se
Product
Reliability engineering So>ware & architecture Big-‐Data
Currently remotely monitoring 3,000+ assets daily Comprehensive, 6ered solu)on
Markets served
Defense & Commercial Mari6me, Process, Power Gen
ESRG overview
Focus & results
Turn data into ac6onable informa6on; examples: • $70,000 annual savings per gas turbine • Defer ~60% of in-‐person equipment assessments
Industrial Analytics to create significant value…
17
Cisco es)mates 50B devices connected,
crea)ng addi)onal $14T in profits over next
decade
McKinsey & Co. es)mates up to $6.2T in annual value
created annually by 2025
General Electric es)mates the market for industrial internet technology and services to grow to $500B by 2020
Sources: Manyika, James, others, “Disruptive technologies: Advances that will transform life, business and the global economy” McKinsey Global Institute, May 2013 Chambers, John, “Internet of Evertyhing”, Cisco, February 21, 2013 General Electric press release, June 18, 2013
…ESRG uses data to improve return on industrial assets
Source: Bringing the industrial internet to the maritime industry and ships into the cloud ;http://www.esrgtech.com/company/ESRGcontent/ 18
Avoid breakdowns & down6me
Op6mize maintenance
Improve environmental compliance
Improve opera6ons
Improve energy efficiency
Marine Vessel $500K-‐$1.5M Per year savings poten6al
Liquid filling 10-‐20% up6me
10-‐50% maint cost Per year savings poten6al
Our technology monitors 3,000+ assets
~3,000 total assets monitored
AC plants Hydraulic systems Compressors GT engines GT lube oil systems Diesel engines GT Generators Reduc)on gears/transmissions Refrigera)on Desaliniza)on Fuel flow meters Misc
3/24/15 ESRG Confidential 19
20
Mechanical equipment
Fuel/energy consump)on
Control & opera)ons
Emissions, Discharges, etc
1000s of sensors per asset… …across an enterprise …automated analy6cs avoid need for large Department to analyze “Big Data”
Automated analy6cs & Experts overcome Big Data challenges
Once per second data = 86,400 points/day 1000 data points = 2.6B points/month
100 ships = 3.1 trillion data points per year
Automated analy)cs provide fuel/energy efficiency and equipment health
ESRG uses analytics to turn Big Data into actionable information
OstiaEdge overview
21
On-‐site / Onboard Local
Central / Shore Central or Cloud
Plant Edi)on Central Edi)on Business Intelligence
• Local data acquisi)on & qualifica)on
• Local analy)cs and presenta)on
• Real-‐)me data viewer
• Machine state and condi)on analy)cs
• Embedded Keble (PDI) • Web/cloud presenta)on • Workflow management • Security/user mgt
• Dashboards & analyzers
• ETL • Email reports • Mobile • NEW: PDI: R & Weka
embedded
Example data flow and integration
3/24/15 ESRG Business Sensitive 22
Data Input
External
ETL PDI
RUL Algorithm & Engine
DSP R-‐Plugin
Customer maintenance
system
External
Os)aEdge Analy)cs
Os)aEdge Presenta)on
Analyzers & Dashboards
Custom Dashboards
BI Cube(s)
Email Reports
Configura)on Management
New
Saving Fuel and Avoiding Failure Case Example: Maritime
23
Op)mize generators
$50K-‐$250K Tune equipment
$50K-‐$150K Avoid failure
$10K-‐$500K
Large ship owner trying to reduce fuel & failures/down)me: • $5-‐10M in fuel cost per year • $10,000 per day for vessel down)me
Solu6on
Situa6on
Use Os)aEdge & embedded Pentaho ETL to make beber opera)ons & maintenance decisions • Embedded Pentaho ETL for generator op)miza)on & dashboards • Os)aEdge analy)cs for failure avoidance
New: Predicting Time to Failure Case Example: Liquid packaging
24
R & Weka based RUL algorithms
Predict Failure Standard & Custom Dashboards
Exec Transparency
Global liquid packaging OEM with two goals: • Improve customer up)me • Reduce unnecessary maintenance and extra parts consump)on
Solu6on
Situa6on
Leverage Pentaho PDI + Os)aEdge to predict Remaining Useful Life (RUL) • ETL to bring in enterprise level data • Data Science Pack (R & Weka) used to design algorithms to predict RUL • Customized embedded dashboard
Industrial Analytics Opportunity
25
Os6aEdge + Pentaho • ETL • Diagnos)cs • Analyzers &
Dashboards • New:
Prognos?cs with R & Weka
Lower Cost & Faster • Small team • Rapid & agile
algorithm development
• Easy integra)on • Flexible
implementa)on
Avoid breakdowns & down6me
Op6mize maintenance
Improve environmental compliance
Improve opera6ons
Improve energy efficiency
Q and A …
Ask Questions. Our team is standing by to help. The webinar slides will be posted to our website and our Slideshare.net/aipmm page. The webinar recording will be posted at AIPMM.net for members.
Product Management Body Of Knowledge
We will pick one winner from our attendees.
15 Compe66ve Intelligence Ques6ons that Product Managers Need To Ask Mar 13
AIPMM Webinar Series: hbp://aipmm.com/aipmm_webinars
Topic Sugges)ons: [email protected] Announcements: hbp://www.aipmm.com/subscribe LinkedIn: hbp://www.linkedin.com/company/aipmm Membership: hbp://www.aipmm.com/join.php Cer)fica)on: hbp://aipmm.com/html/cer)fica)on/
Upcoming Webinar