5 big data use cases for 2013

42
5 Big Data Use Cases for 2013 Tim Gasper Director of Product Infochimps, Inc. #5usecases

Upload: infochimps-a-csc-big-data-business

Post on 22-Apr-2015

7.840 views

Category:

Documents


1 download

DESCRIPTION

Jump start into 2013 by exploring how Big Data can transform your business. Listen to Infochimps Director of Product, Tim Gasper, cover the leading use cases for 2013, sharing where the data comes from, how the systems are architected and most importantly, how they drive business insights for data-driven decisions.

TRANSCRIPT

Page 1: 5 Big Data Use Cases for 2013

5 Big Data Use Cases

for 2013Tim GasperDirector of ProductInfochimps, Inc.

#5usecases

Page 2: 5 Big Data Use Cases for 2013

#1 enterprise cloud for big datasome of our customers our partners

#5usecases

Page 3: 5 Big Data Use Cases for 2013

BI & DataVisualization

Big DataApplications

Reporting &Ad-hoc Business

Questions

#5usecases

Page 4: 5 Big Data Use Cases for 2013

poll

#5usecases

Page 5: 5 Big Data Use Cases for 2013

poll

#5usecases

Page 6: 5 Big Data Use Cases for 2013

#5usecases

Page 7: 5 Big Data Use Cases for 2013

81% of companiessay Big Data is a top 5IT priority in 2013

#5usecases

Page 8: 5 Big Data Use Cases for 2013

#5usecases

Page 9: 5 Big Data Use Cases for 2013

CRM/customer supportPOS/purchasesERP/accountingemail/documents/collab.BI & data warehousesystem & network logsweb logs/clickstreamgoogle analytics/omniturefacebook/twitteryelp/foursquare/googleexperian/epsilon/acxiommobile devicessensorsproduct reviewsgoogle search results+ more

?many terabytes of data,

sometimes many petabytes

more data than ever before

#5usecases

Page 10: 5 Big Data Use Cases for 2013

BIG• volume• velocity• variety• variability

DATA• scalable• intelligent• agnostic• holistic

#5usecases

Page 11: 5 Big Data Use Cases for 2013

poll

#5usecases

Page 12: 5 Big Data Use Cases for 2013

#5usecases

Page 13: 5 Big Data Use Cases for 2013

#5usecases

Page 14: 5 Big Data Use Cases for 2013

14

#5usecases

Page 15: 5 Big Data Use Cases for 2013

1risk analysis and fraud detection

#5usecases

Page 16: 5 Big Data Use Cases for 2013

comprehensive data picture• build comprehensive data picture of customer-

side risk• publish a consolidated set of attributes for

analysis• add additional context, both internal and

externalparse and aggregate data from different sources

• credit and debit cards, product payments, deposits and savings

• banking activity, browsing behavior, call logs, e-mails and chats

merge data into a single view• a “fuzzy join” among data sources• structure and normalize attributes• sentiment analysis, pattern recognition

#5usecases

customer risk analysis

Page 17: 5 Big Data Use Cases for 2013

#5usecases

Page 18: 5 Big Data Use Cases for 2013

activity records in a central repository• centralized logging across all execution

platforms• structured and raw log data from multiple

applicationspattern recognition to detect anomalies/harmful behavior

• feature set and timeline vector are very dynamic

• “schema on read” provides flexibility for analysis

data is primarily served and processed in HDFS with MapReduce

• data filtering and projection in Pig and Hive• statistical modeling of data sets in R or SAS

#5usecases

surveillance & fraud detection

Page 19: 5 Big Data Use Cases for 2013

#5usecases

regulatorycompliance

Source:http://virtualization.sys-con.com/node/101598

Page 20: 5 Big Data Use Cases for 2013

ingest datasearch &

legal discovery

intraday analysis& historical analysis

(production reports +exploratory risk modeling)

TradingData

CustomerData

global investment bank#5usecases

trade risk

Page 21: 5 Big Data Use Cases for 2013

#5usecases

brand and sentiment analysis

2

Page 22: 5 Big Data Use Cases for 2013

the internet generates a lot of chatter about brands

• understanding what’s said is key to protecting brand value

• facebook & twitter generate a flood of data for large brands

capturing and processing direct feedback• better engagement and alerting via sentiment

analysis• integration with other customer service

systemshadoop handles the diverse data types and processing

• sources of data changing and semantics continuously evolving

• sophistication of algorithms is iteratively improving

#5usecases

brand & sentiment analysis

Page 23: 5 Big Data Use Cases for 2013

ingest datasearch &

application

trend analysis

SocialMedia

TraditionalMedia

large media conglomerate#5usecases

News, Blogs, etc.

real-time sentiment,influence, gender,

topic extraction, etc.

Page 24: 5 Big Data Use Cases for 2013

customer insights/behavior

3#5usecases

Page 25: 5 Big Data Use Cases for 2013

#5usecases

Page 26: 5 Big Data Use Cases for 2013

understanding customer behavior and preferences

• rapidly test and build behavioral model of customer

• combine disparate data sources (transactional, social, etc.)

structure and analyze with Hadoop• traversing usage and social graphs• pattern identification and recognition to find

indicatorsfeature extraction to find root causes

• defining attributes and modeling statistical significance

• combinations and sequence of attributes + actions factor in

customer churn analysis

#5usecases

Page 27: 5 Big Data Use Cases for 2013

comparison shopping is making retail hyper-competitive

• discount programs, e-mail correspondence entice shoppers

• brand loyalty means attention to detail and service

customer lifecycle is more than purchases• browsing and online data used to capture

customer attention• loyalty programs bridge the gap between

purchasesreach into online channels

• online engagement is personalized just as in store

• connecting online and in store shows customer awareness

customer loyalty

#5usecases

Page 28: 5 Big Data Use Cases for 2013

ingest datacustomer insight

reports

shopping patternrecognition

Demographics,Geography,

Web Data, etc.

Point Of SalePurchase Data

#5usecases

customer segmentation

Page 29: 5 Big Data Use Cases for 2013

targeted marketing and personalization

4#5usecases

Page 30: 5 Big Data Use Cases for 2013

the checkout lane is everywhere• cookies track users through ad impressions• purchasing behavior is time sensitive

logs collected online and offline• data is ingested incrementally• process happens at a variety of time scales

data logged into HBase and primary store• some events naturally associate, others require

deeper analysis• insights implemented via application logic

targeted offers

#5usecases

Page 31: 5 Big Data Use Cases for 2013

collect and serve personalization information• wide variety of constantly changing data

sources• data guaranteed to be messy

data ingestion includes collection of raw data• filtering and fixing of poorly formatted data• normalization and matching across data

sourcesanalysis looks for reliable attributes and groupings

• interpretation (e.g. gender by name)• aggregation across likely matching identifiers• identify possible predicted attributes or

preferences

recommendations & forecasting

#5usecases

Page 32: 5 Big Data Use Cases for 2013

ingest data

pre-definedweb content

and deals

behavioralcluster analysis

ClickstreamData from Online

Storefront

#5usecases

major apparel brandtargeted discounts

Page 33: 5 Big Data Use Cases for 2013

big data business intelligence

5#5usecases

Page 34: 5 Big Data Use Cases for 2013

poll

#5usecases

Page 35: 5 Big Data Use Cases for 2013

#5usecases

Page 36: 5 Big Data Use Cases for 2013

#5usecases

Page 37: 5 Big Data Use Cases for 2013

#5usecases

traditional datawarehousing

Page 38: 5 Big Data Use Cases for 2013

#5usecases

big datawarehousing

Page 39: 5 Big Data Use Cases for 2013

#5usecases

big datawarehousingThe Infochimps Approach

Page 40: 5 Big Data Use Cases for 2013

#5usecases

big data exploration & visualization

Page 41: 5 Big Data Use Cases for 2013

ingest data

SQL analysiswith Hive & Hue

Retail SiteWeb Logs

BI dashboarding

#5usecases

popular online deal sitebusiness command center