Big Data

…..and its Big Opportunity

What is Big Data?

Volume

Variety

Velocity

800 EB in 2009

2PB/day

Video/ImagesTextNumericalAnalog: Voice calls

The Data Lifecycle

Capture Store Analyze Insights Act

Maturity

Valu

e

High

High

Proprietary systems

The Big Insights Engine

One click, machine learning enabled insights– Interoperability with data sources– Ability to process varied data types– Ability to rapidly perform statistical analysis and

choose winner– Automatic data visualizations of key drivers– Identify anomalies and trends– Ability to feed the data out to other systems

which can act on triggers

The Challenges

Semi Structured Data

Pre-determined list of problems/Patterns- Modeling to specific result or known usecase

Training models takes huge amount of data- Disjoint training, validation and test

sets

Compatibility with existing systems- Disparate data sources within organizations- Ability to consume the results

Big Data: Industry Opportunity

Maturity

Prod

. Im

prov

emen

t

$50B

$80B–$100B

$160B-$200B

$100B

$300BRetail

Financial Services

Utilities

Telecomm

Healthcare

Government & Education

Manufacturing

High

High

Source: Big Data – The next frontier by Mckinsey, 2011

Leader’s Quadrant

Completeness of vision

Abili

ty to

Exe

cute

APPENDIX

SQL vs NoSQL Databases

Traditional Databases• Difficult to scale• Transaction overhead – Inefficient joins– ACID causes latency

• Not optimal at handling diverse data types

• Easy integration with existing BI tools

NoSQL Databases• Easy to scale using cheap

hardware• Distributed parallel

processing of job• Schema independent

enabling semi structured data storage

• Batch oriented – not ideal for real time analytics