big data - big benefit or big waste?

6
PERSPECTIVE 1 Big Data - Big Benefit or Big Waste? Your Big Data strategy will determine your success Big Data – Blessing or Curse? Google, Amazon and Facebook, three of today’s most powerful companies, have been built around Big Data. The information that flows through their veins is directly linked to their ability to analyze vast amounts of data. Exactly this capability makes them market leaders with a position that is hard to attack for any competitor. No surprise that by now, other companies across all industries have likewise started to gather information in hope to boost their business. T-Mobile USA is only one of many examples for which these efforts have already paid off. By analyzing huge amounts of data in real time with SAPs new in-memory database HANA, it is now possible to provide targeted offers to more than 21 million customers. This results in $10-25 savings for each won back subscriber and potentially in billions of additional revenue per year. In the saturated telecommunications market this will be a crucial advantage against the competitors. On the flipside, companies missing this analytical competence will see their market position significantly weakened with the threat of an attack of a much better prepared competitor always lingering around the corner. Since the amount of digital data doubles every five years according to recent calculations, this effect will further increase. By 2012 we have already been faced with about 2.6 Zettabytes of global data. Assuming that a human being is able to memorize 100 GB of data, it would need four times of the entire global population to store this information. With a CAGR of almost 45%, in 2013 already six times of the world population would be needed. This trend is boosted by the increasing distribution of networked devices and sensors in our daily life, smart phones, internet and the social media platforms. The key challenge is that the amount of data companies need to evaluate in acceptable time is nowadays growing faster than the performance of established database technologies and analytical tools (compare Figure 1). In addition, available data becomes more and more semi-structured (XML) and unstructured (documents, e-mails, videos, pictures, etc.) and hence more difficult to analyze. Figure 1: Growing amnesiaof enterprises 1960 1970 1980 1990 2000 2010 2020 2030 1,000,000 1,000 1 1,000,000,000 Analytical Speed 1) 1) # of Data Units that can be evaluated in acceptable time and money 2) # of company’s Data Units to be evaluated to stay competitive Source: xCon Partners analysis Evaluation Requirements 2) World Wide Web Email PC today Enterprise Social Media Internet of Things # of Data Units Data amount exceeds analytical performance resulting in amnesia The ability to rapidly analyze Big Data has become a key competitive advantage to manage the exponential growth of globally available information. Early adopters have already proven benefits both on the cost and on the revenue side. However, a well-thought-out Big Data strategy is crucial to leverage this potential without spending millions on oversized analytic capabilities. xCon Partners has coordinated several hundred Big Data projects for a major solution supplier, always supporting the fast and smooth project execution. We are your ideal partner to tackle the challenges involved in Big Data and to disclose its full potential by giving strategic guidelines as well as execution support.

Upload: lpoliakov

Post on 19-Jan-2015

295 views

Category:

Documents


2 download

DESCRIPTION

The ability to rapidly analyze Big Data has become a key competitive advantage to manage the exponential growth of globally available information. Early adopters have already proven benefits both on the cost and on the revenue side. However, a well-thought-out Big Data strategy is crucial to leverage this potential without spending millions on oversized analytic capabilities. xCon Partners has coordinated several hundred Big Data projects for a major solution supplier, always supporting the fast and smooth project execution. We are your ideal partner to tackle the challenges involved in Big Data and to disclose its full potential by giving strategic guidelines as well as execution support.

TRANSCRIPT

Page 1: Big Data - Big Benefit or Big Waste?

PERSPECTIVE

2012

1

Big Data - Big Benefit or Big Waste? Your Big Data strategy will determine your success

Big Data – Blessing or Curse?

Google, Amazon and Facebook, three of today’s

most powerful companies, have been built around

Big Data. The information that flows through their

veins is directly linked to their ability to analyze vast

amounts of data. Exactly this capability makes them

market leaders with a position that is hard to attack

for any competitor. No surprise that by now, other

companies across all industries have likewise started

to gather information in hope to boost their business.

T-Mobile USA is only one of many examples for

which these efforts have already paid off. By

analyzing huge amounts of data in real time with

SAP’s new in-memory database HANA, it is now

possible to provide targeted offers to more than 21

million customers. This results in $10-25 savings for

each won back subscriber and potentially in billions

of additional revenue per year. In the saturated

telecommunications market this will be a crucial

advantage against the competitors.

On the flipside, companies missing this analytical

competence will see their market position significantly

weakened with the threat of an attack of a much

better prepared competitor always lingering around

the corner. Since the amount of digital data doubles

every five years according to recent calculations, this

effect will further increase. By 2012 we have already

been faced with about 2.6 Zettabytes of global data.

Assuming that a human being is able to memorize

100 GB of data, it would need four times of the entire

global population to store this information. With a

CAGR of almost 45%, in 2013 already six times of

the world population would be needed.

This trend is boosted by the increasing distribution of

networked devices and sensors in our daily life, smart

phones, internet and the social media platforms. The

key challenge is that the amount of data companies

need to evaluate in acceptable time is nowadays

growing faster than the performance of established

database technologies and analytical tools (compare

Figure 1). In addition, available data becomes more

and more semi-structured (XML) and unstructured

(documents, e-mails, videos, pictures, etc.) and

hence more difficult to analyze.

Figure 1: Growing “amnesia” of enterprises

1960 1970 1980 1990 2000 2010 2020 2030

1,000,000

1,000

1

1,000,000,000

Analytical

Speed1)

1) # of Data Units that can be evaluated in acceptable time and money

2) # of company’s Data Units to be evaluated to stay competitive

Source: xCon Partners analysis

Evaluation

Requirements2)

World Wide Web

Email

PC

today

Enterprise Social Media

Internet of Things

# of Data

Units

Data amount

exceeds analytical

performance

resulting in amnesia

The ability to rapidly analyze Big Data has become a key competitive advantage to manage the

exponential growth of globally available information. Early adopters have already proven benefits both

on the cost and on the revenue side. However, a well-thought-out Big Data strategy is crucial to

leverage this potential without spending millions on oversized analytic capabilities. xCon Partners has

coordinated several hundred Big Data projects for a major solution supplier, always supporting the

fast and smooth project execution. We are your ideal partner to tackle the challenges involved in Big

Data and to disclose its full potential by giving strategic guidelines as well as execution support.

Page 2: Big Data - Big Benefit or Big Waste?

PERSPECTIVE

2

To avoid resulting amnesia, companies have to either

dispose part of potentially valuable data untouched or

switch to new innovative analytics techniques.

Therefore it is essential for executives to define the

optimal Big Data strategy tailored towards their

company’s needs.

Big Data for Big Benefits

Even though data intense industries such as

telecommunication, (multi-)media as well as the

banking and insurance sector face the highest

obvious demand for Big Data solutions, basically all

industries and companies can take advantage of the

new and improved analytic possibilities. One

compelling success story is for example written by

the retail industry which is characterized by a lot of

customer interactions. After putting new in-memory

analytics technology in place, the luxury label

Burberry is now able to access millions of records

from multiple sources (customer data, stock

information, social media, etc.) nearly in real time.

This was made possible due to a speed increase

factor of 14,000 – requests now take one second

instead of previously nearly five hours. The real-time

analytics system is used to profile customers,

allowing making tailored customer offerings on the

fly. Hence, it allowed Burberry to implement a very

efficient customer-oriented sales strategy and to build

up a consistent global brand.

Certain cross-industry effects related to Big Data and

new analytics technologies are affecting all types of

companies:

Fact based decisions instead of relying on

assumptions: For business critical decisions, all

historical facts and current events can be

considered instead of relying on estimates and

assumptions.

Interactive scenario simulation: Calculate the

impact of alternative scenarios in almost real time

to assess risks and fine tune the chosen model.

This is especially interesting for price

optimizations which can now be based on all

available historical data.

Real time customer interaction support: Always

having an up-to-date customer behavior profile

available will create a very personal customer

experience and enable you to react in an optimal

way.

Micro personalization: Tap directly into your

customer base by defining very precise customer

segments to run individual micro campaigns and

to measure their success.

New business models: Real-time analytical

capabilities can also be used to access emerging

markets. New sensors like smart phones or smart

meters will for example enable new business

models around location based services or home

automation.

Even in the field of crime reduction, Big Data is

continuously gaining importance: Fraud Mining

applications are used by international finance

institutions with high transaction rates for systematic

identification and prevention of fraud. JP Morgan is

for example successfully deploying this technology in

numerous projects for identifying potential fraud

among traders.

Another example in this field is the joint venture of

Paymint AG and Fraunhofer IAIS that successfully

developed and implemented the application MINTIfy.

This solution protects millions of European credit card

accounts from fraud by analyzing thousands of

attributes in the transaction history and identifying

conspicuous patterns.

Different branch, similar problem: TomTom Business

Solutions, a commercial vehicle fleet specialist, is

Are you faced with Big Data?

Benefiting from Big Data requires analyzing huge

amounts of data from different sources at very

high speed. An exact quantitative definition of Big

Data is difficult since computing power is

constantly growing. For example the amount of

data used by the NASA in 1969 for the moon

landing operation can nowadays easily be

handled by a modern smart phone. Therefore we

recommend this pragmatic quantification: Big

Data cannot be captured, stored, managed and

analyzed by the commonly used software and

hardware within a tolerable period of time. In this

sense the magnitude is specific for your business

model and your Big Data capabilities. A good

indication on the criticality for your business can

be derived by assessing the following three V’s:

Volume: What amount are you faced with?

Variety: How diverse is your data?

Velocity: How fast do you need to analyze it?

Page 3: Big Data - Big Benefit or Big Waste?

PERSPECTIVE

3

facing over 1.5 billion real-time notifications from

175,000 vehicles and more than 1 billion requests per

month. Fujitsu accepted the challenge and

implemented an Oracle based solution which is now

able to handle 200,000 input-output-operations per

second with response times of under one millisecond.

This grants fleet managers and transport planners

access to real time information to optimize the routing

of their vehicles and to reach a higher fleet utilization.

In-Memory, Scale Out or NoSQL?

Once the need to handle Big Data is evident, the

question of the best IT solution(s) arises. To make

use of the company’s Big Data, many different

technologies (e.g. in-memory computing) and

methodologies (e.g. visual analytics) need to be

evaluated to define a comprehensive Big Data

strategy. Depending on the business problem, the

currently most promising technologies to significantly

speed up and broaden data evaluations are in-

memory computing, a horizontal scale-out approach,

NoSQL databases, or a combination of those. The

best fitting technology always depends on the

company’s specific business context.

The “trick” of in-memory technology is to avoid slow

hard disk access by constantly keeping all relevant

data in RAM. As the average access speed between

RAM and disk differs roughly by a factor of 100 to

1,000, data operations can be performed significantly

faster. Obviously, the speed increase depends on the

specific business context and the chosen

implementation. No wonder that currently almost all

of the leading global enterprises evaluate this

technology. In addition, in-memory solutions are often

combined with other powerful concepts to further

boost their performance. Some database vendors for

example use their know-how of specific applications

to optimize the underlying data structure: by using

columnar architectures, only columns that contain the

necessary data to determine the answer are being

processed which leads to significantly faster

response times. Additionally, advanced data

compression capabilities are applied to reduce the

size of information that needs to be stored and

analyzed. In-memory technology “standalone”

currently works best with relational databases and

structured data, providing instant results.

Speed improvements with even higher factors than

1,000 are possible: Automotive Resources

International (ARI) is now able to analyze millions of

data points collected from approx. 923,000 vehicles

3,600 times faster than with traditional technologies.

After a three week implementation of the SAP HANA

data mart solution, the company now benefits from

5% cost reduction in total overhead expense and

from increased contact center performance.

Currently, the in-memory market is still dominated by

data warehouse projects aiming to provide optimal

management decision support (see Figure 2). But

almost as many companies gather their own

experience by experimenting with custom

developments. We expect that some of the most

radical game changing innovations will arise out of

these initiatives.

Figure 2: Market split of current in-memory activities

Complementary or as an alternative to in-memory

computing, Hadoop is one of the most prominent

frameworks to implement scale-out scenarios.

Developed by the Apache group, it allows for the

distributed processing of large data sets across

clusters of computers using simple programming

models. It is designed to scale out from single

servers to thousands of machines, each offering local

computation and storage. At the core of Hadoop is an

implementation of the “MapReduce” algorithm: first,

the “Map” function divides the original query into sub-

segments and calculates their results on any number

of distributed nodes. Second, the “Reduce” function

centrally aggregates these intermediate results and

returns the answer. A whole set of additional Hadoop

components supports the seamless integration into

the enterprise environment: Flume and Sqoop help

with data population, Mahout encapsulates data

mining capabilities, Hive and Pig assist with query

generation. This framework is already used by most

of the leading online companies like Google,

Amazon, and Facebook for searching and analyzing

On-DemandSolutions

DataWarehouse

CustomDevelopment

Applications

StandardizedConfigurations

Side-by-SideAcceleration

Source: xCon Partners analysis

Page 4: Big Data - Big Benefit or Big Waste?

PERSPECTIVE

4

their data. Scale-out technology works especially well

with unstructured and semi-structured data.

NoSQL (Not Only SQL) databases are a good

alternative if Big Data performance and scalability is

most important and 100% consistency is not

required. Compared to the sophisticated relational

databases, they are better suited to handle large

volumes of multi-structured data. The biggest

disadvantage is that the majority of NoSQL

implementations no longer required transactions to

be ACID (atomic, consistent, isolated, durable). This

is one major difference to the in-memory databases

and disqualifies this technology for online-

transaction-processing that does not compromise on

data accuracy. Besides the most prominent NoSQL

implementations BigTable (Google) and Dynamo

(Amazon), it is to mention that with HBase, a version

exists which is especially meant to be deployed on

top of HDFS, the Hadoop Distributed File System. By

allowing low-latency lookups in Hadoop, it combines

these two Big Data technologies.

In certain cases, also Hadoop and in-memory

technologies need to be combined to achieve the

desired computing power. One example is a new

cancer research solution realized by Charité. The

university medical center has proven that it is

possible to reduce the time required to analyze a

tumor by a factor of 1,000, reducing it from hours

down to a few seconds. This offers the possibility to

adjust cancer treatments before the patient leaves

the hospital.

Increasingly, new players position themselves

successfully and offer their own Big Data solutions:

the online retailer OTTO was seeking for a possibility

to improve its warehouse management and its sales

forecast system. The predictive-analytics software

NeuroBayes by Blue Yonder was applied here,

handling over one billion forecasts in a year. The self-

learning system is able to process over 135 GB or

300 million of daily new data sets, boosting the

forecast efficiency up by 40%.

At the same time, established vendors like SAP or

Oracle develop a whole set of new in-memory based

applications that will set the industry standard in the

mid future. Some early adopters and co-innovators

have already migrated.

Additionally to the described major Big Data

technologies, there are various new and innovative

analytic solutions being developed, tailored

specifically towards Big Data. Those analytic

solutions are usually combined and sit on top of one

or several of the described Big Data technologies.

Due to the high number of different approaches,

describing them would go beyond the purpose of this

Perspective and needs to be evaluated individually

for the specific business purpose.

Big Data Market Overview

The Big Data market size strongly depends on the

market definition. Sticking to the Big Data definition

provided above, we size the Big Data market

(Software, Hardware and Services related to and

used by In-Memory, Scale-Out, and NoSQL

technologies) for 2012 at about 10 billion Euros.

Looking in the past and the years to come, we predict

a CAGR of 40% for the upcoming five years which

brings us to a market size above 50 billion Euros in

2017. Major driver for this significant market growth is

the explosion of data due to increasing use of social

media, mobile devices and the internet of things.

There are more than 100 players active in the Big

Data market with solutions tailored specifically

towards Big Data. Figure 3 provides xCon Partners’

view on the most important players in the Big Data

market. Naturally, this matrix is frequently subject to

change due to the fast moving and dynamic market.

Figure 3: Big Data Market Overview

Big Data Capability2)

Challengers Leaders

High Potentials Innovators

SAP

Oracle

Microsoft

Google

Amazon

IBM Netezza

HP Vertica

Teradata Aster

EMC Greenplum

Cu

rre

nt

Ma

rke

t P

os

itio

n1)

Cloudera

1010Data

SAS

10genMapR

Hortonworks

VMware

1) Besides Big Data market share considers also market share inDBMS, Data Warehouse, BI, Enterprise Process Management

2) Considers completeness (In-memory, Hadoop, NoSQL, etc.),maturity and vision of Big Data solution

Source: xCon Partners analysis

FacebookKognitio

ParAccel

MarkLogic

Page 5: Big Data - Big Benefit or Big Waste?

PERSPECTIVE

5

To determine the “Current Market Position”, besides

the still very volatile Big Data revenue, also market

shares in related and established markets like

Database Management Systems (DMS), Data

Warehouses (DW), Business Intelligence (BI), and

Enterprise Process Management (EPM) have been

considered. The “Big Data Capability” is not only

determined by the completeness of the offered

solution, but also by its maturity and the vision of the

vendor.

The seven biggest vendors with complete offerings

for In-Memory or Scale-Out Solutions in the Big Data

market are currently IBM Netezza, Oracle, Microsoft,

SAP, HP Vertica, Teradata Aster, and EMC

Greenplum.

The market position of those seven vendors is on the

one hand threatened by the innovators which are

mature internet companies that have developed their

own and very sophisticated Big Data solutions like

Google, Amazon or Facebook. On the other hand,

there are many emerging players entering the market

with new and innovate Big Data technologies and

solutions.

Companies need to select the most suitable solutions

and best positioned vendors carefully to not run into

the trouble of discontinued products.

Rely on xCon Partners’ Experience

Since 2011, xCon Partners has been involved in and

coordinated several hundred Big Data Projects,

always supporting the fast and smooth project

execution. Therefore, we are your ideal partner for

tackling the involved challenges and for disclosing

the full potential. Besides strategic recommendations,

we also support our clients during the execution to

ensure successful implementations. From having

analyzed hundreds of slipped projects we can give

indications on common implementation risks and help

to set realistic project targets and roadmaps.

From our experience, the Big Data topic is best

addressed with the following 5-step approach (see

also Figure 4):

1) Conduct a Big Data readiness check to

derive a clear picture of available information,

data sources and own analytic capabilities

2) Calculate the business potential for

enhancements to the current business model

and for new business opportunities including

a cost-benefit check

3) Understand which technology and solution is

best suited to implement the Big Data

strategy

Figure 4: xCon Partners Big Data Services

xCon Partners Big Data Services

Big

Data

Strategy

Implement and Track Under-

stand Technology

andMarket

Big Data Readiness

Check Calculate Business Potential

PlanImplementation

Define roadmap and project plan

System & integrator selection

Technology assessment

Overview on Big Data market

(vendors & service providers)

Transparency on current

market adoption of different

solutions

Support proof of concept

activities

Cost-Benefit analysis

Analyze available information

and additional data sources

Evaluate current capabilities for

Big Data analytics

Benchmarking

Analyze potential

enhancements to current

business and identify new

business opportunities

Define additional

information requirements

and analytics capabilities

Estimate effort and

business potential

Project management

Expert insights based on

400+ tracked active

Big Data projects

Avoid project delays

Measure success

1

2

3

4

5

Page 6: Big Data - Big Benefit or Big Waste?

PERSPECTIVE

6

4) Define a realistic implementation plan and

select the right partners

5) Implement and measure the success

It is important that Big Data and the technology to

tackle the involved challenges must not be an end in

itself. As a starting point, the development of a

holistic and company adapted Big Data strategy is

crucial for success and should allow executives to

maintain a clear vision of what they want to achieve.

Based on this, the appropriate data, the proper level

of detail and the best technologies have to be

selected. If this is done correctly, companies have the

chance to gain a significant competitive advantage.

Even increasing the revenues by up to 30% may not

be out of reach. This is the target that the online

game company Bigpoint is aiming for. It will be

reached by analyzing more than 5,000 game events

per second in real time to offer their players an

individualized game environment and to enable

personalized micro sales. This will further strengthen

Bigpoints already strong position in the market.

Securing and strengthening the market position by

leveraging Big Data is also possible for you: with

xCon Partners’ proven holistic Big Data approach

and our comprehensive project history, we are the

partner of your choice to develop a well-thought-out

Big Data strategy for you.

About the Authors

Percy Stocker

Partner

[email protected]

+49 176 21 30 44 50

Leonid Poliakov

Consultant

[email protected]

+49 173 2482 043

About xCon Partners

xCon Partners is a strategy and management

consulting firm focusing on Business-IT-Alignment –

We link business and IT!

We offer a combination of in-depth experience in

international management and strategy consulting

and special know-how in the CIO and CTO area of

information and technology management.

Our clients benefit from a cooperative consulting

approach, always considering the individual and

unique situation of the client. Our extensive partner

network gives us on-demand access to further

industry specific and functional know-how whenever

needed.

With office locations across Germany (Bremen,

Wiesbaden and Munich), we are proud to serve our

DAX 30 and mid-sized customers with close distance

and perfect reachability. For more information, please

visit www.xcon-partners.com.

Copyright © by xCon Partners GmbH 2013.

All rights reserved.