What’s Holding Back Big Data?

CA asked 1,000 IT and business leaders on the successes and challenges they’ve had with Big Data.

Respondents’ organizations must have either implemented a Big Data project, or plan to be doing so within three years.

Over half of the respondents have already implemented a Big Data project. 29 percent of those haven’t yet are planning to do so in the next year.

• Yes, we have implemented Big Data 55%

• Not yet, but we plan to in the next year 29%

• Not yet, but we plan to in one to two years 10%

• Not yet, but we plan to in two to three years 6%

On average, organizations already using Big Data have fully implemented 4 Big Data projects.

However, the majority of existing Big Data projects are standalone, or not integrated across the company.

• Integrated initiative across the company 44%

• Multiple departmental projects, not integrated 35%

• Project limited to a single department or areas 21%

92% of respondents admit that there are major obstacles to implementing Big Data projects in their organization.

Nine in ten (91%), for example, are prevented from including data from both mainframe and non-mainframe sources in their Big Data solutions.

What are the obstacles holding Big Data back? Infrastructure and organizational complexity are tops. • Almost a third report that their firm’s infrastructure

is not sufficient to support Big Data.

• 32% Existing infrastructure is not sufficient

• 27% Organizational complexity

The top environmental challenges? Complexity. • Complexity of managing such a large

implementation 48%

• Complexity of managing so many different solutions 43%

• Cost 40%

• Difficulty reintegrating analysis 36%

• Lack of visibility into information and processes 31%

• May not have the ability to build and manage the evolving infrastructure on premise 15%

Big Data requires a bigger infrastructure. Scalability, capacity, integration, bandwidth, and predictability all featured high in respondents concerns about infrastructure.

Infrastructure challenges

• Scalability to support data set requirements 42%

• Overall capacity 41%

• Ability to integrate/leverage existing infrastructure 39%

• Network bandwidth issues 35%

• Predictability of capacity needs 32%

• Lack of ability to build the evolving infrastructure on-premises 25%

• Lack of ability to manage the evolving infrastructure on premises 12%

• I do not think there will be any major challenges 8%

Organizations are underestimating how much data is involved with Big Data. 41% of organizations with Big Data initiatives report that they have more than 3PB of data, while only 29% of organizations in the planning stages anticipate that much.

• Amount of data being analyzed in organizations with existing Big Data projects (547)

• Amount of data that will be analyzed in organizations planning on implementing Big Data projects (453)

The amount of data handled by today’s organizations is growing by leaps and bounds.Respondents reported the amount of data in their organization had grown 16% in the past two years, anticipated to grow 24% over the next two.

Organizations are underestimating the difficulty of implementing Big Data.

Organizations that already have Big Data projects in place are almost twice as likely to say that there are “significant interdependencies” between their Big Data processes and other processes outside of it (38% to 21%).

What’s Holding Back Big Data?

Read the full research report

Details: Independent research firm Vanson Bourne interviewed a total of 1000 senior managers and users/ technology managers in IT in February 2015. The respondents were from organizations around the world with an annual revenue or budget of more than $500m and from a range of sectors, including retail; financial services; banking; telecoms; insurance; and government.