Challenges of Big Data

Banks have begun dusting off some of the ambitious data management projects they began in the early 2000s, then shelved when the financial crisis and recession hit.

But according to SAP's top banking executives, who met with us last week, these projects are being scaled down to handle a single problem, such as improving same-day liquidity risk reporting, rather than trying to transform data systems across the entire company at once.

The Walldorf, Germany-based software company's executives shared what they're seeing and hearing from their U.S. bank customers about the challenges of managing large data silos (sometimes referred to by the overused yet ill-defined term Big Data).

The Big Data myth. This is not a bank-specific issue, but the basic premise of the hyped-up term Big Data is the idea that companies' (and social media networks') data sets have grown so large and complex that they are awkward to work with using standard database management tools. But often it makes sense to narrow the data set first.

"Big Data is fun, but if you know what you're looking for — liquidity risk data, market risk data, credit risk data, those are kind of different," says Simon Paris, global head of banking. "Then when you think about how you're going to use that risk data, which is the mirror image of regulatory compliance — Dodd-Frank, Basel II and III, FATCA, you name it, it starts not being about the sources of the data but the application of the data."

"There's not a conversation I've had in the past 14 months where a customer has said to me, help me solve my Big Data problem," observes Eric Stine, regional vice president, sales, financial services for North America. "Not a single one of these projects is a Big Data project. It's a business problem, generally a business risk problem, that allows them to create a new paradigm for how they manage data." For instance, by making distinctions between data that must be accessed quickly and that must be stored long-term.

Dirty data. "Banks have forgotten that they don't have clean data," says Don Trotta, global head of banking industry development at SAP and formerly group chief information officer at Barclays Bank. "Data integrity has to be taken on as a big part of the project as well. All the tools and technologies you put on afterward are not going to fix that."

Aged data infrastructures. "Managing data takes up 7-10% of a bank's operating income, it's feeding the beast in the basement," Paris says. "The complexity that causes that cost is also unfathomable — it's 20, 30, and 40-year-old custom-developed code, sitting on a mainframe. We're the world's largest application company and yet we count at least ten banks that are bigger software developers than we are."

The need for real-time data. "When you have a digital multichannel customer, then you can do things like real-time offer management and relationship pricing," Paris says. "That makes you realize that the data from yesterday is no longer relevant." Basel requirements for real-time or at least same-day views of liquidity risk also call for current data feeds and on-the-fly analytics.

The need to get new products to market quickly. "Typically to generate a new product it would take you three to nine months," Paris says. "Now [using SAP's HANA in-memory analytics software, which stages streams of data stored for analysis right in computer memory rather than storing it in a separate disk drive or storage unit, which allows queries to be conducted faster], you can bring that down to three to nine minutes."

Inefficient data management. "There's currently a discussion going on among CFOs and CROs of many large banks around, how can we come up with a more consistent strategy to increase regulatory reporting timeliness and data quality?" observes Falk Rieker, global vice president for banking. "We believe you can do that by rethinking your database strategy and how you manage data. That's the mega topic banks will face over the next couple of years."

Banks are very traditional in the way they manage data, he says. "They replicate data," he says. "They have the same data stored seven to nine times. They put data into a warehouse, it goes to an operational data store, they have aggregation layers. There's an enormous amount of work around it, the cost of data maintenance is huge." Part of the solution, he believes, is to conduct analytics on data temporarily staged in memory, as in SAP's Hana platform. (Oracle has a product, its Exalytics appliance, that does the same thing.)

Seeing the value of data. "In the past, banks have looked at data just as data — it was more or less a pain, of no value," Falk says. "They're starting to change, they're realizing that data is meaningful, as long as they can analyze it."