turning the corner: what it takes to build a modern data warehouse

Grab some

coffee and enjoy the pre-show

banter before the top of the

hour!

Building the Modern Data WarehouseRoundtable Webcast | Oct. 15, 2015

SPONSORED BY

Welcome

Host:Eric KavanaghCEO, The Bloor Group@[email protected]

Guests

TJ LaherProduct Marketing, Cloudera

Dwaine SnowStrategy Lead, IBM Analytics Platforms

Heine Krog IversenGroup CEO, TimeXtender

Findings WebcastDec. 2

The Modern Data Warehouse

Roundtable WebcastToday, Oct. 15

Survey Under Way!Ends Friday, Oct. 14 @ 5pm ET

#MDW

Data Warehousing: Past, Present, Future

Dennis JarvisTibet-5809 - Yak at Yundrok Yumtso Lake

kevin ryderold tractor

Original data warehouses were slow but powerfulSecond generation offered new functionalityThird generation? Agile, Automated, Adaptive

Agile, Automated, Adaptive: How So?

!Agile: How quickly does the warehouse deliver value to key stakeholders? Can new users be onboarded efficiently?

!Automated: Does your team need to do much manual, tedious maintance? What about documentation?

!Adaptive: How easily can the warehouse be modified to add new data sets?

Data Warehousing vs Big Data Analytics

!Data Warehousing uses structured (relational) data to enable accurate reporting, and facilitate ad hoc queries.

!Big Data Analytics focuses on “unstructured” (non-relational) data to enable business analytics.

!These are two very different paradigms, though there is some overlap.

Whither, the Warehouse?

!For the time being, the Data Warehouse will surely remain the “golden source” of curated, governed, trusted data for key executive decision-makers.

!Forward-thinking organizations will find ways to combine warehouse data with insights derived from Big Data, via use of dashboards and data visualizations.

Big Value, Big Hurdles for Big Data

!Big Data offers a second chance for the data management industry

!Because machines don’t lie, Big Data can provide built-in accuracy (except social)

!The toolsets for harnessing Big Data remain relatively nascent

!Though tremendously powerful, the open-source movement is volatile

©"2015"IBM"Corporation

The"Future"of"Data"Warehousing"is"“Logical”

Dwaine'SnowStrategy"Lead,"IBM"Analytics"PlatformsSeptember"25th,"2015

©"2015"IBM"Corporation12 IBM"Confidential

Agenda• The Changing Nature of Data• The Hybrid, Logical Data Warehouse• Fluid Query – Powering Hybrid, Logical

Data Warehousing


The Big Data counter ( ) will aggregate all the data generated from the moment you load the page …

This is Causing an Explosion in Data Types and Volumes


Not'just'Social'Media,'all'of'these'“new”data'sources'–

JSONWeb'Logs

Free'form'TextXML

CDR,'TDRPicturesAudioVideo

True Insight Today Requires All of Your Data Being Combined


Data"Sources

Transactional

Social

Application

User"Generated

Journal

Video"and"Audio

Machine"/"Sensor

Documents

Third"Party

The Market Has Evolved into the Logical Data WarehouseOptimizing access and reducing costs

Internal"Insight

Reporting

Enterprise"Content

Discovery"Exploration

Decision"Management

PredictiveAnalytics

Visualization

ExternalWFacing"ApplicationsWeb"or"Mobile"Systems"of"Engagement

Information"Governance

RealWtime"Analytics

NoSQL"Doc"Store Data"Warehouse Deep"Analytics,"Modeling

Transactional"Systems

Landing,"Exploration,"Archive

Reporting,"Analytics

Logical'Data'Warehouse


On'Premise CloudFluid'Query

IBM"Fluid"Query"– Powering"Big"Data"Analytics

" Intelligently"route"queries"to"the"correct"data"store

" Simplify"and"unify"information"access"for"end"users"and"applications

" Access"all"data"within"the"logical"data"warehouse"for"analytics"and"business"insight

Move'the'query'to'the'data,'not'the'data'to'the'query

Question

Answer

Hadoop

Data'Warehouse

Data'Mart

Operational

Other

In#the#world#of#big#data,#can#you#really#afford#to#move#all#your#data#to#the#analytics?

1"©"Cloudera,"Inc."All"rights"reserved."

Enterprise"Data"Warehouse"

Op=miza=on"

TJ"Laher"|"Product"Marke=ng,"Business"Solu=ons"


Trends"in"the"Market"

16#billion"connected"devices"genera=ng"more"data"

"

“It"will"soon"be"technically#feasible#&#affordable#to"record"&"store"everything…”"

ELT"drives"up"to"80%"of"database"capacity"

"

Internet"of"Things" Data"Storage"Costs" Resource"Intensive"ELT"

Trends"Driving"Change"

Source:(Forbes( Source:(New(York(Times( Source:(Syncsort((


Challenges"with"a"Tradi=onal"Architecture"

1)  Limited#Data#Ingest#


Applica@ons#Data#Sources#

Structured"

Unstructured"

Ingest"

Staging#Environment##

Tradi@onal#Architecture##Enterprise#Data#Warehouse#

Serve"ELT"

Archive"

BI"System"

Modeling"

Repor=ng"

ETL"

Storage"#1"

Storage"#2"

Storage"N"

Ingest"

Process" L

oad"

1"




2)#Inefficient#Data#Processing##



Structured"

Unstructured"

Ingest"



Serve"ELT"

Archive"

BI"System"

Modeling"

Repor=ng"

ETL"

Storage"#1"

Storage"#2"

Storage"N"

Ingest"

Process" L

oad"

1"

2"

2"




2)#Inefficient#Data#Processing##

3)#Data#Archived##



Structured"

Unstructured"

Ingest"



Serve"ELT"

Archive"

BI"System"

Modeling"

Repor=ng"

ETL"

Storage"#1"

Storage"#2"

Storage"N"

Ingest"

Process" L

oad"

1"

2"

2"

3"


A"New"Way"Forward"

1)#Ingest#More#Data#


Structured"

Unstructured"


Modern#Architecture##

Enterprise#Data#Warehouse#

EDH"Ingest"

Ac=ve"

Structured"Data"

Serve"

Serve"

ELT"

Archive"

Load"

1"

ETL"

BI"System"

Modeling"

Repor=ng"


A"New"Way"Forward"


2)#Op@mize#Data#Processing#


Structured"

Unstructured"




EDH"Ingest"

Ac=ve"

Structured"Data"

Serve"

Serve"

ELT"

Archive"

Load"

2"

1"

ETL"

BI"System"

Modeling"

Repor=ng"


A"New"Way"Forward"


2)#Op@mize#Data#Processing#

3)#Automated#Secure#Archive#


Structured"

Unstructured"




EDH"Ingest"

Ac=ve"

Structured"Data"

Serve"

Serve"

ELT"

Archive"

Load"

2"

3"1"

ETL"

BI"System"

Modeling"

Repor=ng"


Thank"you."

Questions?

THANK YOU!

FIND OUT MORE athttp://insideanalysis.com/research/modern-data-warehouse

turning the corner: what it takes to build a modern data warehouse

Technology