What is What is

Data WarehousData Warehousinging

All About? - 2012All About? - 2012

1/7/20121/7/2012

www.InstantBI.cowww.InstantBI.comm

Agenda

Intended AudienceIntended Audience Introduction to the 2012 updateIntroduction to the 2012 update What do you want, really?What do you want, really? What is the Data Warehouse all about?What is the Data Warehouse all about? Management Decision Making ProcessManagement Decision Making Process Role of the Data WarehouseRole of the Data Warehouse What are people saying about Data What are people saying about Data

Warehousing?Warehousing? What went so wrong?What went so wrong? How to make it go right?How to make it go right? Business benefits of Data WarehousingBusiness benefits of Data Warehousing What works?What works?

Intended Audience As much as I would like business managers to listen to As much as I would like business managers to listen to

this I realise that the actual audience will be BI IT this I realise that the actual audience will be BI IT Managers and professionals Managers and professionals

That is kind of sad because it is business managers who That is kind of sad because it is business managers who need to hear this presentation. need to hear this presentation. Most simply will not take the time to listen Most simply will not take the time to listen They are “too busy” to listen and then they wonder why They are “too busy” to listen and then they wonder why

they do not have successful BI projectsthey do not have successful BI projects It takes VERY little time for a business manager to listen It takes VERY little time for a business manager to listen

to what he/she needs to know to make BI successfulto what he/she needs to know to make BI successful When I give an abbreviated form of this presentation to When I give an abbreviated form of this presentation to

business managers they “get it”business managers they “get it” So, @BI IT Managers? So, @BI IT Managers?

I will make a second version for your business managers. I will make a second version for your business managers. It is up to you to get them to listen to it. It is up to you to get them to listen to it.

Introduction to the 2012 Update

The original versionThe original version This presentation was written in 1994 based on a This presentation was written in 1994 based on a

presentation by Keith Fountain of Metaphor in 93.presentation by Keith Fountain of Metaphor in 93. Keith made this presentation to many prospects in Keith made this presentation to many prospects in

Sydney in when I invited him to come and talk to our Sydney in when I invited him to come and talk to our IBM Insurance Clients in SydneyIBM Insurance Clients in Sydney

I upgraded versions of this presentation for 10 yearsI upgraded versions of this presentation for 10 years Now, in 2012, I have upgraded it because it is as Now, in 2012, I have upgraded it because it is as

relevant today as it was nearly 20 years agorelevant today as it was nearly 20 years ago This is a valuable presentation that belongs in the This is a valuable presentation that belongs in the

public domain in the you tube agepublic domain in the you tube age We have not moved forward nearly as far as I expected We have not moved forward nearly as far as I expected

in 1993. Data volumes yes. Data models yes. in 1993. Data volumes yes. Data models yes. But the ability/desire of business people to exploit the But the ability/desire of business people to exploit the data warehouse has gone down, in my opinion data warehouse has gone down, in my opinion

What Do You Want, Really?

Improved profitImproved profitability with sustainable ability with sustainable growthgrowth Attention Attention to both revenue and coststo both revenue and costs

Improved decision making capabilityImproved decision making capability Make more effective decisions more oftenMake more effective decisions more often Consistent measures of business Consistent measures of business

performanceperformance Profit is a fact, not an opinionProfit is a fact, not an opinion

Compliance to government legislationCompliance to government legislation You can be put out of business if notYou can be put out of business if not

Others?Others?

What is Data What is Data Warehousing Warehousing

All About?All About?

What is the DW All About?

The Data Warehouse is all aboutThe Data Warehouse is all about

“Supporting the management decision “Supporting the management decision making process”making process”

It’s that simple, and that complicatedIt’s that simple, and that complicated

Management Decision Making Process

How’s Business?

Why?What if?

Invest $$$

What is the Data Warehouse? (1993)

Time VariantTime Variant Subject OrientedSubject Oriented IntegratedIntegrated Contains non volatile snapshotsContains non volatile snapshots

A Data Warehouse is a collection of data in A Data Warehouse is a collection of data in support of the management decision making support of the management decision making process.*process.*

This definition has stood the test of time This definition has stood the test of time remarkably well. It is as relevant today as it was remarkably well. It is as relevant today as it was in 1993.in 1993.

*Source: Building the Data Warehouse by W.H. Inmon

What Does a DW Look Like? (1993)

HighlyHighlySummarisedSummarised

LightlyLightlySummarisedSummarised

CurrentCurrentDetailDetail

Older Older DetailDetail(Tape/cdrom)(Tape/cdrom)

MMeettaa

DDaattaa

What Does a DW Look Like? 2012

Role of the Data Warehouse

How’s Business?

Why?What if?

Invest $$$

What are People Saying What are People Saying About About

Data Warehousing?Data Warehousing?

Please remember this was in the 90s.Please remember this was in the 90s.

What are People Saying?

““The business case for warehouses is The business case for warehouses is simple: they help turn data into a simple: they help turn data into a competitive tool”competitive tool”

COMPUTERWORLDCOMPUTERWORLD

What are People Saying?

Tom PetersTom Peters We are living through a shift from selling We are living through a shift from selling

virtually everyone the same thing a virtually everyone the same thing a generation ago to fulfilling individual needs generation ago to fulfilling individual needs and tastes... by supplying... customised and tastes... by supplying... customised products and services. The shift [is] from “get products and services. The shift [is] from “get the sale now at any cost” to building and the sale now at any cost” to building and managing ... databases that track the lifetime managing ... databases that track the lifetime value of your relationship with each value of your relationship with each customer. customer.

What are People Saying?

Rapp & Collins Rapp & Collins it may almost be time to replace it may almost be time to replace

“location, location, location” “location, location, location”

with with

“database, database, database”.“database, database, database”.

What are People Saying?

Stanley Davis - Future PerfectStanley Davis - Future Perfect Mass CustomisationMass Customisation Standardise the Commodity and Standardise the Commodity and

Customise the Service that Surrounds ItCustomise the Service that Surrounds It

Example: Telephone callsExample: Telephone calls

What are People Saying?

Philip Kotler - Marketing ManagementPhilip Kotler - Marketing Management Mass markets are fragmenting into micro-Mass markets are fragmenting into micro-

markets; multiple channels of distribution markets; multiple channels of distribution are replacing single channels...are replacing single channels...The winners are those who carefully The winners are those who carefully analyse needs, identify opportunities and analyse needs, identify opportunities and create value-laden offers for target create value-laden offers for target customer groups that competitors can’t customer groups that competitors can’t match.match.

IDC

Perhaps the single most important Perhaps the single most important ongoing occurrence to affect data ongoing occurrence to affect data warehousing has been the information warehousing has been the information explosion. Organisations realise that, explosion. Organisations realise that, given the fundamental relationship given the fundamental relationship between knowledge and power, between knowledge and power, utilisation of this information is key to utilisation of this information is key to their competitive positioning. their competitive positioning.

Meta Group

Bottom Line: Data warehouses are an Bottom Line: Data warehouses are an increasingly critical component of the increasingly critical component of the systems that support the ever-increasing systems that support the ever-increasing tempo of business competition. tempo of business competition.

Let Us Take a Checkpoint We just saw some very strong comments We just saw some very strong comments

made in the 90s about data warehousing.made in the 90s about data warehousing. I made MANY similar comments myself.I made MANY similar comments myself. Who, listening to this, can honestly say Who, listening to this, can honestly say

their company has been able to gain very their company has been able to gain very significant competitive advantage and significant competitive advantage and sustain that competitive advantage via data sustain that competitive advantage via data warehousing and business intelligence?warehousing and business intelligence?

In my experience this is very, very few.In my experience this is very, very few. With the consolidation of BI vendors it will With the consolidation of BI vendors it will

be even fewer in the future.be even fewer in the future. WHY were the promised benefits not WHY were the promised benefits not

realised?realised?

Let Us Take a Checkpoint In my opinion not enough senior business managers listened In my opinion not enough senior business managers listened

closely enough to those of us who knew what we were talking closely enough to those of us who knew what we were talking aboutabout

In the late 90s every man and his dog was a “BI Consultant” In the late 90s every man and his dog was a “BI Consultant” whether he knew how to spell “BI” or notwhether he knew how to spell “BI” or not

I tried my best to put standards into the public to show people I tried my best to put standards into the public to show people what a BI consultant should knowwhat a BI consultant should know

Men like me were widely ignored as we tried to bring in Men like me were widely ignored as we tried to bring in standardsstandards

BI has become “write more useless reports” in the mainBI has become “write more useless reports” in the main Many people have become enamoured with “sexy charts”Many people have become enamoured with “sexy charts” Many people no longer look for competitive advantage and Many people no longer look for competitive advantage and

therefore they do not find ittherefore they do not find it We have snatched defeat from the jaws of victory in the vast We have snatched defeat from the jaws of victory in the vast

majority of BI Projectsmajority of BI Projects The 00s were a frustrating time for me watching the BI Industry The 00s were a frustrating time for me watching the BI Industry

be given a bad name by failure after failure by people who be given a bad name by failure after failure by people who could not spell “BI” let alone build a data warehousecould not spell “BI” let alone build a data warehouse

Case Studies?

I am not going to bore you with case studiesI am not going to bore you with case studies Every vendor has “glamorous case studies” with Every vendor has “glamorous case studies” with

promises of great things to comepromises of great things to come Alas, mostly those promises are not deliveredAlas, mostly those promises are not delivered

Make no mistake SOMETIMES there is successMake no mistake SOMETIMES there is success One of my clients paid for the project before it went live!One of my clients paid for the project before it went live! One of my clients doubled gross profit inside 2 yearsOne of my clients doubled gross profit inside 2 years One of my clients called it the most successful project One of my clients called it the most successful project

everever

But success has been “surprisingly” elusiveBut success has been “surprisingly” elusive Well…it has not been a surprise to meWell…it has not been a surprise to me I can usually tell if a BI project will deliver success by I can usually tell if a BI project will deliver success by

reading the requirements document and or project planreading the requirements document and or project plan It is THAT obvious if the dev team know what they are It is THAT obvious if the dev team know what they are

doingdoing

What Went So Wrong?What Went So Wrong?

What Went So Wrong? In March 1991 I saw the futureIn March 1991 I saw the future

It was called the Data Interpretation System from Metaphor It was called the Data Interpretation System from Metaphor Computer Services (Computer Services (www.meta5.comwww.meta5.com))

It took me less than 30 minutes to realise thisIt took me less than 30 minutes to realise this I understood that my life had just changed I understood that my life had just changed

I was about to embark on another career changeI was about to embark on another career change I would move from getting data into computers to getting I would move from getting data into computers to getting

knowledge OUT of computersknowledge OUT of computers I would move into a career where gleaning knowledge I would move into a career where gleaning knowledge

from masses of data to make decisions would be my jobfrom masses of data to make decisions would be my job And for 5 years it wasAnd for 5 years it was Then what happened in many cases was we built the data Then what happened in many cases was we built the data

warehouse and the customers said “thanks for that, we warehouse and the customers said “thanks for that, we will do our own data analysis now”. And they produced will do our own data analysis now”. And they produced more reports.more reports.

SOME customers did very well. Many did not.SOME customers did very well. Many did not. Those that did not are the ones who claim BI is not very Those that did not are the ones who claim BI is not very

good.good.

What Went So Wrong? People refused to learn what the data warehouse was all People refused to learn what the data warehouse was all

about in the first place. Here it is again. From the late 80s. about in the first place. Here it is again. From the late 80s. The most elegant definition of a data warehouse ever The most elegant definition of a data warehouse ever written down.written down.

““A Data Warehouse is a collection of data in support of the A Data Warehouse is a collection of data in support of the management decision making process.”management decision making process.”

Yet? I can go to 100 DWAs and ask the question Yet? I can go to 100 DWAs and ask the question “can you “can you explain to me the Management Decision Making Process?”explain to me the Management Decision Making Process?” and they will look at me like I just arrived from the moon. and they will look at me like I just arrived from the moon.

They will often say They will often say “What has data warehousing got to do “What has data warehousing got to do with the MANAGEMENT decision making process?”with the MANAGEMENT decision making process?”

Almost NO ONE in BI can properly articulate what the Almost NO ONE in BI can properly articulate what the “Management Decision Making Process” is“Management Decision Making Process” is

I find that quite amazing I find that quite amazing Go ahead. Ask your “BI Consultants”. See how they goGo ahead. Ask your “BI Consultants”. See how they go

How to Make it Go How to Make it Go Right?Right?

How to Make it Go Right? Metaphor history explainedMetaphor history explained This is best explained in two anecdotesThis is best explained in two anecdotes Anecdote 1Anecdote 1

A major Australian company with two failed BI projectsA major Australian company with two failed BI projects In the business requirements discussion a comment was In the business requirements discussion a comment was

made made ““Do you realise the product is different every week?”Do you realise the product is different every week?”

This comment was checked and found to be trueThis comment was checked and found to be true The ERP could not implement allocations on selected weeksThe ERP could not implement allocations on selected weeks Allocations were implemented in the DW and fed to the ERPAllocations were implemented in the DW and fed to the ERP

Anecdote 2Anecdote 2 Qantas Cargo had two failed BI projectsQantas Cargo had two failed BI projects I was engaged to do “third time lucky”I was engaged to do “third time lucky” Complaints were made about me…situation normalComplaints were made about me…situation normal The CEO of Cargo gave me the chanceThe CEO of Cargo gave me the chance We delivered an outstanding prototype and the project We delivered an outstanding prototype and the project

went on to be very successfulwent on to be very successful

How to Make it Go Right? What was different about what I did?What was different about what I did? I was focussed on the “management I was focussed on the “management

decision making process”decision making process” I looked for the decisions being made in the I looked for the decisions being made in the

company that affect both revenue and costscompany that affect both revenue and costs I gathered the data that would support I gathered the data that would support

those decisions both plan and actualthose decisions both plan and actual This is NOT rocket scienceThis is NOT rocket science But as far as I know, VERY few people do But as far as I know, VERY few people do

thisthis Ask you consultants for PROOF they do thisAsk you consultants for PROOF they do this Consider replacing them if they fail this testConsider replacing them if they fail this test

How to Make it Go Right? (90s) Success was virtually assured if the DW was aimed Success was virtually assured if the DW was aimed

at the “management decision making process”at the “management decision making process” Every time a management decision was being Every time a management decision was being

made, data from the data warehouse should be made, data from the data warehouse should be mandatorymandatory

This means PLANS and FORECASTS are a natural This means PLANS and FORECASTS are a natural and REQUIRED part of the DWand REQUIRED part of the DW

Plans vs actuals are NOT optional in a good DWPlans vs actuals are NOT optional in a good DW Capturing reasons for variance is not optionalCapturing reasons for variance is not optional Multiple versions of plans is not optionalMultiple versions of plans is not optional Recording investment decisions is NOT optionalRecording investment decisions is NOT optional Being able to run “business process change” tests Being able to run “business process change” tests

is not optionalis not optional Being able to run test campaigns is not optionalBeing able to run test campaigns is not optional Recording everything you can lay your hands on is Recording everything you can lay your hands on is

VERY desirableVERY desirable

How to Make it Go Right? (10s) All the above PLUSAll the above PLUS

Have a very good data modelHave a very good data model The data model is the foundationThe data model is the foundation

Have a very good idea of the applications to buildHave a very good idea of the applications to build Preferably pre built and easy to customisePreferably pre built and easy to customise

Have segmentation engines to put objects (usually Have segmentation engines to put objects (usually customers) into segmentscustomers) into segments

Disconnection of the components Disconnection of the components Make customisation fast and easyMake customisation fast and easy DO NOT re-invent the wheelDO NOT re-invent the wheel Have a methodology that is consistent with the Have a methodology that is consistent with the

toolstools Focus on supporting decisions wherever smart Focus on supporting decisions wherever smart

decisions might generate business benefitdecisions might generate business benefit Put less emphasis on operational reportingPut less emphasis on operational reporting

Questions to Ask Vendors Get them to explain the “Management Decision Get them to explain the “Management Decision

Making Process” to you. See how they goMaking Process” to you. See how they go Ask them to explain their methodology and how it Ask them to explain their methodology and how it

works and works and whywhy it is the way it is it is the way it is Ask them to explain their data models and applicationsAsk them to explain their data models and applications

If they are talking about “we start with a blank sheet of paper” If they are talking about “we start with a blank sheet of paper” then you should be VERY wary as to then you should be VERY wary as to whywhy that might be that might be

Ask them about business process change testingAsk them about business process change testing Ask them about campaign management and testingAsk them about campaign management and testing Ask them about segmentation and how that worksAsk them about segmentation and how that works Ask them about customisation of existing appsAsk them about customisation of existing apps Ask them about how they liaise with managementAsk them about how they liaise with management Make them talk about Make them talk about HOWHOW they achieve results and they achieve results and

WHYWHY those results are achieved those results are achieved Ask them how it could be you would Ask them how it could be you would RETAINRETAIN your your

competitive advantage if you hire themcompetitive advantage if you hire them

Questions to Ask Vendors Ask them about what recognised business Ask them about what recognised business

frameworks their BI solutions fit intoframeworks their BI solutions fit into We use “Strategic Information Alignment We use “Strategic Information Alignment

Framework” from Donald Marchands book Framework” from Donald Marchands book Competing with InformationCompeting with Information

11

77

77

77 77

11

Add ValueAdd ValueCustomers and MarketsCustomers and Markets

Create New RealityCreate New RealityIntelligenceIntelligence

(Social, Political, Technological etc)(Social, Political, Technological etc)

ReduceReduceCostsCosts

Transactions Transactions and and

ProcessesProcesses

Minimise RisksMinimise RisksMarket, Legal, Market, Legal,

FinancialFinancialOperationalOperational

Questions to Ask Vendors If you are unsure if your BI vendors can deliver then If you are unsure if your BI vendors can deliver then

feel free to hire IBI to advise and guide you on your BI feel free to hire IBI to advise and guide you on your BI projectproject

For a small retainer we can review the vendors For a small retainer we can review the vendors deliverables and tell you if you are getting value or notdeliverables and tell you if you are getting value or not

Example. Example. A recent client spent USD2 M on a project that failed A recent client spent USD2 M on a project that failed As soon as I saw the proposed project plan I knew the vendor As soon as I saw the proposed project plan I knew the vendor

could not be successfulcould not be successful Yet the same vendor wanted a steep discount on my rates Yet the same vendor wanted a steep discount on my rates

because “we do not have enough money to pay you”because “we do not have enough money to pay you”

I have done my best to give companies world wide I have done my best to give companies world wide good advice over many yearsgood advice over many years

Mostly men like me are ignored and BI project failures Mostly men like me are ignored and BI project failures continue to happen at high ratescontinue to happen at high rates Skill levels of the people doing the projects being the #1 Skill levels of the people doing the projects being the #1

causecause

Business BenefitsBusiness Benefits

A Way to Test New Ideas

How do you test new ideas for How do you test new ideas for profitability?profitability?

Data Analysis/Mining “tests” ideasData Analysis/Mining “tests” ideas Data Mining to find “diamonds”Data Mining to find “diamonds” There are “diamonds” in your dataThere are “diamonds” in your data Marketing Data WarehouseMarketing Data Warehouse

Small investment, fast returnSmall investment, fast return Likely source of “diamonds”Likely source of “diamonds”

Business Benefits

More cost-effective decision makingMore cost-effective decision making Better business intelligenceBetter business intelligence Enhanced customer serviceEnhanced customer service Enhanced asset/liability managementEnhanced asset/liability management Aligned with corporate downsizingAligned with corporate downsizing Relationship to Business Process Relationship to Business Process

ReengineeringReengineering

What WorksWhat Works

Likely High Payback Projects

High Volume, Low Cost products High Volume, Low Cost products Telco, retail, web, media (BI4ALL Models)Telco, retail, web, media (BI4ALL Models)

Marketing CampaignsMarketing Campaigns Customer Value IndexCustomer Value Index

Customer profit contributionCustomer profit contribution Tiered servicing of customersTiered servicing of customers

Product launch, modify, termination, Product launch, modify, termination, pricingpricing

Customer/Product understandingCustomer/Product understanding Segmentation for marketingSegmentation for marketing

What Works

Senior Manager(s)Senior Manager(s) Making big decisions on scant dataMaking big decisions on scant data Demanding informationDemanding information

Marketing Manager(s)Marketing Manager(s) Demanding better sales targetingDemanding better sales targeting

Do something you can’t do nowDo something you can’t do now

What works is generally not so easy to doWhat works is generally not so easy to do

What Doesn’t Work

IT technology driven projectIT technology driven project Let’s try this new box, tool, toy ...Let’s try this new box, tool, toy ...

Pie in the sky wishful thinkingPie in the sky wishful thinking There seems to be a lot of this aboutThere seems to be a lot of this about

The strange idea that dashboards will The strange idea that dashboards will solve the problems of managementsolve the problems of management

DW as an answer to operational DW as an answer to operational reportingreporting

Places to Start

75% of DWs start in:75% of DWs start in: Marketing/SalesMarketing/Sales Customer Information SystemsCustomer Information Systems

All benefit from integrating data from All benefit from integrating data from multiple source systemsmultiple source systems

All contribute to new revenue and All contribute to new revenue and profitprofit

Most of the other 25% start in Most of the other 25% start in Performance measurement projectsPerformance measurement projects Financial reporting and analysisFinancial reporting and analysis

Places not to Start

Financial SystemsFinancial Systems eg. Ledger, Accounts Receivable, Product eg. Ledger, Accounts Receivable, Product

CatalogueCatalogue

Not customer focusedNot customer focused These systems are already ‘good’These systems are already ‘good’ Provides ‘better’ numbers for Provides ‘better’ numbers for

accountantsaccountants Little opportunity for new revenueLittle opportunity for new revenue

Cost Justifications

These are hard to do in advanceThese are hard to do in advance If Management is not convinced in the If Management is not convinced in the

value of BI better to just leave it alone in value of BI better to just leave it alone in 20122012

It is hard to believe there is a business It is hard to believe there is a business manager alive today who does not know manager alive today who does not know there is value in BIthere is value in BI

This is as opposed to being able to get This is as opposed to being able to get that value extracted which is a different that value extracted which is a different thingthing

Summary

Intended AudienceIntended Audience Introduction to the 2012 updateIntroduction to the 2012 update What do you want, really?What do you want, really? What is the Data Warehouse all about?What is the Data Warehouse all about? Management Decision Making ProcessManagement Decision Making Process Role of the Data WarehouseRole of the Data Warehouse What are people saying about Data What are people saying about Data

Warehousing?Warehousing? What went so wrong?What went so wrong? How to make it go right?How to make it go right? Business benefits of Data WarehousingBusiness benefits of Data Warehousing What works?What works?

Thank You for Your Thank You for Your Time!Time!