avoid bi catastrophe
DESCRIPTION
BI project have a terrible track record. Historically, 50 to 80 percent of project fail to meet expectations. This presentation itemized several reasons for this problem and proposes a solution.TRANSCRIPT
![Page 1: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/1.jpg)
AVOIDING BI CATASTROPHE
Technology Focus on Backend BI
LeapFrogBI – Making Agile Data Warehousing Possible
Paul B. [email protected]
![Page 2: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/2.jpg)
1998 201420062002 2010
“There is a common perception that the failure rate of data warehousing projects is 70 to 80 percent…”
“…one study reported a 90 percent failure rate.”
“(From now) through 2007, more than 50 percent of data warehouse projects will have limited acceptance, or will be outright failures…”
“Between 70 to 80 percent of corporate business intelligence projects fail, according to research by analyst firm Gartner.”
“During the panel session vendors were asked to estimate the failure rate of analytic projects. They generally agreed that more than 70 percent failed to meet expectations.”
“A staggering 60 percent of BI projects end in abandonment or failure…”
* - Reference Available by Request
?
![Page 3: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/3.jpg)
TODAY’S AGENDA
Success & Failure
Agile ETL Design Pattern
15 Minute Data Mart (Video)
Open Discussion
~ 45 Minutes
![Page 4: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/4.jpg)
SUCCESS & FAILURE
Defining the goal and the most common causes of failure.
![Page 5: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/5.jpg)
A SUCCESSFUL BI SOLUTION
Add ValueValue = Revenue - Costs
1. Improve Decision Making by Providing Relevant Information
2. Respond Efficiently to a Changing Business Environment
![Page 6: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/6.jpg)
SUCCESSFUL BACKEND BI SOLUTION
Data Structure that Supports Requirements
1. Acceptable Performance
2. Easy to Navigate
3. Responsive to Changing Requirements
4. Meet Data Persistence Needs
5. Resolve Data Quality Issues & Apply Business Rules
6. Consolidated Model (One Version of the Truth)
7. Resilient, Accurate, Timely, and Scalable
8. Controlled Ongoing Support/Maintenance Costs
![Page 7: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/7.jpg)
BACKEND BI FAILURE
#1 - Failure to Plan
John Wooden – UCLA Coach (1910 – 2010)
"If you don’t have time to do it right, when will you have time to do it over?"
• Resource gaps – Technical & non-technical
• Cowboy coding – No standards• Building without requirements• No support & maintenance plan• No MDM or data quality plan• The short term solutions
![Page 8: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/8.jpg)
BACKEND BI FAILURE
#2 – Low Level Development
Benjamin Disraeli – British Politician(1804 – 1881)
"There can be economy only where there is efficiency.”
• Expensive; recreate the wheel• Tightly coupled dependencies• Difficult to transfer knowledge• Lower usually equals slower• Difficult to extend; loss due to rework• Developer fatigue
![Page 9: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/9.jpg)
BACKEND BI FAILURE
#3 – Flawed Design Patterns
Mark Twain – Author(1835 - 1910)
"I have never let my schooling interfere with my education.”
• Failure to consolidate• Reliance on natural keys• No separation between stage & load• Focus on optimization before design• Dimensions not conformed• No bus architecture
![Page 10: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/10.jpg)
AGILE ETL DESIGN PATTERN
An architecture that supports agile data warehousing.
![Page 11: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/11.jpg)
AGILE ETL MANIFESTO
Strive for Simplicity
Plan for Change
Default to the Foundation Pattern
Facilitate Custom Needs
Loosely Couple Dependencies
![Page 12: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/12.jpg)
AGILE ETL DESIGN PATTERN
![Page 13: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/13.jpg)
AGILE ETL DESIGN PATTERN
![Page 14: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/14.jpg)
AGILE ETL DESIGN PATTERN
![Page 15: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/15.jpg)
AGILE ETL DESIGN PATTERN
![Page 16: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/16.jpg)
AGILE ETL DESIGN PATTERN
![Page 17: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/17.jpg)
AGILE ETL DESIGN PATTERN
![Page 18: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/18.jpg)
AGILE ETL DESIGN PATTERN
![Page 19: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/19.jpg)
AGILE ETL DESIGN PATTERN
![Page 20: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/20.jpg)
15 MINUTE DATA MART
LeapFrogBI platform demonstration.
![Page 21: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/21.jpg)
METADATA DRIVEN ETL
ID Effective Expired Current CustomerID Name Age City1 1/1/1900 5/12/2010 0 4562 Jack 56 Austin2 5/13/2010 2/4/2013 0 4562 Jack 56 Dallas3 2/5/2013 1/1/9999 1 4562 Jack 56 Houston
Data Warehouse
![Page 23: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/23.jpg)
QUICK RECAP
BI Success = Net Value Add
Avoid Common Causes of Failure
Select an Agile ETL Design Pattern
Consider a Metadata Driven Approach
![Page 24: Avoid BI Catastrophe](https://reader033.vdocuments.net/reader033/viewer/2022061214/54994b61b47959cb298b4755/html5/thumbnails/24.jpg)
OPEN DISCUSSION
Question, Comments, and Experiences.
LeapFrogBI – Making Agile Data Warehousing Possible