Download - Data Vault Whats the Combo-1
-
5/21/2018 Data Vault Whats the Combo-1
1/31
Data Vault
RMOUG Training Days
2006Colorado Convention Center
Denver, Colorado
February 15-16
-
5/21/2018 Data Vault Whats the Combo-1
2/31
Data Vault;Whats The Combination?
Jeff Meyer
Enterprise Data IntegrationOracle DBA
Department of Technology Services
Denver Public Schools
-
5/21/2018 Data Vault Whats the Combo-1
3/31
Data Vault
Who are we? DBAs
Managers
Analysts
Enterprise Data Warehouse Projects Currently in process
Planned
Data Marts
-
5/21/2018 Data Vault Whats the Combo-1
4/31
Data Vault
Brief History and Revisit Some
Definitions
Three Basic Building Blocks of the
Data Vault
Advanced Features Questions
-
5/21/2018 Data Vault Whats the Combo-1
5/31
Data Vault
Brief History and Revisit Some
Definitions
Three Basic Building Blocks of the
Data Vault
Advanced Features Questions
-
5/21/2018 Data Vault Whats the Combo-1
6/31
Data VaultBrief History and Revisit Some Definitions
1970Dr. E.F. Codd of IBM
1979First Working Relational
Database by Relational Software
Incorporated
Oracle v2 1991William H. Inmon published
Building the Data Warehouse
-
5/21/2018 Data Vault Whats the Combo-1
7/31
Data VaultBrief History and Revisit Some Definitions
Legacy System any system that has been put into production.
(para-phrased W.H. Inmon)
Operational Data Store a subject-oriented, integrated, volatile, current
or near current collection of operational data.W.H. Inmon
-
5/21/2018 Data Vault Whats the Combo-1
8/31
Data VaultBrief History and Revisit Some Definitions
Data Warehouse a subject-oriented, integrated, time-variant, non-volatile
collection of data designed for support of business decisions
W.H. Inmon
Data Vault a detail-oriented, historical tracking and uniquely linked set
of normalized tables that support one or more functionalareas of business.
Dan Linstedt
-
5/21/2018 Data Vault Whats the Combo-1
9/31
Data VaultBrief History and Revisit Some Definitions
Data Mart a subset of a data warehouse, for use by a single
department or function.
www.e-formation.co.nz/glossary.asp
Corporate Information Factory
the framework that exists that surrounds the datawarehouse; typically contains an ODS, a data warehouse, data
marts, DSS applications, exploration warehouses, and so forth.
W.H. Inmon
http://www.e-formation.co.nz/glossary.asphttp://www.e-formation.co.nz/glossary.asphttp://www.e-formation.co.nz/glossary.asphttp://www.e-formation.co.nz/glossary.asp -
5/21/2018 Data Vault Whats the Combo-1
10/31
Data VaultBrief History and Revisit Some Definitions
* Source: Bill Inmon and Claudia Imhoff
-
5/21/2018 Data Vault Whats the Combo-1
11/31
Data VaultWhy?
Why do we need it? We finally have a Data Model that will work for small,
medium, or large business Anyone building a Data Warehouse can use these techniques.
Weve got issues in constructing the data warehouse
from 3rdnormal form, or star schema form. There are inherent road blocks to each method that we must
solve technically through our Data Model.
-
5/21/2018 Data Vault Whats the Combo-1
12/31
Data Vault
Brief History and Revisit Some
Definitions
Three Basic Building Blocks of the
Data Vault
Advanced Features Questions
-
5/21/2018 Data Vault Whats the Combo-1
13/31
Data VaultThree Basic Building Blocks
Hubstand alone table; list of unique business
keys; used for business identification Satellitedescriptive data; historical data; used
for descriptive information for the HUB or LINK
Linkassociative table; list of uniquerelationships between keys; used for relationshipsbetween HUBs and LINKs
-
5/21/2018 Data Vault Whats the Combo-1
14/31
Data VaultThree Basic Building BlocksPreview
Hub Employees
Hub Schools
ELAName
EEOCDates
Hub Students
EEOCName
ShotsAddrs
Assign Enrollments
-
5/21/2018 Data Vault Whats the Combo-1
15/31
Data VaultThree Basic Building BlocksHUB
Primary Key
Load DTS
Record Source
Primary Key
Load DTS
Record Source
Sample Data Set CUSTOMER
CONTRACTS2-2-200093KFLLA10
CONTRACTS2-2-2000929ABC29859
CONTRACTS2-2-2000PAFJG28958
FINANCE2-2-2000PPRU_32597
SALES8-3-2001HUJI_BFIOQ6
SALES6-4-2001LLOA_82J5J5
CONTRACTS3-7-2000KKO92854_dd4
CONTRACTS1-25-2000DKEF3
CONTRACTS10-2-2000ABC925_24FN2
RCRD SRCLOAD DTSCUSTOMER #ID
MANUFACT10-12-2000ABC1234561
CONTRACTS2-2-200093KFLLA10
CONTRACTS2-2-2000929ABC29859
CONTRACTS2-2-2000PAFJG28958
FINANCE2-2-2000PPRU_32597
SALES8-3-2001HUJI_BFIOQ6
SALES6-4-2001LLOA_82J5J5
CONTRACTS3-7-2000KKO92854_dd4
CONTRACTS1-25-2000DKEF3
CONTRACTS10-2-2000ABC925_24FN2
RCRD SRCLOAD DTSCUSTOMER #ID
MANUFACT10-12-2000ABC1234561
A Hub is a list of unique business keys.
-
5/21/2018 Data Vault Whats the Combo-1
16/31
Data VaultThree Basic Building BlocksSATELLITE
Primary Key
Load DTS
Detail
Business Data
Aggregation Data
{Update User}
{Update DTS}
Record Source
Primary Key
Load DTS
Detail
Business Data
Aggregation Data
{Update User}
{Update DTS}
Record Source
CONTRACTS10-2-2000ABC925_24FN2MANUFACT10-12-2000ABC1234561
RCRD SRCLOAD DTSCUSTOMER #ID
CONTRACTS10-2-2000ABC925_24FN2MANUFACT10-12-2000ABC1234561
RCRD SRCLOAD DTSCUSTOMER #ID
CONTRACTSWorldwide Suppliers Inc10-14-20002
CONTRACTSWorldPart10-2-20002
CONTRACTSABC DEF Incorporated12-2-20001
MANUFACTABC Worldwide Suppliers, Inc10-31-20001
MANUFACTABC Suppliers, Inc10-14-20001
MANUFACTABC Suppliers10-12-20001
RCRD SRCNAMELOAD DTSCSID
CONTRACTSWorldwide Suppliers Inc10-14-20002
CONTRACTSWorldPart10-2-20002
CONTRACTSABC DEF Incorporated12-2-20001
MANUFACTABC Worldwide Suppliers, Inc10-31-20001
MANUFACTABC Suppliers, Inc10-14-20001
MANUFACTABC Suppliers10-12-20001
RCRD SRCNAMELOAD DTSCSID
CUSTOMER NAME SATELLITE
A Satellite is a time-dimensional table housing detailed
information about the hubs business keys.
-
5/21/2018 Data Vault Whats the Combo-1
17/31
Data VaultThree Basic Building Blocks
Hub Employees
ELAName
EEOCDates
Employees HUB and some of its Satellites
-
5/21/2018 Data Vault Whats the Combo-1
18/31
Data VaultThree Basic Building BlocksLINK
Primary Key
Load DTS
Record Source
CONTRACTS10-2-2000ABC925_24FN2
MANUFACT10-12-2000ABC1234561
RCRD SRCLOAD DTSCUSTOMER #ID
CONTRACTS10-2-2000ABC925_24FN2
MANUFACT10-12-2000ABC1234561
RCRD SRCLOAD DTSCUSTOMER #ID
FINANCE10-14-2000CONT259101
FINANCE10-14-2000CONT212100
RCRD SRCLOAD DTSCONTACT #ID
FINANCE10-14-2000CONT259101
FINANCE10-14-2000CONT212100
RCRD SRCLOAD DTSCONTACT #ID
FINANCE10-14-20001012
FINANCE10-14-20001001
RCRD SRCLOAD DTSCONTACT IDCSID
FINANCE10-14-20001012
FINANCE10-14-20001001
RCRD SRCLOAD DTSCONTACT IDCSID
A Link is an associative or intersection table, representing the
connection between information between business elements.
Link Table
-
5/21/2018 Data Vault Whats the Combo-1
19/31
Data VaultThree Basic Building Blocks
Hub Employees
ELAName
EEOCDates
Hub Schools
Geo CdAddr
FloorBldg
Assign
Sat
Hub and SatellitesHub and Satellites
Link and Satellites
-
5/21/2018 Data Vault Whats the Combo-1
20/31
Data Vault
Brief History and Revisit Some
Definitions
Three Basic Building Blocks of the
Data Vault
Advanced Features Questions
-
5/21/2018 Data Vault Whats the Combo-1
21/31
Data VaultAdvanced Features
Point-In-Time A structure which sustains integrity of joins across time to all
the SATELLITES that are connected to the HUB or LINK.
Bridge A single row table that contains the latest Load Date Time
Stamp (DTS). Similar to Point-In-Time except it spans a
subject-area or a schema.
User Grouping Link The information provides the user with a customized view
from a reporting standpoint and does not affect the
underlying information.
-
5/21/2018 Data Vault Whats the Combo-1
22/31
Data VaultAdvanced Features
Point-In-Time (PIT)
MANUFACT10-12-2000ABC1234561
RCRD SRCLOAD DTSCUSTOMER #ID
MANUFACT10-12-2000ABC1234561
RCRD SRCLOAD DTSCUSTOMER #ID
ABC DEF Incorporated12-2-20001
ABC Worldwide Suppliers, Inc10-31-20001
NAMELOAD DTSCSID
ABC DEF Incorporated12-2-20001
ABC Worldwide Suppliers, Inc10-31-20001
NAMELOAD DTSCSID
123 World Drive12-5-20001
123 World Dr10-14-20001
ADDRESSLOAD DTSCSID
123 World Drive12-5-20001
123 World Dr10-14-20001
ADDRESSLOAD DTSCSID
12-5-200012-2-200012-5-20001
10-14-200012-2-200012-2-20001
10-31-2000
10-14-2000
NAME_LOAD_DTS
10-14-2000
10-14-2000
ADDRESS_LOAD_DTS
10-31-20001
10-14-20001
LOAD DTSCSID
12-5-200012-2-200012-5-20001
10-14-200012-2-200012-2-20001
10-31-2000
10-14-2000
NAME_LOAD_DTS
10-14-2000
10-14-2000
ADDRESS_LOAD_DTS
10-31-20001
10-14-20001
LOAD DTSCSID
A structure which sustains integrity of joins across time to
all the satellites that are connected to the hub.
Customer Name Satellite Customer Address Satellite
Hub Key
Load Date
{Sat Load DTS}
{Sat Load DTS}
{Rec Source}
Hub Key
Load Date
{Sat Load DTS}
{Sat Load DTS}
{Rec Source}
-
5/21/2018 Data Vault Whats the Combo-1
23/31
Data VaultAdvanced Features
Bridge
A single row table that contains the latest
Load DTS with multiple columns. A Bridgeis not a helper table.
Similar to a PIT Table except it spans or
applies to a subject-area or schema. A PITTable is HUB (LINK) and SATELLITE
specific.
-
5/21/2018 Data Vault Whats the Combo-1
24/31
Data VaultAdvanced Features
User Grouping Link
Primary Key
Load DTS
Record Source
Primary Key
Load DTS
Record Source
EXCEL10-2-2000Small Customers2
EXCEL10-12-2000Big Customers1
RCRD SRCLOAD DTSGrouping LabelID
EXCEL10-2-2000Small Customers2
EXCEL10-12-2000Big Customers1
RCRD SRCLOAD DTSGrouping LabelID
FINANCE10-14-2000ABC-1101
FINANCE10-14-2000ABC295882100
RCRD SRCLOAD DTSCustomer #ID
FINANCE10-14-2000ABC-1101
FINANCE10-14-2000ABC295882100
RCRD SRCLOAD DTSCustomer #ID
EXCEL10-14-20001011
EXCEL10-14-20001001
RCRD SRCLOAD DTSCustomer #Grp#
EXCEL10-14-20001011
EXCEL10-14-20001001
RCRD SRCLOAD DTSCustomer #Grp#
The User Grouping Link, allows users to state how they wantroll-ups to occur in situations where source data doesnt exist.
BASE TABLE:
-
5/21/2018 Data Vault Whats the Combo-1
25/31
Data VaultHow is DPS using DVHub_Students
Student_IDSIS_Code
Load_DTS
Rec_SRC
Hub_Students
Student_IDSIS_Code
Load_DTS
Rec_SRC
Hub_Schools
School_IDSchool_Number
Load_DTS
Rec_SRC
Hub_Schools
School_IDSchool_Number
Load_DTS
Rec_SRC
Hub_Employees
Employee_ID
HR_Emp_ID
DPSID
Load_DTS
Rec_SRC
Hub_Employees
Employee_ID
HR_Emp_ID
DPSID
Load_DTS
Rec_SRC
Lnk_School_Enrollments
Sch_Enr_ID
School_ID
Student_ID
Grade_Name
Load_DTS
Rec_SRC
Lnk_School_Enrollments
Sch_Enr_ID
School_ID
Student_ID
Grade_Name
Load_DTS
Rec_SRC
Lnk_Teacher_Schools
Teacher_School_IDSchool_ID
Employee_ID
Load_DTS
Rec_SRC
Lnk_Teacher_Schools
Teacher_School_IDSchool_ID
Employee_ID
Load_DTS
Rec_SRC
The directionof the arrows
equate to
crows feet.
-
5/21/2018 Data Vault Whats the Combo-1
26/31
Data VaultWhy is DPS using DV
Storage considerations.
Vertical partitioning of data (rate ofchange).
All the FACTS all the TIME.
Scalability and Extensibility.
-
5/21/2018 Data Vault Whats the Combo-1
27/31
Data VaultWhat was not covered.
How to apply Data Vault Modeling.
Best practices.
Lessons Learned. Dan Linstedts use of DECODE in
determining changed data capture.
Whos data is it? SLAs?
The new regulations / compliance that willaffect all of us.
-
5/21/2018 Data Vault Whats the Combo-1
28/31
Data VaultQuestions?
-
5/21/2018 Data Vault Whats the Combo-1
29/31
Data Vault - References
DATA VAULT OVERVIEW: THE NEXT EVOLUTION IN DATA MODELING
Dan Linstedt - Core Integration Partners, Inc.http://www.tdan.com/i021hy01.htm
DATA VAULT OVERVIEW THE NEXT EVOLUTION IN DATA MODELING SERIES 2Dan Linstedt - Core Integration Partners, Inc.
http://www.tdan.com/i023hy02.htm
DATA VAULT - SERIES 3 END-DATES AND BASIC JOINSDan Linstedt - Core Integration Partners
http://www.tdan.com/i024hy02.htm
DATA VAULT - SERIES 4 LINK TABLESDan Linstedt - Core Integration Partners
http://www.tdan.com/i027ht04.htm
DATA VAULTTM OVERVIEW THE NEXT EVOLUTION IN DATA MODELING SERIES 5LOADING TABLESDan Linstedt - Core Integration Partners
http://www.tdan.com/i027ht04.htm
Data Vault ModelingClass Materials and Notes; copyright 2002-2003
Dan LinstedtCore Integration Partnershttp://www.coreintegration.com
Home of the Data Vault; www.danlinsedt.com
Audit the Dataor Else. Un-audited Data Access Puts Business at High Risk ; Bloor, Robinand Baroudi, Carol; Lumigent, Inc.; copyright 2004
http://www.tdan.com/i021hy01.htmhttp://www.tdan.com/i021hy01.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.coreintegration.com/http://www.coreintegration.com/http://www.danlinsedt.com/http://www.danlinsedt.com/http://www.coreintegration.com/http://www.coreintegration.com/http://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i027ht04.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i024hy02.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i023hy02.htmhttp://www.tdan.com/i021hy01.htmhttp://www.tdan.com/i021hy01.htm -
5/21/2018 Data Vault Whats the Combo-1
30/31
Data VaultContact Information
JEFFREY MEYER
-
5/21/2018 Data Vault Whats the Combo-1
31/31
Data Vault
RMOUG Training Days
2006Colorado Convention Center
Denver, Colorado
February 15-16