data warehousing theory and modelling techniques building dimensional models
TRANSCRIPT
Data warehousing theory Data warehousing theory and modelling techniquesand modelling techniques
Building Dimensional Building Dimensional ModelsModels
1. Matrix Method for getting1. Matrix Method for getting started. started. 2. 2.
1.11.1 Build the Matrix Build the Matrix
1.21.2 Use the four-step method Use the four-step method
22. Managing the dimensional . Managing the dimensional Modelling Project.Modelling Project.
2.1 Data Warehouse Bus Architecture 2.1 Data Warehouse Bus Architecture MatrixMatrix
2.22.2 Fact Table Diagram Fact Table Diagram
2.32.3 Fact Table Detail Fact Table Detail
2.4 Dimension Table Detail2.4 Dimension Table Detail
2.5 Steps for the Modelling Team2.5 Steps for the Modelling Team
2.6 Identifying the sources2.6 Identifying the sources
2.7 using a Data Modelling Tool2.7 using a Data Modelling Tool
SummarySummary
1.11.1 Build the Matrix Build the Matrix
Table RowsTable Rows : Data Marts : Data Marts
Table ColumnsTable Columns : Dimensions : Dimensions
ExampleExample Data Warehouse Bus Architecture Matrix Data Warehouse Bus Architecture Matrix
1.1 Build the Matrix 1.1 Build the Matrix List the Data Marts (rows)List the Data Marts (rows)
Example: data marts for a telephone co.Example: data marts for a telephone co.– customer billing statementscustomer billing statements– scheduled service and installation ordersscheduled service and installation orders– trouble reportstrouble reports– marketing promotionsmarketing promotions– call detail for a billing perspectivecall detail for a billing perspective– customer inventorycustomer inventory– labor and payrolllabor and payroll– ......
1.1 Build the Matrix 1.1 Build the Matrix List the Dimensions List the Dimensions
(columns)(columns)
Example: Dimensions for the telephone co.Example: Dimensions for the telephone co.in data mart: in data mart: Customer billing statementCustomer billing statement– Time (date of billing)Time (date of billing)– CustomerCustomer– ServiceService– Rate categoryRate category– Local service providerLocal service provider– ......
Data Warehouse Bus Architecture MatrixData Warehouse Bus Architecture Matrix
1.21.2 Use the four-step Use the four-step Method to design Each Method to design Each
Fact TableFact Table
Step 1. Choose the Data MartStep 1. Choose the Data Mart
Step 2. Declare the GrainStep 2. Declare the Grain
Step 3. Choose the DimensionsStep 3. Choose the Dimensions
Step 4. Choose the FactsStep 4. Choose the Facts
Example: Telephone co.Example: Telephone co.
Four-steps e.g.Telephone Four-steps e.g.Telephone co.co.
1. 1. Data Mart:Data Mart: Customer billing Customer billing 2. 2. Grain:Grain: the individual line item on the individual line item on
each monthly customer billeach monthly customer bill 3. 3. Dimensions:Dimensions: Time, Customer, Time, Customer,
Service, PromotionService, Promotion 4. 4. Facts:Facts: Line item amount, Line item amount,
Line item quantityLine item quantity
2.2 Fact Table Diagram2.2 Fact Table Diagram
The fact table Diagram:The fact table Diagram:– Names the fact tableNames the fact table– Clearly states its grainClearly states its grain– Shows dimensions to which it is connectedShows dimensions to which it is connected– Shows all the other dimensions without Shows all the other dimensions without
connectionsconnections Example : fact table diagram for the Example : fact table diagram for the
telephone billing line itemtelephone billing line item Example :Example : Supporting information Supporting information
The telephone billing The telephone billing Fact Table DiagramFact Table Diagram
Dimension information Dimension information TableTable
Supporting information for the Fact Supporting information for the Fact Table Diagram includes the Name Table Diagram includes the Name and Description of each dimension and Description of each dimension
Dimension Name Dimension Description
2.3 Fact Table Detail2.3 Fact Table Detail
Complete list of all the facts Complete list of all the facts available through the fact tableavailable through the fact table
List includes:List includes:– actual facts in the physical tableactual facts in the physical table– derived facts presented through viewsderived facts presented through views– facts calculated from first two groupsfacts calculated from first two groups
Example: Customer billing Line item Example: Customer billing Line item fact table detail for the telephone cofact table detail for the telephone co
Customer billing Line item Customer billing Line item
Fact table detailFact table detail
2.4 Dimension Table 2.4 Dimension Table DetailDetail
Shows attributes in a single dimensionShows attributes in a single dimension Shows explicit grain of the dimensionShows explicit grain of the dimension Shows the approximate cardinality of each Shows the approximate cardinality of each
dimension attributedimension attribute Shows hierarchies and relationships Shows hierarchies and relationships
between the attributesbetween the attributes Example: Time dimension Table detailExample: Time dimension Table detail Example:Example: Dimension attribute detail Dimension attribute detail
descriptionsdescriptions
Time Dimension Table detail Time Dimension Table detail
diagramdiagram
Dimension attribute detail Dimension attribute detail descriptionsdescriptions
Documentation: Full descriptive Documentation: Full descriptive information to support the diagraminformation to support the diagram– Attribute Name, description, cardinality Attribute Name, description, cardinality – Slowly Changing Policy, Sample ValuesSlowly Changing Policy, Sample Values
Example: Time Dimension attributeExample: Time Dimension attribute detail descriptions detail descriptions
Example:Example: Many-to-many relationships and Many-to-many relationships and slowly changing dimension attributesslowly changing dimension attributes
Example:Example: Correlated attributesCorrelated attributes
Time Dimension attributeTime Dimension attribute detail descriptions detail descriptions
Many-to-many relationships and slowly Many-to-many relationships and slowly changing dimension attributeschanging dimension attributes
Correlated attributesCorrelated attributes
2.5 Steps for the Dimensional 2.5 Steps for the Dimensional Modelling TeamModelling Team
Create the Initial draft: data marts, Create the Initial draft: data marts, dimensions, data matrix and diagramsdimensions, data matrix and diagrams
Track Base FactsTrack Base Facts Track derived Facts (Example)Track derived Facts (Example) Present initial design to rest of IS teamPresent initial design to rest of IS team Select some key users to work on Select some key users to work on
projectproject Present to Business users Present to Business users
Derived Fact worksheetDerived Fact worksheet
2.6 Identifying the Sources for 2.6 Identifying the Sources for Each Fact Table and Dimension Each Fact Table and Dimension
TableTable
SourceSource. Name of the source system.. Name of the source system. Business ownerBusiness owner. Name of the primary contact within . Name of the primary contact within
the business who is responsible for this data.the business who is responsible for this data. IS ownerIS owner. Name of the person who is responsible.. Name of the person who is responsible. PlatformPlatform. Operating environment where system runs. . Operating environment where system runs. Location.Location. The actual location of the system. The city The actual location of the system. The city
and the specific machine where this system runs. and the specific machine where this system runs. DescriptionDescription. A brief description of what system does. A brief description of what system does Example: Data Source DefinitionsExample: Data Source Definitions Example:Example: Mapping data from source to Target Mapping data from source to Target
Data Source definitionsData Source definitions
Mapping data from source to Mapping data from source to TargetTarget
Source-to-target data map:Source-to-target data map:
Foundation for the development of Foundation for the development of the data staging processthe data staging process
To document specifically where the To document specifically where the data can be locateddata can be located
Example: Sample Source-to-target Example: Sample Source-to-target data mapdata map
Sample Source-to-target data map Sample Source-to-target data map (1/2)(1/2)
Sample Source-to-target data map Sample Source-to-target data map (2/2)(2/2)
Table nameTable name. The name of the logical table in the data warehouse.. The name of the logical table in the data warehouse. Column nameColumn name. The name of the logical column in the warehouse.. The name of the logical column in the warehouse. Data type.Data type. The data type of the logical column (char, number, date). The data type of the logical column (char, number, date). Length.Length. The length of the field of the logical column. The length of the field of the logical column. Target column descriptionTarget column description. A description of the logical column.. A description of the logical column. Source systemSource system. The name of the source system where data feeds . The name of the source system where data feeds
the target logical column.the target logical column. Source table/fileSource table/file. The name of the specific table or file where data . The name of the specific table or file where data
feeds the target logical column. feeds the target logical column. Source column/fieldSource column/field. The name of the specific column or field . The name of the specific column or field
where data feeds the target logical column.where data feeds the target logical column. Data transformData transform. Notes about any transformations that are required . Notes about any transformations that are required
to translate the source information into the format required by the to translate the source information into the format required by the target column. target column.
2.7 Using a Data Modelling Tool2.7 Using a Data Modelling Tool
Used to develop the physical data Used to develop the physical data modelmodel
Preferably one that stores your Preferably one that stores your model’s structure in a relational model’s structure in a relational databasedatabase
Summary (1/2)Summary (1/2)
Process used to apply dimensional Process used to apply dimensional modelling techniques modelling techniques
Bus Architecture Matrix to lay out Bus Architecture Matrix to lay out the data marts and dimensionsthe data marts and dimensions
Four-step method to design a single Four-step method to design a single data martdata mart
Diagramming techniques Diagramming techniques Data sourcing and mappingData sourcing and mapping
Summary (2/2)Summary (2/2)
In this section we focused on In this section we focused on the the
DATADATA
Next: different path of lifecycleNext: different path of lifecycleTechnical architecture
Supporting Templates Supporting Templates collected on CD-Romcollected on CD-Rom
Template 7.1 Data Mart MatrixTemplate 7.1 Data Mart Matrix Template 7.2 Dimensional Model Template 7.2 Dimensional Model
DocumentDocument Template 7.3 Derived Fact WorksheetTemplate 7.3 Derived Fact Worksheet Template 7.4 Logical table designTemplate 7.4 Logical table design Template 7.5 Data Source Definition Template 7.5 Data Source Definition
DocumentDocument Template 7.6 Source to Target Data MapTemplate 7.6 Source to Target Data Map