1 1 the it perspective: data warehousing, management, and analytical structures rafal lukawiecki...
TRANSCRIPT
11
The IT Perspective: Data Warehousing, Management, and Analytical Structures
Rafal LukawieckiStrategic Consultant, Project Botticelli [email protected]
22
ObjectivesExplain the basics of:1. Master Data Management2. Data Warehousing3. ETL4. OLAP/Multidimensional Data
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material presented is not certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to the information in this presentation.
Portions © 2010 Project Botticelli Ltd & entire material © 2010 Microsoft Corp. Some slides contain quotations from copyrighted materials by other authors, as individually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Project Botticelli Ltd as of the date of this presentation. Because Project Botticelli & Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided after the date of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE.
This seminar is based on a number of sources including a few dozen of Microsoft-owned presentations, used with permission. Thank you to Chris Dial, Tara Seppa, Aydin Gencler, Ivan Kosyakov, Bryan Bredehoeft, Marin Bezic, and Donald Farmer with his entire team for all the support.
33
ANALYZE
INTEGRATE
MANAGE
REPORT
STEAWARDSHIP
44
SQL Services – Why?Install only the ones you needWhich?
Integration ServicesGet your data from the world outside (ETL)
Analysis ServicesCubes, Data Mining, support for PowerPivot on SharePoint
Reporting ServicesDIY Report Builder and traditional “big” reports
Master Data ServicesQuality of critical master data (cities, colours, customers)
Database EngineData warehouse and OLTP relational storage
5
Master Data Management
66
MDM
Ensures consistency of data across all organisational usesImpacts overall data qualityProcesses and tools for:
Collection, aggregation, matching, distribution, and persistence of master data
ConsistentlyRelated to Federated Data Management
Key to MDM: Modelling
88
MDM Processes
• Batched Acquisition from Staging Tables• Members, Attributes,
Parent-Child Relationships• SQL Integration Services
Import & Integration
• Versioning Changes• Auditing• Compliance• Tracking of Instances
Modeling • Subscription Views• Export to:• Operational Systems• Data Warehouses• BI Analytics• Reporting Tools
Export & Subscription
99
Microsoft Master Data ServicesSQL 2008 R2 Enterprise, Datacenter, Developer
Tools:Master Data Manager
Primary tool for managing your master data
MDS Configuration Manager
IT Pro toolMDS Web Service
For developers wanting to extend MDS
Concepts:ModelsEntitiesAttributesMembersHierarchiesCollectionsVersionsDatabase
1010
Modelling Master Data
Model organises data at highest levelAllowing versioning of changes to data
There are typically four categories of models:People (Customers, Staff)Places (Geographies, Cities, Countries)Things (Products)Concepts (Accounts, Behaviours, Transactions)
1111
Example: Product MDM Model
Product (model)
Product (entity)
Name (free-form attr)
Code (free-form attr)
Subcategory (domain-
based attr)
Name (free-form attr)
Code (free-form attr)
Category (domain-
based attr)
Name (free-form attr)
Code (free-form attr)
StandardCost (free-form
attr)
ListPrice (free-form
attr)
Photo (file attr)
12
1. Reviewing a Data Model Using Master Data Services
demos
13
Data Warehouse
1414
OLE DB
ODBC
DB2Oracle
XML
SQL Server
Analysis Services
SQL Server
Report Server Models
SQL Server
Data Mining Models
SQL Server
Integration Services
MySAP
Hyperion Essbase
SAP
NetWeaver BISQL
Server
Teradata
Rich ConnectivityData Providers
1515
Star Schema
1616
Star Schema Benefits
Simple, not-so-normalized modelHigh-performance queries
Especially with Star Join Query OptimizationMature and widely supportedLow-maintenance
1717
Snowflake Dimension TablesDefine hierarchies using multiple dimension tablesSupport fact tables with varying granularitySimplify consolidation of heterogeneous data
Potential for slower query performance in relational reporting
No difference in performance in Analysis Services database
2323
Slowly Changing Dimensions
Maintain historical context as dimension data changesThree common ways (there are more):
Type 1: Overwrite the existing dimension recordType 2: Insert a new ‘versioned’ dimension recordType 3: Track limited history with attributes
27
Integration and ETL
2828
Let’s do ETL with SSIS
SQL Server Integration Services (SSIS) serviceSSIS object modelTwo distinct runtime engines:
Control flowData flow
32-bit and 64-bit editions
3030
Control Flow
Control flow is a process-oriented workflow engineA package contains a single control flowControl flow elements
ContainersTasksPrecedence constraintsVariables
3131
Data FlowThe Data Flow Task
Performs traditional ETL and moreFast and scalable
Data Flow ComponentsExtract data from SourcesLoad data into DestinationsModify data with Transformations
Service PathsConnect data flow componentsCreate the pipeline
32
1. Using SQL Server Integration Services for Splitting Data
demos
33
OLAP/Multidimensional Data
3434
Cube = Unified Dimensional Model
Multidimensional dataCombination of measures and dimensions as one conceptual model
Measures are sourced from fact tablesDimensions are sourced from dimension tables
3737
Hierarchy
Defined in Analysis ServicesOrdered collection of attributes into levelsNavigation path through dimensional spaceVery important to get right!
Customers by Geography
Country
State
City
Customer
Customers by Demographics
Marital
Gender
Customer
3838
Measure Group
Group of measures with same dimensionalityAnalogous to a fact tableCube can contain more than one measure group
E.g. Sales, Inventory, FinanceDefined by dimension relationships
3939
Sales Inventory Finance
Customers X
Products X X
Time X X X
Promotions X
Warehouse X
Department X
Account X
Scenario X
Measure Group
Measure GroupD
imen
sio
n
42
1. Using BIDS to Review Dimension Design
2. Cube Design and Functionality
demos
4343
Summary
As a platform for enterprise Business Intelligence you should consider four services:
Data Warehouse (can be relational)Process for Data Management (MDS)Process for Data Integration (ETL)Analysis (OLAP, Data Mining, Columnar)
= SQL Server 2008 R2
4444
© 2010 Microsoft Corporation & Project Botticelli Ltd. All rights reserved.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material presented is not certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to the information in this presentation.
Portions © 2010 Project Botticelli Ltd & entire material © 2010 Microsoft Corp. Some slides contain quotations from copyrighted materials by other authors, as individually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Project Botticelli Ltd as of the date of this presentation. Because Project Botticelli & Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided after the date of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE.