leveraging the microsoft repository in your warehouse

Leveraging the Microsoft Repository in Your Warehouse

Saeed K. Rahimi

Graduate Programs in Software

University of St. Thomas

DWSoft Corporation

DAMA Presentation, August 2000 © Copyright 2000 Saeed K. Rahimi

Contents

Data Warehousing and Repository The Open Information Model and Meta

Data Coalition Microsoft’s Data Warehousing Strategy Integrating the Repository Into Your

Warehouse


Data Warehouse Architecture

Repository

Metadata Services

OperationalData

• Extract• Cleans• Transform• Load

Data warehouse Life Cycle

DW DM

SQL Server 7.5 - Object Persistence Services

DM

DM


What is Metadata? (Continued)

Information not just data What is 122599? Is it an integer number? A string? Christmas day 1999? Repository provides metadata about this:

mmddyy


What Is Metadata?

Information about your Systems

Source systems, Warehouse systems,ProcessesEtc.

DatabasesSource databasesWarehouse databases

Table Columns


What Is Metadata?

Information about your Transformations

From source system to data warehouseFrom data warehouse to source systems

CalculationsTotal-Sales = amount*quantity – discount +

shipping&handling


What Is Metadata? (Continued)

OLAP information Sales cube

TotalSales measureTime, product, geography dimensions

Other information Reporting Tools Modeling Tools


Metadata Benefits

Roadmap to information in the warehouse What is in the warehouse What it means How to get the information Who owns it When it was created How it was created/generated


Metadata Benefits

All users that interact with the warehouse will benefit from enhanced information

Analysts can better understand what data is available, and how it was calculated

Maintains a history prior to current processes


Metadata Benefits (Continued)

Provides a common frame of reference – “what does total sales mean”

Provides a central storage location of warehouse knowledge (helps with turnover, growth)

Allows enterprises to ensure data marts/warehouses are built using common terms and goals


Open Information Model

The schema for the repository Common set of core definitions to promote sharing

of information among different tools and vendors Covers basic set of sub-models Created with help/input from over 20 partners and

reviewed by 300 Microsoft has transferred rights to evolve the OIM

to the meta data coalition (MDC)


MDC OIM Components Current components

Unified Modeling Language (UML) Relational database schema Data transformations Multidimensional schema (OLAP) English semantics Legacy databases Component descriptions

Components under review Report definitions Entity/relationship diagrams Business engineering


Sample OIM Constructs

ITfmTransformableObject(Columns)

ITfmTransformableObjectSet

TfmObjectSetConsistsOfObject

ITfmTransformation

Tfm

Transform

ationHasTargetColumns

Tfm

Transform

ationHasSourceColum

ns

Transformation – mapping from Source column to Target column


Virtual Relationship

A

C

B

D

This

VS

A

C

B

D

This


Virtual Properties

A

C

B

D

This

VS

This

a b

c

d e

A

a b c d e

C

B

D

c

d e


Microsoft’s Strategy Impact

Prior to SQL server 7.0, DW was primarily available only to very large companies

Tools were expensive, expertise hard to find Microsoft has lowered the bar by bundling

components (with SQL server for free) and making DW available to many more companies


Microsoft Warehouse Components

Data transformation services Extract, transform, load

OLAP services Multidimensional server and analysis

Repository Information sharing

English query English to SQL translation


Repository Maintenance

Importing information into the repository has always been a challenging task Information needs to be mapped to the model

components Information capture needs to be automated

Once in the repository, maintaining versions of the information is needed New information load can not destroy the

information already in the repository


Loading the Repository

Data Transformation Services

OLAP Services

English Query Application

OLTPDatabase

Star SchemaDatabase

DB Schema

Transform

OLAP

Semantics

DB Schema

Repository

Other

OtherSources


Loading the Repository

DTS can save to repository OLAP information can be saved to

repository with SQL Server 2000 OLE DB and ODBC data sources and related

information can be imported into repository with SQL Server 2000


Implementation Issues DTS doesn’t provide complete source to target

mappings for certain transformations For example

When a query used as source,

Interfaces to many third party tools incomplete or not available Modeling tools

ERWin Visio PowerDesigner

Warehouse tools Business Objects Information Advantage


Repository Features

Extensible metamodel Version management Dynamic COM based interface Batch XIF file based interface Workspace concept Some pre-built interfaces to data warehouse

tools


Keys to Success

Ensure repository information is current Provide access to all users of the warehouse Tackle implementation in well defined, small

steps Designate someone with the responsibility

and authority to manage the repository Maintain long term focus


Access to All Users

Show users what they want to see Provide simple, easy to understand information Make warehouse information easily available “in

context” Use familiar interface and UI concepts

Web based access Client/server based access

Provide custom integration to existing tools and interfaces when necessary


Modes of Access

Administrative Access Management Meta-model (schema) changes Meta-data loading and maintenance

End-user Access Browsing Navigation Impact analysis


Types of Access

Local and over the LAN access Metamodel management Repository bulk loading Interface from other tools (loading meta-data)

Remote and over the WEB access Browser based

accessible from anywhere on the globe Navigation oriented Interface from/to other tools May be minor editing capabilities


Access Control

User Classification Different Class of Users Need to Access Different

Objects in the Repository Different User Profiles Different Read/write Access Control

Possible User Classes Business Executive Casual User Database Administrator Data Warehouse Administrator Repository Administrator

Questions?

Comments or discussions?

Emails: [email protected]

[email protected]

leveraging the microsoft repository in your warehouse

Documents