leaving a metadata trail chapter 14. defining warehouse metadata data about warehouse data and...
TRANSCRIPT
![Page 1: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/1.jpg)
Leaving a Metadata Trail
Chapter 14
![Page 2: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/2.jpg)
Defining Warehouse Metadata
Data about warehouse data and processing Vital to the warehouse Used by everyone
MetadataMetadata
![Page 3: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/3.jpg)
The Key to Understanding Warehouse Information Specifies data location Manage data Aids use of information Describes the data Documents the development process Provides a record of changes Records enhancements over time
![Page 4: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/4.jpg)
Metadata UsersIT developers
ETT
Operational
Enduser
Metadatarepository
WarehouseEnd users
![Page 5: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/5.jpg)
Types of Metadata End user: - Key to a good warehouse - Navigation aid - Information provider ETT: - Maps structure - Source and target information - Transformations Operational: - Load, management, scheduling processes - Performance
![Page 6: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/6.jpg)
Developing a Metadata Strategy Define a strategy to ensure high-quality metadata
useful to users and developers. Primary strategy considerations: - Define goals and intended use. - Identify target users. - Choose tools and techniques. - Choose the metadata location. - Manage access to the metadata. - Manage access to the metadata. - Integrate metadata from multiple tools. - Manage change.
![Page 7: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/7.jpg)
Defining Metadata Goals and Intended Usage Define clear goals. Identify requirements. Identify intended usage.
MetadataMetadata
![Page 8: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/8.jpg)
Identifying Target Metadata Users Who are the metadata users? - Developers - End users What information do they need? How will they access the
metadata?
![Page 9: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/9.jpg)
Choosing Metadata Tools and Techniques Tools - Data modeling - ETT - End-user query and analysis Database schema definitions COBOL copybooks Middleware tools
![Page 10: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/10.jpg)
Choosing the Metadata Location
Usually the warehouse server Possibly on operational platforms Desktop tool with metalayer
Metadatarepository
Warehouse End users
ExternalsourcesOperationalDatasources
![Page 11: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/11.jpg)
Managing the Metadata Managed by the metadata manager Maintained by the metadata architect Standards produced by the metadata
architectMetadatarepository
Warehouse End users
ExternalsourcesOperationalDatasources
![Page 12: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/12.jpg)
Integrating Multiple Sets of Metadata Multiple tools may generate their
own metadata. There are many metalayer
integration issues. Metadata exchangeability is
desirable.
![Page 13: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/13.jpg)
Managing Changes to Metadata Different types of metadata have
different rates of change. Consider metadata changes
resulting from refresh cycles.
![Page 14: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/14.jpg)
Examining Types of Metadata ETT metadata End user metadata
ETTEnduser
Metadatarepository
Warehouse
![Page 15: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/15.jpg)
ETT Metadata Business rules Source tables, fields, and key values Ownership Field conversions Encoding and reference table Name changes Key value changes Default values Logic to handle multiple sources Algorithms Time stamp
Extraction
Externalsources
Operational data sources
Staging file
![Page 16: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/16.jpg)
Extraction Metadata Space and storage requirements Source location information Diverse source data Access information Security Contacts Program names Frequency details Failure procedures Validity checking information
Extraction
Externalsources
Operational data sources
Staging file
![Page 17: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/17.jpg)
Transformation Metadata Duplication routines Exception handling Key restructuring Grain conversions Program names Frequency Summarization
TransformExternalsources
Operational data sources
Staging file
![Page 18: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/18.jpg)
Transportation Metadata Method of transfer Frequency Validation procedures Failure procedures Deployment rules Contact information
Externalsources
OperationalData sources
ETT
TransportTransport
Transport
Staging file
Metadatarepository
Warehouse
![Page 19: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/19.jpg)
End-User Metadata 739516 1816 666 17.62 Associate the metadata description Analogous to Oracle Data Dictionary views
Enduser
Metadatarepository
Warehouse
![Page 20: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/20.jpg)
Example of End User Metadata
TableName
ColumnName
Data Meaning
Product Prodid 739516 Unique identifier for the product
ProductProduct
Valid_dateWare_loc
01/971816
Last refresh dateWarehouse location number
Product Ware_bin 666 Warehouse bin number
Product Weight 17.62 Packed shippingWeight in kilograms
![Page 21: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/21.jpg)
More End-User Metadata Information
Location of fact and dimensions Availability Description of contents Algorithms for derived and summary data Owners of data and telephone number
Enduser
Metadatarepository
Warehouse
![Page 22: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/22.jpg)
Historic Context of Data Supports change history Maintains the context of information
Operational Warehouse
Content
Structure
repositoryMetadata
94 95 96 97 98
![Page 23: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/23.jpg)
Types of Context Simple: - Data structure - Naming conventions - Metrics Complex: - Product definitions - Markets - Pricing External: - Economic - Political
Warehouse
94 95 96 97 98
![Page 24: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/24.jpg)
Additional Metadata Content and Considerations Summarization algorithms Relationships Stewardship Permissions Pattern analysis Reference tables
![Page 25: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/25.jpg)
Metadata Management Tools Carleton Evolutionary Technologies Hewlett Packard Informatics Oracle Designer Platinum Technology Prism Solutions Sagent
![Page 26: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/26.jpg)
Common Warehouse Metadata
Design and Administratioon
![Page 27: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/27.jpg)
Common Warehouse Metadata Future
WarehouseBuilder
Oracle8iServer
Discover
ExpressServer
Commonmetadata
![Page 28: Leaving a Metadata Trail Chapter 14. Defining Warehouse Metadata Data about warehouse data and processing Vital to the warehouse Used by everyone Metadata](https://reader035.vdocuments.net/reader035/viewer/2022070323/56649db65503460f94aa8a91/html5/thumbnails/28.jpg)
Summary
This lesson discussed the following topics: Definitions Integration Contents Storage Creation Selection Tools