metadata overview deck - ibm overview deck - ibm

© 2008 IBM Corporation

Last Updated 9-8-08

InfoSphere Metadata Comprehensive Deck

Information Management Software

Agenda

• 2008 Metadata Market Survey

• InfoSphere Information Server Metadata Approach

• InfoSphere Business Glossary

• InfoSphere Metadata Workbench

• Third Party Metadata

• Cognos Integration

• Case Studies


2008 Metadata Market Survey

IBM and Gavilan Research Associates (GRA)


Top 5 areas companies said they need to improve their metadata:• Documenting business metadata (business rules & element descriptions) -

80.8 % of respondents

• Support for Data Governance and Data Stewardship (67.5% of respondents)

• Documenting a Data Warehouse, Data Mart, or Analytical Application (64.8% of respondents)

• Support for Data Lineage reporting and/or Data Mapping analysis (64.9%)

• Implementing a corporate Data Dictionary or Data Glossary (64.7%)

Of those surveyed:Less than 7% said their implementation was highly successful !

IBM & Gavilan Research Associates 2008 Metadata Survey – 287 respondents

IBM and Gavilan 2008 Survey Highlights: Turning your Data into Information with Metadata


What Are Your Top Metadata Management Issues ?

Respondents could choose multiple answers, which created a broad spectrum.

The “top 5” metadata management issues in 2008 are:

• documenting business metadata (business rules & element descriptions) - 80.8 % of respondents

• support for Data Governance and Data Stewardship (67.5% of respondents)

• documenting a Data Warehouse, Data Mart, or Analytical Application (64.8% of respondents)

• support for Data Lineage reporting and/or Data Mapping analysis (64.9%)

• implementing a corporate Data Dictionary or Data Glossary (64.7%)

Respondent Comments:

• “My top issue is getting metadata out of existing toolsets into a repository, where it can be accessed and reported on.”

• “Change Management is our biggest issue! How could you miss this point?”

Source: IBM / Gavilan Research Associates 2008 MetaData Market Survey


InfoSphere Information Server Metadata Approach


So what exactly is Metadata?

Metadata enables you to put context and meaning to things.

It is generated and consumed by every organization and software product.

Made by

Since

Type of food

With a special

ingredientWith many

varieties

Stored in a can

With a trademark label


Metadata Primer for Business

• Business Metadata• Business rules, Stewardship, Business Definitions, Auditing Terminology,

Glossaries, Algorithms and Lineage using business language. Audience: Business users.

• Technical Metadata• Defines Source and Target systems, their Table and Fields structures and

attributes, Documentation for Auditing Derivations and Dependencies. Audience: Specific Tool Users – BI, ETL, Profiling, Modeling.

• Operational Metadata• Information about application runs: their frequency, record counts,

component by component analysis and other statistics for auditing purposes. Audience: Operations, Management and Business Users.

B

T

O

Literally, “data about data” that describes your company’s information from both a business and a technical perspective


MetaData is what describes, defines, & inter-relates your key information assets

• “Social Security Number: a unique identification number issued to citizens, permanent residents, and temporary (working) residents by the US Government ...”

• pic XXX-XX-XXXX

• “Profit Amount: a currency value that is calculated by combining data from the Customer Master database and Wholesale Inventory applications . . .”

• Calculation included on Monthly Report

Let’s put Metadata to an Example

Suppose you have these items of “data”

• 552-90-3456

• $233,567.00


• What data or information exists ?

• Where is it being used ?

• What is its business definition ?

• What other names has it been called or is being called ?

• How is it inter-related to other information ?

• Who is using it ?

• Why do we need it ?

• When was it last updated ?

Metadata helps answer important questions such as:

What is “Profit Amount” ?How is it defined ?

How is it calculated ?Where is it stored or used ?Is it reliable ? Accurate ?

If I make a change to “Profit Amount” – what will be

impacted ?


Metadata Business Drivers

What is happening in the Data Integration Market that requires Metadata?

Governance and Compliance Regulations are increasing

- How do organizations comply and meet documentation requirements?- How can organizations ensure accountability and responsibility?

Business Competition continues to grow

- How do organizations individualize their customer experience?- How can organizations get access to information to make correct decisions?

Costs and system complexities are expanding

- How can organizations drive optimization with integration?- How do organizations manage complex software environments?

Information Management SoftwareIBM Data Governance Maturity Model - measurable results

Business Transformation


13

1990

2008

First Metadata RepositoriesIBM Data Dictionary

Single EnterpriseRepositoryIBM led AD Cycle

DataWarehouseRepositoriesETL, DQ, DPe.g. Ascential Metastage

1980

2000

1970

2007

2002Federated RepositoriesAnd Registriese.g. Unicorn

Understand how software and data services related to business processes workflows, and the requirement to publish, reuse and govern these services.

Steve Born

Metadata ConsolidationSolutions

Community FocusedRepositories… IBM buys Ascential

IBM buys Unicorn

less-ambitious more-scoped metadata

management solutionsConsolidation to central repository using

parsers, bridges and generators

e.g. Dovetail

1/3 failed miserably, 1/3 succeeded marginally, 1/3 demonstrated good ROI

$50 million to $100 million mainly maintenance

and most organizations decided live with metadata definitions in multiple

technologies, and manually synchronize

Information Server LaunchedA Brief History of Metadata – Where did we come from?


InfoSphere Information

Server MetadataIBM Clean, controlled metadata supply

Where IBM Fits in the Metadata Life Cycle

Enterprise Repositories


Why Businesses Care


Metadata Business ChallengesMetadata is naturally a very complex subject which virtually all organizations address at some point and time. Some successfully and some unsuccessfully.Key challenges:

• Obtaining agreement on what metadata means• What does metadata mean to a specific organization or division?• What metadata is important to track and manage?• How does each group use metadata for their particular job?

• Selecting the correct metadata strategy for particular business requirements• How many and what kinds of silos of metadata exist today in

organization? • Are there revenue $ at risk, compliance issues, regulatory rules which

must be addressed?• Do we have the flexibility to assess the impact of changes with our

current architecture?

• Promoting adoption of a metadata strategy and associated technology• How does our approach address our different user’s needs?• Is our approach one that easy to use and facilitates adoption rather

than hinders it?• Do we have business and technical sponsorship?


Understand Cleanse Transform Deliver

Parallel ProcessingRich Connectivity to Applications, Data, and

Content

IBM InfoSphere Information Server

Discover, model, define, and govern

information structure and

content

Standardize, merge,and correct information

Combine and restructure

information for new uses

Synchronize, virtualize and move information for in-

line delivery

Unified Deployment

Unified Metadata Management

Delivering information you can trust

The IBM Solution: InfoSphere Information Server


InfoSphere Information Server: Unified Metadata Management

Store Enhance Exchange

Active, centrally managed repository with secure access via services layer

Share and deliver relevant information across the

organization

Define relationships, control extensibility, and link 3rd party metadata

BusinessUnified Metadata Management| |Technical Operational

Any User

• Data integration metadata repository – core to InfoSphere Information Server

• Facilitates change management & reuse

• Simplifies integration

• Increases trust and confidence in information

• Enable data governance initiatives


Store Enhance

Active, centrally managed repository with secure access via services layer

Define relationships, control extensibility, and link 3rd party metadata

BusinessUnified Metadata Management| |Technical Operational

DevelopersSubject Matter Experts

DataAnalysts

Business Users

Architects DBAs

Enhance Collaboration & Alignment

Role-based Tools with Integrated Metadata

Exchange

Share and deliver relevant information across the

organization

InfoSphere Information Server: Unified Metadata Management


InfoSphere Business Glossary


Business Glossary Overview

• Applications

• Users

• Business Glossary

• Business Glossary Anywhere


Applications of a Business Glossary

Simply put, a Business Glossary is created to represent the language of the business, independent of technology

Three Primary Applications:Three Primary Applications:

1.1. OwnershipOwnership Identifying stewardsIdentifying stewards

Managing contentManaging content

2.2. CollaborationCollaboration Common, approved vocabularyCommon, approved vocabulary

Sharing domain expertise - Business & ITSharing domain expertise - Business & IT

3.3. AuditabilityAuditability Evolution of languageEvolution of language

Centralized managementCentralized management

All key enablers to regulatory compliance and All key enablers to regulatory compliance and support the IBM Data Governance Maturity Modelsupport the IBM Data Governance Maturity Model


Business Glossary: For Every Enterprise User

• General Business Users

• Business Analysts

• Data Analysts

• BI and ETL Developers

• Data Stewards

• Subject Matter Experts

• Business Glossary Authors

• Business Glossary Administrators

Examples of most common users include:Examples of most common users include:


Understanding the value of Business Metadata• In the language of the business, independent of

technology• Documents the business meaning of data & related

technology assets• Used to

• define a shared meaning• standardize names• establish responsibility, accountability, and traceability• govern access• share insights & experiences among users• represent business hierarchies• document business descriptions, examples,

abbreviations and synonyms

• Must be managed by those that understand the meaning and importance of the information assets to the business

• Better aligns the efforts of IT with the goals of the business


InfoSphere Business GlossaryCreate and manage business vocabulary and relationships

Steward

Features

• Facilitate business & IT communications by creating & managing a common business vocabulary

• Web based interface shared across enterprise business teams

• Allows creation of stewards & assignment of their responsibilities for terms & assets.

• Link business terms to information assets

Benefits

• Aligns the efforts of IT with the goals of the business • Provides business context to information technology

assets• Establishes responsibility and accountability in

accordance with data governance policies

Steward Console

Business Users

Business Interface


Populate: Import IBM Industry Models

Jump-start your Business Glossary with thousands of industry standard terms from the IBM’s Industry Models. Refined by real customer usage, leverage years of

experience and expertise to provide the best foundation for your glossary.

Insurance Healthcare

TelcoIndustrial Retail

Financial Services

Available now!Available now!


Populate: Import InfoSphere MDM for PIM

Share your product category information structures from

Master Data Management with Business Glossary enabling

access by all enterprise users.


Populate: Create and Assign Stewards

• Create and Assign Stewards to Terms, Categories or Assets

• Who is responsible for this Term?

• Who is responsible for this Category?

• Who is responsible for this Asset?

• View Contact Information for Steward


I can set this up so that my people immediately see the most used

business hierarchies

I didn’t need any special training to use the business glossary. It’s laid-

out exactly as I expected.

Access: Simple to use – business browser


Real-time access to business glossary from any desktop application

Features• Click on a term and view definition in a pop-up

window without loss of context or focus• Intelligent matching returns best candidates in a

single search

• Search engine for terms and categories

• Access steward contact information directly

• Security enforced via the Information Server common security layer

Benefits• Increased trust and acceptance of information by

delivering definitions in context• Expanded adoption of enterprise glossary outside of

Information Platform technologies• Improved information availability with multiple access

mechanisms for electronically stored information (ESI)

ANY User

From Any Application...

Pop the Definition!

InfoSphere Business Glossary Anywhere


Business Glossary Anywhere with Cognos

1. Highlight Term Active and click to look up in Business

Glossary

2. Candidate Terms returned.

3. Select Term to view full details.


Shared Information Server metadata repository

Category: CostsTerm: Tax ExpenseFull Name: Tax to be paid on Gross Income“The expense due to taxes …..”(John Walsh is responsible for updates. 90% reliable source)Status: CURRENT

Database = DB2

Schema = NAACCT

Table = DLYTRANS

Column = TAXVL

data type = Decimal (14,2)

Derivation: SUM(TRNTXAMT)

Achieve a common vocabulary between business & technical users!

InfoSphere DataStage InfoSphere Business Glossary

Manage: Business and IT understanding

33


Business Glossary Business Benefits

• Enables data governance• Common language supports compliance regulations such as Basel II• Represent and expose business relationships

• Accountability and responsibility• Assign stewards as single point of contact for control

• Improved productivity• Administrators can tailor the tool to the needs of their business users • Access information you need when you need it

• Increased collaboration• Capture and share annotations between team members• Greater understanding of the context of information.

• Greater trust in information• Immediate, in-context access to terms and definitions, improves decision making


InfoSphere Metadata Workbench


Metadata Workbench Overview

• Applications

• Users

• Metadata Workbench


Applications of Metadata WorkbenchSimply put, Metadata Workbench allows you to understand where

information came from and where it is used.

Three Primary Applications:Three Primary Applications:

1.1. TrustTrust Data lineage for traceabilityData lineage for traceability

Understand relationshipsUnderstand relationships

2.2. ProductivityProductivity Assess impact of changeAssess impact of change

Maximize re-use Maximize re-use

3.3. GovernanceGovernance Visibility and reportingVisibility and reporting

Centralized managementCentralized management

All key enablers to regulatory compliance and All key enablers to regulatory compliance and the IBM Data Governance Maturity Modelthe IBM Data Governance Maturity Model


Metadata Workbench: For Projects leads & Architects

• Project Leaders

• Data Architects and Modelers

• DataStage Developers

• BI and OLAP Developers

• Business Subject Matter Experts

• Business Users and Compliance Officers

Examples of most common users include:Examples of most common users include:


InfoSphere Metadata Workbench Exploration of assets generated and used by Information Server

Features

• Cross-suite view and administration of Information Server assets

• Explore, analyze and manage metadata graphically and textually

• Comprehensive reporting on data lineage

• Assess dependencies across Information Server & 3rd party tools

• Perform searches and build customized queries

Benefits

• Assess and mitigate change management risk

• Support compliance and governance initiatives requiring data lineage (eg Sarbanes-Oxley, Basel II)

• Proactively manage and administer complex data integration environments

IT DevelopersAdministrators

Project Managers

& DBAs


Information Server: Optimizing Application DevelopmentImport Industry

Data Models

Search and Profile Source Data

Information AnalyzerBusiness GlossaryDataStage and QualityStage

Transform and Cleanse

Map Sources to Target Model

FastTrack

Services Oriented Architecture

Information Services Director

Populate

Share

Deploy

Share Share Share

LinkDefi

ne

Create

Unified metadata for active administration, management and reporting Metadata Workbench offers complete visibility and control of

metadata

Exchange Data Structures

Link

Common Enterprise Vocabulary

Rational Data Architect


Metadata Workbench Integration Asset Categories


Metadata Workbench Feature Overview

EXPLORE

Explore key Integration Assets:

Jobs, Reports, Databases, Models, Terms, Stewards, Systems, Specifications, Quality Rules

Easy navigation of key Integration Assets

Simple and advanced search

Integrated cross-view of Information Server and 3rd party assets

Graphical view of Asset Relationships

ANALYZE

Trace data movement to and from databases, jobs and reports for full lineage

Understand business meaning of columns, tables, and other assets

Assess the impact of change across Integration assets

Robust query builder

Analyze dependencies and relationships between key Integration assets, Business Intelligence Reports and data models

ADMINISTER

Manage Integration Assets to enable in-depth analysis

Assign security roles

Link together multiple viewpoints of design assets from ETL, business, BI and modeling with operational metadata

Edit names and descriptions of Integration Assets

Access runtime information to enrich reporting

Import export manager for 3rd party integration


Explore: Homepage

The homepage offers direct access to Discover key Information Assets, Find any Information Asset or execute Queries.


Explore: Graphically Browse

Browse Database, Files, DataStage/QualityStage Jobs,

Table Definitions or other objects within their Folder and

Project placement.

Flexible browsing capabilities enable users to find information

easier and better manage complex environments


Explore: Textually Search Where and how do I find out details about assets?

• Full search and explore capabilities

• View in textual list or graphical format

• Search multiple ways including by asset class, name or by asset property


Analyze: Data Lineage Where does a Field of Data in this Report Come From?

Source TablesIBM Information Server

• Import & Browse Full BI Report Metadata

• Navigate through report attributes

• Visually navigate through data lineage across tools

• Combines operational & design viewpoint


Analyze: Cross-Tool Impact Analysis Where Happens if I Change this Column?

• Show complete change impact in graphical or list form

• Includes impact on reports in BI tools

• Visually navigate through impacted objects across tools

• Allows impact analysis on any object type


Analyze: Business MeaningWhat does this Field Mean?

• Show relationships between business terms, data model entities, and technical and report fields

• Provides cross-tool mapping of business terms

• Allows field meaning to be understood

• Allows business term relationships to be understood


Metadata Workbench Business Benefit

• Good exploitation and coordination of metadata across tools is very compelling

• Tool Integration & metadata collaboration saves time and money, and improves Quality of results.• An organization’s ability to Govern Data is significantly improved.• A organization is better able to Manage Change, more Agile.• A Shared, Common, Vocabulary saves time and effort, helps development, and makes data more

accessible & more understandable.

The key is making the metadata capture and share process effortless


Third Party Metadata


Import Export Manager for Information Server

Expand visibility of metadata touch-points in support of data integration projects

Features• Security enforced via Information Server common security

layer as well as the 3rd party application security layer

• Metadata Bridges interchange metadata with each specific application a consist of a model, a decoder, and an encoder which require no coding.

• Import capabilities for 3rd party BI tools (Cognos, Business Objects, MicroStrategy), data modeling tools (ERwin, RDA) and databases (ODBC connections to all major RDBMS)

• Support a variety of import formats including XMI, XML, UML, CWM and CSV metadata exchange formats

Benefits• No manual interface coding required for 3rd party

metadata visibility• Visibility of data modeling to ETL to report layer

minimizes risks of overlooking critical dependencies• Leverage common metadata exchange environment for

application development consistency

IT Developers

IT Administrators


Meta Integration Technologies, Inc. (MITI)

• OEM of 3rd party metadata bridges for import- More than a dozen major vendors OEM MITI bridges*

• IBM and MITI jointly certified and tested bridges

• Additional (MITI) bridges

- Many bridges are available “as-is” and can be easily enabled post installation.

* http://www.metaintegration.net/Partners/Directory.html


Fully Supported Bridges

IBM and MITI both perform QA testing independently

As of 8.1 releaseAs of 8.1 release


Bridges Provided As-Is

MITI performs QA testing independently



Bridges Provided As-Is (continued)

MITI performs QA testing independently



ETL Operational Metadata (Job Run Information)

Physical Schemas

ETL Job Design

BI Reports

Business Glossary & IS Users

The Areas of Metadata


ETL Operational Metadata (Job Run Information)

Physical Schemas

ETL Job Design

BI Reports

The Areas of Metadata Connected

Business Glossary & IS Users


Case Studies


Melbourne Health (MH)Advances clinical research with next generation genetic information infrastructure

• Melbourne Health’s legacy I/T infrastructure had limited ability to access and share medical data.

• Their research data, diagnostic results, and clinical documentation were stored in mixed formats across mixed environments: MS SQL Server, MS Excel and MS Access

• They managed disparate information silos without being able to gain insight and meaning from them – key to unlocking the advancement of medical research and improving treatments.

Challenge

• Melbourne Health is deploying their BioGrid Australia project in 3 phases from 2003-2009 in collaboration with hospitals and research institutes.

• The project utilizes the following key software:

• IBM DB2

• IBM InfoSphere Information Server: InfoSphere Business Glossary, IBM InfoSphere DataStage, and IBM InfoSphere Federation Server

• SAS Enterprise Business Intelligence Server, SAS Web Report Studio, and SAS Enterprise Guide

Solution

Benefits• The BioGrid Austalia virtual platform currently links

research databases across 16 organizations. A world’s first and milestone achievement for the research community.

• BioGrid provides access to a comprehensive set of research data which ultimately leads to improved research accuracy and treatment quality, by:

• Linking data across disparate clinical and biomedical data sources within and across institutions

• Linking patient/subject records by assigning data Unique Subject Identifiers (USI’s)

• Providing a uniform interface and access mechanism

• BioGrid is a building block of a larger national grid initiative to share research data for the priority disease group of cancer.

APPROVED FOR EXTERNAL USE


Melbourne HealthScreenshot: BioGrid Australia



Melbourne HealthClient Quotes

"At this stage, our researchers are already able to carry out previously impossible studies to stratify patients by genetic types. Researchers are already seeing the initial fruits of collaboration, which will stimulate further cooperation as BioGrid is expanded. Ultimately, BioGrid equips us with capabilities to accelerate clinical research through collaboration.“

– Robert Merriel, Director of Business Development, Melbourne Health

“IBM InfoSphere Information Server is a key toolset for our multi-phase, multi-year Molecular Medicine Informatics Model project. We are leveraging the capabilities of InfoSphere Business Glossary, a module of IBM InfoSphere Information Server, to share information about the business and technical metadata stored in our research repositories so that users can have the relevant information at their fingertips.”

– Robert Merriel, Director of Business Development, Melbourne Health



Business challenge Business intelligence reports are used to make some of this company's important business decisions. Understanding how these reports are created to ensure validity is a must have requirement. Today, when errors or questions about reports come up, a manual , time consuming, and error prone discovery process is completed by DataStage Developers to track the lineage of the BI report field in question.

SolutionAs a long-time DataStage user, this Customer saw immediate value in leveraging the Metadata Workbench to better understand lineage and impact analysis across thousands of DataStage jobs. The capabilities of the Metadata Workbench could be used in place of tedious and error prone documentation currently used to trouble shoot errors or questions about business intelligence reports. In addition, the creators of BI reports saw value in accessing the Metadata Workbench on their own to do their own research about reports in question.

Benefits Improved TRUST in BI reports

Time saved to do manual troubleshooting of errors in BI reports

Automated the documentation process

“The Metadata Workbench helps this customer prove through lineage and impact analysis that their BI reports contain Trusted Information.”

—Desiree Ruiz-Amirgholi IBM Lab Advocate and Product Manager for the Metadata Workbench

Major Insurance Company in the UK – Metadata WorkbenchImproved Trust of BI Reporting Through Lineage and Impact Analysis


Summary


Information Server Metadata Differentiators

Auditing and traceability - single data integration repository

Enhanced collaboration – seamlessly share information across user roles

Simplified development – reduce manual steps with integrated platform development

Streamlined information access – promote enterprise adoption through ease of use

Strategic integration with IBM brands – roadmap for success