mdm dg bestpractices techgig dc final cut - copymaster data management data governance leadership...

© Copyright 2010 Hitachi Consulting

www.hitachiconsulting.com

Master Data Management (MDM) Data Governance Leadership and Best Practices

Dinesh Chandrasekar Practice Director CRM & MDM Hitachi Consulting , GDC

1


Agenda

2 2

Impact of Poor Data & Need for DQ Why MDM & Customer Hub Customer Data Problems & Solutions Significance of Data Governance Data Governance Leadership Strategies Data Stewardship Best Practices Open Forum


Acronyms

Commercial in Confidence 3

EIM – Enterprise Information Management EDM – Enterprise Data Management MDM – Master Data Management DM – Data Management DG – Data Governance DQ – Data Quality SOR – System of Record KPI – Key Performance Indicators UCM – Universal Customer Master CDH – Customer Data Hub PDH – Product Data Hub SH – Supplier Hub & Site Hub CH – Customer Hub


How clean is your Wind Shield ?


“ Ultimately, poor data is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some

point, you either have to stop and clear the windshield or Risk everything.” - Ken Orr Institute


Impact of Poor Data Quality


“… Fortune 1000 enterprises will lose more money in operational inefficiency due to data quality issues than they will spend on data warehouse and CRM

initiatives.”

“Data integration and data quality are fundamental prerequisites for the successful implementation of enterprise applications,

such as CRM, SCM, and ERP.”

Ineffective Cross-sell/Up-sell

Lower call center productivity

Increased marketing mailing costs

Reduced CRM adoption rate

Customer Service

Risk, Compliance Management

Heightened credit risk costs

Potential non-compliance risk

Increased report generation costs

Increased data management costs

Increased sales order error

Delayed sales cycle time (B2B)

Mediocre campaign response rate

Operational Efficiency

Reduced IT Agility

Increased integration costs

Increased the time to bring new projects and services to market

Proliferation of data problems from silos to more applications

© Copyright 2010 Hitachi Consulting Commercial in Confidence 6

Ever proliferating islands of information

…in disparate applications covering multiple channels, divisions & functions

…duplicated, incomplete, inaccurate data

• Key enterprise processes based on unclean / incomplete data Marketing, sales, service & customer retention processes, regulatory compliance, new product introduction,…

• Unclean data makes Analytics invalid

• Error prone integration

• Slows enterprise agility and innovation

Web site

Call Center SFA Partner Fusion

App

Fusion App

SCM ERP2 Legacy ERP 1

Fragmented data is the source of the problem


Consolidate/Federate shared information into one place

Cleanse data centrally

Share data as a single point of truth as a service

Consistency siloed environments (Integrated Best of Breed) Lower data management costs Better reporting Enterprise foundation for agility & innovation

ETL

ETL

Middleware

Application Integration Architecture BI

Analytics

Web site

Call Center

SFA Partner Fusion App

Fusion App

SCM ERP2 Legacy ERP 1

MDM

MDM : The source of clean data for the enterprise Nurture one of your most valuable asset


The New Age Digital Customer


Why Customer Hub ?

Commercial in Confidence

Unify your Customer View with Customer Hub

9

Maximize Customer Retention Provides complete knowledge of customers value and history to improve customer loyalty Ensures effective marketing and selling while avoiding missteps Enables sharing of customer information with applications, business processes and point of

contact personnel

Increase Selling Efficiencies Facilitates accurate up-selling and cross-selling of products and services Provides accurate product data which reduces order entry errors and decreases days sales

outstanding Delivers full quality customer and product information at the point of contact

Reduces Cost and Risk Provides clean data to all applications and business processes increasing ROI from existing

investments Enables data governance to insure compliance and reduce risk Accelerates time-to-market of new products and services


Why Organizations engage in Customer Hub Projects?


Benefits

10

GROWTH Improve CRM

performance to increase revenue and

market share

EFFICIENCY

Operational efficiency across

multi-functions of an enterprise

IT AGILITY

Increase IT resiliency in a changing

business landscape

COMPLIANCE

Reduce operational risk and improve

regulatory compliance

CUSTOMERS ON AVERAGE

GENERATED 2%-5% INCREASED

REVENUE FROM SALES WITH

MDM

EFFICIENCY OF OPERATIONS

INCREASE WITH IMPROVED

PROCESSES AND DATA

GOVERNANCE

EFFICIENCY OF IT

OPERATIONS RESULTING IN

GREATER AGILITY OF

BUSINESS MODELS

EFFICIENCY OF IT OPERATIONS

RESULTING IN GREATER

AGILITY OF BUSINESS MODELS


Customer Hub Styles


Registry Style •Various Source System publish their data and a Subscribing Hub stores only the Foreign Keys , Source System Ids and Key data values needed for matching •The Hub runs the cleansing and matching algorithms and assigns unique global identifier to the matching records , but does not send any data back to the Source Systems •The Registry Style Hub is to build the “ Virtual Golden View of the master entity from the Source Systems”

Consolidation Style • The Consolidation Style MDM Hub has a physically instantiated, "golden" record stored in the central Hub • The authoring of the data remains distributed across the spoke systems and the master data can be updated based on events, but is not guaranteed to be up to date. •The master data in this case is usually not used for transactions, but rather supports reporting; however, it can also be used for reference operationally.

Transaction Style • In this architecture, the Hub stores, enhances and maintains all the relevant (master) data attributes. • It becomes the authoritative source of truth and publishes this valuable information back to the respective source systems. • The Hub publishes and writes back the various data elements to the source systems after the linking, cleansing, matching and enriching algorithms have done their work. Upstream, transactional applications can read master data from the MDM Hub, and, potentially, all spoke systems subscribe to updates published from the central system in a form of harmonization. •The Hub needs to support merging of master records. Security and visibility policies at the data attribute level need to be supported by the Transaction Style hub, as well.

Simple & Faster Medium Complex Complex

Short term Gain Mid term Gain Long term Gain


Oracle Enterprise Master Data Management


Gartner Magic Quadrant for Customer Hub Solutions

“UCM has the strength of the Oracle name behind it, leading to an impressive number of commitments from blue chip names in the Siebel customer base across a range of industries”

John Radcliffe, Gartner, May 2008


Unclean to clean data(Initial & Delta load) Operational exchanges

Hub / Apps

Application Integration

Architecture

Siebel

EBS

SAP

JDE

Custom

MDM Aware Apps

Hyperion DRM for Customer Hub

Data Governance Manager

MDM Analytics

Customer

Hub 8.2

Oracle Customer Hub (Siebel UCM) 8.2 Best in Class MDM Solution

Oracle Data Quality

Source

Systems

Siebel

EBS

SAP

JDE

Custom


Key Components of Oracle Customer Hub



Example of Customer Data Quality Issue A Simple Customer Table Sample

Name Address City State Zip Phone Email

Bob Williams 36 Jones Avenue Newton MA 02106 617 555 000 [email protected]

Robert Williams 36 Jones Av. MA 02106 617555000

Burkes, Mike and Ilda 38 Jones av. Nweton MA 02106 617-532(9550) [email protected]

Jason Bourne,

Bourne & Cie. 76 East 51st Newton MA 617-536-5480 6175541329

… … … … … … …

Mis-fielded data

Matching Records

Typos Mixed business and

contact names

Multiple Names

Non Standard formats

Missing Data


COMPLETENESS

CONFORMITY

CONSISTENCY

DUPLICATION

INTEGRITY

ACCURACY

Customer Data Problems today


Functionality Feature

Profiling/Pattern Detection

Parsing and Standardization

Address Validation / Cleansing

Matching and Linking

Enrichment

* OEDQ is formerly known as Datanomics Data Quality Application

Understand data status & deduce meaning from unstructured patterns

Create structured records from unstructured data Spot and correct errors; transform to std format

Valid address identification and correction

Spot / eliminate duplicates & identify related entities

Attach additional attributes and categorizations

Examples

Name: LN+FN (CHS, KOR, JPN); FN+MN+ PN+LN (Latin); Tel# is null 30%

Address field -> Address Line 1, City, State,… Nationality: US, USA, American-> USA

809 Newel rd, PALO ALTO 94301 -> 809 Newel Road, Palo Alto, CA 94303-3453

Oracle Offering

Universal DQ Connector + D&B connector + AIA 2.5 PIP for Acxiom

OEDQ Matching Server

OEDQ Cleansing Server

OEDQ Parsing & Standardization Server

OEDQ Profiling Server

Haidong Song = 宋海东 =

Haidong Song: “single, 1 child, Summit Estate, DoNot Mail”

Oracle Enterprise Data Quality Functionality in a Glance


Data Governance Leadership



DG is all about establishing the strategies, objectives and policies to effectively manage corporate data by specifying accountability on data and its related processes including decision rights.

For example, DG defines

• Who owns the data;

• Who creates records;

• Who can update them; and also,

• Who arbitrates decisions when data management disagreements arise.

People, processes and technologies are the building blocks for Data Governance

Data Governance ( DG )


Data Governance Technology Requirements

Define, Communicate & Enforce

Easily Operate hub

Monitor hub operations Fix data issues

Define enterprise master data

Define and view data policies

Data accountability

Escalation process

Administer hub

• Execute day-to-day hub operations (Consolidate, Cleanse, Share & Master)

• Perform data steward tasks, such as merge/unmerge

• Analyze hub DQ metrics

• Track sources of bad data

• Monitor hub transaction load

• Fix import errors and resubmit corrected data

• Proactively watch & repair data

• Tune data quality rules


Potential Data Governance Leadership Council

22

Data Governance Committee

Client DG Leadership Council

Lead / Business Data Managers

Roles and Responsibilities

Subject Area Business OwnersCustomer/Contact, Booking, Services etc.

IT Domain OwnersClient IT Systems

Data Stewards

· Source Steward

· End User Steward

· Data Hygiene

Steward

Process Stewards

· Sales Process

· Service Process

· Orders/Bookings

· Cancellation

Consumer Base

Executive Layer· Approve Strategy Roadmap

· Align Business and IT Goals

· Align to Client Strategy

· Approve Project Prioritization

· Advocate Compliance

Management Layer· Recommend Strategy and Goals

· Prioritize and Execute Projects

· Define Standards and Policies

· Advocate Compliance

· Act as Subject Matter Experts (SMEs)

Operations/Execution Layer· Stewardship of Data, Data SME

· IT/System/Database Administration (DBAs)

· Interface Daily with Customer Groups

· Ensure Compliance

Business IT Enterprise Wide

IT Architect

· DBA

· ETL Specialist

· Data Modeler

Development

& Maintenance

Manager

· Application Leads

· Technology Leads

· Project Delivery

Technical

Manager

IT Data

Personnel

IT Application

Personnel

· MDM Specialist

· DQM Specialist

· DQ Tools

Specialist

IT Integration

Personnel

Leadership Layer· Sponsorship, Oversight & Approval



DG Council Task Force


23

Leadership Council

• Champions of the DG Council provides the Leadership, Sponsorship and Overall Vision & Direction Serves as the Final Authority on all decisions

• The council would typically consists of a Chief Sponsor ( MDM )and top leadership from Business & IT (for e.g. CIO, VP Operations etc.)

Governance Committee

• Defines business strategies and champions the importance of data governance & data quality domain-specific data, processes, and business rules throughout Client Organization

• Sets priorities for domain-specific data quality improvement projects

• Arbitrates competing interests and makes final decisions regarding issues the Management Layer is unable to resolve

Business Data Managers & IT Administrators

• Responsible for managing specific domain-data sets and is responsible for the data stewardship and quality of that data

• Recommend specific data projects to support better Data Governance and Data Quality efforts

• Responsible for assigning IT resources to support various data projects and initiatives

• Responsible for the upkeep of IT systems and tools to support better Data Management

Data Stewards

• Stewardship of the data for a particular domain (e.g. Customer)

• Perform data cleansing, and other data quality activities for that data domain

• Ensure data standards and compliance

• Perform audits and security checks

• Serve as a liaison between IT & business with regards to data

Process Stewards

• Responsible for entering data for each business process (e.g. Sales , Marketing, Order Entry, Service Request etc.)

• Aid better data quality by supporting data corrections and communication

• Provide inputs to data collection process improvements for the specific process domain

• Serve as SME for specific data sets within the process domain


Data Governance Program Activities


High-level Activities

1. Establish Data

Governance Leadership

Organization

Define Data Governance

Organization Framework

Detailed tasks

Data Governance Activities

2. Establish Data

Governance Charter &

Vision

Establish Data

Governance Committee

Establish

Leadership Council

Identify DG Council

Champions

Formalize & Kick off Data Governance

Leadership Organization internally

Establish Governance

Charter & Vision


Goals & Objectives

Refine Data Governance Charter after

socializing with the LeadershipReview & Refine Data

Governance Goals & Objectives


Foundations & Framework

Define & Refine Leadership

Roles & Responsibilities

Nominate Data

Governance Lead

Subject Area Owners & IT Domain Owners

Communicate Charter & Vision to their teams

3. Establish the Data

Governance Framework

Processes

Identify Business Data

Managers for Customer Master


Framework Process

Identify IT Management

Resources

Review & Refine Data Governance

Framework Processes

Define Standards,

Policies & Procedures

Establish Data Governance

Compliance & Monitoring Framework

5. Establish the

Stewardship Processes

& Organization

Identify and Align

Process Stewards

Identify IT, Technical

& Project Resources

Identify/Recruit

Data Stewards

Define & Refine Stewardship

Processes including DQ Processes

Formalize the operational Data

Governance Organization

6. Formalize & Kick Off

Customer Master Data

Governance Initiative

Formalize & Kickoff Customer

Data Governance Initiative

Define Stewardship

Roles & Responsibilities

Publish, Communicate and Kick Off Data

Governance Organization across the Enterprise

4. Operationalize

Standards & Policies

Align standards with vision &

strategy; Refine standards;Establish processes to manage

and monitor standards & policiesDefine/Refine additional policies

around audit & security


Process Definitions and Improvement Activities



1. Establish Data

Governance Processes

Refer & Align with Data

Governance Roadmap

Detailed tasks

Process Definitions & Improvement Activities

2. Refine Program/

Project Management

Processes

Identify Current Program

management Framework

Identify Current Change

Management Framework

Refine/Redefine Program


Refine/Redefine Change


Establish Change

Control Processes

Identify project Management

processes in place and refine/

adopt to MDM/DG projects

3. Refine Business

Processes to support

MDM/DG Processes

Inventory current Business Processes

with touch point to customer data

Identify process improvements

for each process

Refine/Redefine business process to

align better with future state MDM

Implement Identified

Changes


Metrics Definition & Monitoring Activities



1. Establish Governance

Metrics

Detailed tasks

Metrics Definitions & Monitoring Activities

3. Refine System SLAs

and System Metrics

Identify & Define Governance

& Stewardship Metrics

Monitor & Report Governance


Operationalize

Governance Metrics

2. Establish Data Quality

Metrics

Identify & Define Data Quality

Metrics for Customer Domain

Monitor & Report Governance


Operationalize DQ Metrics for each system

(Oracle CRM on Demand , BRM etc..)

Refine/Define System SLAs

and Metrics

Monitor & Report System

SLAs and Metrics

Operationalize System

SLAs Metrics


Data Governance – Key Takeaways

Establish Data Governance Leadership Council

Establish Data Governance procedures To ensure data standards and compliance around

Data Consolidation

Data Cleansing

Data Governance

Data Sharing

Data Protection

Data Analysis

Data Decay



Addition of any global languages needs DGC approval

Rules to curtail data decay need to be formalized .e.g.. All golden records that are not updated for the last 6 months needs revisit from customer calls.

Hierarchy Management of customers needs to be visited occasionally, as new branches can be added to accounts.

Exception management process (DQ Assistant)related functionality needs revision and monitoring from DGC.

Any updates for Transports and Connectors w.r.t. change, upgrade etc needs DGC approval

Any changes to Authorization and Registry services needs approval of DGC

Some Examples of DG Council Action Items


Customer Hub

Data Stewardship Best Practices



Data Stewardship with OCH 8.2 v …


Data Stewardship with OCH 8.2 v

Data Steward performs the following operations on a day to day basis using the Data Stewardship application screens provided with OCH 8.2 o Suspect Match o Merge Request o Incoming Duplicate Overview o Guided Merge & Unmerge o Incomplete Records o Survivorship Rules o Data Decay Management

The idea is to present the features available and supported by Oracle Customer Hub 8.2 v

This is only sample set of functionalities and you may choose to

explore other options and enhancements available with the product



Merge

There are 3 possible outcomes:

Threshold Type Threshold Score Description

Auto Threshold

(Auto-merge)

>= 90 UCM will automatically merge the two records (except for Sales Records)

Manual Threshold <90 and =>70

UCM will flag the records to have a Data Steward review and determine whether or not to merge

Auto Threshold

(Create New Record)

<70 UCM will create a new record and publish the record to the boundary systems

UC Matching Threshold Scores M Merging Process

Record is sent back to boundary system

UCM process the record based on

the Matching Threshold

UCM calculates Matching

Threshold score based on the

defined attributes

Record is updated based on

Survivorship Rules

Record is sent back to boundary

system


Merge Criteria used within UCM

Threshold Score: 90% or above - the incoming record will merge with the existing record using the

survivorship rules* Less than 90% greater than 70% - the incoming record will be potentially merged depending

on the Data Steward’s decision

Matching Threshold

Accounts Attributes Survivorship Rules

• Recent – Incoming value will always

survive

• History – Existing value will always

survive

• Source – The value from the

source will survive., External

Systems or Siebel.

>=90%

<90%

>=70%

<70%

UCM Merging Process

• Account Name

• Main Phone

• Address

• City

• State

• Postal Code

If the Matching Threshold score falls within this range, the Survivorship Rules will apply

* Sales Records will never be auto merged


Data Stewards needs to review the record within the “Incoming Duplicates” screen when a Matching Threshold score is within the range of >= 70 and < 90

Data Stewards will determine if the record needs to be merged with another record

or should be treated as a new record

Matching Threshold

Accounts Attributes

• Account Name

• Main Phone

• Address

• City

• State

• Postal Code

>=90%

<90%

>=70%

<70%

Data Steward

Survivorship Rules

Create New Record

Link and Update

Create New

Create and Merge Accounts


Manual Link and Update Process

Yes

No

Record

Matches?

End

Data Steward

logs onto

Incoming

Duplicates

Screen in UCM

Data Steward

reviews

incoming record

Data Steward

selects “Link and

Update”

UCM updates

record using

Survivorship

Rules

Data Steward

selects “Create”

UCM updates

record as a new

record

Data Steward

queries for their

record


All Data Stewards will see the same records within the “Incoming Duplicates” Screen

Incoming Duplicate Process


Link and Update a Record After reviewing the record information, the Data Steward can return to

the “Incoming Duplicates” Screen to “Link and Update” or “Create New” When a Data Steward selects “Link & Update”, UCM will update the

record based on the predefined survivorship rules

Link and Update


Create a New Record After reviewing the record information, the Data Steward can return to the

“Incoming Duplicates” Screen to “Link and Update” or “Create New” If the Data Steward selects “ Create New”, UCM will update the record as a new

record and no survivorship rules are applied

Create New


UCM Existing Duplicates The “Existing Duplicates” screen is only used when records are loaded into UCM

using a batch process Only potential duplicates will be displayed in the “Existing Duplicates” screen Potential duplicates can be view “Duplicate Contacts” under Administration-

Data Quality and “Existing Duplicates” under Administration – Universal Customer screen.


Potential Duplicate Records

Merge Button

Guided Merge and Un Merge Process


Unmerging Records

The Unmerge Profile Screen is where the account and contact records can be unmerged:

Unmerging Records

Records that were merged within the “existing Duplicate”

screen

Un Merge Button


Reject Button

Guided Merge Button Merge Button

Merge, Un Merge and Reject Records


Guided Merge allows end-user to review duplicate records and propose merge by presenting three versions of the duplicate records and allows end user to decide how the record in the UCM should look like after the merge task is approved and committed.

• Victim: the record that will be deleted (from master BC)

• Survivor: the record that will be (from master BC)

• Suggested: output from Surviving Engine (transient to the task)

Guided Merge


Incomplete Records processing

Data Steward will analyze and re-process the Incomplete data through UCM Batch process.


43

Survivorship Rules

Survivorship Rules are used to automate the quality of the master customer data.

Once a record is determined to be merged, UCM will compare each attribute within a record and update the record accordingly

Data Steward will change the Survivorship rule weight age depends on source system’s and surviving field in Master record level.

There are three comparison methods used by Survivorship rules: • Recent – Incoming value will always survive

• History – Existing value will always survive

• Source – The value from the source will survive a.k.a., External Systems or Siebel.

UCM Merging Process UCM process the record based on

the Matching Threshold

UCM calculates Matching

Threshold score based on the

defined attributes

Record is updated based on Survivorship

Rules

Record is sent back to

boundary system

Remember that whether a record is auto merged by UCM or manually selected to be merged, the survivorship rules will apply.

UCM Survivorship Rules

© Copyright 2010 Hitachi Consulting 44

Survivorship Rule Example - Source

Name Verizon

Phone Number 4085467880

Fax Number 4086548980

Street Address 5649 Tasman Drive

City San Jose

State CA

Postal Code 93425

Country USA

Name Verizon




City San Jose

State CA

Postal Code 93425

Country USA

Name Verizon




City San Jose

State CA

Postal Code 93425

Country USA

Best version UCM record

New incoming record from Siebel (primary source) Existing Record within UCM ( from Siebel )




UCM Survivorship Rule set View


Enhanced Data Stewardship Capabilities

48 © Copyright 2009 Hitachi Consulting Commercial in Confidence

For any Questions & Clarifications Twitter : din2win Email : [email protected] [email protected]

mailto:[email protected]

mailto:[email protected]