Data Governance and Data Quality

1

Stewardship

Agenda

Discuss Data Quality and Data Governance

Considerations for future technical decisions

2

3

Portal Embedded InfoApps™

Applications Legacy Systems Relational/Cubes Big Data Columnar/In Memory Unstructured Social Media Web Services Trading Partners

Integration

SocialHot

BadFeedback

Predictive Analytics

Sentiment and Word Analytics

Search Location Analytics

Mobile Write-Back

Data Discovery Reporting Dashboards Casting and Archiving

Active Technologies

High-Performance Data Store

Data Quality

Data Governance

Master Data Management

Batch ETL Real-Time ESB

Integrity

Intelligence

4

5

Tell yourself they are the same, it doesn’t matter!

Which ones are bigger?

Data Value Chain

Connect

Data

Application

E-Business

Legacy

SaaS

Big Data

Data

Move

Batch

Transactional

Event Driven

SOA

Automate

Orchestrate

Fix

Profile

Clean

Enrich

“DQ Firewall”

Relate

Master Data

Organize

Synchronize

360 View”

Govern

Monitor

Visualize

Alert

Remediate

Report

History

Business Intelligence

Dashboards

Analytics

Ad Hoc Reports

Enterprise Search

Mobile

Visualize

Predictive

Social Intelligence

Performance Mgt

Business Value

Integration Integrity Intelligence

Anyone Facing these Challenges?

7

Difficult to produce accurate customer

count

Not clear who

“owns” data

Different answers for the same

question

LOB “manage” their own

data in Excel

Duplicate data across

systems

Resources tied up

researching and fixing

data issues

Errors in processing data due

to incomplete

data No single version of the truth

Why Information Management

Barriers to Information Management Address disparate, dirty and timeliness of data

Data Spread Across Too Many Apps and Systems 67%

Multiple Versions of the Truth 64%

Data Not Timely Enough 60%

Data Not Clean Enough To Use 58%

Technology Not Able to Meet Needs 57%

Source: Ventana Research Information Management Benchmark Research

Data needs to be treated as a Business Asset

9

Gartner has researching the concept that information is an under-managed, under-utilized asset because it's not a balance sheet asset.

Gartner – 25% of critical data is flawed

Not ALL data should be managed equally

How can we think about Data Quality?

Potential Energy = m * h * g

10

There will not be a test after the talk!

How can we think about Data Quality?

11

Dat

a Q

ual

ity

Time

Pote

nti

al o

f D

ata

Optimal Data usage gives continual energy to your organization

Available

Accessible

Authorized

Timely

Efficient

Usable

Defined

Recognized

Structured

Reliable

Consistent

Accurate

Complete

Auditable

What are common aspects of Objective Data Quality?

Data Stewardship enables Data Quality

12

What are common aspects of Subjective Data Quality?

Data Stewardship enables Data Quality

13

Trust

Understandability

Interpretability

Objectivity

Timeliness

Relevance

Benefits of Data Quality

14

Improved productivity with efficient processes Reduced errors HR

Reduced staff for data cleansing tasks Improved productivity of standards based application development IT

Single View of Customer (increases customer satisfaction) Enable better interaction with customers across touch points Ability to cross sell and up sell products and services Accurate Install Base information

Sales, Marketing and Customer Service

Enhanced and Accurate Reporting Efficient Planning and Budgeting with increased granularity Enhanced ability for regulatory compliance Improved decision making based on accurate data

Finance and Corporate

Business Function Benefit of Data Quality, Governance, and MDM

Remediation and Data Stewardship

16

Data stewardship is the management and oversight of an organization's data assets to help provide business users with high-quality data that is easily accessible in a consistent manner.

How is it maintained?

How is it used? How is it

consumed?

People

Process

Technology

Success in Data Quality relies on the harmony of…

People

Process

Technology

Data Governance is supported via Remediation

The “Harmony” is supported via Data Governance

People

Process

Technology

People…

Element of Success: Data Governance

20

Start with a realistic goal

Provide a plan

Be clear with roles & responsibilities

Marketing! Marketing!

Early

Executed

Replicated

Governed

Where is your organization?

Information Management Maturity Quadrant

Wide-spread

Siloed

Sporadic

Systemic

5 Roles of a Data Steward

22

Promote and drive practical governance guidelines throughout an organizations data ecosystem Lead

Work with the business to understand data needs and find better or more efficient processes. Then, the steward can craft appropriate processes to the data uses. Map Per Forrester, “A data steward should drive the implementation and enforcement of requirements for services including, but not limited to, data management, data brokering, customer data integration and data appends. To further drive organizational consistency, the data steward should participate in the vendor evaluation process as necessary to ensure compliance.”

Define

A data steward must keep up-to-date on changes to data-related legislation for external business implications as well as internal communications and compliance.

Be an Expert

Along with daily roles, the data steward should be a point person on the organization’s data evolution. Meaning as new data requirements arise, they must advocate that data governance best practices continue to be utilized.

Advocate

People

Process

Technology

Success in Data Quality relies on the harmony of…

Which Factors to consider

Deciding on the right “Process”

24

Current data skills Company culture

Data reputation Current opinion on data ownership

Maturity of KPI Culture

Reusability of data.

Data Stewardship can be broken into 5 different patterns of allocating the remediation process responsibilities

Allocating Resources to the Stewardship Process

25

By Subject Area

By Function

By Business Process

By System

By Project

• Each Data Steward is in charge of their own subject area. One is in charge of customer and another is in charge of product.

Stewardship by Subject Area

26

Product

Location

Customer

Vendor

Data Management &

Data Governance Processes

Stewardship by Subject Area

27

Cons Focus may be at expense

of broader business benefits (Customer Retention for example).

Size differences of Domains.

Might be difficult to tie Data Steward back to business initiatives.

Pros Boundaries are clear Subject Knowledge Grows

over time

• Each Data Steward focuses on their line of business or department. Such as Marketing or Finance.

Remediation by Function

28

Business Rules & Standards

ERP CRM Inventory FMS

Finance Sales Customer

Service Logistics Marketing

Remediation by Function

29

Cons

Multiple data stewards in different

departments may be managing

and manipulating the same data.

The nature of this model means

that data stewards are rarely

motivated to collaborate across

functional boundaries

Functional data stewardship won’t

work in companies that have

prioritized enterprise-class “single

view” initiatives or consolidation

programs.

Pros

Bounded by the organization

means easier to establish

definitions and rules.

Will be business-savvy and familiar

with the data’s context

They know the team

• Each Data Steward is assigned to a single business process. For example Sales or Enrollment.

Remediation by Business Process

30 Tip : For very mature data-driven organizations

Start End

Start End

Start End

Start End

Data Management & Data Governance Processes

Sales

Enrollment

Procurement

Reporting

Remediation by Business Process

31

Cons

Data ownership is more difficult to

assign. A broader data governance

program is critical for managing such

situations.

Business constituents can get confused.

Consistency around similar types of

data.

In this model, data stewardship is only

as effective as the company is clear

about its processes.

Pros

Extension of exiting processes

Success measurement is more

straightforward

The process oriented model is a very

effective way to entrench data

stewardship.

Data Steward is assigned to the system that they manage the data for. Such as SAP ERP or Salesforce.

Remediation by System

32

ERP

CRM

Inventory

FMS

Data Management &

Data Governance Processes

Tip : This may have caused some of the data quality issues in the first place.

Remediation by System

33

Cons

Business people may equate data

ownership with data stewardship,

thus assuming stewardship to be

“an IT issue”

Data stewards can become myopic

as they maintain the integrity of

the data on their systems

A systems orientation doesn’t

ensure data sharing or

reconciliation.

Pros

IT can take a leadership role

Drives from a Bottom Up approach

Assigning multiple data stewards at

once is more realistic: “each core

system will have a data steward”

becomes an established practice.

• Data Steward is assigned to a project that they will manage the data for. Can be assigned through the PMO office. Examples are a Data Warehouse Implementation or ERP Migration.

Remediation by Project

34 Tip : This can be the fastest way to introduce the role to the organization

Data Management &

Data Governance Processes Project

Management Office

Remediation by Project

35

Cons

“Project” implies ‘ending’

Are skills in house?

Pros

Speed! It is part of the Project and

most organizations can add that as part

of a process easily.

Start with Project then Grow

Clear definition of success

People

Process

Technology

Success in Data Quality relies on the harmony of…

Changing the landscape

37

Data Integration, Quality, and Mastering

38

Typical Historical Approach

End State MDM hub

Data warehouse

Partner interface

Quality process

Operational systems

BI/analytics app

Data Integration, Quality, and Mastering

39

Agile Approach to MDM, Data Quality, and Data Integration

End State MDM hub

Data warehouse

Partner interface

Quality process

Operational systems

BI/analytics app

Traditional in Transition to Modern

40

Fewer use cases

More use cases

Modern Traditional

Hadoop

IoT

Streaming

Virtual DW

Data Lake

OLTP

OLAP

Data warehouses

Data marts

Point-to-point Integration

EII

The Evolution of Integration

41

Hand Coded Integration

ETL Messaging Bus

ESB EAI Hadoop-Based Integration

We Have Some Pretty Simple Problems…

According to a May 2015 Gartner Survey…

26% are deploying Hadoop, 11% in 12 months, 7% in 24 months

49% cite trying to find value as their biggest problem

57% cite the Hadoop skills gap as their biggest problem

To summarize…

Companies are investing in Hadoop, but not sure why

Companies are investing in Hadoop, but don’t know how to use it

42

Big Data Under the Control of Master Data

Hadoop

Master Data Repository

Golden Records

Hadoop can be: • Staging area for application data • Source for mastered subjects • Source for transactional subjects

Master data can: • Provide context to Hadoop data • Establish trust in big data • Guide extraction of Hadoop data

Want More Information?

44