Why is Reference Data “Mission Critical” Now?
Aaron Zornes Chief Research Officer The MDM Institute Conrad Chuang Director Product Marketing Orchestra Networks
We help organizations manage their most important data
Reference Data
Thin
gs Parties
Relationships
Master & Reference Data
Support over the entire life cycle of your master & reference data
Measure Integrate Manage Control Design Define
Business Glossary
Data and workflow modeling
Data quality and validation
Workflow -driven data & hierarchy
management
Integration and
distribution
DQ KPIs and
dashboards
+ Full version control on models and data + Fine-grained role-based security
Orchestra Networks has wide array of customers
Agenda
n Why is RDM mission critical today?
n How does RDM differ from (how is it similar to) MDM?
n What are the top business drivers for RDM?
n What are the “top 10” technical evaluation criteria?
n Where are most organizations focusing their RDM efforts?
n Q&A
© 2014 The MDM Institute The-MDM-Institute.com
What is Reference Data?
Reference data =“coded, semantically stable, relatively static data sets shared by multiple constituencies”
(people, systems, & other master data domains)
Customers
Product
Industry
Sales Person
Geo
Cost / Revenue
Acct
Business Unit ID
In the logical view, private & public forms of reference data connect domains & application; consistent values
(& semantics) required for multi-domain views & hierarchies © 2014 The MDM Institute The-MDM-Institute.com
n Errors in reference data will ripple outwards affecting quality of master data in each domain, which in turn affects quality in all dependent transactional systems
n RDM needed in both operational & analytical MDM use cases where capability often used to provide attributes, hierarchies & KPIs
Why Reference Data? Why Now?
Central role of reference data means RDM becoming “starting point” for many organizations planning MDM & DG
Systemic Failure
Inconsistent Reporting
Transaction Failure
Regulatory Non-compliance
© 2014 The MDM Institute The-MDM-Institute.com
Social Data
TXN Data
Master Data
Enterprise data relies upon clean & consistent reference data.
Reference Data
© 2014 The MDM Institute The-MDM-Institute.com
Many analyst inquiries cite poor reference data a leading cause of errors in the organization
Reference Data Categories Multi-Domain
RDM Use Cases Real-Time / Transactional
RDM Use Cases
Public (External)
Countries & Subdivisions (FIPS10) Currencies (ISO 4217) Time Zones (ISO 8601)
Industry Classification (NAICS, ISIC) Security Prices
SWIFT BIC Codes (Payments) ICD-9/10 Codes (Healthcare)
ACORD/ISO Codes (Insurance)
Private (Internal)
Legal Entities Chart of Accounts
Organizations Employees
(i.e., much of HR & Finance Data)
Reference data required for transaction processing
Semi-Private? (Shared)
Customized Public Reference Standards (e.g. customized D&B)
Shared Private Data (Finance)
© 2014 The MDM Institute The-MDM-Institute.com
RDM programs need to support multiple domains (kinds) of reference data to avoid RDM silos.
FIRST POLL
© 2014 The MDM Institute The-MDM-Institute.com
Multi-Domain MDM
Multidomain RDM
Vendor RDM Approach Indicates Level of Multi-Domain Ability
Customers
Product
RDM
RDM
Two main approaches to RDM = Synchronization-driven (buy lots of ETL) vs. Multidomain (buy commercial RDM).
Synchronization-driven requires $$$ for data integration & lacks semantic stability/governance of hub approach.
Suppliers
RDM
Synchronization-Centric
Private RD
RDM
ETL/DI
Private
Public
Customers
Product
Suppliers
RDM Hub or Multi-Domain MDM Implementation
© 2014 The MDM Institute The-MDM-Institute.com
Critique of Current Approaches for Multidomain Reference Data
RDM Solution Drawback Recommendation
Custom-built, manual solutions
Heavy TCO burden Avoid unless reference data demands are truly unique
Spreadsheets Difficult to govern, secure, version, & audit; no modeling, poor hierarchy management
Distribute data in spreadsheets; govern data in RDM solution
Repurpose hierarchy management solution (MSFT MDS, ORCL DRM)
Poor cross-domain support, no classification mapping, few enterprise integration options
Seek out multi-domain RDM solution with hierarchy management
Customize existing domain-specific MDM (Customer or Product)
Rudimentary data modeling, lifecycle mgmt capabilities, & governance features (esp. authoring & workflow)
Use multi-domain RDM solution to maintain connections & govern/update into CDI & PIM via data services
ERP / Enterprise Application
Limited governance, versioning, distribution; also reference data customized use in app may have limited appeal in other systems
Master in external platform. RDM can be used to govern baseline set, versions and adaptations
Real-time / industry-specific RDM
Premium priced R/T RDM solutions do not represent good economic sense
Leverage R/T RDM solutions for R/T use cases (trading, claims processing, payments)
© 2014 The MDM Institute The-MDM-Institute.com
“Top 10” RDM Technical Evaluation Criteria
1. Administration of diverse reference data types 2. Ability to map reference data 3. Management of reference data sets 4. Architecture/performance 5. Hierarchy management over
reference data sets 6. Connectivity/integration 7. Import & export 8. Versioning support 9. Security & access control 10. E2E lifecycle management
Coming to market are RDM solutions characterized by multiple, diverse levels of integration w/ market-dominant MDM hubs as well as repackagings of existing mid-market
MDM solutions – HOW TO EVALUATE? © 2014 The MDM Institute The-MDM-Institute.com
“Administration of Diverse Reference Data Types”
From R. Thompson,/Credit Suisse, “Multidomain Enterprise Reference Data,” 7th Annual MDM & Data Governance Summit New York 2013
Private Ref Data
Public Ref Data
RDM solution should support a wide mix of data structures from name:value pairs to hierarchies (see criteria #5)
RDM Top 10 Eval Criteria #1
© 2014 The MDM Institute The-MDM-Institute.com
FINANCE LOCATIONS
“Ability to Map Reference Data” – pt. 1 (cross-domain mapping)
Issuing Country (ISO3166)
Name ISO 4217 Code
USA US Dollar USD CHN Yuan Renminbi CNY JPN Japanese Yen JPY
Official Currency
ISO3166
USD ASM USD IOT USD ECU USD SLV USD GUM USD HTI USD MHL USD FSM USD MNP USD PLW USD PAN USD PRI USD TLS USD TCA USD USA USD VIR CNY CHN JPY JPN
ISO 3166 Code
Name
USA United States of America CHN People’s Republic of China JPN Japan ASM American Samoa IOT British India Ocean Terr. ECU Ecuador SLV El Salvador GUM Guam HTI Haiti MHL Marshall Islands FSM Micronesia MNP Northern Mariana Islands PLW Palau PAN Panama PRI Puerto Rico TLS East Timor TCA Turks and Caicos Islands VIR Virgin Islands
RDM solutions need to preserve values & mappings between reference data sets – both in
domain & across domains
RDM Top 10 Eval Criteria #2
LOC
ATI
ON
& F
INA
NC
E
© 2014 The MDM Institute The-MDM-Institute.com
2013 VERSION 2007 VERSION
“Ability to Map Reference Data” – pt. 2 (temporal referential integrity)
2013 NAICS
Description
311224 Soybean and Other Oilseed Processing
221114 Solar Electric Power Generation
221115 Wind Electric Power Generation
221116 Geothermal Electric Power Generation
221117 Biomass Electric Power Generation
221118 Other Electric Power Generation
2007 NAICS
Description
311222 Soybean Processing
311223 Other Oilseed Processing
221119 Other Electric Power Generation - solar electric power generation
RDM solution needs to maintain links between versions, creating a migration path between versions of reference data … “Crosswalks”
are important for understanding how something changed
MERGE
SPLIT
RDM Top 10 Eval Criteria #2
© 2014 The MDM Institute The-MDM-Institute.com
RACI Tasks User
R Update sales hierarchies Rogers
R Change industry classifications Romanova
A Approve hierarchies and effective dates Stark
A Approve industry classifications Banner
A Approve merge into effective dated
Fury
“Mgmt of Reference Data Sets” (Governance Workflows)
RDM Top 10 Eval Criteria #3
An RDM solution needs to support governance workflows … includes defining: responsible & accountable parties (including systems),
permissions & area of responsibility for each party (field, instance, container level), how parties interact/tasks & auditing/history…
Sequence of Interactions
Permissions
Responsibilities
© 2014 The MDM Institute The-MDM-Institute.com
“Hierarchy Management Over Reference Data Sets”
RDM solution should harness relationships between reference data sets & existing party or thing data to create hierarchies
SIC Codes
Customer & SIC Code Mapping
ICD-10 Codes
Active Ingredients & ICD-10 Mapping Active Ingredients & Product Mapping
Viewing customers by industry classification
Viewing drugs by Active Ingredient interactions & ICD-10 Codes
RDM Top 10 Eval Criteria #5
© 2014 The MDM Institute The-MDM-Institute.com
EMEA OPS DEU Cost Ctr
FRA Cost Ctr
APLA OPS
TUR Cost Ctr
JPN Cost Ctr
MEX Cost Ctr
NA OPS CAN Cost Ctr
USA Cost Ctr
“Versioning Support” (a.k.a. time travel)
EMEA OPS
TUR Cost Ctr
DEU Cost Ctr
FRA Cost Ctr
APLA OPS JPN Cost Ctr
MEX Cost Ctr
NA OPS CAN Cost Ctr
USA Cost Ctr
Cost Centers (As-Of 2013 Q2)
EMEA OPS DEU Cost Ctr
FRA Cost Ctr
AP OPS TUR Cost Ctr
JPN Cost Ctr
CALA OPS MEX Cost Ctr
NA OPS CAN Cost Ctr
USA Cost Ctr
RDM solution needs versioning & “as of” / effective dating to support recall of reference data values, relationships or hierarchies
(versioning has *major* implications for analytics/ BI!)
Cost Centers (Current)
Cost Centers (Effective 2013 Q1)
RDM Top 10 Eval Criteria #8
© 2014 The MDM Institute The-MDM-Institute.com
2014 RDM Market Survey Results (Overview)
Aaron Zornes Chief Research Officer
The MDM Institute [email protected]
www.linkedin.com/in/aaronzornes @azornes
+1 650.743.2278
SECOND POLL
© 2014 The MDM Institute The-MDM-Institute.com
Agenda
n Survey methodology
n Attitudes about RDM & current plans
n Trending key features
n Vendor rankings
© 2014 The MDM Institute The-MDM-Institute.com
Partial List of Survey Respondents
Amgen CIT Fireman’s Fund Insurance Sabre
ANZ Bank Citi Mortgage Grant Thornton Spectrum Health
Aspen Re Citibank Hellmann Worldwide Logistics
Standard Bank South Africa
Bank Audi Sal Citizens Financial Group IBM (CIO Office) State Farm Automobile Insurance
Bank of Montreal COLT Technology Services Insurance Commission of British Columbia TD Ameritrade
Barclays Credit Suisse Louis Dreyfus Commodities
The Bank of New York Mellon
Blue Cross Blue Shield Association Cummins Nippur Volvo AB
BPI France Dept of Homeland Security/CBP
NJ Office of Information Technology WellPoint
British American Tobacco DNB ASA OP-Pohjola Group World Vision Cargill Electronic Arts Royal Mail Group Yorkshire Building Society
Carolinas Healthcare Fidelity Investments Royal Melbourne Institute of Technology Zurich Insurance
© 2014 The MDM Institute The-MDM-Institute.com RDM is a key business initiative
in some of the world’s largest companies!
>50% of Respondents Plan to Invest in RDM Within 2 Years
MYTH: There’s no interest in RDM Attendees to summits have been requesting RDM sessions,
Megavendors (and more) have RDM products … and now most of the responding organizations are planning on investing in RDM?
© 2014 The MDM Institute The-MDM-Institute.com
<50% of Respondents = Financial Services Organizations
MYTH: RDM is only for financial services & finance departments Surveys & analyst inquiry load
show increasing interest across the board © 2014 The MDM Institute The-MDM-Institute.com
Accountability for Multi-Domain RDM Centers at the Enterprise Level
MYTH: RDM projects LOB-centric or departmental projects RDM is gaining enterprise level relevance
& are often directly coupled to business users © 2014 The MDM Institute The-MDM-Institute.com
Multi-Domain RDM Projects Have Enterprise-Appropriate Budgets
MYTH: RDM is small scale, small investment Multi-phase RDM programs are in multi-million USD range … Wide range in investment levels reflect where respondents are in RDM
journey © 2014 The MDM Institute The-MDM-Institute.com
Interest in RDM to Support Transactional & Operational Enterprise Applications
MYTH: RDM is an isolated program Considerable interest in integrating enterprise applications with RDM
… Analyst inquiry load shows interest in tying analytical enterprise applications (BI/Big Data) to RDM
© 2014 The MDM Institute The-MDM-Institute.com
Interest in RDM to Support Traditional CDI, PIM & Other MDM Programs
MYTH: RDM is an isolated program, Part II Considerable interest in RDM to support traditional MDM initiatives in Customer & Product (Why maintain country codes everywhere?)
© 2014 The MDM Institute The-MDM-Institute.com
Categories of Data Managed by RDM Multi-Domain
RDM Use Cases Real-Time / Transactional
RDM Use Cases
Public (External)
Private (Internal)
Semi-Private (Shared)
88%
59%
80% 12%
2%
RDM programs are not only managing public forms of reference data, but are focused on public & private in equal measure … also interest
in using RDM to syndicate data services internally © 2014 The MDM Institute The-MDM-Institute.com
THIRD POLL
© 2014 The MDM Institute The-MDM-Institute.com
Top 5 RDM Business Drivers
Reporting, Reconciliation & Regulatory Compliance = key business reasons for pursuing RDM (Bad reference data = bad reports)
© 2014 The MDM Institute The-MDM-Institute.com
Top 3 RDM IT Drivers
n Developing standard metadata management to support enterprise application integration— e.g., common, standard data domain definitions
n Providing common reporting dimensions to support analytics across departments/LOBs, application systems, data marts & data warehouses
n Providing common RDM support services to support transactional application integration
Reporting & reconciliation are also top IT drivers for RDM … Goal for IT teams =providing source of consistent & accurate
reference data to organization & applications
© 2014 The MDM Institute The-MDM-Institute.com
What are the Top Cost Drivers in RDM Projects?
Plan to benchmark development time (screens, BPM/workflow, models) as part of PoC … high development time = high TCO!
Also key cost factor = vendor licensing models © 2014 The MDM Institute The-MDM-Institute.com
Top Evaluation Requirements for Commercial RDM Solutions (Ranked Order)
Multi-domain requirements (cross-domain, inter-temporal), hierarchy management, governance & versioning = top requirements
Rank Table Data
1 Ability to map reference data — e.g., between multiple domains, between time periods or versions of reference data
2 Management of reference data sets — e.g., implementing governance workflows 3 Hierarchy management over sets of reference data
4 Administration of reference data types — e.g., handling public, semi-private & private
4 Versioning support — e.g., support for four main version types: as-of, effective, current, proposed/hypothetical
6 Connectivity & integration with other enterprise applications or MDM platforms
7 Architecture/performance — e.g., enterprise scalable, Cloud-based deployment/licensing option
7 Import & export
9 E2E lifecycle management — e.g., onboarding, managing, retiring, redacting reference data
10 Security & access control — e.g., fine grained/field level permissioning for create, read, update, & delete of reference data
© 2014 The MDM Institute The-MDM-Institute.com
Teams Using Commercial RDM Solutions = Most Satisfied
Tools vs. Sa*sfac*on Effec*ve Program Planning Ineffec*ve Program
Not Planning
Commercial RDM solu0on 23% 6% 10% 0% Exis0ng MDM pla<orm 3% 0% 3% 3% Custom-‐built solu0ons or spreadsheets 0% 23% 13% 0%
ERP or other Enterprise Applica0on 0% 10% 0% 3%
Don't know 0% 0% 0% 3%
Teams that are currently using a commercial RDM solution are the most satisfied, while teams using XLSs are planning on change
© 2014 The MDM Institute The-MDM-Institute.com
Which Vendors are Commonly Short Listed & Which are Being Implemented?
Multiple vendors are shortlisted for RDM (megavendors, incumbents). Incumbent does not mean best fit.
© 2014 The MDM Institute The-MDM-Institute.com
MDM Vendors IBM & Orchestra Networks = Over 2/3 of Implementations Surveyed
RDM focus/components (vs. generic CDI/PIM) seem to be cause, not correlation of implementations
© 2014 The MDM Institute The-MDM-Institute.com
BOTTOM LINE
n Impact of poor or non-existent RDM is profound n RDM is vital for analytical, transactional & operational
systems— reflected in investment levels within the organization
n Initial RDM investments are manageable— getting started won’t kill your budget like CRM, ERP, CDI or PIM
n 50% of successful MDM implementations happen on non-mega vendor solutions … RDM is tracking to the same market dynamics
n RDM is independent from traditional MDM (CDI, PIM)—forward looking MDM providers have a distinct RDM strategy n e.g., IBM, Oracle, Orchestra Networks
© 2014 The MDM Institute The-MDM-Institute.com
BUY DON’T BUILD RDM— seek out multi-domain MDM providers with an RDM solution
Next Steps
n Web research, blogs, investigate social media SIGs, RDM field reports on slide share & MDM Institute website
n Read market studies, analyst reports, investigate academic resources, e.g., DAMA-DMBOK
n Attend industry conferences & network with veteran RDM practitioners
© 2014 The MDM Institute The-MDM-Institute.com
Contact vendors & explore their solution Unlike CDI & PIM, RDM can be tested with real business data & users
with very little setup
MDM & Data Governance Summit™ Conference Series
“More MDM programs get their successful start at MDM & Data Governance Summits than anywhere else”
MDM & Data Governance Summit Europe London ▪ May 19-21, 2014
MDM & Data Governance Tokyo Belle Salle Kanda– Tokyo ▪ June 12-13, 2014
MDM & Data Governance Summit San Francisco Hyatt Fisherman’s Wharf – June 17-18, 2014
MDM & Data Governance Summit Canada The Carlu – Toronto ▪ July 8-9, 2014
MDM & Data Governance Summit Asia-Pacific Doltone House, Darling Harbour – Sydney ▪ July 29-31, 2014
MDM & Data Governance Summit New York Sheraton Times Square ▪ October 5-7, 2014
MDM & Data Governance Summit Shanghai Shanghai Int’l Convention Ctr ▪ Fall 2014
© 2014 The MDM Institute www.the-MDM-Institute.com