ai-powered data cataloging virtual summit navigating data ... · navigating data lineage. with...
TRANSCRIPT
`
AI-Powered Data Cataloging Virtual Summit
Navigating Data Lineage
With Rabobank and Informatica
Requirements:
Data Quality
Data Governance
Data Analytics
Data Privacy and Security
Regulatory Compliance
Data Lineage traces data from source to destination, covering the entire lifecycle of data. It includes
information about changes to data during its journey.
Data Lineage: A Business Imperative
3 © Informatica. Proprietary and Confidential.
Data Lineage: The Foundational Use Case
• Dev Operations: Change Management & Impact Analysis - what-if analyses for changes
• Operational Efficiency: Eliminate proliferation, duplication, data silos, reduce costs
• DW/Apps Modernization: Complete understanding of the data landscape to enable app modernization & cloud migration
…and AI use cases
• Explainable AI & Data Science Governance: Track and assess data used to train models, govern AI projects. Support Explainable AI. Ensure training data variety.
Increasingly “IT” use cases are coming to the forefront…
4 © Informatica. Proprietary and Confidential.
Data Lineage: Key Informatica CapabilitiesUser experience is key to understanding lineage, custom views at all levels and detail for every use case
• Business and logical views
• Dataset level lineage
• Detailed field level lineage
• Drill down details on data transformation logic
• Summarized, level-wise views
5 © Informatica. Proprietary and Confidential.
Data Lineage: Key Informatica CapabilitiesComprehensive lineage to support all use cases requires wide and deep metadata connectivity as well as active metadata-driven intelligence
• Automatic data lineage stitching
• Discovered and curated
• Impact analysis with shareable exports
• Automatic lineage derivation from code: SQL Scripts, stored procedures, BI reports, ETL jobs & mappings
• Change notifications
• Data Similarity discovery
• Other relationships (joins, usage)
Gaurav Pathak Anil Bandarupali
Speakers
Senior Director,Product Management
Senior Software Engineer, Data Management
Rabobank experience with EDC
Anil Bandarupalli
Enterprise Data Catalog
• Introduction to Rabobank
• Rabobank vision & strategy
• Data Management Office
• Data Governance team
• Scope of Data Lineage
• Data Lineage use case#1
• Data Lineage use case#2
• Lessons learned
• EDC next steps for Rabobank
Reading Guide
8
2nd largerst bank in Netherlands101 local Rabobanks409 offices1.9 million members6.5 million private customers0.8 million commercial customers
Rabobank in the Netherlands
Mission Growing a better world together
Domestic Retail Banking
Almost 8.3 millioncustomers
7.3 million Dutch customers 1.0 million international customers
41 countriesInternational
Market Leader in Financing, Food and Agriculture
Private Sector Lending to Trade, Industry And Services
Rabobank at a GlanceSituation on December 31, 2018
9
Growing a Better World Together
10 StrategicTop Priorities
• 100% Digitalconvenience in everything
• Top customeradvice nearby
• Growth with innovation
• Top performance• Optimal balance sheet• Exceptionally good
execution
• Focus on social responsibility and sustainable contribution
• Involved members andcommunities
• Inspired employees• One-Rabobank culture
HOW?Through our values and behaviours
I dare to make a difference for the world
I make you betterI am doing the right thing
exceptionally wellI go the extra mile
for my clients
Banking for the Netherlands Banking for Food
Excellent Customer Focus
Rock-solidBank
EmpoweredEmployees
Meaningful Cooperative
We are client-driven and action-oriented
We are purposeful and courageous
We are professional and considerate
We bring out the best in each other and keep learning
10
Data Management Office
11
Data Management Office
Dat
a Li
neag
e
Dat
a A
rchi
tect
ure
Dat
a Q
ualit
y
Dat
a D
efin
itio
ns
Ente
rpri
se D
ata
Rec
ord
Keep
ing
Ref
eren
ce D
ata
Man
agem
ent
Dat
a Ex
chan
ge
Proc
ess
& C
ontr
ols
The Data Management Office supports the data governance structure within Rabobank.
The Rabobank Managing Board has mandated the Data Governance Board to increase the management of data with the following goals:• Define, approve and communicate data
strategies, data policies, data standards, data architecture, procedures and metrics
• Track and enforce compliancy of data policies, standards and architecture
• Initiate, track and oversee the delivery of data management projects and services
• Manage the resolution of data related issues
• Increase understanding and promote the value of data assets
Data Management & Data Logistics Department
12
Data Management
& Data Logistics
Data Quality
Why Data Lineage So Important?
13
Having Data Lineage for Enterprise will create business value in four areas:
Regulatory Requirements
Use end-to-end lineage BCBS 239 and other regulatory mandates
Data Quality Management
Data lineage improves the overall data quality by reducing the duration of root cause analyses process of Data Quality Issues
Change management
Provides the possibility to view the impact of proposed changes to the data
integration environment
Data Integration
Creates a better understanding what the data means, where it came from, where it is used
and how it has been transformed
Business Value of Data Lineage
Group Data layer
Group Reporting and Calculation
** ** ******
**** **** **
** **
GROUP
** ** **
****
**
**
**
**
Group Risk and Finance sources
**
****
****
**
****
** **
****
******
** **
**
Group
** **
LBB Risk LBB Finance
**
**
**
**
**
**
**
**
****
**
**
**
** ** **
**
**
**
**
BU Reporting and Calculation
BU Data Layer
BU Risk or finance sourcesWRR
** Data Exchange function
**
Data Warehouse level
Back / Mid office level
Front office level
FinancierenSparen Beatalingsverkeer Klant
** **
**
**
**
**
**
** **
**
**
** **
**
**
**
**
** **
**
**
**
**
****
* *
**
**
********
Dat
a Fl
ow
** System names are masked for security reason
PowerCenter
Scope of Systems f0r Data Lineage
14
EDC — GUI interface
15
Using EDC for ComplianceUse case for BCBS#239 compliance
*BCBS#239 is Regulatory compliance on Data Governance
Manual vs Automated Lineage
17
Existing situation manual lineage in Excel
Automated lineage in EDC
Understanding Lineage Can Be Too Complex
18
Deriving Business Lineage Out of EDC
19
EDC for Data GovernanceUse Case, Integrating EDC into Data Governance Center
Integrated View in Data Governance Center
21
Data Lineage starts in EDC
Reporting Repository
Business Data Element
Logical Data Attribute
Physical Attribute
FIND
YOU
R DATA
Horizontal Lineage in EDC— Technical Lineage
22
FOLLOW YOUR DATA
• EDC catalog search is very powerful like Google for enterprise catalog
• EDC has a big list of connectors
• EDC has open API, makes it easy to integrate with other tools
• Technical data lineage can be too much information for business users
• When you are looking at a big lineage diagrams, it is not possible to restrict lineage for selected tables
Lessons Learned
23
• Continue expanding data lineage for enterprise
• Adding business terms to data attributes
• Using data domain discovery for privacy regulations
• Deriving business lineage out of EDC
EDC Roadmap for Rabobank
24
2525
Bedankt voor uw aandacht!
Demo
27 © Informatica. Proprietary and Confidential.27 © Informatica. Proprietary and Confidential.
Learn More
1. Don’t miss Keynotes and Deep-Dives at the AI-Powered Data Cataloging Virtual Summit:• Market and Analyst Perspectives featuring New York Life, Tableau, and Amalgam Insights
• Data Cataloging Solution Theaters featuring Maersk, Nissan, Rabobank and Biogen
2. Stop by an Informatica World Tour near you:• Chicago Sept-11 | Washington, DC Oct-15
• Frankfurt Oct-8 | London Oct-9 | Paris Oct-10
3. Watch a Product Webinar:• Advancing Analytics Maturity with an Intelligent Data Catalog: with Mattel and Aberdeen
• Meet the Expert PM Webinar: EDC 10.2.2 Release Deep-Dive & Demo
`
Thank You