use of predictive analytics in business and internal audit · 2019. 4. 23. · targeted marketing...
TRANSCRIPT
1
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Use of Predictive Analytics in business and internal Audit
Sudharshana Balasubramanian
Director – Advisory Services
—
16th April 2019
2
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Synopsis of the presentation
Game Changers and KPMG’s Global D&A Survey results
Case studies using Predictive Analytics with client benefits
KPMG’s Global Data and Analytics Platform
Predictive Analytics and Machine Learning1
2
3
4
3
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Top 10 breakthrough technologies – MIT Technology Review Predicting Preemies – every year over 15 million babies are born
preterm
Custom Cancer Vaccinations– Conventional Chemotherapies
replaced with Predictive custom medicines
Robot Dexterity– Robots are teaching themselves to handle the
physical world using predictive data analytics
ECG on your wrist–to continually monitor heart condition with
wearable devices using supervised machine learning
Smooth-talking AI assistants: will be able to perform
conversation-based tasks through supervised machine learning
4
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Types – Machine Learning
Machine
Learning
Dimensional
Reduction
Clustering
Unsupervised
Learning
Classification
Regression
Supervised
Learning
Reinforced
Learning
Meaningful
Compression
Big Data Visualization
Structure Discovery
Feature Elicitation
Recommender Systems
Targeted Marketing
Customer Segmentation
Robot Navigation Learning Tasks / Decision Trees
Skill AcquisitionReal-Time Decisions
Image Classification
Fraud Analytics
Diagnostics
Customer Retention
Advertisements and
Popularity Prediction
Revenue, Cost and
Expense Prediction
Market ForecastingGrowth Prediction
https://youtu.be/f_uwKZIAeM0
5
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
What is Machine Learning?
Supervised Learning Unsupervised Learning Reinforced Learning
—Algorithms apply what has been learned
in the past to predict future events on
labelled and classified data.
—Learns by comparing with intended
output and finds errors to modify the
model accordingly.
—Algorithms train information that is
neither classified nor labelled.
—Studies how systems can infer a function
to describe a hidden structure from
unlabeled data
—Algorithms that interact with the
environment by producing actions and
discovers errors or rewards.
—Automatically determines ideal behavior
within a specific context to maximize
performance
Key Features Key Features Key Features
Database Marketing Information Extraction
Pattern RecognitionOptical Character
Recognition
Cluster Analysis Anomaly Detection
Multivariate AnalysisGenerative Topographic
Map
Error Driven LearningDistributed Artificial
Intelligence
Temporal Difference
Learning
State-Action-Reward-
State-Action
“Machine Learning is the science of getting computers to learn and act like humans do, and improve their learning over time in autonomous
fashion, by feeding them data and information in the form of observations and real-world interactions.”
Techniques Techniques Techniques
Linear regression Random Forests
Naïve Bayes
Classification
Support Vector
Machines
Gradient Boosting Artificial Neural Network
K-Means Clustering Local Outlier Factor
Principal Component
AnalysisSelf Organizing Map
Expectation –
Maximization AlgorithmHierarchical Clustering
Brute Force Monte Carlo Methods
Direct Policy Search Q Learning
Predictive D&A around the world
7
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Game changers in Prediction
IBM: Developed “predictive attrition
program”, to predict employees most likely
to resign with 95% accuracy using AI.
Gro Intelligence: Agricultural information
group uncovers trends it hopes can counter
looming food shortage.
https://www.ft.com/content/e6530830-2b9f-
11e9-9222-7024d72222bc
Australian Securities and Investment
Commission: Analyze large amounts of
speech and text to identify patterns to
detect / predict misconduct and improve
regulation
German Trains: Sensors and analytics are
making predictive maintenance
work, for engineers and carmakers –
https://www.ft.com/content/9fb0d378-6ad4-
11e6-ae5b-a7cc5dd5a28c
Rolls-Royce: “There are massive savings if
you can pre-emptively perform maintenance
on these aircraft during a stop”.
NYPD: New York Police Department use
Patternizr, to validate and refer “hundreds of
thousands” case files using OCR.
8
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Game changers: Using data analyticsMedical Industry: Developed a FDA
cleared platform for Patient Monitoring
using Predictive Analytics for trials.
https://hitconsultant.net/2019/04/03/mic-
fda-cleared-patient-monitoring-
funding/#.XKnaUuZlLIU
Law firms’ sifts and summarizes data with
speed and precision to replace routine tasks –
contract reviews using Natural Language
Processing.
FDA: Federal Govt of US leveraged
predictive analysis to discover patterns and
associations to identify the occurrence of
food based infections
https://intellipaat.com/blog/7-big-data-
examples-application-of-big-data-in-real-life
China seeks glimpse of citizens’ future with
crime-predicting data and analytics
Companies and police develop technology to
stop criminals before they act
https://www.ft.com/content/5ec7093c-6e06-
11e7-b9c7-15af748b60d0
Insurance: Employ Anti-Fraud arsenals
through advanced data analytics, as CAGR
spiked to 64% in 2019, compared to 16% in
2016
Sports analytics: Xebia developed a
predictive fitness model and logistics
operations management tool for athletes in
Special Olympics.
Implementation of Predictive Analytics
10
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Types - Data analytics
Probability depiction of future outcomes and trends on analysis of
historical data / categorical data
Used by FEW COMPANIES
Tools used: SAS, Python, R, SPSS
3
Critical Analysis of a problem statement to derive solutions
Used by MANY COMPANIES
Tools used: Python, R, Alteryx
2
Preliminary interpretation of Raw Data to provide an accurate
assessment
Used by ALMOST EVERY COMPANY
Tools used: ACL, Tableau, Shiny R
1
Descriptive
Data AnalyticsDiagnostic
Data Analytics
Predictive
Data Analytics
Prescriptive
Data Analytics
Illustration of possible actions to mitigate risks and guide future
decisions
Used by VERY FEW COMPANIES
Tools used: CPLX, LINDO, SAS programming
4
1
2
3
4
11
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Predictive Data AnalyticsPredictive Data Analytics leverages the use of statistical models and forecasts techniques to understand a future state to answer the question “what
could happen?”.
Techniques Limitations When to Use
— Regression
— Neural Networks
— Depends on availability of labeled data –
training data.
To make an informed decision on probable
results with cluster/regression/hidden
analysis
Predictive
Data
Analytics
Identify Outliers through an
algorithm rather than random
guessing of Audit samples
In-build auditor-on-demand
to provide greater assurance
with the limited resources
Enhance the First and Second
lines of defense with
embedding predictive analytics
Leverage Flexible, Low-Cost
Storage Technology
Data Models and Algorithms
Growing Volumes and Types of
Data
Enablers Outcomes for Auditors
Limitations
12
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Explanation of Predictive AnalyticsPredictive Analytics is a branch of advanced analytics that enables to make predictions about unknown future events. It incorporates
techniques such as, data mining, modeling, machine learning and artificial intelligence to analyze current data.
Historical DataPredictive
AlgorithmsModel
New Data Model Predictions
Model Building
Model Prediction
Key Feature of Predictive Analytics
https://hbr.org/video/5299994733001/the-refresher-regression-analysis
The technology that learns from past data to predict future
outcomes
13
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
“the technology that learns from experience (data) to predict
likelihood of… Manipulation or Major Outliers... to provide:
1. ESTIMATED values with confidence intervals; and
2. CLASSIFICATIONS on High, Medium and Low risks.
Use of Predictive analytics in
Detailed Case Studies Predictive Data Analytics
Case Study Predictive Cost Analytics:“Spotting expenses Outliers from huge loads of data”
16
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Executive Summary (1/4) – Project overview
1. Perform comprehensive analysis
for 5 years Cost and expenses
2. Reconcile cost information from
varied data bases
3. Establish relationship amongst
major cost centers and create
predictive algorithm
4. Compare inventory consumption
by type across various periods and
suppliers
5. Develop methods to establish the
acceptable range of the cost to
identify outliers.
1. Reviewed the overall cost for year
2012-2017
2. Performed cost cross-validation
between Materials-services
Requests with Payments and GL
3. Integrated information from GL,
Work Schedule, Information
Management System, Electronic
Data, and Report.
4. Identified key cost drivers for all
major well cost centers
5. Developed a predictive well cost
algorithm to identify outliers
1. Identified major cost centers
contributing to the majority total well
cost
2. Observed mismatches in the
overall reconciliation from varied
data bases
3. Developed acceptable range for
cost centers per asset
4. Identified wells outside the
acceptable range or with abnormal
expenses
Preliminary Scope
of Work
01
OurApproach
02
Results
03
17
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Review of Databases (2/4) – Access to various data centersDuring this assignment, we reviewed and analyzed the data from the below mentioned
reports/data sheets for the last 5 years
Monthly operational reports Various Information
Management System
Daily Reports
KPI and BSC workings Sub-LedgersGeneral Ledger from ERP
Payments Desk Top Requests Invoice
18
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
c
Internal Audit
Data Analysis
Integrate Procurement and
Consumption Data spread across many
systems using Clustering techniques
Establish the Correlation plots
amongst the cost elements to
establish degree of relationships
Use Regression Techniques
to detect the abnormality and
spot outliers
Establish both detailed & big-
picture perspectives of Purchase &
Consumption using Descriptive &
Visual Analytics
Cost Saving
Opportunities
“the goal is to
turn data into
information and
information into
insights which
leads to better
decisions.”
Approach (3/4)
19
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Summary of Findings
Value added (4/4)
Predict Abnormalities: Built predictive
algorithms to identify abnormal cost
centers and specific outliers in individual
cost elements with threshold limits
amounting on 5 years of data.
In-built three-way matching: Spotted
major irregularities amongst: General
Ledger, Payables and Inventory records.
Identified major Cost Drivers:
Identified 12 major cost centers (out of
140) that contributed to 80% of the total
cost & established one to many and
many to one correlation factors.
On-going monitoring: Developed
Interactive dashboards to stakeholders
on discrepancies in three-way matching,
irregular well costs and expenses
outliers, which are accessible even in
hand-held devices.
Case StudyPredictive Maintenance:“Understand how likely an equipment needs maintenance
and preempt failure”
21
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Using Predictive Risk Analytics (1/2)Problem statement
How to use Predictive Risk Analytics to gain insight into:
Maintaining large-scale refinery infrastructure in energy industry is a
challenge, and achieving optimum asset performance even more so:
— Optimize asset management through effective preventative and
predictive maintenance
— Improve reliability by avoiding asset failures (which require corrective
maintenance)
— Identify chronically problematic assets with a high ratio of repair-to-
replacement costs
Predictive Maintenance: “To predict the probability of failure in the
next cycle based on operational / maintenance data and other
data sets”
I want to
understand
how likely a
machinery /
equipment is
going to
need
maintenance
in the next
cycle and
which
equipment is
critical for
maintenance
requirements
22
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved. 22© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Document Classification: KPMG Confidential
Dashboard (2/2)
Devices with high
propensity of
failure identified
using the output
of statistical
models
Detailed analysis
of each of the key
service
parameters which
are indicative of
the failure
Key indicators
which are used
by the model
to predict
failure
Case Study Detect Financial Statement Risks Analysis
24
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Computation of over 40 + financial measures
25
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Establishing the intricate relationship between metrics
26
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Cash Coverage Ratio
Interest Coverage Ratio
Interest Coverage
Days Inventories Outstanding
Total Debt to Total Shareholder's Equity
Debt to Equity
Return on Common Equity
Equity Multiplier
Capitalisation Ratio
Long term Debt to Equity
Return on Assets
Inventory Turnover
Total Asset Turnover
Operating Profit Margin
Net Profit Margin
Free Cash Flow to OCF Ratio
Gross Profit Margin
Long term Debt to Total Assets Ratio
Debt Service Coverage Ratio
Operating Return on Assets
Capital Expenditure Coverage Ratio
Operating Cycle
Net Working Capital to Sales
Inventory to Net Working Capital
Working Capital Turnover
Currrent Ratio
Quick Ratio
Short Term Debt Coverage Ratio
Operating Cash Flow to Sales
Accounts Receivable Turnover
Average Collection Period
Fixed Assets Turnover
Total Debt to assets
Applying Machine Learning to group the financial metrics
27
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Predictive data analytics to spot potential anamolies
Case StudyVulnerable IT assets for Cyber Defense“Intrusion Detection System for Cloud Systems”
29
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Network log
Executive Summary – Assignment Overview
Network packet sized Service requests Network logNetwork packet
sizedService requests
Intrusion Attackers or Non-Intrusion Service seekers
Network Log
Packet Size
Service Request
Observed Sequences
Intrusion Attackers
Non Intrusion Attackers
Unobserved States or Hidden
States Generating the
Observations
Real World Scenario: Intrusion Detection System for Cloud Services
Cloud Systems suffer from lot of security vulnerabilities and it is necessary to detect and alert intrusion attacks in advance.
Problem: Normal Service activities and intrusion attackers activities, both generate the data points. But, there is a different pattern for intrusion attackers and normal
service users.
Observed Data Points: Network Logs, Packet Size and Event Logs
Case Studies Predictive Data Analytics from an IA perspective
31
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Case Studies
First and Third Party Fraud1 Cognitive Analytics4
“Time off Work” using PA6Lease Accounting and IFRS3
Predictive Auditing2 Behavioral Model to Predict Renege5
Following are case studies that made immense strides within their business with the use of Machine Learning and Predictive Data Analytics.
Internal Audit Risk Management
Internal Audit and Risk Management
Manage Demand using PA Consumer Preferences7 8
32
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
International Case studies
— Analyze the patterns of the
consumers for a leading bank to
identify fraudulent consumers
— Develop a predictive model to
identify potential future fraudsters
— Highlight first and third hand
fraudulent events within the data
source
Engagement Goals Benefits to the Auditor
First and Third Party Fraud
— Deployed Social Network Analysis
system, to highlight individual entities
that would potentially perpetrate
large scale frauds – which were
taken-up for audits.
— Reduced the annual costs by
decreasing investigation time to
highlight “high-likelihood” cases
— Develop robust auditing platform to
identify trends of emerging risks –
moving away from control testing
— Use of both proactive and
predictive auditing
— Analyze associations and attributions
between different departments
processes
— Incorporate group company set of
criteria within the platform
— Enhancing efficiency & effectiveness,
channeling auditing resources to
more value adding activities
— Proactive auditing on the go, moving
from hindsight to foresight
— Predictive mapping trends to forecast
and identifying emerging risks
— Prevent incidents from occurring
Engagement Goals Benefits to the Auditor
Predictive Auditing for a larger Asian BankIA IA
33
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
International Case studies
— Analyze the real-time sensor data to
monitor the health of the aircraft
during flight
— Predictive model to assess the
maintenance or replacement of the
aircraft parts
Engagement Goals Benefits to the Risk Manager
Cognitive Analytics for Aircraft Maintenance
— Improved the risk categorization
through consolidated information for
all the parts required for aircraft to be
replaced
— Risk assessment was conducted on
the list of parts with “no fault” from
contractor, yet identified as faulty in
the current system – built a key risk
indicator profiles.
— Highlight aircraft parts due for
maintenance in advance in order to
saving procurement costs for new
parts
— Automated processing lease account
documents
— Extraction of key facts: contractual
parties, length of contract etc.
— Digitized, indexed and searchable
document corpus
— Classification of lease documents
categories
— Additional information like
geographical overview, notification on
expiring contracts etc.
Engagement Goals Benefits to the Auditor
Leasing Accounting / IFRS 16
— Manual processing is time-
consuming and can be (partially)
automated using cognitive
technology
— System is trained by leasing experts
and available around the clock for
IFRS 16.
— All contracts details centrally available
and linked to the respective original
contract
— Use of Natural Language Processing
in predicting potential leasing
clauses in contracts.
RM IA
34
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
International Case studies
— Identify the leading designs in the
fashion industry, to help predict the
upcoming trends
— Analyze real time sales of stores, to
determine items to reorder or
abandon
— Incorporate e-commerce statistics, to
gain valuable insights on preferences
from browsing history and customer
interaction
— Ability to gauge the “Risk Factor” of
upcoming season trends for the
organization
— Real time response initiative to
changing trends and delay mass
production to reduce losses
— Focused decisions based on country /
demographic
Engagement Goals Benefits to the Risk Manager
Consumer Preferences
— Predict the influx of patients in a
Health Center, to plan for the
appropriation of staff and medical
supplies for cardiovascular
examinations
Engagement Goals Benefits to the Auditor
Predictive Analytics to Manage Demand
— Identify the baseline to maintain and
manage inventory depending on
the influx of patients
— Plan and assess the organization’s
staff appropriation for different health
centers within a geographical
region
— Ability to schedule medical
procedures and surgical treatments at
an optimal level
IA/RM IA/RM
35
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
International Case studies
— Identify the key drivers that influence
the candidate joining/not-joining the
organization
— Descriptive analytics on the renege
— Predictive model to analyze the
probability to accept an offer and join
the company
— Identified 6 key driver stages to
gauge a candidate joining/not-
joining the organization
— Insights provided on each stage the
probability of the candidate to join
the organization
Engagement Goals Benefits to the Risk Manager
Behavioral Model to Predict Renege
— Predict the time required for
claimants to return to work to
pinpoint cases that need additional
support
— Analyze the additional time off work
required by claimants
Engagement Goals Benefits to the Risk Manager
“Time off Work” using Predictive Analytics
— Identified high risk claims (over 90
days required) with 90% confidence
— Cost reduction through earlier
assignment of project
managers/occupational
therapists/temporary workers to
maintain business continuity
— Process improvement opportunities
and policy reconsideration strategies
— Work force planning and business
continuity model.
RM RM
Challenges in implementing Predictive Analytics
37
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Issues and mitigation optionsActions to
avoid issues
Frequently
observed issues
Lack of skilled resources Hire and / or train to use case need
Extensive data detective workPut data ‘under governance’ enable high quality data
for top use cases
Purchasing the ‘wrong’ technology
Understand use cases prior to purchasing the ‘right’ tools
are not always ‘new’
Confusion on services offered
Creation of a service catalog with defined business
alignment
Inability to describe value generated
Identify high priority use cases Outline value tracking
mechanisms
‘Lack-of’ or slow adoption
Demonstrated executive commitment Engage
stakeholders early
38
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
How I imagine my data lake to be… What it’s really like
Copyright Dale Williams data lake memes
© 2018 KPMG International Cooperative (“KPMG International”). KPMG International provides no client services and is a Swiss entity with which the independent member
firms of the KPMG network are affiliated.
KPMG AI in Control Methodology
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Integrity
Track the lineage and provenance of raw data,
training data, model experiments, ongoing
changes made by SME’s. Model training incl
changes by stored in a immutable ledgers.
Strategy & Governance Understand & Design Model & Train Evaluate & Assure Deploy & Evolve
Explainable
Models that can explain the knowledge learned.
Can provide explanation in business terms on how
decisions were made. Interpretations are gained
or inferred from explanations.
Free from prejudice
Models as well as the training data that must be
free of bias, are inclusive and avoids unfair
treatment of certain protected groups. Be certain
that the models incl the trainer comply with
policies & regulations
Agile and robust
Models are interoperable between various
runtimes, providers, or frameworks. Consumable
from apps & processes. The models, ground truth
and feedback are safe & secure from harm or
adversarial attacks.
— Identified Business / LOB
owner
— Defined key business
objectives
— Modeled measurement
metrics (business)
— Model Capability mapped to
biz reqs
— Involved persona metadata
— Regulatory mandates and
requirements
— Explainability result template
— Features comply with
policies
— Features comply with biz
reqs
— Model usage restrictions
— Need to know, groups, users
etc.
— Model SLA’s
— Required skills and support
to manage and maintain
— Identified raw & training
data sources
— Pre-processing performed
on data
— Training data provenance
— Data SME’s Involved?
— Training data metadata
— Training data descriptive
data
— Training data outliers
— Define explainability schema
— Population sampling methods
— Sampling size
— Training data completeness
— Test data completeness
— Features list
— Data usage guidelines
— Confidential features?
— Model weights transferable
— Model metadata
— Model trainer profile
— Training methodology
— Techniques and algorithms
applied
— Feature changes
— Model metadata
— Knowledge representation
ontology
— Type of explainability
— Technical vs business lingo
used
— Bias detection techniques
— Bias remediation
techniques
— Model weights verification
— Training data protection
— Training data access
— Frameworks and libraries
used
— Tooling used
— Experiment setup and
config
— Model experiments reports
— QC & assurance on model
— Model evaluation reports
— Feature change log
— Model evaluation metrics
and scores
— Human validation reports
— Handling unexplainable
events
— Redundant coding
— Inclusiveness listing
— Model drift evaluation
— Simulation on new data
— Approved framework and
runtime
— Concept drift audit log
— Intermediate representation
— Framework/runtime
vulnerability testing
— Feedback & usage logs
— Experiment iterations
from feedback
— Model
improvement/change log
— Continuous testing &
monitoring of explainability
— Training vs usage data
differences
— Model serving access
protection
— Model serving monitoring?
— Cloud vs on-premise
— Feedback retention policies
— Data stationarity checks
KPMG’s D&A Tools: e-Audit, Ignite, Clara and Sofy
41
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved. 41© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Document Classification: KPMG Confidential
Reinforcement Learning
Supervised Learning
Unsupervised Learning
Knowledge-Based Systems
Natural Language Generation
Natural Language Processing
Deep Learning
Data Components Algorithms and Tools Human-in-the-Loop
Training
Automation
Acceleration
Enhanced
Insights
Text/Semi-
Structured
Speech
Image
Structured
Data
Ignite employs advanced analytical techniques and algorithms to train computers how to
use data from a wide variety of sources and formats to enhance, accelerate, automate,
and augment decisions that drive growth and profitability.
41© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Document Classification: KPMG Confidential
Ignite Methodology
42
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
KPMG Clara MethodologyScaling the base Stabilizing the core Innovating the future
We are on a journey
Data & Analytics
Knowledge &
Collaboration
KPMG Clara
Validation &
Integration Predictive
Analytics
Prescriptive
Analytics
Machine
Learning
Cognitive
Analytics
Growing global solutions ecosystem
43
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
KPMG Sofy
Access
Management
Controls
Management
3rd party Risk
Management
Policy & Regulation
Management
Risk
Assessment
Continuous Controls
Monitoring
Process
Insights
My Tasks
Spend
Analytics
VAT
Analytics
VAT
Compliance
Transfer
Pricing
Customs
Manual Journal Entry
IFRS 16
Data
Governance
Data Quality
Monitoring
Tax Management
Data ManagementGovernance, Risk & Compliance
Strategy & Operations Performance
Finance Excellence
Risk
Management
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
44
Document Classification: KPMG Confidential
© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved. 44© 2019 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with KPMG International Cooperative (“KPMG International”), a Swiss entity.
All rights reserved.
Document Classification: KPMG Confidential
– IT’S OUR IMAGINATION
It’s not technology that limits us today
Document Classification: KPMG Confidential
kpmg.com/socialmedia kpmg.com/app
This approach note is made by KPMG, the United Arab Emirates member firm of the KPMG network of independent firms affiliated with KPMG International
Cooperative (“KPMG International”), and is in all respects subject to the negotiation, agreement, satisfactory clearance of KPMG’s client and engagement
evaluation process, and signing of a specific engagement letter or contracts. KPMG International provides no client services. No member firm has any authority
to obligate or bind KPMG International or any other member firm vis-à-vis third parties, nor does KPMG International have any such authority to obligate or bind
any member firm.
© 2017 KPMG Lower Gulf Limited and KPMG LLP, operating in the UAE and member firms of the KPMG network of independent member firms affiliated with
KPMG International Cooperative (“KPMG International”), a Swiss entity. All rights reserved.
The KPMG name and logo are registered trademarks or trademarks of KPMG International.
Sudharshana Balasubramanian
+971 52 559 0100
Thank you