(rutgers business school - transportation.org...big data analytics •the right tools and skills can...

Emerging Audit Technologies

Hussein Issa(Rutgers Business School)

Outline• What is Artificial Intelligence (AI)

• Technological Process Reframing (TPR)

• Research Questions:

– 1. How will the field of artificial intelligence change the audit process through TPR?

– 2. How to analyze the cost and benefit of the investment in AI?

– 3. How to make auditors, who lack data mining and AI knowledge and skill, master AI techniques and tools?

– 4. What are the differences and similarities between expert system and deep learning?

– 5. What can be learned from the research of the application of expert system to auditing to support the application of deep learning to auditing?

– 6. Is it appropriate to directly employ existing deep learning trained with nonfinancial data, to analyze financial content?

– 7. What are the changes in audit conceptualization that will be facilitated by sensing, archiving, and predictive technologies?

– 8. What are the lines of defense in the modern continuous and (partially) intelligent audit?

– 9. What are the modules of the modern (intelligent) assurance process?

– 10. What are these more detailed “modern assertions”?

– 11. What data will be evidence?

– 12. What is an adequate way to taxonomize audit judgments that is appropriate for intelligent automation?

– 13. To what degree can audit judgment be automated?

– 14. Are audit populations a large enough sample for deep learning?

– 15. How can you do deep learning of financial statement fraud identification if the known frauds & restatements are very limited?

– 16. If less independence can result in better assurance, should the standards be modified in that direction?

– 17. What are the parts of auditing that can be divided into a series of automatable production processes?

– 18. Would a different organization of the audit process be more appropriate to the AI enabled audit (AIEA)?

– 19. What are the subcategories of audit judgments?

– 20. Which of these can be formalized?

– 21. Which of these can be supported by expert systems / neural networks / deep learning methodologies?

– 22. How does the evolution of technology and its adoption affects the audit process? Is there a substantive amount of TPR?

– 23. Will automation cause workforce replacement or supplementation in the auditing field?

• Exogenous measurement and quality of measurement

• How will AI affect the Auditing Profession

• Formalization of Audit through Automation

• Workforce replacement or supplementation

• List of additional Research Questions

7/26/2019AASHTO Audit Annual Meeting 2019 2

Outline• What is Artificial Intelligence (AI)

• Technological Process Reframing (TPR)

• Research Questions:

– 1. How will the field of artificial intelligence change the audit process through TPR?

– 2. How to analyze the cost and benefit of the investment in AI?

– 3. How to make auditors, who lack data mining and AI knowledge and skill, master AI techniques and tools?

– 4. What are the differences and similarities between expert system and deep learning?

– 5. What can be learned from the research of the application of expert system to auditing to support the application of deep learning to auditing?

– 6. Is it appropriate to directly employ existing deep learning trained with nonfinancial data, to analyze financial content?

– 7. What are the changes in audit conceptualization that will be facilitated by sensing, archiving, and predictive technologies?

– 8. What are the lines of defense in the modern continuous and (partially) intelligent audit?

– 9. What are the modules of the modern (intelligent) assurance process?

– 10. What are these more detailed “modern assertions”?

– 11. What data will be evidence?

– 12. What is an adequate way to taxonomize audit judgments that is appropriate for intelligent automation?

– 13. To what degree can audit judgment be automated?

– 14. Are audit populations a large enough sample for deep learning?

– 15. How can you do deep learning of financial statement fraud identification if the known frauds & restatements are very limited?

– 16. If less independence can result in better assurance, should the standards be modified in that direction?

– 17. What are the parts of auditing that can be divided into a series of automatable production processes?

– 18. Would a different organization of the audit process be more appropriate to the AI enabled audit (AIEA)?

– 19. What are the subcategories of audit judgments?

– 20. Which of these can be formalized?

– 21. Which of these can be supported by expert systems / neural networks / deep learning methodologies?

– 22. How does the evolution of technology and its adoption affects the audit process? Is there a substantive amount of TPR?

– 23. Will automation cause workforce replacement or supplementation in the auditing field?

• Exogenous measurement and quality of measurement

• How will AI affect the Auditing Profession

• Formalization of Audit through Automation

• Workforce replacement or supplementation

• List of additional Research Questions


Outline

• Big Data & Its Implications

• Big Data & Audit Analytics

• Artificial Intelligence

• Impact of AI On the Auditing Profession

• The Emerging Technological Landscape

• The Future!


DISRUPTIVE TECHNOLOGY

A technology that significantly alters the way that businesses

operate. A disruptive technology may force companies to alter

the way that they approach their business, risk losing market

share or risk becoming irrelevant (ISACA 2017).


• Current Technological Landscape

– Big Data

– Data Analytics

– Artificial Intelligence

• Emerging Technological Landscape

– Drones

– Blockchain

– Visualization

– Audit Production Line

BIG DATA & ITS IMPLICATIONS

What is Big Data?

• No unified definition:

– Data exceeding the level of efficient manageability within traditional DB

(Harris 2013)

– Process of analyzing a large volume of diverse data, in any variety of form,

using ground-breaking apparatus to identify opportunities to improve overall

value (Miller 2012; Moore et al. 2013; Wyner 2013)

• Common trait: large population of data

• Components: volume, variety, velocity, veracity

• New to accounting and audit industry: no formal means to evaluate it, has not applied it in assessments.

• Correlations vs. causation

The power of Big Data lies in the ability to find patterns, which drives the way in which Big

Data is analyzed (Alles 2013)7/26/2019AASHTO Audit Annual Meeting 2019 8

Big Data explosion

AASHTO Audit Annual Meeting 2019

12 ZBDATA CREATED

IN 2018

0.5 %OF AVAILABLE DATA

USED BY BUSINESSES

© Copyright 2019, MindBridge Analytics Inc.

7/26/2019 9

EXAMPLES OF EXOGENOUS (BIG) DATA

• GPS receiver in your cell phone

• Cash registers when you make a purchase

• Cameras in public places

• Your car

• Your digital photos

• Your IoT devices

• Sensors


Challenges of Auditing Big Data

• Pattern recognition using unstructured data vs. deriving intelligence from

benchmarks and models derived from structured data

• Identified exceptions and anomalies are expected to increase dramatically

• Lack of adequate training and necessary skills to analyze Big Data

• Increasing complexity of Big Data leads to increased cost for companies

(hiring data scientists and investing in additional software)

• Some analytical tools are like a black box to auditors (e.g. Neural Networks),

leading to decreased popularity.


BEHAVIORAL IMPLICATIONS

• Audit by Exception (Vasarhelyi and Halper, 1991)

• Continuous Auditing literature is rich with studies that propose statistical and

machine learning techniques to identify exceptions (Dull et al., 2006; Groomer

& Murthy, 1989a; Alexander Kogan et al., 1999; Vasarhelyi & Halper, 1991)

• Problem?

• Result?


Large numbers of Exceptions!

-Information overload!

-Pattern recognition!

-Ambiguity!

-Information relevance!

Behavioral Implications-Information Overload


Behavioral Implications-Information Relevance


Behavioral Implications-Pattern Recognition


Behavioral Implications-Ambiguity


Exceptional Exceptions Framework


How will the audit approach change?

Sampling

Retroactive – Point in Time

Traditional Audit Evidence

Processing Few Notable Items

Full Population

Predictive – More Frequent

Non-Traditional Audit Evidence

Processing Numerous Notable

Items


BIG DATA AND AUDIT ANALYTICS

What is Audit Data Analytics (ADA)?

Audit data analytics (ADA) is data analytics applied to the audit

process to produce audit evidence and assist in auditor judgements.

Audit Evidence Auditor Judgements

Firm-Relevant

Data


Ratio

Analysis

Trend

Analysis

Descriptive

Statistics

SamplingLinear

Regression

Logistic

Regression

Machine

Learning

Expert

SystemsClustering

Content

Analysis

Visualization

Robotic Process

Automation

DATA ANALYTICS TECHNIQUES

• Descriptive – provides an analysis of past

performance: “what happened”

– Example: traditional financial reporting

Newer techniques for

audit data analytics


Predictive – provides an

estimate of future performance:

“what may happen”

Example: management or

analyst earnings forecasts

Prescriptive – provides a specific action to take:

“what to do”

Example: if inventory levels reach “x” amount,

purchase more

Example: if variance greater than threshold, flag

as an exception

Retroactive vs. Predictive

• Auditing emphasizes a retroactive, backward-looking approach

• With technology, close to real-time (predictive) assurance is

possible

• Reduce expectation gap between auditors and financial statement

users


Predictive Analytic (cont.) Clustering Using Store Sales by Peer Group


Few vs. Numerous Notable Items

• Sampling methodologies limit the number of items to test

• Full population expands coverage, but generates numerous

notable items

• How to process these items?

– Human involvement

– Technology


Challenges to Applications of ADA

• Data capture – need a streamlined process

– IT capabilities – will determine manual vs. automatic capture

– Data standards – all data fields must be in the same format

– Client approval / privacy concerns

• Data validation – verifying the completeness and accuracy of

data extracts

• Data volume – need capacity to process and store big data

– Size of data set

– Computational complexity


Challenges to Applications of ADA

• Data exceptions – too many exceptions in full population testing

– Must develop proper filters for exceptions

– Acceptable level of precision

• Audit evidence – documentation and support concerns

– “Black Box” of data analytics – how to document ?

– Hierarchy of evidence – where does ADA fit in?

• Compliance – existing audit standards and regulations

– Current standards do not provide explicit guidance on analytical procedures

– Standards may need to be transformed


How Can ADA Impact Audit Quality?

• Assist in risk assessment during the acceptance and planning stages of

the audit

• Increase efficiency and effectiveness of controls and substantive testing

• Provide increased coverage through full population testing vs. sampling

methodologies

• More timely analysis through continuous auditing methodologies vs.

year-end audit approach


Big Data Analytics

• The right tools and skills can help the auditor take advantage of Big Data, such as

– Predictive analytics

– Data visualization

– Text mining

– Dashboards (real-time analytics)

– Data warehouses and DBMS

– Expert systems

• Technology is available and being used by other industries

– Audit organizations should look to some of these companies to evaluate how they may be

able to leverage these technologies to integrate Big Data in their audit process.


ARTIFICIAL INTELLIGENCE

What is Artificial Intelligence?

Definition:

“Intelligence exhibited by machines. A

flexible rational agent that perceives its

environment and takes actions that

maximize its chance of success at some

goal. the term ‘artificial intelligence’ is

applied when a machine mimics ‘cognitive’

functions that humans associate with other

human minds, such as ‘learning’ and

‘problem solving’”


Introduction

The increased use of automation and artificial intelligence in

auditing will result in a shift in the auditor’s roles and level of

involvement in the audit, but not the auditor’s responsibility.

“Hiring of auditors and accountants could fall by as much as 50% by 2020 due to the impact of artificial

intelligence” - Steven B. Harris, Board Member PCAOB ~ quoting Big 4 executive

“30% of corporate audits [will be] performed by AI” by 2025 - World Economic Forum survey of 800

executives and experts

“AI technologies are rapidly outpacing the organizational governance and controls” – EY Assurance and

Advisory Leaders


Computerization of Occupations


Adapted from: “The Future of Employment: How Susceptible are Jobs to Computerisation?” (Frey and Osborne, 2013)

History of AI


© Copyright 2019, MindBridge Analytics Inc.

Supervised learning

Human expert feeds the computer with training

data. From that data the computer should learn

the pattern.

Unsupervised learning

No expert input, the computer identifies

pattern in data and looks for outliers. Particularly useful where the human

expert doesn’t know what to look for.

Reinforced learning

Reinforced learning algorithm continuously

learns from the environment in an iterative fashion.

What is machine learning?

AASHTO Audit Annual Meeting 2019 7/26/2019 36

Machine Learning

37

https://www.linkedin.com/pulse/building-machine-learning-infrastructure-pat-alvarado/

7/26/2019

AI Enablers

– Faster technology

– Larger yet cheaper storage

– Computerization

– High level of investments by industry (Google, Baidu, Microsoft, etc)

Deepmind developed AlphaGo

IBM Watson uses in healthcare

Deloitte and Kira systems in contract analysis


Examples of What Machine Learning can do

INPUT A RESPONSE B APPLICATION

Picture Are there human faces? (0 or 1) Photo tagging

Loan Application Will they repay the loan? (0 or 1) Loan approvals

Ad plus user information Will user click on ad? (0 or 1) Targeted online ads

Audio clip Transcript of audio clip Speech recognition

English Sentence French Sentence Language translation

Sensor from plane engine, etc Is it about to fail? Preventive maintenance

Car camera and other sensors Position of other cars Self-driving cars

Source: Andrew Ng


IMPACT OF AI ON THE AUDITING PROFESSION

Artificial Intelligence and Auditing

• What will it enable?– Deep learning in Image recognition (Inventory checks, security footage)

– Natural Language analysis (text mining)

– Speech recognition

– Data from videos (e.g. drones, security footage)

– Sensor data (e.g. RFID)

• What will it impact?– Sampling

– Auditor independence

– Manual preprocessing and examination of certain documents

– Current training and accounting education


Exogenous (Secondary) Evidence Integration

Measurements Measurement variables Assurance of Quality compared with

traditional

Facebook/twitter/news mentions Name mentions

Positive / negatives

Sentiment

Text meaning

Risk faced

Product popularity

Sales level

Different

Calls / mails to customer

services

Classification of type and outcome by

agent

Reserve for product replacement

Bad debt estimates

Different

Internet of Things (IoT) records

of equipment usage

Sensor data (e.g. weather data) External Verification Better

Face recognition of clients Metadata of videos and pictures: time,

location, identity of the person

Fraud Less accurate but exogenous

so it is not intrusive

Video footage Number of cars in parking lots Estimates of sales revenue Less accurate, but more

difficult (costlier) to falsify

Geo-locational data GPS coordinates

Zip codes

Efficiency

Fraud (collision)

FCPA (kickbacks)

Accurate


What data will be considered evidence?

THE EMERGING TECHNOLOGICAL LANDSCAPE

• Inspection

• Damage assessment

• Surveillance

• Bridges

Drones


• Inspection


• Surveillance

• Bridges

Drones


E&Y University Drones for Inventory Case Studies! Bryan’s

Amazing Animals


Blockchain & Smart contracts

• Distributed ledger technology

• Ability to share databases and processes

– MLS database

– Title records

• Smart contracts:

– Real estate contracts

– Escrows

– Property records

– Money7/26/2019AASHTO Audit Annual Meeting 2019 49

Blockchain & Smart contracts

• Distributed ledger technology

• Ability to share databases and processes

– MLS database

– Title records

• Smart contracts:

– Real estate contracts

– Escrows

– Property records

– Money7/26/2019AASHTO Audit Annual Meeting 2019 50

Process Map

Purchase-to-Pay Data

Source:

(1) Purchase-to-Pay: Mieke Jans

7/26/2019 51

Visualization


Audit as a production line


Formalization of Audit through Automation

Issues to be considered

• Billing by hour

• Rigidity of the standards

• Formalization of audit steps:1. Pre-planning Phase

2. Contracting Phase

3. Understanding Internal Controls and Identifying Risk Factors

4. Control Risk Assessment

5. Substantive Tests

6. Evaluation of Evidence

7. Audit Report


Audit Production Line


Phase AI-Enabled Automated Audit Process Traditional Audit Process

Pre-planning -AI collects and analyzes Big Data (exogenous)

-Data related to the client’s organizational

structure, operational methods, and accounting

and financial systems feed into AI system

-Auditors examines client’s industry

-Auditor examines client’s organizational structure,

operational methods, and accounting and financial

systems

Contracting -AI uses the estimate of the risk level (from phase

1) and calculates audit fees, number of hours

-AI analyzes a database of contracts and prepares

the contract

-Auditor and Client sign contract

-Engagement Letter prepared by the auditor based on

the estimated Client risk

-Auditor and client sign contract

Understanding Internal

Controls and Identifying

Risk Factors

-Feed flowcharts, questionnaire answers,

narratives, into AI and use image recognition and

text mining to analyze them

-Use Drones to conduct the walkthrough, then

use AI to analyze the generated video

-Use visualization and pattern recognition to

identify Risk factors

-AI aggregates all this data to Identify Fraud and

illegal acts risk factors

-Document understanding (flowcharts,

questionnaires, narratives, walkthrough)

-Auditor aggregates this information and uses their

judgment to identify risks factors

-Understanding of IC to determine the scope, nature,

and timing of substantive tests.

Control Risk Assessment -Continuous Control Monitoring Systems examine

controls continuously

-AI runs Process mining to verify proper IC

implementation

-Logs are automatically generated to ensure their

integrity.

-Examination of the client’s IC policies and

procedures

-Risk assessment for each attribute

-Test of controls

-Reassess risk

-Document testing of controls.

Substantive tests -Continuous Data Quality Assurance to ensure

quality of data and evidence

-AI examines data provenance

-Continuous test of details of transactions on

100% of the population

-Continuous test of details of balances (at all

times)

-Continuous pattern recognition, outlier

detection, benchmarks, visualization

-Periodical Sampling-based tests, and nature, extent,

and timing depend on IC tests

-Tests of details of a sample of transactions

-Test of details of balances (at a certain point in time)

-Analytical procedures

Evaluation of Evidence -This becomes part of the previous phase -Auditor must evaluate the sufficiency, clarity, and

acceptability of collected evidence. Accordingly, the

auditor may either collect more evidence, or

withdraw from engagement.

Audit Report -AI uses a predictive model to estimate the

various risks identified

-Audit report can be continuous (graded 1-00 for

example) rather than categorical (clean, qualified,

adverse, etc.)

-Auditor aggregates previous information to issue a

report

-Report is categorical: Clean, qualified, adverse, etc.





-Data related to the client’s organizational

structure, operational methods, and accounting

and financial systems feed into AI system

-Auditors examines client’s industry

-Auditor examines client’s organizational structure,

operational methods, and accounting and financial

systems

Contracting -AI uses the estimate of the risk level (from phase

1) and calculates audit fees, number of hours

-AI analyzes a database of contracts and prepares

the contract


-Engagement Letter prepared by the auditor based on

the estimated Client risk


Understanding Internal

Controls and Identifying

Risk Factors

-Feed flowcharts, questionnaire answers,

narratives, into AI and use image recognition and

text mining to analyze them

-Use Drones to conduct the walkthrough, then

use AI to analyze the generated video

-Use visualization and pattern recognition to

identify Risk factors

-AI aggregates all this data to Identify Fraud and

illegal acts risk factors



-Auditor aggregates this information and uses their

judgment to identify risks factors

-Understanding of IC to determine the scope, nature,

and timing of substantive tests.

Control Risk Assessment -Continuous Control Monitoring Systems examine

controls continuously

-AI runs Process mining to verify proper IC

implementation

-Logs are automatically generated to ensure their

integrity.


procedures


-Test of controls

-Reassess risk


Substantive tests -Continuous Data Quality Assurance to ensure

quality of data and evidence


-Continuous test of details of transactions on

100% of the population

-Continuous test of details of balances (at all

times)



-Periodical Sampling-based tests, and nature, extent,

and timing depend on IC tests


-Test of details of balances (at a certain point in time)


Evaluation of Evidence -This becomes part of the previous phase -Auditor must evaluate the sufficiency, clarity, and

acceptability of collected evidence. Accordingly, the

auditor may either collect more evidence, or

withdraw from engagement.



-Audit report can be continuous (graded 1-00 for

example) rather than categorical (clean, qualified,

adverse, etc.)

-Auditor aggregates previous information to issue a

report

-Report is categorical: Clean, qualified, adverse, etc.





-Data related to the client’s organizational structure,

operational methods, and accounting and financial systems

feed into AI system

-Auditor examines client’s industry

-Auditor examines client’s organizational

structure, operational methods, and

accounting and financial systems

Contracting -AI uses the estimate of the risk level (from phase 1) and

calculates audit fees, number of hours

-AI analyzes a database of contracts & prepares contract


-Engagement Letter prepared by the

auditor based on the estimated Client risk


Understanding

Internal

Controls and

Identifying Risk

Factors

-Feed flowcharts, questionnaire answers, narratives, into AI

and use image recognition and text mining to analyze them

-Use Drones to conduct the walkthrough, then use AI to

analyze the generated video

-Use visualization and pattern recognition to identify Risk

factors

-AI aggregates all this data to Identify Fraud and illegal acts

risk factors



-Auditor aggregates this information and

uses their judgment to identify risk factors

-Understanding of IC to determine the

scope, nature, and timing of substantive

tests.

Control Risk

Assessment

-Continuous Control Monitoring Systems examine controls

continuously

-AI runs Process mining to verify proper IC implementation

-Logs are automatically generated to ensure their integrity.


procedures


-Test of controls

-Reassess risk


1

2

3

4

Audit Production Line (Continued)



Substantive tests -Continuous Data Quality Assurance to

ensure quality of data and evidence


-Continuous test of details of transactions

on 100% of the population

-Continuous test of details of balances (at

all times)



-Periodical Sampling-based tests, and nature,

extent, and timing depend on IC tests


-Test of details of balances (at a certain point

in time)


Evaluation of

Evidence

-This becomes part of the previous phase -Auditor must evaluate the sufficiency, clarity,

and acceptability of collected evidence.

Accordingly, the auditor may either collect

more evidence, or withdraw from

engagement.



-Audit report can be continuous (graded

from 1-100 for example) rather than

categorical (clean, qualified, adverse, etc.)

-Auditor aggregates previous information to

issue a report

-Report is categorical: Clean, qualified,

adverse, etc.

5

6

7

THE FUTURE!

Actor or Assistant?


What about the impact on Accounting Education?


•AACSB’s New A5 Standard!–“Accounting degree programs include learning

experiences that develop skills and knowledge

related to the integration of information technology in

accounting and business. This includes the ability of

both faculty and students to adapt to emerging

technologies as well as the mastery of current

technology.”

Carlab Research (Rutgers Accounting Department)


Hussein Issa

[email protected]

(rutgers business school - transportation.org...big data analytics •the right tools and skills can...

Documents