Automação de Auditoría

Ryan Teeter, Ph.D. Student, Rutgers University

18th World Continuous Auditing Symposium6th CONTECSI São Paulo, Brasil – June 4, 2009

18th WCARS – June 4, 2009

Aiding the Audit

Outline1. Introduction

– Continuous Controls Monitoring (CCM) and COSO Guidance

2. Automating the IT Audit– Evaluating monitoring software platforms

3. Implementation of CCM at Siemens PLM– Classifying audit requirements into degrees of automation– Creating rules from Audit Action Sheets– Reengineering audit processes– Feedback loop

4. Preliminary Results– Time and resource commitments– Successes & Challenges

5. Conclusion

2

18th WCARS – June 4, 2009

Aiding the Audit

1. Introduction

• Continuous Controls Monitoring (CCM)– Evaluating control settings on business processes that provide

compliance with regulation and/or internal management objectives.

– Proof of concept expands existing research in continuous audit streams (see Brown, Wong, and Baldwin 2007, Alles et al 2006)

• COSO “Guidance on Monitoring Internal Control Systems” (2008):– Effective monitoring involves (1) establishing an effective

foundation for monitoring, (2) designing and executing monitoring procedures that are prioritized based on risk, and (3) reporting the results, and following up on corrective action where necessary.

3

18th WCARS – June 4, 2009

Aiding the Audit

4

2. Automating the IT audit

• Why the IT audit?– Cost considerations and effectiveness– Internal audit team spends approximately 70 days

manually checking tables, authorizations, and documentation

• Vasarhelyi et al (2004) indicate firms are likely to adapt existing internal audit programs

• Alles et al (2006) suggest utilizing the expertise of experienced audit professionals

• Verifiability of automated controls against results from the manual audit

18th WCARS – June 4, 2009

Aiding the Audit

• CCM aids compliance for SOX sec. 201 and 404

• Large accounting firms unable to source CCM

• Third-party platforms installed as monitoring and control layer– Minimal impact on performance– See Vasarhelyi, 2004

2.1 Evaluating monitoring software platforms

5

Type ERP Topography Example

System-specific Homogenous Approva BizRights

Modular and mapping Homo/heterogenous ACL IDEA, OverSight

Custom Homo/heterogenous Siemens’ e-AuditAlles, et al 2006

18th WCARS – June 4, 2009

Aiding the Audit

Siemens’ Current SAP Audit Model• Use text file output and transaction checks on line to audit

SAP• Report findings and recommendations for remediation• Use follow-up audits to assure appropriate controls are

in place and remain in place

6

Company ASAP SYS.

PD2

Company BSAP SYS.

P88

Company CSAP SYS.

P51

Company DSAP SYS.

P40

Common –“E -Audit” Extractions on a request basis.

Text FileStore

Text FileStore

Text FileStore

Text FileStore

18th WCARS – June 4, 2009

Aiding the Audit

3. Implementation of CCM at Siemens PLM• Rules were created in Approva BizRights based on

~300 audit action sheets provided by Siemens• Siemens Corporation wants universally-adaptable sets

of rules and control tests for use in different divisions

7

18th WCARS – June 4, 2009

Aiding the Audit

8

18th WCARS – June 4, 2009

Aiding the Audit

9

18th WCARS – June 4, 2009

Aiding the Audit

3.1 Classifying rules by degree of automation• Authorization

– users with access to screens or functions– Approx 30% of audit effort

• Baseline• Separation of duties• Transaction

– Frequency of code use

• User Activity Insight– Timeliness and correctness

• Configuration– ERP settings

• Manual10

18th WCARS – June 4, 2009

Aiding the Audit

3.2 Creating rules from Audit Action Sheets• Low-hanging fruit

– Authorization requests– Separation of duties checks– Example: See who can create and approve purchase orders

• Partial automation– Example: See who has access and whether that is appropriate

• Non-automatable– Evaluation of documentation– Interviews with managers

11

18th WCARS – June 4, 2009

Aiding the Audit

3.3 Reengineering audit processes

• Creation of custom rules in Approva InsightStudio• Combination of existing controls tests• Partial automation of manual controls

– “Gain an understanding of X process. Verify Y function isn’t allowed.”

12

18th WCARS – June 4, 2009

Aiding the Audit

3.4 Feedback loop

• Rule descriptions were added to aid the audit• Rules were tested and compared to results from the

manual audit• Adjustments were made based on results

13

Rule Type AAS Ref #

Short Description Description of rule to be made

Conditions used

Status

Authorization 1.02.X Unauthorized access to SAP system – emergency user concept

Test these authorizations:1. S_TCODE=SM18, S_ADMIN_FCD=AUDA2. …

AI rules Rule Built 1.02.X

Configuration 1.02.X System admin/completeness verification

Set up 3 rules to test the following:1. parameter rdisp/vbdelete=02. parameter rdisp/vbreorg=03. …

Parameters are listed in report RSPFPAR

Rule Built 1.02.X

18th WCARS – June 4, 2009

Aiding the Audit

4. Results

• Time and resource commitments • Successes• Challenges• Firm characteristics

14

18th WCARS – June 4, 2009

Aiding the Audit

4.1 Time and resource commitments

• Time commitments:– 70 days for the manual audit– 3 months preparation

• Platform installation• AAS classification

• Resource commitments– Travel, lodging, etc.

• 3-5 researchers – 3 Full-time equivalent• 2 internal auditors at PLM• 2 IT auditors from Siemens• 1 support staff from Approva

15

18th WCARS – June 4, 2009

Aiding the Audit

4.2 Successes

• Initially approximately 63% of controls automated• Rules were used to provide support for the IT audit• Initial evaluation of cost savings (A&D PL specific)1:

– For 3 of every 4 years, eliminate ~ 500 man-hours of IT GCC and application control testing (@ $137/hr = $68,750/year)

– With system certified, 80% reduction in 500 man-hours of annual external IT audit hours (@ ~$200/hr, $80,000/year)

1 Siemens IT audit pool billing rate is $137/hour; Approx $200/hr Big 4 blended rate.

16

18th WCARS – June 4, 2009

Aiding the Audit

4.2 Successes

ModuleTotal Controls

Authorizations Controls

Percent Automated (%)

Business Process Controls

Percent Automated (%)

Overall Percent Automated (%)

Basis System (BC) 104 20 100% 84 44% 55%

Financial Accounting (FI) 55 8 100% 47 51% 58%

Asset Accounting (AA) 26 4 100% 22 64% 69%

Sales and Distribution (SD) 21 5 100% 16 50% 62%

Materials Management (MM) 32 8 100% 24 54% 66%

Project System (PS) 32 9 100% 23 70% 78%

Human Resources (HR) 14 14 100% N/A N/A 100%

Total 284 68 100% 216 52% 63%

17

18th WCARS – June 4, 2009

Aiding the Audit

4.3 Challenges

• Audit priority– Non-applicable rules ignored because of time constraints

• CCM platform issues– Bugs or unimplemented features– Identified when comparing automated with manual results– Vendor vs. auditor priorities– Issues addressed in future releases

• Properly functioning controls– Control failure resulted in lack of support for the audit

18

18th WCARS – June 4, 2009

Aiding the Audit

4.4 Firm characteristics

• Siemens PLM – Technology firms generally have better IT controls– Already using SAP R/3

• Degree of success may depend on the amount of IT systems and support.

19

18th WCARS – June 4, 2009

Aiding the Audit

5. Conclusion

• The IT audit is a feasible starting point for CCM implementation– Existing audit plan– Knowledge of experienced auditors– Real-time performance comparison

• 63% of audit controls automated – 100% of authorizations, which comprise 30-35% of audit

commitment– vs. 75% proposed by Alles et al. (2006)

20

18th WCARS – June 4, 2009

Aiding the Audit

Expanding this paper

• Weighting control risk? – (Cushing 1974, Cash et al, 1977, Vasarhelyi 1980, Srindini and

Vasarhelyi 1986, Vasarhelyi and Srindini 1989)

• Cost savings reallocation to auditing rulebooks?

21