importing data - the complete course in all file types and data tricks to get your data ready for...

2/19/2015

1

AuditNet® Training without Travel™IDEA Software Training via Practical Application

Importing Data - The Complete Course in All File Types and Data Tricks to Get Your Data Ready for Analysis

Guest Presenter:Richard Cascarino, MBA, CIA, CFE,CRMA, CISMRichard Cascarino & Associates

About Jim Kaplan CIA CFE

President and Founder of AuditNet®, the global resource for auditors (now available on Apple, Microsoft and Android devices)

Auditor, Web Site Guru,

Internet for Auditors Pioneer

Recipient of the IIA’s Bradford Cadmus Memorial Award.

Author of “The Auditor’s Guide to Internet Resources” 2nd Edition

2/19/2015

2

Richard Cascarino MBA CIA CISM CFE

Principal of Richard Cascarino & Associates based in Colorado USA

Over 30 years experience in IT audit training and consultancy

Past President of the Institute of Internal Auditors in South Africa

Member of ISACA, ACFE, IIA

Author of Auditor's Guide to IT Auditing and Corporate Fraud and Internal Control

Copyright © 2014 AuditNet® and Richard Cascarino & Associates

Webinar Housekeeping

This webinar and its material are the property of AuditNet® and Richard Cascarino and Associates. Unauthorized usage or recording of this webinar or any of its material is strictly forbidden. We are recording the webinar and you will be provided with a link access to that recording as detailed below. Downloading or otherwise duplicating the webinar recording is expressly prohibited.

Webinar recording link will be sent via email within 5‐7 business days.

NASBA rules require us to ask polling questions during the Webinar and CPE certificates will be sent via email to those who answer ALL the polling questions

The CPE certificates and link to the recording will be sent to the email address you registered with in GTW. We are not responsible for delivery problems due to spam filters, attachment restrictions or other controls in place for your email client.

Submit questions via the chat box on your screen and we will answer them either during or at the conclusion.

After the Webinar is over you will have an opportunity to provide feedback. Please complete the feedback questionnaire to help us continuously improve our Webinars

If GTW stops working you may need to close and restart. You can always dial in and listen and follow along with the handout.

2/19/2015

3


Disclaimers

The views expressed by the presenters do not necessarily represent the views, positions, or opinions of AuditNet® or the presenters’ respective organizations. These materials, and the oral presentation accompanying them, are for educational purposes only and do not constitute accounting or legal advice or create an accountant‐client relationship.

While AuditNet® makes every effort to ensure information is accurate and complete, AuditNet® makes no representations, guarantees, or warranties as to the accuracy or completeness of the information provided via this presentation. AuditNet® specifically disclaims all liability for any claims or damages that may result from the information contained in this presentation, including any websites maintained by third parties and linked to the AuditNet® website

Any mention of commercial products is for information only; it does not imply recommendation or endorsement by AuditNet®

Today’s Agenda

Importing a number of different data types and formats Access the Import Assistant Select the File to ImportFile Type Specify Record Length Specify Field Delineators Field Details Create Fields Import CriteriaAppending a Virtual Field

Page 6

2/19/2015

4


Starting IDEA Pre Version 9

Start

Programs

IDEA

Setting the Working Folder ‐ contains data files

Equations

Views

import definitions

File

Set Working Folder

C:\User\<username>\My Documents\IDEA\Samples

7


Starting IDEA Version 9

Start

Programs

IDEA

Setting the Working Folder ‐ contains data files

Equations

Views

import definitions

Select

Managed Projects

Samples

8

2/19/2015

5


Importing a Text File

Use Desktop (+) to import a text file called sales.txt from the Tutorial folder

Next

Next

You will need the file layout

9


Sales.txt Layout

Field Name Type Start Len Dec Mask Description

INV_NO 1 7 Invoice Number

TRANS_DATE D 8 8 YYYYMMDD Transaction Date

PAY_TYPE C 16 4 Type of Payment

SALESMAN N 20 3 Salesman ID

CUST_NO N 23 5 Customer Number

PROD_CODE C 28 2 Product Code

AMOUNT N 30 11 2 Transaction Amount

10

2/19/2015

6

Polling Question 1


Defining Files

Next

If data to be filtered click Equation Editor

Next

We want to: Import the File

Generate Field Statistics

Call the database Sales Transactions

Finish

(If you make a mistake Data then Field Manipulation)

12

2/19/2015

7



Use Desktop (+) to import a text file called sales.txt from the Tutorial folder using the Advanced Record Definition Editor

Next

Use the same file layout until all fields are defined

Call the database Sales Transactions

13



Use Desktop (+) to import an text file called EBCDIC.DAT from the Samples folder

Next

Use the same file layout until all fields are defined

Call the database Sales Details

14

2/19/2015

8


EBCDIC.DAT Layout

• Field Name Type Start Len Dec Mask Description

• SURNAME C 1 20 Surname

• INITS C 21 21 Initials

• ACCT NO N 42 9 Account Number

• TR DATE N 51 8 YYYYMMDDTransaction Date

• AMOUNT N 59 14 2 Amount

15


Importing from an AS400 (iSeries)

Use Desktop (+) to import an AS400 file

Next

Input data file name AS400.dat from the Samples folder

Input definition file name AS400def.fdf from the Samples folder

Call the database AS400 Details

16

2/19/2015

9


Importing from dBase

Use Desktop (+) to import a dBase file

Input data file name DBASE.DBF from the Samples folder

Next

Next

Finish

17

Polling Question 2

2/19/2015

10


Importing from ACCESS

Use Desktop (+) to import an Access table

Input data file name ACCESS.MDBfrom the Samples folder

Open

Next

OK

19


Importing from Excel

Use Desktop (+) to import an Excel file

Input data file name Sample.xls from the Samples folder

Open

Next

Select the first 4 sheets

First Row is Field Names

OK

20

2/19/2015

11


Importing from ODBC

Use Desktop (+) to import a VISIO file

Next

Input table name Network -Computers

Next

Finish

21


Importing from XML

Use Desktop (+) to import an XML file

Next

Input table name OrderDetails.xml from the Samples folder

Next

Expand All

Collapse All

OK

22

2/19/2015

12


Importing from Comma Delimited

You may not have one

Use Desktop (+) import a Text file

Input table name Interviewing.cms from the Samples folder

Next

Next

Next

OK

23

Polling Question 3

2/19/2015

13


Skipping Print Report and PDF

That’s the next webinar

25


Why?

Sort Index

Execution Speed slower faster

Resulting File Size larger smaller

Required Disk Space more less

Processing the Whole File much faster much slower

Processing a Few Records much slower much faster

Sorting vs Indexing files

26

2/19/2015

14

Using IDEA for Data Analysis

What are the objectives of the project?

Who is the custodian of the data (data owner)? i.e. Finance, Purchasing

Who is the IT person overseeing the system?

Consider the data issues

When should I request the data?

If confidential or sensitive information then need to consider security over the data files

First step in detecting, investigating and auditing for fraud using data analysis: Gaining access to the data Identify investigation objectives Arrange how to get the dataMeet with the data custodian (owner) and IT

Define the required data parametersData fields/files neededFormat of the filesRecord layout of the fileTiming of the transferTransfer method

Verifying the data received – QABest to do this BEFORE processing it


2/19/2015

15

IDEA can be used to identify almost any type of data-related anomaly: Example:

Inventory… Calculate total of inventory balances

Calculate totals by product category

Generate exception reports, including negative values, quantities or costs, extension errors, and so on

Select items from perpetual stock records for test counts

Report transactions after period end

Prepare aged schedules of inventory items

Report months of inventory on hand from usage history

Report variations between test counts and perpetual records


Polling Question 4

2/19/2015

16

Ensuring the Data is CleanUse of control totals and Hash totals

Use of Data Verification

Cleaning up data for interrogation

Recording all clean-ups

Using IT to help Keep it Clean

Acquiring the Data

Obtaining Access Rights Read-only

If IT won’t give access, approach the user management. It’s their dataThey can permit whoever they wantYou have Carte Blanche to look at anything but keep that in reserve

TemporaryYou only need access to live data for the duration of the audit

ConfidentialityYou will ensure the confidentiality of the data at all times is in audit’s possessionAll archive copies of the data will be kept encrypted

Acquiring the Data

2/19/2015

17

Acquiring the DataGetting it From the Computer Raw dataMainframe

EBCDICPC

ASCIIDiffering sourcesDisc

ODBCDelimited TextACCPACSAP Private formatAS/400 FDFCOBOL format

Tape

Acquiring the DataNeeded to Obtain the Data Access to the DataWhat file is it?What disc is it on? Can I have read access?What does the data look like?

“Owners’” Arguments against You have no right to the data files You actually don’t know what you want You will probably damage the data You won’t understand what you are looking

at …continued

2/19/2015

18

Acquiring the Data Needed to Obtain the Data (continued)

Your Counterarguments As audit we have unrestricted access to any data we

require for the auditWe know what we want to know, you should be able to tell

me where it isWe want read-only access – we cannot damage the live

dataWe also require data file layouts

Acquiring the Data

Further arguments … and counterarguments Tell us what you want and we’ll extract it for youWe don’t want it extracted, we want it in unaltered format

Why don’t we just give you a printout – tell us what you wantWe may need to analyze it in several ways depending on

what we find

There is too much data to analyze Heard of computers?

We don’t know how to access that data Tell us who does and we’ll speak to them

…continued

2/19/2015

19

Acquiring the Data

Further arguments … and counterarguments (cont’d)

There is no way to access that dataWhat you are saying is that you don’t know how – see above

We don’t have files going that far back How far back can you go?Who decided the retention and on what basis?

Polling Question 5

2/19/2015

20

Acquiring and Importing Data

You’ve got the data – now what? Make sure it’s what you asked for

Timeliness – does it reflect the right period? Accuracy – is it the live data? Completeness – is it all the data?

It’s embarrassing to come to an adverse conclusion only to find you were given the “wrong” file / layout etc.

Its even worse if you came to a non-adverse conclusion

Check against known Control totals Dates Transactions

Never believe what the first printout tells you


It seems to be the right data – now what? You know what you wanted to find You knew where the data resided Now you’ve got it Go ahead with the analysis you planned You have the answer NOW CHECK IT Remember – Never Believe What The First Printout

Tells YouParticularly if its what you want to believe

2/19/2015

21


If all you can get is the hard copy Can they print it to a file instead

Comma Delimited if possible Fred Smith, Internal Audit,3/13/2011, Individual data fields separated by commas Easy for the software to identify individual fields

If it’s a printout scan it 1 field of 120 characters for example The audit software will allow you to define fields within the 120

characters You can even define different layouts for different rows

Polling Question 6

2/19/2015

22


Questions?

• Any Questions?Don’t be Shy!


Training Schedule

IDEA Software Training via Practical Application

1. IDEA Basics, Getting Started, and Basics of Importing Data

2. Importing Data ‐ The Complete Course in All File Types and Data Tricks to Get Your Data Ready for Analysis

3. Conducting basic Data Analysis with IDEA

4. Reporting using IDEA

5. Importing Data from Reports using the IDEA Report Reader

6. Advanced Report Importing

7. Using Statistics in IDEA

8. Continuous Monitoring using Advanced Statistical Analysis

9. Basic Script Writing in IDEA

10. Using advanced @Functions

11. Advanced Script Writing in IDEA Part 1

12. Advanced Script Writing in IDEA Part 2

2/19/2015

23


Training Schedule

IDEA Software Training via Practical Application

1. Conducting basic Data Analysis with IDEA 3/18

2. Reporting using IDEA 3/20

3. Importing Data from Reports using the IDEA Report Reader 3/25

4. Advanced Report Importing 3/27

5. Using Statistics in IDEA 4/1

6. Continuous Monitoring using Advanced Statistical Analysis 4/3

7. Basic Script Writing in IDEA 4/8

8. Using advanced @Functions 4/10

9. Advanced Script Writing in IDEA Part 1 4/15

10. Advanced Script Writing in IDEA Part 2 4/17

Supplemental Information

CAATS – Web site from Dave Coderre Generic Approach to the Application of Data Analysis to Auditing

©CAATS 2007 http://www.caats.ca

Computer Assisted Audit Tools and Techniques http://www.auditnet.org/audit-library/computer-assisted-audit-tools-and-techniques-caatt

Data Analysis Software Demo Evaluations ACL – http://www.acl.com IDEA - http://www.caseware.com/products/idea Arbutus - http://www.arbutussoftware.com/

AuditNet® Documents (by request) AuditNet® Guide to Downloading Data Data Request Sample Letter CAATT Application Checklist

2/19/2015

24


Thank You!

Richard Cascarino, MBA, CIA, CISM, CFE Richard Cascarino & Associates

[email protected]

Jim KaplanAuditNet LLC®800-385-1625

[email protected]

importing data - the complete course in all file types and data tricks to get your data ready for...

Data & Analytics

institute of internal

iia author of auditors

data ready

data tricks

auditors pioneer recipient

founder of auditnet

jim kaplan cia cfe president

number of different