eurosai it working group - michel huissoud 1 training in portugal (1)

EUROSAI IT Working Group - Michel Huissoud

1

Training in Portugal (1)Training in Portugal (1)

EUROSAI IT Working Group - Michel Huissoud

2

Project:Project:« « Design a self-assessmentDesign a self-assessmenttool for SAI’stool for SAI’s based on based on » »

EUROSAI IT Working Group - Michel Huissoud 3

What we wantWhat we want

Improve IT audit (methodology and practical approach with CobiT)

IT Governance (with self-assessment) by the SAIs


1. Genesis of a success story1. Genesis of a success story

the Hague, 1 October 2002


Our mandate Our mandate

The objective of this project is to design and pilot a self-assessment tool for all SAIs. It is based on 'CobiT', which is a governance (and audit) framework for the domain of information technology. The self-assessment tool we are developing should enable us to measure the maturity of the IT control of our own offices.


2. Why...2. Why...

... a self-assessment ?

... of Information Technologies ?

... based on CobiT ?


Why a self-assessment?Why a self-assessment?

It allows « proximity ». The evaluation is carried out by the people :

– who know the subject– who are interested in solving the problems

It is confidential. The organization is in control of the results of the evaluation and their distribution. Self-assessment is not an audit.

The extern moderation encourages the people to speak freely.


Why IT?Why IT?

As in every organisation or company, it is in the interest of the SAI to maintain control of its IT system. The latter is of fundamental importance, whether this has to do with managing dossiers, planning auditor tasks, communication or knowledge management.

Issues concerning communication and defining the roles between the different partners represent one of the main challenges in IT governance. The SAIs, together with other enterprises, need better communication between the sponsors and the IT specialists.


like the other organisations...like the other organisations...

we lose time because of system shutdowns... we type the same information in different

systems two or three times... we develop projects which don’t meet

expectations... we manage expensive service providers... we use IT without enough training...


Why “based on CobiT”?Why “based on CobiT”?

CobiT is a well accepted standard Cobit can be downloaded free from www.isaca

.org CobiT is also available in French www.afai.asso.

fr , German www.isaca.ch and Spanish www.isaca.org

but our group wanted to be sure that CobiT is the best choice ...


Studies of other tools: ISO 9001 European Foundation for Quality Management (EFQM) Excellence

Model ITIL / Process Maturity Self-Assessment & Action Plan CMM Capability Maturity Model Common Assessment Framework (CAF), result of the cooperation

among the EU Ministers responsible for Public Administration

Contact with specialists: Philips, The Netherlands Swisslife, Switzerland Prof. W. van Grembergen (University of Antwerp, Belgium)

What have we done?What have we done?

.... our.... our research confirmed the legitimacy of choosing CobiT research confirmed the legitimacy of choosing CobiT


3. Looking for the gaps and 3. Looking for the gaps and use CobiT as a bridge!use CobiT as a bridge!...the problem is always by the interface

Management IT

IT Audit

IT audit Financial audit


CCOBIOBIT includes 36 national T includes 36 national and international standardsand international standards

Codes of conduct issued by Council of Europe, OECD, ISACA, etc.

Qualification criteria for IT systems and processes: ITSEC, TCSEC, ISO 9000, SPICE, TickIT, Common Criteria, etc.

Professional standards in internal control and auditing: COSO Report, IFAC, AICPA, IIA, ISACA, PCIE, GAO standards, etc.

Industry practices and requirements from industry forums (ESF, I4) and government-sponsored platforms (IBAG, NIST, DTI), etc.

Technical standards from ISO, EDIFACT, etc.

Emerging industry-specific requirements such as from banking, electronic commerce and IT manufacturing


the three most the three most important sources:important sources:

audit standards (IFAC, IIA, COSO, GAO, ...)

IT security standards (ITSEC, BS7799, etc...)

qualification standards (ISO, SPICE, ITIL,...)


CControl ontrol OBOBjectives for jectives for IInformation nformation and Related and Related TTechnologyechnologywith CobiT, they can

communicate together!...


Service levelService levelfor examplefor example

Management GuidelineKey Performance Indicators•Time lag of resolution of a service level change request •Time lag to resolve a service level issue•Number of times that root cause analysis of service level procedure and subsequent resolution is completed within required period•Significance of amount of additional funding needed to deliver the defined service level (...)

Control Objectives•The service level agreement should cover at least the following aspects: availability, reliability, performance, capacity for growth, levels of support provided to users, continuity planning, security, minimum acceptable level of satisfactorily delivered system functionality, restrictions (limits on the amount of work), service charges, central print facilities (availability), central print distribution and change procedures. (...)

Audit Guideline•Considering whether recourse process is identified for non-performance•Testing that historical performance against prior service improvement commitments is tracked (...)


or Informationor InformationArchitectureArchitecture Management Guideline

Key Goal Indicators•(...)•Reduction of data redundancy•Increased interoperability between systems and applications (...)Control Objectives

•Data Classification SchemeA general classification framework should be established with regard to placement of data in information classes (i.e., security categories) as well as allocation of ownership. The access rules for the classes should be appropriately defined.(...)

Audit Guideline•Considering whether a medium is used to distribute the data dictionary to ensure that it is accessible to development areas and that changes are reflected immediately•Identifying data items where ownership is not clearly and/or appropriately defined. (...)


or manageor managethe operationsthe operations

Management GuidelineCritical Success Factors•Changes to job schedules are strictly controlled•There are strict acceptance procedures for new job schedules, including documentation delivered•Clear and concise detection, inspection and escalation procedures are established(...)

Control Objectives•Job SchedulingIT management should ensure that the continuous scheduling of jobs, processes and tasks is organised into the most efficient sequence, (...). The initial schedules as well as changes to these schedules should be appropriately authorised.•Remote OperationsFor remote operations, specific procedures should ensure that the connection and disconnection of the links to the remote site(s) are defined and implemented..(...)

Audit Guideline•Review of a sample of limited IT operations and determining whether they meet policy and procedures requirements. •Identifying a sample of abnormal ends (ABENDS) for jobs and determining resolution of problems which occurred. (...)


Quality Cost Delivery

Confidentiality Integrity Availability

Effectiveness and Efficiency of operations or projects

Reliability of Information Compliance with laws and regulations

CobiT is specialCobiT is special

.... .... this framework goes further than this framework goes further than the other ones!the other ones!


Navigation in CobiT: How can you select the right process?Navigation in CobiT: How can you select the right process?« availability » for example« availability » for example


or « human ressources » ?or « human ressources » ?


Warm up…Warm up…

Who doesn't know what the EUROSAI IT Working Group is?

Who doesn't know what CobiT is? Who doesn't know what self-assessment is? Is self-assessment a questionnaire or an

interview method? Are we looking for problems in efficiency or

in security?


4. Our method4. Our method


How do we proceed?How do we proceed?

2 weeks before Workshop

Documentation to study will be

provided on CobiT, self-assessment,

etc..)

The instructor will provide more

information, the structure of your business will be discussed and

then forms will be filled in

The instructor will consolidate the results and a

discussion of the results will

follow

An action plan for the future

will be prepared together and the

exercise will then be

evaluated

The results of the workshop

are then presented to

the top management

of the SAI

Post ws


business process 2

business process 3

business process 4

business process 5

business process 6

business process 1

business process 7

first

dim

ensi

on =

bus

ines

s

The problem has 2 dimensionsThe problem has 2 dimensions

PO

1

AI1

AI2

PO

2

Planing ans organisation

acquisition and implementation

Etc...

Etc…

Etc…

second dimension = IT


the first formthe first formidentify the business processidentify the business process

Business added-value chain

high

(5)

impo

rtanc

e le

vel (

4)im

porta

nce

leve

l (3)

impo

rtanc

e le

vel (

2) lo

w (1

)no

app

licat

ion

softw

are

(0)

high

(5)

impo

rtanc

e le

vel (

4)im

porta

nce

leve

l (3)

impo

rtanc

e le

vel (

2) lo

w (1

)no

app

licat

ion

softw

are

(0)

very

low

(0)

qual

ity le

vel (

1)qu

ality

leve

l (2)

qual

ity le

vel (

3)qu

ality

leve

l (4)

very

hig

h (5

)

B1 Audit Risk Management

B2 Organise the missions

B3 Analyse the data

B4 Test the IT by the IT-Audit

B5 Report the results to the auditee

B6 Track the implementation of the recommandations

B7 Manage the knowledge

B8 Manage finances and human ressources

B9 Administer and archive the dossiers

B10 Publish the results of the audits

B11 Communicate

B12 other …

B13 other…

What is the importance of the current IT systems for this business process?

In which IT-process (see in Form 2) is the problem (especially if quality level = 0 or 1)?

What is the quality of the current IT systems ?

What is the importance of the future IT systems for this business process?

BVC Form 1. Does the IT help to achieve the SAI's strategic goals?


What do we understand by What do we understand by “business process”? examples:“business process”? examples:Audit Risk ManagementOrganise the missionsAnalyse the dataTest the IT by the IT-

AuditReport the results to the

auditeeTrack the

implementation of the recommendations

Manage the knowledge

Manage finances and human resources

Administer and archive the dossiers

Publish the results of the audits

Communicate Automated data inputs Automated relations

between different audits


then, we evaluate the then, we evaluate the importanceimportance and and the the qualityquality of the current IT systems of the current IT systems


high

(5)

impo

rtanc

e le

vel (

4)im

porta

nce

leve

l (3)

impo

rtanc

e le

vel (

2) lo

w (1

)no

app

licat

ion

softw

are

(0)

high

(5)

impo

rtanc

e le

vel (

4)im

porta

nce

leve

l (3)

impo

rtanc

e le

vel (

2) lo

w (1

)no

app

licat

ion

softw

are

(0)

very

low

(0)

qual

ity le

vel (

1)qu

ality

leve

l (2)

qual

ity le

vel (

3)qu

ality

leve

l (4)

very

hig

h (5

)



B3 Analyse the data








B11 Communicate

B12 other …

B13 other…






Importance of the IT

systems?

Quality of the IT systems

?


the second formthe second form

very

impo

rta

nt (

2)im

port

ant

(1)

not

imp

orta

nt (

0)

not

sure

(0)

COBIT's Domains and Processes non

-exi

sten

t (0

)in

itial

/ a

d h

oc (

1)

repe

ata

ble

but

intu

itive

(2)

defin

ed p

roce

ss (

3)m

anag

ed a

nd m

easu

rabl

e (4

)op

timis

ed (

5)

Planning and OrganisationPO1 Define a Strategic IT Plan

PO2 Define the information architecture

PO3 Determine the technological direction

PO4 Define the IT Organisation and Relationships

PO5 Manage the IT investment

Which business processes (see in Form 1) are affected by this problem (especially if level = 0 or 1)?

Maturity level of the process

Importance of the process

CobiT Form 2: What is the maturity level of the

IT-processes?

. . .

Importance of the IT

systems?

Quality of the IT systems

?


6 maturity levels6 maturity levels


Maturity model? Maturity model? Example: “Example: “DS04 Ensure continuous serviceDS04 Ensure continuous service””0 Non-existent.

There is no understanding of the risks, vulnerabilities and threats to IT operations or the impact of loss of IT services to the business. Service continuity is not considered as needing management attention.

5 Optimised5 Optimised Integrated continuous service processes are proactive, self-adjusting, automated and self-analytical and take into account benchmarking and best external practices. Continuous service plans and business continuity plans are integrated, aligned and routinely maintained. Buy-in for continuous service needs is secured from vendors and major suppliers. Global testing occurs and test results are feed back as part of the maintenance process. Continuous service cost effectiveness is optimized through innovation and integration. Gathering and analysis of data is used to identify opportunities for improvement. Redundancy practices and continuous service planning are fully aligned. Management does not allow single points of failure and provides support for their remedy. Escalation practices are understood and thoroughly enforced.


Example 2: “Example 2: “PO10 Manage projectsPO10 Manage projects””0 Non-existent.

Project management techniques are not used and the organization does not consider business impacts associated with project mismanagement and development project failures.

5 Optimised5 Optimised A proven, full life-cycle project methodology is implemented and enforced, and is integrated into the culture of the entire organization. An on-going program to identify and institutionalize best practices has been implemented. There is strong and active project support from senior management sponsors as well as stakeholders. IT management has implemented a project organization structure with documented roles, responsibilities and staff performance criteria. A long term IT resources strategy is defined to support development and operational outsourcing decisions. An integrated program management office is responsible for projects from inception to post implementation. The program management office is under the management of the business units and requisitions and directs IT resources to complete projects. Organization-wide planning of projects ensures that user and IT resources are best utilized to support strategic initiatives.


very

impo

rtan

t (2

)

impo

rtan

t (1

)

not

impo

rtan

t (0

)

not

sure

(0)

COBIT's Domains and Processes non-

exis

tent

(0)

initi

al /

ad

hoc

(1)

repe

atab

le b

ut in

tuiti

ve (

2)

defin

ed p

roce

ss (

3)m

anag

ed a

nd m

easu

rabl

e (4

)

optim

ised

(5)

Planning and OrganisationPO1 Define a Strategic IT Plan

PO2 Define the information architecture

PO3 Determine the technological direction

PO4 Define the IT Organisation and Relationships

PO5 Manage the IT investment

PO6 Communicate management aims and direction

PO7 Manage human resources

PO8 Ensure compliance with external requirements

PO9 Assess risks

PO10 Manage projects

PO11 Manage quality

Acquisition and ImplementationAI1 Identify automated solutions

AI2 Acquire and maintain application SW

AI3 Acquire and maintain technology infrastructure

AI4 Develop and maintain procedures

AI5 Install and accredit systems

AI6 Manage changes

Delivery and SupportDS1 Define and manage service levels

DS2 Manage third-party services

DS3 Manage performance and capacity

DS4 Ensure continuous service

DS5 Ensure system security

DS6 Identify and allocate costs

DS7 Educate and train users

DS8 Assist and advise customers

DS9 Manage the configuration

DS10 Manage problems and incidents

DS11 Manage data

DS12 Manage facilities

DS13 Manage operations

MonitoringM1 Monitor the processes

M2 Assess internal control adequacy

M3 Obtain independent assurance

M4 Provide for independent audit

Which business processes (see in Form 1) are affected by this problem (especially if level = 0 or 1)?

Maturity level of the process

Importance of the process

CobiT Form 2: What is the maturity level of the IT-processes?


high

(5)

impo

rtanc

e le

vel (

4)im

porta

nce

leve

l (3)

impo

rtanc

e le

vel (

2) lo

w (1

)no

app

licat

ion

softw

are

(0)

high

(5)

impo

rtanc

e le

vel (

4)im

porta

nce

leve

l (3)

impo

rtanc

e le

vel (

2) lo

w (1

)no

app

licat

ion

softw

are

(0)

very

low

(0)

qual

ity le

vel (

1)qu

ality

leve

l (2)

qual

ity le

vel (

3)qu

ality

leve

l (4)

very

hig

h (5

)



B3 Analyse the data








B11 Communicate

B12 other …

B13 other…






matching the results ...matching the results ...

Where are the reasons for the dissatisfaction?

What impacts do the IT problems have?


5. what you get5. what you get

gaps analysis a good discussion ! action plan


For example: satisfaction with the IT support For example: satisfaction with the IT support of the business processesof the business processes

B10 2.29B12 confidential.... 2.00B6 confidential... 1.75B5 confidential... 1.38B3 confidential... 1.33B4 confidential... 1.29B1 confidential... 1.00B9 confidential... 1.00B7 confidential... 0.86B2 confidential... 0.83B11 confidential... 0.60B8 confidential... 0.00

confidential....


0

1

2

3

4

5

6

B1

B2

B3

B4

B5

B6

B7

B8

B9

B10

B11

B12

B13

process

What is the quality of thecurrent IT systems ?

What is the importance ofthe future IT systems ?

identification of the problems identification of the problems (business point of view)(business point of view)


0

1

2

3

4

5

6

PO

1

PO

2

PO

3

PO

4

PO

5

PO

6

PO

7

PO

8

PO

9

PO

10

PO

11 AI1

AI2

AI3

AI4

AI5

AI6

DS

1

DS

2

DS

3

DS

4

DS

5

DS

6

DS

7

DS

8

DS

9

DS

10

DS

11

DS

12

DS

13 M1

M2

M3

M4

process

identifying the problems identifying the problems (IT point of view)(IT point of view)

Importance of the process ?Maturity level of the process ?


An action planAn action plan

Findings and Actions form

Finding/ Gap Description

Risk / implication Recommendation / action description

Person in charge

Deadline for finishing activity

Priority 1-10


and perhaps in the future:and perhaps in the future:a benchmarkinga benchmarking

Big SAIs

Middle SAIs

Small SAIs


A reasonable time management (first day)A reasonable time management (first day)

14.00 Start of the workshop15.00 Identify the business processes15.30 Coffee break (moderator) Adaptation of the form 1 and print them16.00 Fill form 116.15 Presentation CobiT17.15 Select the most important IT processes18.00 Fill form 218.30 End of the first day

Then, put the results in your EXCEL sheet, prepare the presentation of the results and the discussion of tomorrow…


A reasonable time management (second day)A reasonable time management (second day)

09.00 Presentation of the results09.30 Discussion (validation of the results, looking for

consensus)10.00 Listing the most important problems and strengths10.15 Coffee break 10.45 Prepare an action plan11.30 Fill the evaluation forms

Finalization of the action plan12.30 Discussion and end of the workshop

Preparation of the final presentation15.00 Presentation and discussion with the head of the

SAIWrite the evaluation report!


2 weeks before Workshop

Documentation to study will be

provided on CobiT, self-assessment,

etc..)

The instructor will provide more

information, the structure of your business will be discussed and

then forms will be filled in

The instructor will consolidate the results and a

discussion of the results will

follow

An action plan for the future

will be prepared together and the

exercise will then be

evaluated

The results of the workshop

are then presented to

the top management

of the SAI

Post ws

We will now focus We will now focus on the following pointson the following points

Get the right

persons!

Identify the

processes!

Ask the right

questions!

Get a good action plan!

Use the EXCEL sheet

correctly!

eurosai it working group - michel huissoud 1 training in portugal (1)

Documents