KPI Project Progress Report (Project I: Key Performance Indicator

Methodology for Auditing IT Programs)

China National Audit Office

January, 2012

1. Background

2. Project history

3. Project objectives

4. Project scope

5. Project plan

6. PKI beta database

Contents

1. Background

the 19th of the INTOSAI Working Group on

IT Audit

Development of IT Performance Indicators

Performance measures of IT Solutions

implemented in government organizations

Index System about IT Performance Audit

1. Background

the 19th of the INTOSAI Working Group on

IT Audit

The above three projects were put into a new

one. “Key Performance Indicator Methodology

for Auditing IT Programs”.

China volunteered to be the team leader.

team members : SAI Bhutan, China, Ecuador,

Japan, Kuwait, Malaysia, Pakistan, Poland,

Russia and USA.

1. Background

2. Project history

3. Project objectives

4. Project scope

5. Project plan

6. PKI beta database

Contents

2. Project history

the first group meeting on Apr 16, 2010 in

Beijing.

a new name for the project, “Key

Performance Indicator Methodology for

Auditing IT Programs”.

the minutes of the first team meeting.

2. Project history

SAI Russia, Japan , USA , Kuwait and

Ecuador explained their own understanding

about the project.

the project initiation document.

2. Project history

SAI Kuwait submitted the beta database of

KPI indicators.

the original framework for comments.

SAI Kuwait, Bhutan and Japan gave their

comments and suggestion.

a new framework.

2. Project history

the Second group meeting on Apr 16th,

2011 in South Africa

the minutes of the second team meeting

2. Project history

SAI Kuwait submitted KPI beta database

categorized.

SAI Bhutan gave comments on draft framework of

KPI.

SAI China composed the draft of indicators based

on the team members’ deliverables and send it to

the team members for comments by the end of

November.

SAI Kuwait and Japan gave their comments and

suggestion.

1. Background

2. Project history

3. Project objectives

4. Project scope

5. Project plan

6. PKI beta database

Contents

3. Project objectives

develop the mechanisms

implement a KPI’s in the SAI’s

exchange views, know-how, and information

two principal outcomes of the project

1. Background

2. Project history

3. Project objectives

4. Project scope

5. Project plan

6. PKI beta database

Contents

4. Project scope

a set of specific and measurable IT-related

indicators

the ideas of KPIs application

feasible KPI setting and evaluation

methodology

audit case studies of IT projects using KPIs

Investigation and development of possible

areas of performance indication

Analysis, conclusions and decision making

based on results of performance

measurement

Analysis, conclusions and decision making

based on results of performance

measurement.

4. Project scope

1. Background

2. Project history

3. Project objectives

4. Project scope

5. Project plan

6. PKI beta database

Contents

5. Project plan

Milestone

the project framework

the beta database

the draft project Guidance

the final project Guidance

5. Project plan

Next detailed work plan

By the end of April 2012, SAIs submit their own

draft project guidance.

Before June 15, 2012, SAI China composes the

first draft project guidance.

By the end of August 2012, SAIs give comments

and suggestions.

Before October 15, 2012, SAI China composes

the second draft project guidance.

By the end of November 2012, SAIs give further

comments and suggestions.

By the end of January 2013, SAI China composes

the final project guidance.

1. Background

2. Project history

3. Project objectives

4. Project scope

5. Project plan

6. PKI beta database

Contents

6. Framework of PKI project

11 parts, 712 indicators.

1. Decision (20 indicators) Compliance with the laws (21 indicators)

Feasibility study (5 indicators)

Participation in decision (4 indicators)

6. Framework of PKI project

2. Requirement analysis (17 indicators) Organization target (9 indicators)

Core business coverage (1 indicator)

Response/change (7 indicators)

3. Design / Planning (95 indicators) Requirement coverage (10 indicators)

Time limit (2 indicators)

Capacity Planning (18 indicators)

Cost estimation (42 indicators)

IT risks (Added sub-category, 22 indicators)

Not decided(1 indicator).

6. Framework of PKI project

4. Procurement/Development (102 indicators)

Selection for partner or supplier (12 indicators)

Cost control(16 indicators)

Process Control(12 indicators)

Code control(2 indicators)

Selection for software and hardware(1 indicator)

Outsourcing (3 indicators)

Quality Control (26 indicators)

Testing (10 indicators)

Training (19 indicators)

Upgrading (1 indicator)

6. Framework of PKI project

5. Product (41 indicators)

User satisfaction(2 indicators)

Price (3 indicators)

Delivery (5 indicators)

Performance (26 indicators)

Integration (1 indicator)

Technology applicability(4 indicators)

6. Framework of PKI project 6. Maintenance (281 indicators)

Follow the management rules (23 indicators)

Incident management (57 indicators)

Easy to be used (5 indicators)

Availability (18 indicators)

IT Continuity (9 indicators)

Maintenance cost (12 indicators)

Website (54 indicators)

Monitoring(42 indicators)

Change Management (34 indicators)

Data Center (12 indicators)

Configuration management (Added sub-category, 12 indicators)

Not decided(3 indicators)

6. Framework of PKI project

7. Security (27 indicators)

Security plan

Identity management

User account management

Security testing monitoring

Security incident definition

Malicious software prevention

Network security

6. Framework of PKI project

8. Backup and Disaster Recovery (14

indicators)

Backup plan

Recovery plan

Backup operation management

Recovery operation management

6. Framework of PKI project

9. Service (56 indicators)

Response (41 indicators)

Usage by user(0 indicator)

Ratio of performance to price (0 indicator)

Satisfaction (11 indicators)

Service request(Added sub-category, 4

indicators)

6. Framework of PKI project

10. Effectiveness (38 indicators)

Coverage of the core business (7 indicators)

Usage of the employees for the certain application(1

indicator)

Output (6 indicators)

Benefit (10 indicators)

Internal Management optimizing (6 indicators)

Leaning and growth capability for the organization (1

indicator)

Public Service(3 indicators)

Not decided(3 indicators)

11. Others (21 indicators)

Q & A?