MEASURING EFFICIENCY AT FIELD AUDIT OFFICES1

By

Muhammad Akram KhanFormer Deputy Auditor General of Pakistan

Makram1000@gmail.com

1. OBJECTIVE

The objective of this paper is to develop a methodology for measuring efficiency in Field Audit Offices (FAOs) in the Department of the Auditor General of Pakistan. Of late, the Department is under considerable pressure both from within itself and also from the various clients it serves to improve its efficiency. The legislators and executive heads of the governments are asking for the audits which are professionally competent and which also add value to the society at large. The gap between what the society expects the state auditors to perform and what the auditors think about their own role is widening with the passage of time. The auditors are complaining that they do not have adequate physical and financial resources to meet the challenge, while the clients of the department and the government think that the department is wasting its resources and is doing a ‘useless’ work with very little utility. In this scenario, where the need of more resources cannot be denied, the department also needs to do some soul searching. The fact is that the department has never tried to measure its own efficiency either at the level of individuals or at the organizational level. The department is operating like any other public sector organization with all the handicaps and built-in inadequacies. The department of the Auditor General is supposed to act as a watchdog of the other departments. With the addition of performance auditing into its role, it has also started commenting on the economy and efficiency of other departments. While it is a welcome addition to the role of the traditional role of government auditors, it has also made them more vulnerable. Those who try to evaluate the efficiency of others must not display significant inefficiencies in their own operations. That is why, the Auditor General’s department should emerge as an office of highest efficiency and excellence. For this purpose a high degree of efficiency-consciousness is necessary among the officers of the department. The present paper tries to develop a system of efficiency measurement in the Field Audit Offices. At present no such system exists. As a result, the efficiency of the offices is not measured. No one can say with any precision about the quality and efficacy of the work done by an office or its officers. In this scenario, some people are able to score a higher mileage not because of actual output but by their acumen to influence their audience. Also, those who are really hard working but cannot do enough of “marketing” for their work, seem to have not done enough. As a result, it is difficult to define merit and excellence. Such a situation promotes a value system that is conducive to cronyism and corruption. The present paper is an attempt to devise a mechanism that defines merit in as objective terms as possible.

1?* document.doc

2. THE CHALLENGE OF DEVELOPING A COMPOSIT INDEX OF EFFICIENCY

While trying to devise a suitable mechanism for an audit office, the real challenge is to develop a compost index of efficiency. In the literature on performance measurement, there is hardly anything on the efficiency of government audit offices. The usual symbols of efficiency in an audit office in Pakistan are number of audit observations, number of audit reports, recoveries affected at the instance of audit and the number of audit paragraphs printed in the audit report. Although these variables are valuable but they do not tell the entire story. For example, they do not tell anything about the cost of audit, time taken to develop the audit observations, the ratio of resources expended on financial audit and performance audit, resources spent on developing audit infrastructure for improving the effectiveness of office in the long run, etc. Thus there is a need for developing an index of efficiency that takes care of multiple aspects of efficiency. A further challenge is to reduce all the variables to one common denominator so that they can be interpreted. Another problem is to express a judgement on the importance of different variables. This requires assigning weights to different variables. By its very nature such a task is subjective. But we have still adopted this method, since having subject and measurable weights is better than not having any system of measurement. If we apply the subject weights consistently the bias is neutralized to some extent.

3. SCOPE

The present paper covers the economy and efficiency aspects and does not include effectiveness aspects. The effectiveness of a FAO has several dimensions and its measurement requires much extensive data than can be collected economically in the present scenario. It will be a subject of some future exercise.

The present paper covers the efficiency of the FAOs and not individuals working in these offices. We shall address to the question of individual efficiency in a subsequent paper.

4. PARAMETERS OF EFFICIENCY

The parameters of efficiency of a FAO are as follows:

Output Parameters

Quantitative Output Quality of Audit Developing Management Infrastructure

Input Parameters

Number of person days consumed Cost of audit operations

2

As we shall illustrate later, we shall adopt person years as the main determinant of efficiency in each case. Therefore, it will not emerge as an independent index efficiency.

We shall elaborate these parameters in the following discussion.

4.1 Quantitative Output

The quantitative output of a FAO consists of the following:

Number of audit reports printed for the PAC Work-in-progress

4.1.1 Number of Audit Reports

The number of audit reports prepared by a FAO is the foremost indicator of its performance. But since audit assignments differ in size, significance, and scope of work, it is not possible to adopt the number of audit assignments as a variable for comparing the efficiency of two FAOs. Moreover, it will generate a tendency to increase the number of audit reports on flimsy grounds. Therefore, we need to adopt a more uniform variable, keeping intact the issue of audit reports printed by a FAO. We think, the Audited/ Expenditure (AE) is a more uniform measure. It is not under the absolute control of the FAO also, although the FAOs have a limited flexibility to increase or decrease the coverage of AE. Thus we can adopt the AE of the audits completed and printed as audit reports as one variable.

4.1.2 Work-in-progress

At the close of a year, every FAO will have some work done in the pipeline. We should take that into account. Or this purpose, following factors may be adopted to convert the work-in-progress (WIP) into complete audit reports:

Audits only planned: 10% of the AE of the planned audit

Audits in progress 20 % Audits completed but AIR not issued to the auditee 35% Audits completed and AIR issued to the auditee 50% Audit Report issued to the PAO 75% Audit Report approved by the AGP 85% Audit Report sent to press 95%

Once we are able to add up the AE we can work out a ratio of AE with person days and cost of audit.

3

For performance audits we can use these percentages to add up to the number of performance audit reports instead of the AE.

4.2 Quality of Audit

4.2.1 It is very difficult to measure quality of audit directly. We have to adopt a proxy measure. One such measure could be the extent of supervision of audits. We can adopt the person-days spent on audit supervision by the Deputy Director, Directors or Directors General. Information on this will be available from their tour/ local visits program, which should be documented properly. This can be compared with the planned supervision person-days and an index can be calculated.

4.2.2 Recoveries made at the instance of audit

One important indicator of quality of audit is the amount of recoveries made at the instance of audit. This can be compared with the cost of audit. We can then say for each rupee spent on audit led to so many rupees of recoveries.

4.2.2 Savings made on recommendations of audit

Another indicator of quality of audit is to the acceptance of audit recommendations by the executive leading to savings. This can then be compared with the cost of audit. We can then say that for each rupee spent on audit led to so much saving in terms of government expenditure.

4.3 Management Infrastructure

It consists of the FAO's efforts in three areas:

Development of master files Arrangement for weeding out old records System for keeping the office manuals updated

It is difficult to measure output on this count directly. Therefore, we need to adopt a proxy measure. We think all these areas are equally important. Therefore, we can calculate an efficiency by dividing the planned person-days by actual person-days spent on these activities.

5. ANALYSIS

5.1 Number of Person-days consumed on various activities

This will be the most important input measure. It will mean adding up all the working days of the staff from B-5 to B-20 in a FAO. We should classify the entire time into various activities. For example, first we should calculate the time for regularity audit. The time planned for regularity audit will be the time net of other activities, such as

4

administration, audit of Foreign Aided Projects (FAP), Third Party Validation of SAP (TPV) and Performance Auditing.

5. 2 Cost of Audit

It is a simple concept. The total budget of a FAO is the cost of audit. But for applying this concept to efficiency measurement, we need to refine it. The FAOs carry out several types of audits, like regularity audit, special audit, audit of foreign aided projects, Third party Validation, audit of SAP expenditure, special assignments requiring detailed investigations, etc. Therefore, we need to find out a mechanism for allocating cost to various activities. We do not have detailed cost accounting system. Therefore, it is not possible to have a precise and refined allocation system of cost. In the present scenario, trying to develop a cost accounting system will be self-defeating and the whole exercise of efficiency measurement will suffer. Therefore we can adopt a proxy system of allocating cost. We can adopt the concept of Average Cost of a Person Day (ACPD). This can be determined by dividing the total budget of an office by the total number of person days for audit activities. It means we shall first calculate the total number of staff from B-5 to B-20, multiply it with the number of working days in a year and divide the total budget of the FAO by these working days. This will give us Average Cost per Person Day (ACPD). The ACPD can then be used to allocate cost to various types of audit activities according to the number of person days required by each activity.

6. HYPOTHETICAL EXAMPLE OF PERFORMANCE MEASUREMENT OF A FIELD AUDIT OFFICE

Planned Data

Actual Data

Total Budgeted of the Auditee Formations: 75000 M 78000Total number of formations 836 836Budget /Expenditure of the Formations included in the audit plan

28000 M 32000

No. of auditee formations 380 360Person days available with the FAO (B5- B-20)

32000 30500

Budget of the FAO 24 M 25MNo. of regularity audit reports 9 6Average time for one audit assignment* 24 weeks 30 weeksAE of regularity audit reports 22000 M 18000MWIP at the end of the year 6000 M 4000Performance Audit Reports 3 1Person days for regualrity audit 16000 18000Person days for performance audit 4000 2400Average time for one performance audit report*

20 weeks 28 weeks

5

Person days for SAP TPV 3200 3600Sites for validation 280 225Absenteeism cases 325 280Procurement cases 115 105Recruitment 70 65Person days for FAP 2800 2000No. of projects 45 30Person days for administration & co-ordination, etc

6000 4500

Person days for Supervision [included in Regularity Audit Time ]

3200 2800

No. of audits supervised 125 90Person days for Audit Management Infrastructure [included in Administration & Co-ordination Tim]

800 300

* It means the time taken for all stages of audit, from planning through printing of the report. This will be relevant to those reports, which have been printed. Work in progress will not be considered for this purpose.

7. ANAYLYSIS

7.1 Quantitative Outputs:

(a) AE Ratio: Actual AE/ Planned AE= 22000 / 28000 = 0.79 [Ratio 1]

(b) Cost of Regularity Audit:

Average Cost of Person day = Planned: Rs 24000,000 /32000 = Rs 750 Actual: Rs 25000,000 /30,500 = Rs 820

Total planned cost of Regularity Audit: 16000 x 750 = 12 MTotal actual cost of Regularity Audit 18000 x 820 = 14.75 MRegularity Audit Cost Ratio: 12 /14.75 = 0.81 [Ratio 2]

(c) Timeliness of Regularity Audit:Average planned time for completing one audit assignment: 24 weeksAverage actual time for completing one audit assignment: 30Average delay: 30 - 24 = 6 weeksEfficiency in finalizing the audit assignment [ 1- (6 / 24 ) ] = 0.75 [Ratio 3]

(d) Overall Efficiency Index in Regularity Audit: 0.81 x 0.79 x 0.75 = 0.48 [Index 1]

(e) AE person day ratio:

6

AE planned per person day: 28000 / 16000 = 1.75 M per person dayAE actual per person day: 22000/ 18000 = 1.22 M per person day

AE Person day Efficiency Ratio: 1.22 /1.75 = 0.70 [Ratio 4]

(f) Cost of Performance Audit

Total planned cost of performance audits: 4000 x 750 =Rs 3000,000Total actual cost of performance audit: 2400 x 820 = Rs 1968,000Achievement in terms of reports: 1/ 3 = 0.33Average planned cost of producing one performance audit report: 3000,000 /3 = 1000,000Average actual cost of producing one performance audit report: 19680000Performance audit cost ratio: 1000,000/ 1968000 =0.51 [Ratio 5]

(g) Timeliness of performance audit:

Average planned time for finalizing a P.A. = 20 weeksAverage actual time for completing a P.A = 28 weeksAverage delay = 8 weeksEfficiency in finalizing a P.A. = [ 1 - (8 /20 ) ] = 0.60 [Ratio 6 ]

(f) Overall Efficiency Index in Performance audit: 0.33x 0.51 x 0.60 = 0.10 [Index 2]

(g) Social Action Program Third Party Validation Work

Site verification ratio: 225/280 = 0.80Absenteeism ratio: 280/325 = 0.86Procurement cases: 105/115 = 0.91Recruitment cases: 65/70 = 0.93Overall achievement in SAP TPV Work: (0.81+0.86+0.91+0.93) /4 = 0.88 [Ratio 7]

(h) SAP Cost ratio: (3200 x750) / (3600 x820) =0.81 [Ratio 8]

(i) Overall Efficiency Index in SAP= 0.88 x 0.81 = 0.71 [Index 3]

(j) FAP Efficiency Formations audited: 30Formations planned for audit3=45 Achievement rate: 30/45 = 0.66 [Ratio 9]

(k) Cost Efficiency in FAP Work: (2000 x750)/(2800 x820) =0.65 [Ratio 10]

(l) Overall Efficiency Index in FAP = 0.66 x0.65 = 0.43 [Index 4]

7

(m) Contribution of Audit

1. Recoveries: Cost ratio

This ratio measures the relationship between recoveries made at the instance of audit and the cost of audit. It will be worked out as follows:

Recovery: Cost Ratio = 1 - (Total Cost of Regularity Audit/ Total Recoveries at the Instance of Audit), where the amount of recoveries is larger than the cost of audit. Where the recoveries are less that the cost of audit, this ratio will be ignored.

2. Savings: Cost ratio

This ratio measures any savings caused by accepting the advice of audit. This will usually be the case in respect of performance audit reports. Therefore, the cost figure in this ratio will be the cost of performance audit. although it does not debar us from adopting the figure of cost of regularity audit, if the recommendation was made by the regularity audit and accepted by the auditees.

Savings: Cost ratio = 1 - (Total Cost of Performance Audit/ Total Savings on the Recommendation of Audit), where the amount of savings is larger than the cost of audit. Where the savings are less that the cost of audit, this ratio will be ignored.

7. 2 Quality of Audit

(a) Supervision -Person days Ratio: 2800/3200 = 0.88 [Ratio 11]

(b) Supervision Coverage ratio: 90/125 = 0.72 [Ratio 12]

(c) Overall Supervision Efficiency Index: 0.88 x 0.72 = 0.63 [Index 5]

6.3 Audit Management Infrastructure

(a) Infrastructure Achievement ratio: 300/ 800 = 0.38 [Ratio 13]

8. OVERALL ASSESSMENT

We have calculated 13 ratios. By using these ratios we have worked out 5 efficiency indices. But since all activities are not of equal importance, we think, we should assign some weights to various efficiency indices. In the following table column 3 has a set of suggested weights. They are purely for illustrative purposes.

Particulars Efficiency Index Weight Weighted Index

Regularity Audit 0.48 0.40 0.19

8

Performance audit 0.10 0.20 0.02SAP TPV Work 0.71 0.15 0.11FAP work 0.43 0.10 0.04Supervision 0.63 0.10 0.06Management Infrastructure

0.38 0.05 0.02

Total 0.44

Overall Efficiency of the FAO = 44%____________________________________

The conclusion is that if we have arrangements for collecting necessary data on planned and actual achievements of a FAO we are able to determine its efficiency, howsoever crude it may be. It will still be better than mere conjectural assertions.

9. APPLYING THE METHODOLOGY IN THE FIELD AUDIT OFICES

The next question is: how do we apply this methodology in the FAOs. It requires collection and validation of a large amount of data. As a practical measure, the FAOs should be informed in the beginning of the year about the parameters on which they would be evaluated. They should then be asked to maintain necessary record for collecting information on various variables. At the year-end, the FAOs should summarize the information for each variable and submit it to the Auditor General’s office. The Deputy Auditor General concerned should then visit each FAO and ask it to produce evidence in support of the information submitted by the office. The task of the Deputy Auditor General is that of an auditor. He has to verify most of the information presented by the office with reference to the original documents maintained by it. This is a detailed exercise and can take more than a day. At the end of the examination, the Deputy Auditor General should tick off all valid information and engage one of his assistant to work out various ratios. For the facility of the FAOs, we have provided specimens of the forms required for submitting summary to the Auditor General’s.

Once the Deputy Auditor General completes his round of all FAOs and prepares the analysis of various ratios, it would be possible to present the results in the form of a spreadsheet and place different offices in comparison to one another. This spreadsheet can, then, also become basis for any incentives, rewards or prizes to be given by the Auditor General for better performing offices.

It is important to add here that the forms annexed to this paper are only a summary information. The information in these forms cannot be provided until every FAO has a detailed information, which is kept updated on a daily basis. For this purpose, each FAO must set aside human and physical resources. Preferably, the FAOs should create a small cell responsible for monitoring of the plan. This cell should maintain detail information, either manually or on computer, which should be summarized for the office of Auditor General on the forms annexed with paper.

9

10. CONCLUDING REMARKS

The performance measurement system proposed in this paper should be taken with a grain of salt. The performance of an organization or a person is determined by a host of factors. Some of these factors , such as will to work, proper organization of assignments, discipline and sense of urgency , are in the control of a person or an organization. But some of these factors are beyond one’s control. Examples are: lack of financial and human resources, poor incentive system, and inadequate salary structure, lack of appropriate training facilities, political interference and human temperaments, etc. Although performance measurement is a current rage in the administration theory, yet not much attention has been given to the impact of these controllable and non-controllable factors. The moral of the story is that it is very easy to draw conclusion about the efficiency of an organization but rather difficult to understand and analyze reasons for the particular level of efficiency. It is imperative, therefore, that once the above analysis is made, the Deputy Auditor General concerned should hold a detailed session with the field audit office to analyze the level of efficiency achieved by that office. The ultimate decision regarding the performance of an office should be seen in the light of the review session.

10

Evaluation-1

EVALUATION OF FIELD AUDIT OFFICES

GENERAL INFORMATION

NAME OF THE AUDIT OFFICE

------------------------------------------------

YEAR: October 19----September 19----

Particulars Planned2 Actual RemarksTotal No. of Assignments of Regularity Audit3

Total No. of Formations Total Expenditure/Income audited4

Total No. of Available person-days (B-5 to B-20)

*5

Total Budget/Expenditure of the FAONo. of Regularity Audit Reports printed by the FAOPerson-days Spent on Regularity Audits (All Assignments)6

Person-days spent

2 It means the data provided in the original approved plan. Please ignore all subsequent revisions for the purpose of evaluation of your office.3 Also include any special assignments relating to regularity audit4 It means the total of expenditure or income audited in case of government organizations. It does not mean the total of the sample. In case of organization which keep accounts on double entry basis and prepare a balance sheet and an income statement, it means total of all assets plus total of all expenses. Include any special assignments relating to regularity audit.5 Please account for any transfers without replacements, leave by the staff and officers as actually availed or posting in of any personnel.

11

on Performance Audit7

No. of P.A. AssignmentsPerson-days spent on SAP (TPV)8

Person-days spent on SAP (FA)9

Person-days spent on FAPSites for validation under SAP TPVAbsenteeism Cases for Validation under SAP TPVProcurement cases for validation under SAP (TPV)Recruitment cases for validation under SAP TPVNo. of Projects under FAPPerson-days spent on Supervision10

No. of Audits supervised 11

Person-days for Administration & Co-ordination workPerson-days for master filesPerson days for weeding out old recordPerson days for updating office manuals

6 Fill in Form Evaluation-2. The time spent on Regularity Audit will include all time spent on various stages of Regularity Audit from planning to printing of the reports. It will also include time in the processing of the reports, DACs, Pre-PACs, PACs, and inter-departmental meetings, etc.

12

Recoveries made at the instance of audit12 Savings affected on the advice of audit13

13

Evaluation-2

EVALUATION OF FIELD AUDIT OFFICES

DATA ON ASSIGNMENTS COMPLETED14

POSITION AS ON 30.9.19----

Title/No. of Assignment Planned ActualPerson-

daysExpenditure/Income Audited

Calendar

days15

Person-

days

Expenditure/Income Audite

d

Calendar days

Total

14

7 It will include time spent on all stages of P.A. Include any special assignments pertaining to performance audit.8 It will include time spent on all stages of TPV.9 It will include time spent on all stages of SAP (FA).10 It refers to the supervision in the field by officers like Deputy Directors/ Directors/ Directors General. This is already included in various categories of person-days enumerated above. The person-days under regularity audit stated above will include the time for supervision. At this stage, we mention it as a part of the total mentioned above.11 It refers to the number of formations where the supervisory officer went. 12 Recoveries made during the period under report, even though were pointed out in earlier audit reports. Please give only actual figures. This cannot be planned.13 Please give actual figures only. There cannot be any planned figure. Also support it with evidence.14 Assignment includes all types of assignments such as regularity audit, performance audit, SAP TPV, special assignment. Please mention them in blocks of information under proper headings. Classify any special assignment into regularity audit, performance audit and others as the case may be.15 It means the time from planning of audit assignment to printing of the audit report.

15

Evaluation-3

EVALUATION OF FIELD AUDIT OFFICES

DATA ON AUDIT ASSIGNMENTS IN PROGRESS

AS ON 30.9.19----

Title of Assignment Total Expenditure Audited

Stage of Work16

Stages of Work;

1. Audits only planned2. Field work in progress3. Audit completed but Audit Inspection Report not yet issued4. Audit completed and Audit Inspection Report issued to the auditee5. Audit Report issued to the principal accounting Officer6. Audit Report approved by the Auditor General7. Audit Report sent to press for printing

16 Please indicate the stage no. from the list given below.

16