world bank document · 6/21/2016 · menpan-rb ministry of state apparatus and bureaucratic reform...

Document of The World Bank

Report No: ICR00003795

INTENSIVE LEARNING IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD 4762 – IND/IDA 4026 – PHRD TF053556)

ON A

LOAN & CREDIT IN THE AMOUNT OF US$65 MILLION (IBRD 4762: US$55 million; IND/IDA 4026: SDR 3.14 million (equivalent to US$5 million)

IDA credit; PHRD: US$5 million)

TO THE

REPUBLIC OF INDONESIA

FOR

GOVERNMENT FINANCIAL MANAGEMENT AND REVENUE ADMINISTRATION PROJECT (GFMRAP)

June 21, 2016

Governance Global Practice East Asia and Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective as of April 15, 2016)

Currency Unit = Rupiah (IDR) US$1.00 = Rp13,500

FISCAL YEAR

January 1 – December 31

Exchange Rate at Appraisal (November 19, 2004): US$1 = Rp8,997 Exchange Rate at First Restructuring (June 5, 2009): US$1 = Rp9,961

Exchange Rate at Second Restructuring (December 19, 2013): US$1 = Rp12,116

Vice President: Victoria Kwakwa Country Director: Rodrigo A. Chaves Practice Manager: Robert R. Taliercio

Task Team Leader: C. Bernard Myers ICR Co-authors: Hannah Kim, Lina Lo, and

Carlos D. C. Ferreira

ii

Abbreviations and Acronyms

ADB Asian Development Bank APL Adaptable Program Loan AusAid Australia Agency for International Development Bappenas National Development and Planning Agency BER Bid Evaluation Report BI Bank Indonesia BPK Supreme Audit Agency BPKP Finance & Development Supervisory Agency CAS Country Assistance Strategy CIDA Canadian International Development Agency COTS Commercial-Off-The-Shelf CPF Country Partnership Framework CPS Country Partnership Strategy CTF Child Trust Fund DIPA Approved budget allotment document DO Development Objective DPR` House of Representatives or Parliament DG Director General DGB Directorate General of Budget DGT Directorate General of Tax DGCE Directorate General of Customs and Excise EIRR Economic Internal Rate of Return EPP Economic Policy Package ERP Enterprise Resource Planning FMIS Financial Management Information System FOA Final Operational Acceptance FPO Fiscal Policy Office FY Fiscal Year GAAP Government and Accountability Action Plan GDP Gross Domestic Product GFS Government Financial Statistics GFMRAP Government Financial Management and Revenue Administration Project GGP Governance Global Practice GOI Government of Indonesia HR Human Resources IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results ICT Information and Communication Technology IDA International Development Association IEG Independence Evaluation Group IG MOF Inspector General of the Ministry of Finance IMF International Monetary Fund IP Implementation Progress ISR Implementation Status and Results Report ITB Invitation to Bid IT Information Technology IV&V Independent Verification and Validation JBIC Japan Bank for International Cooperation JICA Japan International Cooperation Agency

iii

Kanwil State Treasury Regional Office KSAP Public Sector Accounting Committee KPPN State Treasury Local Services Office or Local Treasury Branches (LTB) LKPP National Public Procurement Agency MDTF Multi Donor Trust Funds M&E Monitoring & Evaluation Menpan-RB Ministry of State Apparatus and Bureaucratic Reform MOF Ministry of Finance MPN State Revenue Module MIS Management Information System MTEF Medium Term Expenditure Framework NOL No Objection Letter NPPA National Public Procurement Agency (LKPP) NPV Net Present Value OM SPAN On Line Monitoring SPAN PAD Project Appraisal Document PBB Performance Based Budgeting PCN Project Concept Notes PDO Project Development Objective PEFA Public Expenditure and Financial Accountability PFM Public Finance Management PHRD Policy and Human Resources Development PIU Project Implementation Unit PINTAR Project for Indonesian Tax Administration Reform PMQA Project Management Quality Assurance PPF Project Preparation Facility PSC Project Steering Committee PSSU Project Services and Support Unit QAG Quality Assurance Group QEA Quality at Entry QER Quality Enhancement Review QSA Quality of Supervision RF Result Framework RPJMN Medium Term Development Plan RTM Requirement Traceability Matrix SAKTI Spending Unit Financial and Accounting Application Secgen Secretary General SPAN State Treasury and Budget System SPIRIT Scholarships Program for Strengthening Reforming Institutions Project for Indonesia SIL Specific Investment Loan SU Spending Units TA Technical Assistance TF Trust Fund TOR Terms of Reference TSA Treasury Single Account USAID United States Agency for International Development

iv

INDONESIA

GOVERNMENT FINANCIAL MANAGEMENT AND REVENUE ADMINISTRATION PROJECT (GFMRAP)

CONTENTS

Abbreviations and Acronyms ............................................................................................ iii Data Sheet .......................................................................................................................... vi Intensive Learning ICR Abstract ..................................................................................... xvi 1. Program Context, Development Objectives and Design ............................................ 1

2. Implementation and Outcomes ................................................................................... 6

4. Assessment of Risk to Development Outcome ......................................................... 21

5. Assessment of Bank and Borrower Performance ..................................................... 22

6. Lessons Learned ....................................................................................................... 29

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 33

Annex 1. Project Costs and Financing .............................................................................. 34

Annex 2. Outputs and Activities by Components ............................................................. 36

Annex 3. Assessment of Achievements by Objective ...................................................... 45

Annex 4. Economic and Financial Analysis ..................................................................... 51

Annex 5. Bank Lending and Implementation Support/Supervision Processes ................ 54

Annex 6. Beneficiary Survey Results ............................................................................... 56

Annex 7. Inputs from Consultations with Stakeholders ................................................... 58

Annex 8. Borrower’s Completion Report and/or Comments on Draft ICR ..................... 61

Annex 9. Comments of Co-financiers and Other Partners/Stakeholders .......................... 75

Annex 10. List of Supporting Documents ........................................................................ 76

Annex 11. Online Monitoring SPAN Module .................................................................. 77

Annex 12. Map of Indonesia ............................................................................................. 78

v

Data Sheet A. Basic Information

Country: Indonesia Project Name:

Government Financial Management and Revenue Administration Project

Project ID: P085133 L/C/TF Number(s): IBRD-47620,IDA-40260,TF-53556,TF-90047,TF-91414

ICR Date: 06/06/2016 ICR Type: Intensive Learning ICR

Lending Instrument: APL Borrower: REPUBLIC OF INDONESIA

Original Total Commitment:

USD 60.00M Disbursed Amount: USD 59.97M

Revised Amount: USD 60.00M Environmental Category: C Implementing Agencies: Ministry of Finance Cofinanciers and Other External Partners: Government of Japan - Ministry of Finance DFATD European Union Government of the Netherlands SECO USAID B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 11/18/2003 Effectiveness: 10/27/2005 10/27/2005

Appraisal: 09/14/2004 Restructuring(s): 06/05/2009 12/20/2013

Approval: 12/21/2004 Mid-term Review: 11/30/2012 02/01/2013 Closing: 06/30/2009 12/31/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory

vi

Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments (if

any) Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 93 97 Law and justice 7 3

Theme Code (as % of total Bank financing) Judicial and other dispute resolution mechanisms 14 3 Other public sector governance 29 1 Public expenditure, financial management and procurement

29 96

Tax policy and administration 14 Trade facilitation and market access 14 E. Bank Staff

Positions At ICR At Approval Vice President: Victoria Kwakwa Jemal-ud-din Kassum Country Director: Rodrigo A. Chaves Andrew D. Steer Practice Manager/Manager:

Robert R. Taliercio M. Helen Sutch

vii

Project Team Leader: C. Bernard Myers Amitabha Mukherjee ICR Team Leader: C. Bernard Myers ICR Primary Author: Hannah Kim Lina Lo Carlos D. C. Ferreira F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To strengthen efficiency, governance and transparency in public financial management and revenue administration. It will provide medium-term support for two of the three pillars of the Economic Policy Package (EPP), namely (i) macroeconomic stabilization, and (ii) increasing investment, exports, and employment. Revised Project Development Objectives (as approved by original approving authority) To improve efficiency, governance, integrity and transparency in public financial management. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : National government policy priorities are reflected in a Medium Term Expenditure Framework and annual budgets

Value quantitative or Qualitative)

No MTEF. An elaborate planning process was in place but the link between planning and budgeting was weak.

Mostly Achieved. MOF issued its regulation No. 143/2015, which will "obligate" line ministries to submit their3-year forward estimates along with the annual budget proposals.

Date achieved 06/01/2004 12/14/2015 Comments (incl. % achievement)

This new regulation will increase the compliance of the line ministries in submitting MTEF information to be included in the financial note documents.

Indicator 2 : Reduced leakage in expenditure flows to end-users as measured by Public Expenditure Tracking Surveys

Value quantitative or N/A Mostly Achieved.

TSA is now

viii

Qualitative) operational in all MOF Treasury offices, with all state receipts deposited in and all state expenditures withdrawn from the TSA, through electronic transfer to ensure reduction of leakage in expenditure flows to end-users.

Date achieved 12/16/2015 Comments (incl. % achievement)

Indicator 3 : Automated treasury payment system enables accurate and timely financial reporting, reduces incidence and size of idle cash balances, and reduces corruption in payments


No automated treasury payment system

Achieved. Rollout of SPAN was completed by February 2015 to all 222 locations across Indonesia. SPAN now manages100% of all financial transactions of over 24,000 government spending units.


This indicator represents the core of the GFMRAP operation. The completion of SPAN signifies a major achievement, underscoring the difficulties and complexities of this type of interventions.

Indicator 4 : Improved customs revenue performance and time-for-release performance Value quantitative or Qualitative)

N/A N/A

Date achieved Comments (incl. % achievement)

Per the first project restructuring effective Jun 30, 2009, the revenue administration related component was dropped. Disbursement under this component was negligible with less than 0.1% of the project amount.

Indicator 5 :

Demonstrable evidence of improved performance of tax and customs collection and greater transparency in implementation of tax and customs regulation, as indicated by increased ratio of revenues to GDP, increase in registered tax payers and filer, reduced

ix


N/A

This tax administration sub-component of the GFMRAP operation spun off as a SIL called PINTAR in 2009.


Per the first project restructuring effective Jun 30, 2009, the revenue administration related component was dropped. Disbursement under this component was negligible with less than 0.1% of the project amount.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Increase policy development capacity within the Fiscal Policy Office as verified by the completion of several high-quality policy analysis products

Value (quantitative or Qualitative)

0 4

Achieved. 37 officers trained from FPO: (a) 12 Masters degrees completed; (b) 7 PhD degrees completed; (c) 18specialized training.

Date achieved 02/12/2005 12/31/2015 01/02/2012

Comments (incl. % achievement)

Indicator reformulated during the first project restructuring. Scholarship programs have contributed a lot in the policy development capacity within FPO. Some staff who completed the scholarship programs were promoted to produce high quality analysis.

Indicator 2 : First Medium Term Expenditure Framework endorsed by the Government of Indonesia and forward estimates presented to the DPR


No MTEF Mostly Achieved. MTEF operational


This indicator was dropped during the first restructuring. MTEF was introduced in 2011. Further fine-tuning of the implementation of MTEF will be needed for decision making purpose.

Indicator 3 : SPAN Stage I evaluated with user input with respect to content, timeliness and verifiability; specified periodic financial reports prepared for concerned entities


Treasury system did not enable accurate and timely financial reporting

SPAN fully functional Achieved. SPAN

fully functional

x

and could not be used for reducing incidence and size of idle cash balances, and reduce corruption in payments

accordance with ITB specifications

accordance with ITB specifications


SPAN now manages 100% of all financial transactions of over 24,000 government spending units across Indonesia.2015 will be the 1st year of using SPAN and the 2015 financial reports are being audited with results in mid 2016.

Indicator 4 : TSA system functioning with acceptable parameters: all on-budget central government bank accounts and own-revenue bank accounts of line ministries are brought into the TSA system


A baseline of greater than 18,000 government cash operations with no cash management standards

Mostly Achieved. TSA has been fully implemented since FY 2009


While significant progress has been made, there remains challenges: optimizing idle cash, cash forecast, electronic money products, and link between cash and debt management.

Indicator 5 :

Central Government financial statements meet Government Accounting Standards through Regulation and the consolidated management reports produce by the DG Treasury include financial assets, liabilities, cash-based reports ofextra-budgetary funds


cash-based accounting

Mostly Achieved. full accrual-based accounting being implemented in 2015


Preparation for accrual accounting started in 2010. MOF implemented full accrual accounting in 2015 that will be audited in mid-2016.

Indicator 6 : An evaluation of the MOF e-procurement pilot is completed and an action plan is adopted by the MOF for wider rollout


0% of MOF procurement through e-procurement system

Partially Achieved. An e-procurement system, which was developed by LKPP for mandatory use by all government agencies, is being implemented for MOF. An Echelon 2 unit responsible for e-procurement was established under

xi

the MOF Secretariat General.


This indicator was dropped during the first restructuring. After restructuring, support to this subcomponent with LKPP as the implementing agency has been under the PFM MDTF.

Indicator 7 : A survey of procurement prices relative to international benchmarks is completed Value (quantitative or Qualitative)

N/A N/A N/A


This indicator was dropped during the first restructuring.

Indicator 8 : The DGCE has adopted a comprehensive and consultatively prepared modernization action plan with inputs from stakeholders


N/A N/A N/A



Indicator 9 : The DGCE has institutionalized an annual survey of stakeholders, and annually published a summary of the results


N/A N/A N/A



Indicator 10 : The DGCE has implemented Web-based filing for about 75 percent of trade transactions


N/A N/A N/A



Indicator 11 : The DG Tax has adopted a Results-oriented Modernization Strategy and Plan towards a redesign of single tax payer account lines based on the results of the pilot re-engineering of one rep district office


N/A A roadmap was developed and DG Tax is geared for the

xii

next wave of reforms


This indicator was dropped during the first restructuring. This subcomponent to support DG Tax spun off as a SIL operation named PINTAR that came on line in 2009.

Indicator 12 : Verification of strengthened basis for ICT development in the DPR


N/A

Not Achieved. ICT related consultancies never took off


This indicator was modified as the result of the first restructuring and was eventually dropped during the second restructuring.

Indicator 13 : Annual surveys of stakeholder satisfaction with Tax Court administration and case management established; an action plan has been adopted to address systems weaknesses


Est. user satisfaction > 10% as no formal surveys, no website or case management and tracking system etc.

Partially Achieved. A first survey on user satisfaction with the Tax Court administration and case management system was completed in December 2009. 68%indicated satisfaction.


This indicator was dropped during the second restructuring in 2013.

Indicator 14 : Tax case decisions are accessible to the public through the Tax Court website


Est. user satisfaction > 10% as no formal surveys, no website or case management and tracking system etc

Achieved. The Tax Court started to digitize its decisions with about 4,700 decisions available on line in 2010


This indicator was dropped during the second project restructuring. The Tax Court is currently thinking of developing a web-based application for managing and accessing database of Tax Court decisions.

Indicator 15 : Verification of effective project governance, external oversight, change management and implementation

Value (quantitative

No GAAP, PSC or investigation unit Achieved. Project

governance and

xiii

or Qualitative) activities are consistent with the GAAP; Project procurement has been consistent with procurement plan; PSC and PSSU and the investigation unit are all operational and functioning well.


As of November 2008, an investigation unit in IG-MOF was established and fully operational.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 06/27/2005 Satisfactory Satisfactory 0.00 2 06/01/2006 Satisfactory Satisfactory 4.16 3 12/11/2006 Moderately Satisfactory Moderately Satisfactory 4.86 4 08/09/2007 Moderately Satisfactory Moderately Satisfactory 4.86 5 05/10/2008 Moderately Satisfactory Moderately Unsatisfactory 4.86 6 02/18/2009 Satisfactory Moderately Satisfactory 4.86 7 03/02/2009 Satisfactory Moderately Unsatisfactory 4.86 8 03/24/2010 Satisfactory Moderately Satisfactory 11.07 9 07/30/2010 Moderately Satisfactory Moderately Satisfactory 11.07

10 10/05/2011 Moderately Satisfactory Moderately Satisfactory 25.30 11 04/04/2012 Moderately Satisfactory Moderately Satisfactory 25.30 12 11/19/2012 Moderately Satisfactory Moderately Satisfactory 25.30 13 06/03/2013 Moderately Satisfactory Moderately Satisfactory 26.19 14 12/14/2013 Satisfactory Satisfactory 26.19 15 06/30/2014 Moderately Satisfactory Moderately Satisfactory 46.19 16 01/13/2015 Moderately Satisfactory Moderately Satisfactory 51.06 17 07/30/2015 Moderately Satisfactory Satisfactory 59.26 18 12/30/2015 Satisfactory Satisfactory 59.97

H. Restructuring (if any)

xiv

Restructuring Date(s)

Board Approved PDO

Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD

millions

Reason for Restructuring & Key Changes Made DO IP

06/05/2009 Y S MU 4.86 To streamline scope 12/20/2013 N S S 26.19 to extend project closing date

If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Highly Unsatisfactory Against Formally Revised PDO/Targets Moderately Satisfactory Overall (weighted) rating Moderately Satisfactory I. Disbursement Profile

xv

Intensive Learning ICR Abstract GFMRAP has been instrumental in improving the PFM landscape in Indonesia by providing the foundation for reforms in budget planning, budget execution, reporting, accounting, tax administration, and e-procurement at a critical time in its development history, when economic growth lingered below 4 percent, poverty reduction remained a challenge, and governance concerns continued to overshadow its achievements. GFMRAP provided the necessary long-term financing platform and convening power necessary to lock in important PFM reforms. The core achievement of GFMRAP was the deployment of an FMIS system as the primary tool to reduce informality, curb opportunities for corruption, slash transaction costs, and optimize management of public resources at the center of government. Government banking was consolidated through a Treasury Single Account and the roll out of SPAN, the FMIS system, helped bring financial discipline and transparency to 182 local treasury offices, 33 regional treasury offices, and 8,000 Treasury staff. GFMRAP also generated substantial efficiency gains, including $300 million in interest savings from TSA, productivity savings from reduced printing and internal communication costs, direct deposits of wages for 1.62 million central government civil servants, improved predictability of budget execution, and reduced payment errors to unintended recipients. SPAN triggered the introduction of an electronic state revenue management system, an online monitoring and reports generation tool, and a feeder module to link expenditure data from all 24,000 spending units. These applications are anchored around SPAN and signal ownership and sustainability of GFMRAP. Daily cash balances and comprehensive transaction data of central government treasury offices are captured on a real time basis and used for informed decision making, including by President Joko Widodo, showing a strong sense of ownership beyond the Directorate General of Treasury. However, implementation of GFMRAP presented significant challenges to the Government and to the Bank. While the project’s broad scope responded to the Government’s need to lock in major PFM reforms at the time of project preparation, it took time for the reforms to take place. Pockets of resistance were identified upfront but building consensus among a large group of stakeholders took time. Challenges remain to ensure sustainability of the investment made and to promote further use of SPAN, including the creation of cadre of knowledgeable ICT staff to continue to support the maintenance, operational support, and system enhancements needed to ensure other modules can be fully integrated. That said, the broad-based reforms envisioned in the original 12-year APL design yielded benefits mostly on the expenditure side of public financial management by helping modernize the core Treasury functions. Moreover, the areas of reform that GFMRAP first identified—budget formulation, tax administration, customs, support to Parliament, training of auditors, and e-procurement—while pursued through different funding mechanisms, still remain relevant to this day. As a result, the overall project outcome is Moderately Satisfactory. This overall outcome rating is based on substantial relevance, efficacy, and efficiency achieved after the first restructuring in 2009. The original results framework was reorganized in Annex 3 in order to include unanticipated results and to strengthen the results chain supporting the theory of change underlying the GFMRAP operation. The real economic internal rate of return over a 15-year period is 50 percent, and the project has a positive economic net present value of US$632 million at a 15 percent discount rate. Performance of both the Bank and the Borrower was moderately satisfactory. In this

xvi

Intensive Learning ICR, the team derived lessons learned that may be applicable to other PFM operations in the Governance Global Practice.

xvii

1. Program Context, Development Objectives and Design 1. The theory of change underlying the Government Financial Management and Revenue Administration Project (GFMRAP) was that improved expenditure and revenue management would make a substantial contribution to sound public financial management (PFM), which in turn would contribute to better governance and increase the potential for economic growth and poverty reduction. 2. The revenue side of GFMRAP was dropped halfway through the project cycle with the tax administration subcomponent, which was originally envisioned as the third phase of the APL, converted into the Project for Indonesian Tax Administration Reform (PINTAR) Specific Investment Loan (SIL) operation. Implementation of GFMRAP continued with a focus on the expenditure side. Within this reduced scope, GFMRAP’s key achievements on public expenditure were: (a) implementation of new budget classification system and chart of accounts conforming to international standards; (b) design and successful rollout of an automated system (SPAN) for budgeting, payment processing, accounting, and reporting functions of the central treasury offices serving 24,000 spending units across the country; and (c) consolidation of Government banking through a Treasury Single Account. 3. This section of the Intensive Learning ICR provides a synopsis of the overall context, development objectives and project design. An overview of the project costs and financing is given in Annex 1, whereas outputs achieved by components is given in Annex 2 with assessment of achievements by objectives in Annex 3. Annexes 6 and 7 provide insights from beneficiary surveys on the functionalities and use of SPAN and stakeholder consultations on how key reforms on public expenditure took place. List of supporting documents on file is given in Annex 10. 1.1 Context at Appraisal 4. At the time of project preparation in 2003, Indonesia was continuing its transition from an autocratic, centralized state, to a democratic, decentralized one. While it successfully regained macroeconomic and political stability following the 1998 Asian Monetary Crisis, economic growth remained below 4 percent, poverty reduction remained a challenge, and governance concerns continued to cloud its achievements. 5. The Government had recognized that weak governance was at the heart of Indonesia’s poverty and development challenges and future prospects were clouded by perceptions of corruption in financial management practices. Knowing that the manual, paper-based systems created high risk of informality and discretion, a reform-minded Minister of Finance capitalized on reforms to the public finance law to pursue a long-term partnership with the World Bank to finance reform and to sustain reform across future changes of government. They launched actions to address critical issues. The Ministry of Finance was reorganized so that better focus could be brought to bear on core standard functions to ensure fiscal discipline, efficient resource allocation and reliable fiscal reporting. 6. The Minister of Finance was well aware of potential obstacles and resistance to a broad-based PFM reform agenda and put forth the concept of creating efficient nuclei and islands of integrity to roll out reforms in stages. Those units more ready for reform would proceed first and

1

serve as role models for the others. It was envisioned by senior officials that a World Bank-financed project would provide the basis for the eventual expansion of governance, transparency and accountability throughout the government, starting with the Ministry of Finance. Accordingly, the World Bank was requested to support the government to “lock in” and implement its long-term PFM reform agenda in October 2003. 1.2 Program Development Objectives, Key Indicators (as approved), and Beneficiaries 7. GFMRAP’s Project Development Objective (PDO) was to improve efficiency, governance, integrity and transparency in public financial management and the revenue administration.1 Key indicators were:

(a) National government policy priorities are reflected in a Medium Term Expenditure Framework and annual budgets

(b) Reduced leakage in expenditure flows to end-users as measured by Public Expenditure Tracking Surveys

(c) Automated treasury payment system enables accurate and timely financial reporting, reduces incidence and size of idle cash balances, and reduces corruption in payments

(d) Improved customs revenue performance and time-for-release performance (e) Demonstrable evidence of improved performance of tax and customs collection and greater

transparency in implementation of tax and customs regulation, as indicated by increased ratio of revenues to GDP, increase in registered tax payers and filers, reduced tax arrears, and reduced clearance time in customs

Revised objectives (as approved by original approving authority) and Key Indicators, and Reasons/Justifications: 8. The PDO was revised to the following in 2009: to improve efficiency, governance, integrity and transparency in the Borrower’s public financial management. Accordingly, only the three PDO indicators related to expenditure management remained (7.a-7.c), while the two related to revenue administration were dropped (7.d-7.e).

1 Slight inconsistencies were identified in the wording of the PDO within and across documents. For the purpose of this ICR, we have adopted the project objective as stated in Schedule 2 of the Loan Agreement because the sub-objectives—efficiency, governance, integrity, and transparency—remain the same throughout the subsequent restructuring exercises. Below are three other versions of the PDO.

• Loan Agreement second paragraph (para. (A): “The objective of the Project is to implement policies and reforms to strengthen efficiency, governance, and accountability in public financial management and revenue administration.”

• Main text of the PAD, section B.3: “The objective of GFMRAP-I is to strengthen efficiency, governance and transparency in public financial management and revenue administration.”

• Annex 3 of the PAD, Results Framework: “Strengthened effectiveness, efficiency, accountability and transparency in public financial management and revenue administration.”

2

Primary beneficiaries of the project: • Directorate General of Treasury and Treasury offices at the local and regional level. Processes

were improved to enhance efficiency for budget preparation, execution, reporting, and accounting.

• Ministries, line departments and agencies at the central government level across the country in 24,000 spending units receiving support in budget preparation, execution, accounting, reporting, and auditing.

• Central government civil servants, suppliers, cash transfer recipients who benefit from improvements in PFM through enhanced efficiency and transparency in the payment process.

• Citizens, elected representatives, civil society, and development partners who can access timely budget execution information made public on the MOF website.

Secondary beneficiaries of the project: • Directorate General of Budget in terms of enhanced budget preparation with performance

targets and annual ceilings imposed to spending units fully integrated with SPAN. • Fiscal Policy Office through capacity building of staff who benefited by receiving updated

training facilities, formal degree scholarships, and programs of internationally accepted professional standards.

• Inspectorate General through the establishment of a special investigations unit to handle corruption allegations involving MOF staff.

• Tax Court through the design of a case management system and related training. • Bappenas through improving the transparency of the Medium Term Expenditure Framework

(MTEF) • Office of the Secretariat General of the Parliament (DPR) through support with improved

capacity for budget analysis and oversight. 1.3 Original and Revised Components (as approved) 9. GFMRAP underwent two restructuring processes. At the time of the first, a Level 1, restructuring in June 2009, component activities were revised and the component related to revenue administration was dropped. At the time of the second restructuring the component related to governance and accountability was dropped. Table 1 below summaries the original and revised components:

Table 1: Original and Revised Components2 Intermediate Indicators (II) Action per 1st Restructuring Action per 2nd Restructuring Component A: Public Financial Management A.1: Policy Capacity Development Policy capacity development (particularly in tax policy and trade/tariff policy) within FPO II-1: Survey feedback (including from Minister) verifies the completion of at least four high-quality policy analysis products in the priority areas (tax policy, trade policy), published on the web site (silent in the loan agreement)

Modified Increase policy development capacity within the Fiscal Policy Office as verified by the completion of several high-quality policy analysis products

Unchanged

2 There are two parts to the numbering format of these intermediate indicators separated by a hyphen. The first part II signifies Intermediate Indicator, and the second part is the indicator number.

3

Intermediate Indicators (II) Action per 1st Restructuring Action per 2nd Restructuring A.2: Budget Planning and Development Strengthened budget planning and development (closer links between policy, planning and budgeting) through reformed budget process and unified budget preparation, signifying implementation of law and regulations on state finances II-2: First Medium Term Expenditure Framework endorsed by the Government of Indonesia and forward estimates presented to the DPR

Dropped This sub-component was closed as of June 30, 2009

A-3: Budget Implementation and Treasury Modernization Design and implementation of a KPPN-based automated treasury payment and budget system (SPAN), establishment of a treasury single account in the central bank, and the production by treasury of comprehensive accounting reports consistent with international standards on both cash and commitment basis, all signifying implementation of the Law on State Treasury II.3: SPAN Stage I (as such is described in the invitation for bids for the contract for the procurement thereof) evaluated with user input with respect to content, timeliness and verifiability; specified periodic financial reports prepared for concerned entities

Unchanged Unchanged

II-4: A Treasury Single Account (TSA) system functioning with acceptable parameters: all on-budget central government bank accounts and own-revenue bank accounts of line ministries are brought into the TSA system

Unchanged Unchanged

II-5: Central Government financial statements meet Government Accounting Standards as promulgated by the MOF, and the consolidated management reports produce by the DG Treasury include financial assets and liabilities, and cash-based reports of extra-budgetary funds

Modified Central Government financial statements meet Government Accounting Standards as promulgated by Government Regulation and the consolidated management reports produce by the DG Treasury include financial assets and liabilities, and cash-based reports of extra-budgetary funds

Unchanged

A.4: Procurement Reform Implementation of pilot e-procurement system in the MOF II-6: An evaluation of the MOF e-procurement pilot is completed and an action plan is adopted by the MOF for wider rollout


II-7: A survey of procurement prices relative to international benchmarks is completed


Component B: Revenue Administration B.1: Customs Modernization Improved compliance, revenue collection and stakeholder consultation II-8: The DGCE has adopted a comprehensive and consultatively prepared modernization action plan with inputs from stakeholders


II-9: The DGCE has institutionalized an annual survey of stakeholders, and


4

Intermediate Indicators (II) Action per 1st Restructuring Action per 2nd Restructuring annually published a summary of the results II-10: The DGCE has implemented Web-based filing for about 75 percent of trade transactions


B.2: Tax Administration Improved collection, compliance and adoption of a Plan for nation-wide rollout II-11: The DG Tax has adopted a Results-oriented Modernization Strategy and Plan with measurable performance indicators, aimed at the eventual redesign of processes along single taxpayer account lines and based on the results of the pilot re-engineering of one representative district office


Component C: Governance and Accountability C.1: House of Representatives (DPR) Strengthened capacity and transparency (a) in the Budget Committee for budget analysis and oversight, and (b) for oversight of budget execution II-12: Survey verifies strengthened capacity of the Budget Committee of the DPR

Modified Verification of strengthened basis for ICT development in the DPR

Dropped This sub-component was closed as of December 31, 2013

C.2: Tax Court Efficient and transparent resolution of revenue disputes II-13: Annual surveys of stakeholder satisfaction with Tax Court administration and case management have been established; an action plan has been adopted to address systems weaknesses

Unchanged Dropped This sub-component was closed as of December 31, 2013

II-14: Tax case decisions are accessible to the public through the Tax Court website

Unchanged Dropped This sub-component was closed as of December 31, 2013

Component D: Project Governance and Implementation Verification of effective project governance, external oversight, change management and implementation II-15: Periodic reviews verify that: • Project governance and all project

activities are consistent with the Governance and Accountability Action Plan (GAAP), and project procurement is consistent with the Procurement Plan

• The Project Steering Committee (PSC) is strengthening project governance and oversight

• An investigation unit is established and fully operational in the Office of the Inspector General of MOF, including the establishment of the necessary legal powers to investigate fully allegations of misconduct and/or corruption against MOF staff

Modified Periodic reviews verify that: • Project governance and all

Project activities are consistent with the Governance and Accountability Action Plan and project procurement is consistent with the Procurement Plan

• The Project Steering Committee (PSC) and the Secretary General of the Ministry of Finance are strengthening project governance and oversight

Unchanged

5

1.4 Other Significant Changes 10. The project was adaptive to conditions on the ground and was accordingly restructured twice during implementation:

• First restructuring. With US$4.86 million or 8 percent of the total loan plus credit disbursed by first quarter of 2009, a Level 1 restructuring was completed in June 2009 to reduce the scope of the PDO, change the lending instrument from APL to SIL, and to extend the closing date from June 30, 2009 to December 31, 2013. This allowed the project to focus on the rollout of the new treasury system (SPAN), as well as support to the Tax Court and DPR, and to exclude other project components related to revenue administration that would be funded through different sources.

• Second restructuring. Level 2 restructuring of the project was done in December 2013

to extend the project closing date from December 31, 2013 to December 31, 2015. Project activities related to Component C–Governance and Accountability–supporting the Tax Court and DPR were dropped during this restructuring.

• Adaptability. Most of the subcomponents dropped as a result of restructuring—

including the budget planning and development, tax revenue administration, legislative oversight, and procurement reform—were adapted in scope and financed in part through the PFM MDTF (see Annex 1).

2. Implementation and Outcomes 2.1 Project Preparation, Design, and Quality at Entry 11. Lessons learned incorporated. As documented in the PAD, the task team recognized lessons from the implementation of similar financial management information system (FMIS) in Kazakhstan, Mongolia, Pakistan, Ukraine, and Vietnam, which included the need for high level government commitment to project objectives consistent with the country’s absorptive capacity; thorough needs assessment; intensive TA support with close monitoring of project preparation actions; and rigorous supervision to handle challenges throughout the project cycle. GFMRAP incorporated these lessons by: (i) focusing first on areas the government was committed to at appraisal (public expenditure); (ii) building on the government’s efforts in these areas; (iii) providing TA to support the government’s project preparation efforts; and (iv) anticipating intensive supervision. 12. Design features. GFMRAP was a response to the government’s objective to realize its poverty reduction goals through increased resource mobilization and more effective public resource management as articulated in the 2002 White Paper.3

3 The White Paper was a key output from the MOF which laid out the rationale behind the public financial management reforms in Indonesia. It states that transparency in government budget preparation and accountability in treasury management would strengthen the responsive, efficient and effective allocation and use of resources, and constitute an essential element of Indonesia’s anti-poverty program.

6

• A phased operation: The operation was envisioned to be a three-phased Adaptable

Program Loan (APL) covering a period from 2004 to 2015 at a total cost of about US$250 million to lock in the government’s entire PFM reform agenda in one instrument. The core of Phase I was to roll out an automated treasury payment and budget preparation information system (SPAN) aimed to revamp resource management and mobilization. Phase II was to start in 2006, when Phase I was still ongoing, to focus on the improvement of the customs administration. The last phase would start in 2010, or earlier depending on the government’s achievements of predefined triggers, to concentrate on the modernization of tax administration.

• Maximizing stakeholders’ involvement to sustain government’s commitment: In order to lock in commitment of all relevant stakeholders in PFM reforms, it was crucial to involve DG Treasury, DG Budget, DG Tax, DG Customs and Excise, Fiscal Policy Office, Tax Court, Parliament, and Bappenas in the project Steering Committee. Phase I of GFMRAP started with an ambitious agenda to maximize coverage of the intended PFM improvements, which in turn required the involvement of a wide range of stakeholders and multiple implementing units within and outside the Ministry of Finance.

• Introducing an owner’s agent to safeguard the single large information system procurement package: An independent verification and validation (IV&V) consultancy was introduced to support the MOF and other stakeholders to handle the complex procurement, contract management and implementation processes.

• Partnerships: The government of Japan provided US$5 million as a co-financing PHRD grant. A US$20 million multi-donor trust fund4 was under discussion at loan signing and became effective in 2007 to complement and broaden the PFM scope envisioned in the GFMRAP design. About US$4.8 million of the trust fund was executed directly by the GOI. This trust fund is still ongoing in its second phase since mid-2014, with some donors having departed and new ones joining. Roughly 50 percent of the first phase of the trust fund was used to support the SPAN development.

13. Risk mitigation. GFMRAP was considered to be a high-risk, high-reward project because of its potential to transform core government systems for a rapidly growing nation. An almost exhaustive list of risks to the PDO and component results were carefully considered and presented in the PAD with risk mitigation measures identified. 14. Design shortcomings.

• Ambitious and ambiguous PDO: There were three sub-objectives in the PDO—

efficiency, governance, and integrity and transparency.5 These are at once ambitious

4 The PFM MDTF Phase I with total fund of about US$20 million was established at the World Bank in 2007 with initial funds pooled from the government of the Netherlands and the European Union (EU). Switzerland joined the trust fund in 2009, followed by the United States Agency for International Development (USAID) in 2011. The Canadian Department of Foreign Affairs, Trade and Development (DFATD) joined up during the phasing out of the PFM MDTF Phase I in 2014. PFM MDTF Phase II is funded by the government of Canada, the European Union, and the government of Switzerland since 2014.. 5 Integrity and transparency were grouped together into a single PDO sub-objective as these were more difficult to unpack and separate from each other than efficiency or governance.

7

and ambiguous end goals for PFM reforms. GFMRAP had four components and a total of nine subcomponents covering both sides of the PFM equation, including support to the parliament and the tax court to strengthen the checks-and-balances of the overall public financial management system. However, the original results chain linking the project’s activities, intermediate results, and PDO sub-objectives was weak. Therefore, it was necessary for the purpose of the ICR to reconstruct the results framework to determine the achievements of this operation (see Section 3 and Annex 3).

• Procurement strategy: Procurement of the SPAN information system, which represented 70 percent of project disbursements, was appropriately identified as high risk and it took almost five years to complete. A number of other procurement packages that were not related to SPAN were cancelled midway due to failure to resolve procurement issues between the Bank and the government teams. The rigorous fiduciary safeguards and mitigation measures that were built into the procurement processes were not conducive to expediency. These processes were further hindered by the difficult political and governance climate related to public procurement (for example, risk aversion due to concern over ex-post audits) that almost led to project failure.

• Implementation arrangements: GFMRAP was designed with each of the nine subcomponents having its own Project Implementation Unit (PIU) to manage its day-to-day agreed activities needed to achieve the subcomponent objectives. PIU-Treasury also served as the Project Services and Support Unit (PSSU) to coordinate all PIUs.6 However, over time, the convening power of the PSSU under the Directorate General of Treasury waned. As a result, the PSSU was eventually moved under the Secretariat General, and a new PIU was set up to service the subcomponent under the Directorate General of Treasury in 2009 as part of the first restructuring.

• Institutional readiness: The readiness of the revenue administration institutions—Directorate General of Customs and Excise (DGCE) and Directorate General of Tax (DGT)—to undertake major reforms was overestimated at appraisal. This resulted in negligible progress and led to the first restructuring in 2009, when both subcomponents on revenue administration strengthening were removed and one was pulled into a separate stand-alone operation (PINTAR).

• Unrealistic costing and scope of FMIS turnkey contract: The appraised cost of the SPAN turnkey solution was US$42.4 million covering design, supply, installation, training, and post-implementation services. Actual costs were US$57 million, of which US$42 million was funded by the loan and the balance by the GOI,7 an overrun of about 36 percent. Computer hardware infrastructure technically depreciates in about four years. The procurement of the hardware infrastructure and SPAN application as one procurement package, and the miscalculation in the timing of the completion of the development of the SPAN application resulted in the premature delivery of the hardware infrastructure. As a result, by the time the SPAN application was ready to be

6 As stated in the PAD, GFMRAP implementation arrangements utilized existing government structures and processes with PSSU set up within DG Treasury initially and then the SecGen’s office after the first restructuring, to provide administrative, logistical, procurement, and financial management support for the overall project implementation, and to coordinate with the various designated subcomponent leaders. PIUs were set up within each unit responsible for its corresponding subcomponent. PSSU and PIUs were staffed entirely by government officials from the respective units. 7 The GOI portion also fully covered operating costs such as maintenance.

8

piloted and rolled out, the hardware infrastructure, while still operational, was already outdated. Furthermore, server capacity was significantly underestimated. Even though the Bank supervision team advised the MOF of the original underestimations, the ministry did not want to take a quick decision to procure new servers without factual evidence supporting the capacity shortage due to fear of negative ex-post audits. This indecisiveness led to severe delays. It took the MOF 1½ years to finally request and obtain the Bank’s no-objection for a tenfold increase in server capacity after the SPAN pilot had to be stopped due to lack of server capacity to manage the current volume of transactions. Moreover, in spite of the significant cost overrun, the SPAN contract failed to include activities essential to support the FMIS professional operation such as: system security, ICT audit, independent testing prior to final acceptance, risk management, and legal advisory services to support implementation and changes in business processes.

2.2 Implementation 15. The GFMRAP operation was originally envisioned to be a 12-year operation addressing both sides of the PFM equation—public resource management and revenue generation. It eventually took 11 years, from the initial conceived 4-1/2 years, to complete the budget planning and execution parts of the operation. The overall implementation of GFMRAP was wrought with delays, though both the government and the Bank committed to working through the numerous challenges to bring it to completion. Missions were conducted regularly, and the government produced semi-annual monitoring reports through the PSSU. The GFMRAP operation was financed through a combination of US$55 million IBRD loan, SDR 3.14 million (equivalent to US$5 million) IDA credit, US$5 million PHRD Japan Grant, about US$12 million of the PFM MDTF (both Bank and recipient executed portions), and about US$30 million in government’s own resources. The operation was signed on December 22, 2004, and became effective in October 2005 after two extensions. As part of the Level 1 restructuring, the closing date of the operation was extended from June 2009 to December 2013, with the scope significantly streamlined. The second, Level 2, restructuring in December 2013 further extended the closing date to December 2015 to facilitate the SPAN rollout across Treasury offices. 16. Implementation highlights.

• Major scope reduction to focus project goals: The loan included eight effectiveness conditions that had to be completed within 90 days after loan signing.8 It took the government almost ten months to complete these effectiveness conditions and some were waived. Once the SPAN contract was signed in July 2009, GFMRAP was

8 These were complicated undertakings. They included finalizing a project management manual and establishing structural units within the Ministry of Finance, Bappenas, and Secretariat General of the Parliament (totaling nine PIUs) to implement the various components of the project. They also included the issuance of Specific Procurement Notice and Request for Proposals for two key procurement packages—change management and communications services consultancy and the US$45 million SPAN information system—and signing of a contract for the independent verification and validation (IV&V) consultancy. Furthermore, a new Minister of Finance, who did not have the institutional memory and ownership of the government’s PFM reform agenda, was sworn in around the time of project negotiation. This affected project implementation during the entire effectiveness period until another Minister, who embraced the reforms on the outset, came on board.

9

streamlined to focus on the development and implementation of the SPAN system. This realignment was formalized by a Level 1 restructuring in June 2009 that removed the entire revenue administration component and shifted most of the remaining components under the PFM MDTF.

• Top management commitment resolved the SPAN procurement impasse: The SPAN contract was originally planned to be awarded in 2006 but it took more than twice the amount of time anticipated. Of the 25 firms that purchased the bid documents, only four bids were submitted under the first stage of the SPAN procurement. After careful evaluation, the procurement committee determined that only one bidder was qualified to advance to the second stage of the procurement process. The Bank disagreed with the committee’s decision to disqualify one of the bidders, and this impasse was not resolved until the then Minister of Finance intervened.

• Project management support through PMQA: The PMQA consultancy was mobilized and played a large role in assisting the government with project management and quality assurance needed to implement the design and roll out of SPAN. The initial phase of the PMQA consultancy was not very effective because it started six months after the SPAN contract and a number of implementation conditions had already been agreed. It was critical during customization of the commercial off-the-shelf software (COTS) 9 when most technical experts were brought on board, it provided much-needed advice and guidance to the structural unit established to lead the SPAN implementation.

• PFM MDTF provided additional funds and a convening platform to broaden the scope of the PFM reform envisioned in the original GFMRAP design: In 2007, the public financial management multi donor trust fund (PFM MDTF) became effective, which allowed the Bank to assign full-time senior advisors in treasury, budget, revenue, and information systems to support the government’s PFM reform agenda. Over time, a number of capacity building activities, particularly those with the Directorate General of Budget and Bappenas, were shifted to the trust fund, allowing more loan resources to be channeled to SPAN implementation. The PFM MDTF has been instrumental in supporting performance-based budgeting, initial adoption of a medium-term expenditure framework, TSA, and design of the tax court’s case management information system and e-procurement portal. It also helped spur bureaucratic reform covering MoF, Bappenas, and the Ministry of State Apparatus and Bureaucratic Reform (Menpan-RB) and the development of a national single window to facilitate trade.

• Multiple change requirements during SPAN implementation: Continuous changes were requested during the design of SPAN. Business process improvements were constantly introduced due to the broader PFM reforms underway, and new features that were

9 Commercial-off-the-shelf (COTS) Enterprise Resource Planning (ERP) are software packages offered by commercial vendors that support core administrative processes such as budgeting, accounting, procurement, performance and human resource management by integrating the data required for these processes in a single database. COTS ERP systems are based on the premise that the software vendor can support common business processes more effectively and efficiently than customer organizations. Because ERP software is maintained by the vendor and is based on a reference model of business processes defined by the vendor, total cost savings and return on the buyer’s investment are predicated on the organization adopting the vendor’s model. While ERP software supports limited customization through changes to configuration settings, unsupported modifications to the software only serve to increase maintenance costs, thereby reducing the overall return on the buyer’s investment.

10

originally not anticipated were requested. The contract with the SPAN developer never specified a date by which system requirements should be frozen, resulting in more change requests well after the developer requested a freeze. At the same time, the MOF project team believed that the SPAN contractor lacked technical experience specific to COTS implementation, had high turnover of key staff, and had difficulty understanding user needs.10

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 17. M&E design. The core of the M&E design was the results framework (RF) with original indicators in Annex 3 of the PAD. The RF provided the basis for the government to monitor progress in meeting the project implementation objectives. The PSSU submitted semi-annual reports providing progress status of all indicators throughout the entire operation on a timely basis. Significant progress was made due to the commitment of the Bank team and the implementing agencies. Outcomes that were not originally envisioned when the project was prepared were achieved. Accordingly, the results framework was reformulated with the PDO indicators and intermediate outcome indicators rearranged in Annex 3 against each of the three sub-objectives—efficiency, governance, and integrity and transparency—to capture the full achievements at the time of the ICR. 18. M&E implementation. No management information system or any system was set up to systematically collect data. The PSSU engaged an M&E consultant on a peripatetic basis to conduct M&E reporting when semi-annual reports were due. Starting in May 2008, three different M&E consultants were engaged, a practice not conducive to preserving institutional memory. However, the PSSU was able to retain a full-time project manager, who remained with the project through its completion. 19. M&E utilization. The government’s semi-annual reports, with mostly qualitative monitoring of progress against pre-defined indicators, were not the basis to inform decision-making and resource reallocation. Mid-course adjustments were done based mainly on the day-to-day interactions between members of the Bank’s task team and their respective government counterparts. Bank’s ISRs did not report against M&E indicators. Project adjustments were formalized through periodic supervision missions. 20. The overall rating for M&E is modest. 2.4 Safeguard and Fiduciary Compliance 21. The GFMRAP operation was rated as an environmental category C with no resettlement or displacement of communities or indigenous peoples encountered during the course of the project.

10 The relationships between the SPAN contractor and the ministry deteriorated to the point that at end-2013, the SPAN contractor submitted a case for adjudication with a US$18.7 million compensation claim. This was eventually resolved when both MOF and the SPAN contractor agreed to accept the adjudicator’s decision to continue completing the project and stop seeking additional compensation from one to the other since both parties contributed to the cause of delays.

11

22. The primary implementing agency was the Ministry of Finance. Other implementing agencies included Bappenas, Tax Court, Procurement Agency, and the Office of the Secretariat General of the DPR. Both project procurement and financial management capacity assessments were completed at appraisal. The project complied with fiduciary covenants during implementation with internal control arrangements in place.

23. GFMRAP employed satisfactory financial management systems overall, though there were a few problems at the beginning of project implementation, particularly at the PIU level. The PSSU adopted a verification checklist and used it prior to issuing payment requests sent to the treasury office (KPPN). In addition to reviewing and verifying supporting documents, the financial staff also reviewed and verified the substance of the payment requests and requested additional documents from requesters when necessary. In general, financial staff assigned to GFMRAP had adequate knowledge and background to manage the financial management aspects of the project. PSSU was also agile in rectifying deficiencies. Auditors expressed an Adverse Opinion for the FY2013 project financial report due to a dispute on accounting principles and adequacy of supporting documents. PSSU took immediate actions by providing all needed supporting documents and improving its financial management system. Since then, auditors have expressed unqualified or clean opinions on all project financial reports up to project closing. 2.5 Post-completion Operation/Next Phase 24. The tax administration component, which was originally envisioned as Phase III of the APL, was converted into the PINTAR SIL. There were other activities funded by the PFM MDTF that provided the basis for ongoing work in tax administration, which was included as a major pillar under an ongoing DPL. The Directorate General of Treasury recognizes there remains work to be done in system enhancements and operational support for the SPAN system and is exploring options to finance these activities. 25. As pointed out above under “Design Shortcomings” sustainability of SPAN is affected by post-implementation challenges. These deficiencies would not be unusual for Phase I of an APL operation because they would be naturally addressed in subsequent phases of the APL. The change of this project to a SIL with no additional contingency funding left these issues unattended. However, Treasury, recognizing the problem, has signaled to the Bank it would welcome a follow-up project. 3. Assessment of Outcomes 26. The scope of GFMRAP was extremely broad, covering virtually all aspects of public financial management, including budget preparation, budget execution, tax administration, customs, e-procurement, accounting, and reporting. The outcome indicators that were developed in 2004 could not have anticipated the full extent and scope of PFM reforms, so new indicators have been introduced in the analysis below to fully capture extent of the benefits of the project.

12

3.1 Relevance of Objectives and Design Relevance Rating: Substantial Relevance of Objectives: High (Pre-restructuring: High, Post-restructuring: High) 27. The Level 1 restructuring that took place in 2009 reduced the scope of the PDO by dropping the work related to revenue administration. The analysis on the ratings for relevance, efficacy, and efficiency will be given pre-restructuring and post-restructuring. However, since the amount disbursed pre-restructuring was about 8 percent of the loan amount, the post-restructuring ratings will be given a much higher weighting. 28. The original project objectives focusing on public financial management and revenue administration remain highly relevant at project closing. GFMRAP’s objectives mirrored the government of Indonesia’s development priorities. Weak institutional capacity continues to be one of Indonesia’s key development constraints, and good governance remains a key ingredient in the Government’s National Medium-Term Development Plan 2015-2018 (RPJMN). In the World Bank’s 2016-2020 Country Partnership Framework (CPF), collecting more and spending better is one of the core engagement areas. 29. The revised project objective focusing only on the Borrower’s public financial management remains highly relevant as well. The 2016-2020 CPF acknowledges a number of positive outcomes in the Bank’s support for public financial management, including development of medium-term budget forecasting tools, M&E processes to inform future budget allocations, regulations for accrual-based accounting policies and chart of accounts, and the roll out of the automated treasury payment and budget preparation information system. Relevance of Design: Modest (Pre-restructuring: Negligible, Post-restructuring: Modest) 30. Various aspects of the design had shortcomings, including the choice of lending instrument, scope of the development objective, and results chain. 31. The original program was overly ambitious and the choice of lending instrument reflected this. The project was originally envisioned as a three-phased Adaptable Program Loan, covering a period of 11 years that would provide a platform to lock in major reforms related to public financial management in the first phase, customs in the second phase, and tax administration in the third phase. The project was designed this way given the challenging governance environment, uncertainty over elections, and lack of strong ownership and commitment of the Directorate General for Tax and Directorate General for Customs and Excise (DGCE) for reforms. While a better approach would have been to implement individual, inter-related but self-standing Specific Investment Loans (SILs), the flexibility of the APL instrument gave the Bank a strategic entry point to engage with the Ministry of Finance on many key public sector reforms.

32. The framing of the development objective was broad-based in order to allow for flexibility needed for the APL. However, about half way into the project life cycle, the tax reform component was converted into the Project for Indonesian Tax Administration Reform (PINTAR), as a SIL, which was considered a more appropriate instrument for the Borrower to establish a focused

13

project implementation structure independent from the broad-based program. Customs administration reform was dropped after the Government decided to implement this component without donor assistance. To accommodate this change in scope, a Level 1 restructuring took place where the revenue administration portion of the PDO objective, components, and indicators were all dropped and the project was converted to a SIL. The work with the Tax Court and DPR were later dropped in the Level 2 restructuring that took place two years later.

33. The restructuring was a good opportunity for the project to rework the results framework given the reduced scope of the PDO objective, but this was not done, making it harder to establish a clear and convincing causal results chain between funding and outcomes.11 The statement of objectives would have been made clearer and better linked to intermediate and final outcomes. 3.2 Achievement of Project Development Objectives 34. GFMRAP has been a major contributor to an improved PFM landscape in Indonesia, bringing basic financial discipline and transparency to 182 local treasury offices, 33 regional treasury offices, and 8,000 Treasury staff. The roll out of the automated treasury payment and budget preparation information system (SPAN) improved the efficiency of the central government’s operations. However, progress on revenue administration was modest. Activities intended to modernize DGCE were barely executed. A national single window to facilitate trade was established with funding from the PFM MDTF but without the participation of DGCE. Although registered taxpayers increased from 19 million in 2010 to 30 million in 2014, the compliance rate of tax return filing is still low at 59 percent and there is no evidence of increased tax revenue to GDP. 35. The PDO contained several compound objectives and these were unpacked into three sub-objectives:

(a) To improve efficiency in the Borrower’s public financial management (b) To improve governance in the Borrower’s public financial management (c) To improve integrity and transparency in the Borrower’s public financial management

36. The project results framework with PDO indicators and intermediate outcome indicators was reconstructed in Annex 3 order to better reflect the unanticipated results of GFMRAP and to strengthen the results chain that supports the project’s theory of change. (a) Improved efficiency in the Borrower’s public financial management Rating: Substantial 37. SPAN has helped contribute to timely and transparent payments, which has discontinued the informal “fees” that were given to expedite payments. Salaries of 1.62 million civil servants

11 There was uncertainty whether GFMRAP was to be continued given that the award of the SPAN contract was a critical path activity. With the project end date approaching and award of the SPAN contract becoming more imminent, the task team then quickly prepared the restructuring package. The M&E arrangement was not considered to be critical to finalize the restructuring process to ensure reform momentum and project continuity.

14

and payments to 300,000 suppliers are processed through direct deposit. As a result, approximately US$55 million is saved in printing and internal communication costs per year. The Treasury also developed OM-SPAN, an online monitoring system for payment transactions that can be accessed to check on payment status on a real-time basis. Furthermore, with a TSA fully implemented since 2009, over US$300 million is saved in interest earnings as a result of having consolidated cash balances. Cash balances have been consolidated, off-budget accounts have been brought into compliance, and existing miscellaneous accounts of line ministries have been closed. New accounts can only be opened after MOF authorization. SPAN has enabled senior government officials (including President Joko Widodo) to see their budget execution progress in almost real time, when previously this was a cumbersome task that involved the reconciliation of multiple reporting systems from multiple databases. (b) Improved governance in the Borrower’s public financial management Rating: Modest 38. GFMRAP also helped enhance legislative scrutiny and oversight of the annual budget by improving the capacity of members of Parliament through analytical support to the budget formulation process. The review undertaken by Parliament covers Indonesia’s macroeconomic framework, main fiscal policies and expenditures and revenues. A detailed discussion of the annual work plans of line ministries takes place directly with the relevant parliamentary sectoral budget commissions. The budget review is undertaken over a period of about seven months. Parliament has some 8-10 weeks (compared to 6-8 weeks before) to review the draft budget once it is tabled in mid-August until it is formally adopted in October. 39. While the medium-term expenditure framework (MTEF) was introduced in the 2011 budget, further fine-tuning of its implementation along with the operationalization of performance-based budgeting (PBB) remains a priority to fully unleash potential benefits of these governance strategies. The medium-term expenditure framework takes the performance indicators from the previous year as a baseline for the subsequent year. On performance-based budgeting, outputs are defined for each program area every year and progress against each output is reported. 40. GFMRAP also supported training and professionalization of 37 Fiscal Policy Office staff, of which 7 completed PhD programs, 12 completed aster’s programs, and 18 completed specialized training. GFMRAP also helped set up a special investigation unit in the Office of the Inspector General in order to investigate allegations of corruption against MOF staff, including tax officers. In addition, the PSSU has been successful in ensuring project activities are consistent with the government’s anti-corruption plan and that project procurement activities are consistent with the Procurement Plan. The project steering committee and the Secretary General of the MOF have been consistently carrying out project governance and oversight with frequent and regular meetings.

15

(c) Improved integrity and transparency in the Borrower’s public financial management Rating: Substantial 41. SPAN helped curb corruption by reducing the opportunities for discretion and informality that are more common in manual, paper-based systems. In addition, it reduces leakages due to erroneous payments or over-spending by budget users. SPAN introduced a new module called the supplier database in order to reduce payment errors to unintended recipients. As a result, manual checks of payments for 300,000 suppliers has been replaced with regular system reporting and fiscal discipline over payments has increased. Commitment control has also been significantly strengthened. Annual budget ceiling data is integrated with SPAN so spending units cannot disburse beyond this limit, helping strengthen expenditure control. However, discrepancies in the recording of certain transactions still exist (e.g. intergovernmental revenues, non-tax oil and gas revenue, and foreign exchange debt) but these are all within the internal auditor’s materiality threshold of 0.5%. Individual line ministers have their own internal auditors and these audits have been on time and cover all ministries and agencies. 42. The Treasury has also made progress transitioning from cash-based accounting to accrual based accounting. The processing and recording of government transactions in a single database has also helped ensure financial data integrity. Monthly budget execution reports are generated on time and the reconciliation mechanism between regional and local Treasury offices is now done electronically. Progress has also been made in terms of ensuring greater access to the semi-annual budget report and to contract awards above a threshold of IDR 50 million on the website of agencies. 43. The overall achievement of objectives given above is assessed as Substantial. 3.3 Efficiency Rating: Substantial (Pre-restructuring: Negligible, Post-restructuring: Substantial) 44. A comparative study of FMIS implementation among countries shows that Indonesia’s investment per user on FMIS is in line with its comparators with equivalent or better utilization of features.12 Another study shows that FMIS implementation time (2009-2015)13 under GFMRAP is comparable to the average period to implement an FMIS (7.9 years) and that 80 percent of the Bank projects financing FMIS were extended.14 On the benefits side, examples shown in Section 3.2 demonstrate that operational and administrative efficiencies were derived through proactive results oriented implementation approaches adopted under the project. The project has improved

12 Ali Hashim, A Handbook on Financial Management Information Systems for Government – A Practitioners Guide for Setting Reform Priorities, Systems Design and Implementation (World Bank, 2014) 13 FMIS implementation was mainly completed from 2009 to 2015. This excludes the initial period up until 2009 spent on policy and institutional reforms and includes the period for which GFMRAP supported the design and roll out of SPAN, systems and applications upgrades, capacity building aimed at sustainability of operation of these systems, and enhancing the functionality of SPAN. 14 Cem Dener, Joanna Watkins and William Dorotinsky, Financial management information systems: 25 years of World Bank experience on what works and what doesn't (World Bank, Apr 2011)

16

the efficiency of Indonesia’s public financial management system through the provision of comprehensive, reliable, timely, and accurate data for informed decision-making. 45. Aimed at strengthening of financial management and revenue administration, GFMRAP-inspired investments allow for more effective public resource management benefits. As such, the project’s successful implementation results in a number of important economic benefits, including the following:

a) Improved predictability of budget execution and reductions on payment errors to unintended recipients. Invoice payments to 300,000 suppliers are issued by SPAN per year through direct payments to individual employee accounts instead of manual checks. SPAN requires suppliers to be registered and recorded in a supplier database before payments can be processed, reducing the number of payments wrongly made to unintended recipients.

b) Direct deposits of civil service wages. Salaries of 1.62 million government employees are now paid electronically by SPAN directly to each individual account. This has helped increase governance and control over accuracy of staff salary payment details and reduced the number of bank accounts being managed for payment of salaries from 750 to 3 banks. The volume of staff salaries has also doubled not due to staff increase but due to inflation.

c) Reduction in cash balances. US$300 million saved per year since 2007 on interest earned or remuneration given by the Central Bank from consolidation of cash balances and reduction of idle cash with the Treasury Single Account.

d) Increased administrative efficiency. Automation created savings on wage bill as the total number of Treasury staff during the last decade remained the same while government budget and expenditure increased three times and the number of transactions doubled. Online real-time data availability has also reduced the staff time and effort to manually collect and aggregate data from local offices.

e) Improved collection and access to information. Utmost reliance on SPAN information by MOF to make expenditure authorization decisions in the capacity of the financial adviser for various ministries.

f) Full reconciliation of audited financial statements. In 2006, the Supreme Audit Agency (BPK) reported that tax revenues as determined by DG Tax were higher than those reported by the Treasury by Rp1.9 trillion (approximately 0.5 percent of aggregate revenues) and considered this discrepancy a cause for recording a disclaimer. Through the revenue collection module introduced in SPAN this discrepancy disappeared and BPK reported that the tax revenue data recorded is clean with an unqualified opinion.

g) Increase transaction processing capacity. Increased processing volume from 32 million revenue payment transactions being processed in the amount of Rp590 trillion in 2008 to 44 million transactions with Rp1,400 trillion in 2013.

h) Better coordination between cash and debt management. Improved coordination between DG Treasury and DG Debt Management leads to greater efficiency in debt issuance, redemption programs, and use of cash. Having a consolidated balance in the TSA helps DG Debt Management and DG Treasury to decide on the borrowing strategy, cash plan and placement. This can also support efficient cash management by altering the profile of debt redemptions through buy-backs or swapping securities of different maturity structures. While DG Debt Management used to set a schedule for bonds issuance in a particular year, the issuance of bonds was usually seen as a “front loading” policy, since DG Debt Management based the schedule around securing its own financing needs at the

17

beginning of year without consultation with DG Treasury on their cash needs. This imposed unnecessary carrying costs resulting from the increase in the idle cash under management at the beginning of the fiscal year.

i) Reduction in corruption. The elimination of thousands of government bank accounts through the consolidation of balances in the TSA has reduced the potential for corruption resulting from the authorities managing large cash balances in commercial bank accounts. Since the TSA has been in place, the local treasury branches (LTBs) and spending units have had much less discretion than before, because of the direct disbursements and minimal balance kept in their accounts. Furthermore, it is not possible for a public agency or unit to open an unauthorized bank account.

46. The economic benefits derived from these administrative improvements alone would suffice to derive economic feasibility. However, as many financial aspects have proven difficult to quantify, a much narrower economic analysis is being presented. 47. Such an economic analysis, based solely on the calculation of NPVs derived from project costs net of (i) interest savings and opportunity savings from centralized cash accounts held in the Treasury Single Account at the Central Bank; and (ii) productivity savings from decreased reliance on manual checks and paper documentation and staff time saved,15 confirms that the returns to investment under GFMRAP are substantial. The rates of return on investment over a 15-year period, applying a 15-percent discount rate,16 are estimated at 50 percent, with a positive economic NPV of US$632 million. High rates of return are mostly due to savings from consolidated cash accounts through the creation of a Treasury Single Account and efficiency gains of local and regional Treasury offices through automation. Costs include actual project costs, including government co-financing, implementation support costs, as well as support provided by the PFM MDTF. Low and high scenarios are considered for benefits and costs. Details on assumptions and calculations are presented in Annex 4. The value added arises mainly from the Bank’s technical input based on international experience, particularly on the FMIS implementation. 48. Nevertheless, in spite of all the difficulties, SPAN was implemented and deployed throughout the country including the most remote Treasury offices. It is being used by DG Treasury every day, and GOI is reaping most of the benefits anticipated for the project. GFMRAP APL design called for a second phase of SPAN implementation to integrate it with many of the other MOF systems and bring its benefits to all areas of the MOF. The MOF is considering alternatives for this expansion.

49. The broad-based positive impact of the project given above benefits is assessed as Substantial.

15 E-transmittal of salary slips and payments is expected to save millions of dollars on paper, printing, communication, and postage. This impact is expected to increase each year, once regulations are changed to no longer require paper forms be dropped off (including SAKTI connection to line ministries). The calculations used in this economic analysis assumes this gradual increase in savings from 2015 to 2019. See Annex 4 for more details on assumptions. 16 GFMRAP has a positive net present value even for a very high discount rate. The NPV ranges from $413 million when applying a 20-percent discount rate to $1.6 billion applying a 5-percent discount rate. The economic analysis uses a high discount rate of 15% given the project’s long implementation period and because assumptions behind productivity gains will need to be confirmed in the next few years.

18

3.4 Justification of Overall Outcome Rating Overall Outcome Rating: Moderately Satisfactory 50. The justification for ratings of relevance, efficacy, and efficiency are elaborated in subsections 3.1, 3.2, and 3.3, respectively. All ratings were either Modest (M) or higher, with a High (H) rating for relevance of objectives and efficiency.

Pre-restructuring

Relevance Efficacy Efficiency Overall

Outcome Objectives Design Efficiency Governance Integrity and Transparency

High Negligible Modest Negligible Negligible Negligible Highly Unsatisfactory Modest Negligible

Post-restructuring

Relevance Efficacy Efficiency Overall

Outcome Objectives Design Efficiency Governance Integrity and Transparency

High Modest Substantial Modest Substantial Substantial Moderately Satisfactory Substantial Substantial

Combined Rating

Against Original PDO

Against Revised PDOs

Overall Comments

1 Rating Highly Unsatisfactory

Moderately Satisfactory Significant

improvement 2 Rating Value 1 4 3 Weight (%

disbursed before/after PDO change)

8% 92% 100%

4 Weighted value (2 x 3) 0.08 3.68 3.76

5 Final rating (rounded) Moderately

Satisfactory

Early revision preserves an MS rating

3.5 Overarching Themes, Other Outcomes and Impacts 51. The GFMRAP operation was prepared at a time when Indonesia’s annual economic growth was stalled at less than 4 percent, and future prospects were clouded by perceptions of corruption in financial management practices. Though designed with a very inclusive set of stakeholders, GFMRAP’s originators were aware that some functions were more ready to embrace reform than others. DG Customs and Excise and DG Tax, for example, quickly revealed an ambivalence toward the project as it was structured. In turn, the government pushed ahead at the opportunity

19

to develop a modern budget and treasury system that could be a foundation for other reforms promoting transparency and efficiency. Built into an FMIS is an automatic audit trail that reduces room for discretion in payments. While some reforms such as the TSA, accrual accounting, MTEF, PBB, e-procurement, and other PFM policies could have been developed without Bank financing, the project provided a convening platform to rally multiple stakeholders within and outside of the Ministry of Finance to dialogue on PFM issues and a focal point for trust fund resources that had been mobilized in parallel from donors. The generous trust fund resources in turn enabled the Bank to leverage the impact of Development Policy Lending (DPLs) and to support other institutional and policy reforms of the Government.

(a) Poverty Impacts, Gender Aspects, and Social Development 52. Strengthened PFM systems introduced under GFMRAP ensured that money was spent within established annual budget ceilings. Enhanced commitment control in turn contributed to services being delivered more efficiently. While there are many other factors at play, strengthened PFM institutions and systems helped encourage transparency and accountability in budget execution. In addition, transparent financial reporting, which leads to informed decision-making, economic viability, and credibility of the state, provides a conducive environment to attract direct and indirect private investment. Because of the project, payments to suppliers and salaries of government employees are now directly deposited to individual bank accounts. This has resulted in reducing petty corruption as well as providing a convenient facility for beneficiaries. The project was also instrumental in identifying ghost vendors and workers, which has helped save millions of rupiahs for the government. (b) Institutional Change/Strengthening 53. The project set the foundation for enhanced capacity in PFM. While the legal and regulatory framework was in place before SPAN was designed, business processes were constantly changing and it was difficult for Treasury to envisage how they could fit into a COTS package. In turn, this resulted in significant delays in freezing the business requirements, increased costs of the supplier, and lengthened SPAN development time. Nevertheless, over time officials from DG Budget and Treasury became better informed of international good practice and were able to steer institutional changes within their departments, helping to ensure the SPAN was rolled out successfully. GFMRAP led the way for better tracking expenditures, enhancing commitment control, increasing efficiency of payment systems, and reducing corruption. Furthermore, SPAN is a catalyst for the ministry’s ICT integration. Without the GFMRAP operation, individual units within the ministry would continue to develop their own standalone and in-house applications and a single integrated database would not be realized. (c) Other Unintended Outcomes and Impacts

54. The development of SPAN triggered the introduction of three applications—MPN, OM-SPAN, and SAKTI—developed by DG Treasury at the same time as SPAN development to enhance the usability of SPAN. MPN, the electronic state revenue management system, was developed to collect state revenues, generate timely revenue collection reports, and provide a more complete picture of Treasury’s financial position. OM-SPAN was developed to ease the load of SPAN to allow for online monitoring and reports generation. SAKTI, the institutional-level

20

financial application system, scheduled to be rolled out to all spending units in 2017, was developed as a feeder module to link all currently standalone systems used by the more than 24,000 spending units across Indonesia to the SPAN system. All these applications anchor around SPAN, expand its utility, and signal ownership and sustainability of the investment.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 55. Insights from stakeholders showed that while a number of reforms were initiated in the early 2000s, it took time for these reforms to be fully internalized. It took time for the regulations to be translated and incorporated into business process improvements, which in turn led to many report customizations required of the COTS system. Pockets of resistance were identified upfront but building consensus among a large group of stakeholders took time. Inputs from different outcomes of the reform can be found in Annexes 6 and 7.

4. Assessment of Risk to Development Outcome Rating: Substantial 56. The key output of the GFMRAP operation is SPAN, a fully operational budget preparation and treasury payment system. SPAN has transformed how budget execution is carried out to such an extent that the potential of going back to the former business processes and previously discrete standalone legacy systems is minimal. There has been overwhelming enthusiasm from key stakeholders that its sustained use is indisputable. The President of Indonesia officially launched SPAN in front of his cabinet showing a strong sense of ownership beyond the Directorate General of Treasury. There is newly acquired skill sets within the DG Treasury, as evident in its in-house development of the report-generating application OM-SPAN to ease the load of SPAN. However, the development of OM-SPAN could also be interpreted as a risk that Treasury lacks the knowledge and experience with SPAN’s full functionalities and could lead to the development of parallel modules outside SPAN. 57. Pusintek, the information system department of the Ministry of Finance, has significantly increased its capacity and capability to become a reliable information communications technology (ICT) service provider for the ministry. It now manages a multi-site disaster recovery center and was successful in developing a case management system for the Tax Court after a failed procurement under the GFMRAP operation. Pusintek’s role in integrating ministry-wide databases and providing ICT needs is likely to continue to broaden, which in turn would lead to strengthened and concentrated ICT capacity in Pusintek to service the rest of the ministry, one of the unspecified benefits of the operation. 58. However, there are some serious sustainability issues set at the Ministry of Finance level that directly affect DG Treasury: (i) the lack of an HR Policy that creates a solid basis for staff expertise to be developed and retained; (ii) the inability to establish a strategic long-term (multi-year) relationship with providers of the technologies being used to ensure proper maintenance, care, and continuous enhancement of SPAN and the infrastructure that supports it; (iii) lack of

21

information security management which could potentially put SPAN at high risk; and (iv) no business continuity plan in place in case of a natural disaster.

5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 59. GFMRAP was prepared at the request of a reform-minded Minister of Finance who was keen to lock in major PFM reforms before elections. Project preparation borrowed heavily on the results of technical assistance that the IMF had been providing to the MOF, particularly in relation to modernization of the Treasury and implementation of SPAN. However, the Bank recognized early on that availability and quality of strategic analyses to support the GOI development objectives for the project’s various components varied significantly. Moreover, the clear recognition of the need for institutional reforms at the top level of the MOF was not fully shared by DG Customs and Excise and DG Tax, nor by lower echelons of the same ministry. Bank management recognized from the outset that this would be a high-risk operation and the project team reflected these risks in a well-prepared risk matrix. The Bank made excellent use of its ability to mobilize resources to promote increased quality at entry: a PHRD grant was obtained in record time, as well as a PPF to increase the MOF capacity to prepare the project and to create the conditions for successful implementation. The adoption of the APL investment instrument was important to mitigate the unevenness of strategic approach to and interest in the reforms in the different MOF directorates general. The project design considered international best practices and lessons learned by the Bank in the implementation of PFM, Customs and Tax projects. A candid QER process provided extensive guidance to the Bank team. Project preparation seemed to have been conducted under considerable time pressure to support the GOI need to lock-in and implement its long-term public financial management and revenue administration reform agenda.17 Project components were aligned with the PDO’s intended outcomes, and the outcomes responded to the government’s development priorities at time of appraisal. The project design also included an extensive analysis of related risks and a thorough evaluation of fiduciary capacity, sustainability, and safeguards issues. However, project indicators did not adequately capture how project activities were contributing to the PDO. 60. Although project design fit well with the PDO, the project scope was over-ambitious even for an APL.18 The preparation of each component—Treasury, Customs and Tax—required highly specialized experts to guide the preparation process and define their development impact. In addition, the project called for heavy investments in information technology, an area repeatedly

17 The PAD indicated “Solid progress in economic policy-making, yet limited strengthening of governance and public institutions, has characterized Indonesia’s continuing transition from an autocratic, centralized state to a democratic, decentralized one.” 18 Peer reviewer Bill Dorotinsky, noted: “The areas of focus and reform seem appropriate, and likely to add value. Treasury reform, customs and revenue administration, and improving oversight. However, this is a big agenda, and larger, more developed MOF’s would be challenged to see this broad agenda through to successful completion.

22

flagged in the Bank for procurement difficulties. Moreover, from the outset the agents responsible for two project subcomponents—DG Customs and DG Tax—were reluctant to engage in open dialogue during preparation and failed to demonstrate ownership and commitment. Guidance to the team, at PCN and QER stages, that these two components should be dropped under these circumstances and that the packaging of planned IT investment on SPAN be broken into smaller packages was not heeded. 19 As a result, these two subcomponents were prepared but barely executed and had to be subsequently cancelled through project restructuring. Procurement and implementation stages of SPAN were plagued with problems. The Monitoring and Evaluation system included several indicators with no baseline or clear targets and could not be adequately monitored by the MOF. As a result, little attention was paid to M&E by the GOI during implementation or by Bank missions as part of supervision. Detailed milestones and triggers were prepared before negotiations but were of little value during implementation because they were so numerous and so detailed that non-observance became a common and acceptable occurrence. Cost estimation was overly optimistic. By the second year of implementation, it was already clear there was a need for additional financing to cover the difference between original cost estimates of US$48 million and revised estimates of US$66 million, a 37.5% increase. (b) Quality of Supervision Rating: Moderately Satisfactory 61. Adequacy of supervision inputs and processes. During the almost 12-year implementation period, the Bank conducted 18 regular implementation support review missions and completed a mid-term review in 2009. Given its wide scope and complexity, the project was placed under intensive supervision from the outset. The Bank’s implementation support review missions were frequent and helpful in identifying and proposing alternative solutions to key implementation challenges that plagued all project activities with no exception. Bank management provided excellent support to these missions. Support missions were fully staffed bringing to the fore subject matter expertise in public financial management, customs administration, tax administration, and the key area of procurement: information technology. Bank management also facilitated access to the top authorities in the Ministry of Finance as needed to address implementation road blocks and encourage the Borrower to put in place the necessary structures, processes and procedures needed for the MOF to carry out its project implementation responsibilities. A well-endowed Multi Donor Trust Fund (PFM MDTF of approximately US$20 million) was put in place to supplement the loan resources and aid the MOF with project execution. Implementation support missions invested substantially in the coordination with the many initiatives financed by other donors.20 Generous resources were provided to the implementation support teams21 to ensure implementation. Twice Bank management restructured the project to add focus on the key objective of implementing a modern Treasury system (SPAN) and extended the loan closing date. Intensive Bank supervision was often required to delve in-depth on key project management, procurement and contract management issues and continuously proposed

19 Project preparation missions noted repeatedly the difficulties in getting DG Tax and DG Customs to prepare their components or even agree to receive the mission for meetings. 20 For example, a 2006 donor meeting included representatives of the ADB, AusAid, Australian Treasury, CIDA, GTZ, IMF, Japanese Embassy, JICA JBIC, the Netherlands Embassy, and USAID. 21 For example, in 2008 the team had three international staff resident in Jakarta supporting project implementation (and development of GFMRAP-II aka PINTAR) in addition to three local staff dedicated to the same activity

23

refined and improved project management methods and technical solutions to the MOF. Fiduciary responsibilities of the Borrower were often verified following Bank guidelines. 62. The project implementation period must be seen as two very different periods. The first period from loan approval by the Board in 2004 through 2009 was very difficult for the implementation team which had to face the consequences of a hastily prepared project, with a broad scope, and lacking in strong ownership by some of the MOF Directorates General being reformed. In 2009 the project was fundamentally restructured with the Tax Administration and Customs components being dropped after having practically no disbursement over the period, cancellation of the Budget Preparation, e-Procurement activities and reduction of the scope of intervention in the Fiscal Policy Office. The second period from 2010 to loan closing in 2015 had the implementation team focus on the key objective of improving Treasury operations through the implementation and deployment of SPAN. 63. The following deficiencies were noted during project implementation:

a) Failure to maintain commitment. Supervision missions failed to convince key stakeholders responsible for the implementation of several components and subcomponents to maintain their commitment to the project resulting in cancellations of all activities other than Treasury’s either at the first or the second restructuring.

b) Misunderstanding of Bank procurement guidelines. DG Treasury, DG Tax and DG Customs and Excises had weak capacity to fulfill their implementation responsibilities in project management and procurement management in particular, which often led to misunderstandings of Bank procurement guidelines and conflict with Bank missions. A much improved and successful project management structure was agreed and put in place for the second period of the project. Yet, management of the large SPAN contract was contentious, with MOF and the contractor having conflicting interpretations of contract clauses, which almost led to contract cancellation.

c) Bundling of software and hardware in a single contract. This led to delivery of a technology infrastructure that was insufficient in capacity and near obsolescence. Procurement of a tenfold increase in hardware capacity to overcome problems with initial estimates took a year and a half which contributed to delays.

d) Lack of focus on development impact. Significant deficiencies were also noted in the arrangements for M&E with lack of baseline data and weak systematic monitoring of results. Although the government prepared semi-annual monitoring reports, baseline data were missing for several PDO and intermediate indicators. In fact, aide memoires and regular ISRs paid little attention to the formal tracking of the indicators. Furthermore, the results framework was not re-worked during the first restructuring and no M&E specialist was engaged as part of the restructuring process.

e) Candor and quality of performance reporting. The IP ratings did not accurately and realistically reflect the procurement challenges the project faced during the selection of the SPAN contractor as well as the PMQA consultancy.

f) Failure to address sustainability issues. Post-implementation challenges were well known to the supervision teams but could not be addressed and still remain as problems to solve: (i) instability of ICT staff caused by the government’s HR staff rotation policy; (ii) lack of technical strength to continue improving SPAN toward a full FMIS due to inability to issue international contracts to improve quality of operation, system maintenance, and develop

24

additional functionalities; (iii) lack of technical knowledge to execute professional testing and acceptance of SPAN, which caused significant delays because Treasury management became reluctant to take the responsibility for the action fearing the position of auditors; (iv) weak security management against hacking type of threats; (v) lack of business continuity plans in the case of disasters (Jakarta is considered high risk of natural disasters); and (vi) insufficient ICT infrastructure capacity to deal with monthly peaks in SPAN usage.

64. These issues were identified timely by the supervision team, but were not easy to address. Some issues like the loss of IT skills due to rotation policies and international procurement constraints would require policy changes effected above the level of our direct Treasury counterpart. Other issues were due to faulty project preparation, including lack of planning and support for post-implementation arrangements, as well as insufficient budget resources. These deficiencies were pointed out and were the main reason for the MU rating for the Bank at Entry. Moreover, these deficiencies were difficult to mitigate during the supervision period due to lack of support in the Loan Agreement for policy exceptions (the alternative of having top Bank management to intervene at high levels of the government was not contemplated), and lack of available financing for significant investment in consulting, computer infrastructure, and technical services because loan funds had already been fully committed.

65. Rating of Bank supervision activities is an unusually complicated case: On one hand Bank management allocated a generous amount of resources to the supervision effort and the supervision teams provided significant day-to-day technical assistance that allowed Treasury to complete the implementation of SPAN and put it in operation countrywide. On the other hand it is a legitimate expectation that the lengthy and costly supervision activity would deliver a cleaner outcome. The rating becomes therefore highly dependent on contrasting the weight given to the supervision effort made by the Bank and the direct outcome of SPAN implementation against the weight given to the problems left on the table. The ICR team also considered that at entry and for the first five years of the project, the supervision team was running an APL and therefore, the expectation would be that the second phase of the APL would have taken care of outstanding post-implementation issues. All things considered Bank supervision is rated as Moderately Satisfactory.

(c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 66. The Bank Performance in Ensuring Quality at Entry was Moderately Unsatisfactory and Quality of Supervision was Moderately Satisfactory. Given the overall outcome is Moderately Satisfactory, the Overall Bank Performance for the GFMRAP project is considered to be Moderately Satisfactory with respect to IEG’s Harmonized Ratings Criteria.

25

5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 67. The Ministry of Finance demonstrated a strong commitment to GFMRAP at preparation and implementation. Six Ministers were involved during the life of GFMRAP and all recognized its importance in supporting Indonesia’s transition towards improved public financial management, improved revenue collection, and reduced levels of corruption. A significant effectiveness delay of ten months, mostly due to a large number of difficult effectiveness conditions that had to be addressed by a new government appointed soon after Board approval of the project, reduced the enthusiasm for the project among the core DGs responsible for implementation. Faced with serious project start-up difficulties and delays on the procurement process, the MOF carried out a review of GFMRAP to decide whether or not to continue the implementation of SPAN. As a result, GOI’s interest in the SPAN project was reconfirmed in 2008, and a new and improved Steering Committee structure was put in place. The government demonstrated continued strong support for the project, even when the project had to be restructured twice. 68. After the second restructuring in 2009, special attention was given to the SPAN component. A new directorate of Treasury Transformation was established under DG Treasury to undertake responsibility for the SPAN component of the project, with about 100 dedicated staff. Higher level MOF officials frequently participated in Steering Committee meetings to assess progress and address issues. The MOF Internal Auditor (MOF-Inspectorate General) assisted implementation through a financial integrity review to ensure the reliability of SPAN to produce high quality financial statements and a thorough administrative due diligence through the RTM (requirements traceability matrix), to guarantee that all suppliers’ obligations had been delivered. 69. However, despite these areas of noteworthy commitment, top echelons of the Government, particularly the Ministers of Finance, had mixed success in compelling the core MOF Director- Generals in charge of the execution of the Customs and Tax Administration components to fulfill their responsibilities as defined in the loan agreement and project appraisal documents, which eventually resulted in the cancellation of these activities.

70. In retrospect, GFMRAP’s focus on Treasury modernization and the implementation of SPAN proved to be a very important and sensible decision. Dropping the other GFMRAP components created the space for reallocation of loan funds to finance the higher than anticipated cost for the SPAN contract, which incurred an overrun of about US$17 million mostly to procure additional servers. GFMRAP reached a total disbursement rate of over 99%, of which about 96% corresponded to SPAN implementation. This allowed SPAN to be fully implemented and deployed to 182 local Treasury offices across Indonesia. Nevertheless, it has been noted during the ICR mission that the integration with the budget preparation system is weak, and that Treasury is having difficulties ensuring the long term sustainability of SPAN because: (a) the organizational needs for staff promotion and rotation force DG Treasury to rotate its ICT cadre out of the SPAN unit resulting in a continuous loss of technical expertise and experience; and (b) the government’s

26

procurement framework makes it difficult for the MOF to enter into multi-year contracts with reputable solution providers that are critical for a reliable operation of SPAN.

(b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory

71. As indicated above, the project implementation period must be seen as two very different periods. During the first period from the loan signing in 2004 to the first restructuring in 2009, the project, designed as a complex APL, had many implementation agencies22 involved: FPO, DG Treasury, DG Budget, DG Tax, DG Customs and Excise, Inspectorate General, and Secretariat General. Subcomponent A1-Fiscal Policy Office made some progress, but planned technical assistance was cancelled as FPO decided instead to concentrate on a scholarship program to send its staff to PhD and Masters degree programs both locally and internationally. Subcomponent A2-Budget Reform also made some progress in the implementation of the MTEF and Performance Based Budgeting using government funding, the PFM MDTF, and assistance from other donors. DG Budget requested cancellation of this GFMRAP activity. Subcomponent A3-Treasury Modernization (SPAN) showed good progress, albeit slowly, in the execution of the SPAN procurement before the contract was finally signed in July 2009. Subcomponent B1-Customs Reform was executed for very minimal amount and B2-Tax Administration Reform left the GFMRAP umbrella. 23 Subcomponent C1-Parliament was shifted to being financed under the PHRD grant and the multi-donor trust fund. Subcomponent C2-Tax Court had slow progress. Minor subcomponent C3-Inspectorate General implemented the planned training. Overall, very little was accomplished in this first period of the project with many of the core MOF Directorates cancelling their participation in the project. Accordingly, the performance of the implementing agencies during this period of implementation is considered to be Unsatisfactory.

72. The second implementation period started in 2009 following project restructuring to concentrate primarily on the implementation of SPAN. Three minor activities remained: DPR ICT development capacity verification, FPO scholarship program, and the Tax Court case management system. DPR activities were partly executed but the funding shifted from the PHRD grant until the end of 2012 and then to the MDTF. FPO scholarship program was effectively executed, which provided a model for the subsequent lending operation SPIRIT (Scholarships Program for Strengthening Reforming Institutions Project for Indonesia). The Tax Court requested cancellation because of its lack of procurement success and the component was dropped at the end of 2013. However, the Tax Court benefited from the case management specifications developed by GFMRAP consultants and eventually developed a case management system using government’s own resources. Accordingly, the performance of the three implementing agencies responsible for the three minor components during this period of implementation is considered to be Moderately Unsatisfactory.

73. The concentration of GFMRAP on Treasury Modernization and the implementation of SPAN proved to have been a successful decision with the system fully implemented and deployed.

22 Agency being used here as the MOF Directorate with immediate decision making authority. 23 Activities of this subcomponent were to be financed from DGCE’s own resources. DG Tax asked instead for a separate project and hence APL Phase III was prepared as a new SIL called PINTAR in 2009.

27

SPAN development, including testing, piloting, acceptance, and roll-out, was much more difficult and complex than anticipated as reflected in the Lessons Learned section of the ICR. Contractual disputes almost derailed the implementation effort. An additional two-year extension of the closing date was necessary due to delays. Delivery of an obsolete infrastructure equipment, in spite of conforming to the terms of the contract, by the contractor was another serious source of delays and conflict. Despite the difficulty faced by the team, along the way, all required tests (Unit Test, Integration Test, and User Acceptance Test) were concluded to ensure the reliability of the system in meeting the government requirements. The SPAN deliverable was finally “accepted” in June 2015, and the full hand over from contractor to MOF started soon thereafter, yielding the planned benefits as the government’s core public financial management system. At the time of the ICR, SPAN is being fully used by 182 local Treasury Offices, 33 regional Treasury Offices, and MOF headquarters to serve over 24,000 Spending Units. The launch of SPAN by Indonesia’s President Joko Widodo in April 2015 at the Presidential Palace reflected the importance of SPAN as a landmark achievement for the country to improve its public finance management.

74. Although SPAN has proven to be a comprehensive system to integrate GOI’s tasks of budget preparation, budget allocation, budget execution and reporting in one integrated database system, it has been noted during the ICR mission that the integration between the Treasury and the Budget Preparation module remains a challenge. While Custom Web (for budget allotment process including virements) is being fully used, the future of Hyperion (for budget formulation analytics, versioning and monitoring evaluation) is questionable because of the need for modifications to comply with the continued changes in budget formulation business processes.

75. In spite of its implementation success, Treasury faces difficulties ascertaining the long term sustainability of SPAN because: (i) the government’s HR staff rotation policy makes it difficult for DG Treasury to retain trained staff with technical knowledge and experience in SPAN; and (ii) the government’s procurement framework makes it challenging for the MOF to continue upgrading its ICT infrastructure and enhance existing SPAN applications (including the improvement of quality of operation, system maintenance, and development of additional functionalities) all of which are impossible to achieve without the procurement of highly qualified international contractors at international salaries, in foreign currency, and multi-year contracts. DG Treasury is seeking support from the PFM MDTF for immediate needs and considering a possible new loan to cover the long term needs. Accordingly, the performance of the implementing agency, DG Treasury, during this period of implementation is considered to be Moderately Satisfactory.

76. In summary, the implementation performance of the three minor implementing agencies, namely DPR, FPO, and Tax Court in the second period of implementation was Moderately Unsatisfactory. The performance of the implementing agency DG Treasury in the second period of implementation was Moderately Satisfactory. Considering that DG Treasury’s component was critical to the achievement of the planned revised PDO’s and corresponded to 92% of disbursements, the overall implementing agency performance for the second period is rated Moderately Satisfactory.

28

Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 77. As Government Performance was Moderately Satisfactory, Implementing Agency Performance during the first period was Unsatisfactory, and during the second period was Moderately Satisfactory, the Overall Borrower Performance for the GFMRAP project is considered to be Moderately Satisfactory with respect to IEG’s Harmonized Ratings Criteria.

6. Lessons Learned

78. Context. There have been three presidents, six finance ministers, and four World Bank task team leaders over the life of GFMRAP. Successful completion of this 12-year operation is owed singularly to top management commitment with a collective belief that failure would not be an option, and to the PFM MDTF that financed extensive and remedial technical assistance that DG Treasury and DG Budget required for effective execution of the project. Even with high staff turnover, project champions remained stable with the same head of PSSU since project inception, stable management of SPAN in DG Treasury, and stable core specialists at the Bank, which is unusual in Bank projects globally. Lessons from the GFMRAP operation are viewed from the project design and implementation aspects and may be generally applicable for similar operations.

6.1 Design

79. Scope of Design. The APL was not an appropriate instrument given treasury, tax, and customs are very distinct systems operating under different political economy contexts. GFMRAP was designed as an ambitious multi-stakeholder multi-component APL operation. At preparation, the APL instrument was considered necessary to lock in the entire scope of the government’s PFM reform program as requested by GOI. All potential stakeholders were allotted a seat at the onset of the envisioned three-phased GFMRAP operation to provide the Bank with a platform for dialogue. The inclusion of all stakeholders, totaling nine PIUs, was cumbersome and led to a sluggish decision-making process. Supervision of nine PIUs and their programs proved to be a challenge. Eventually, 4½ years later and faced with severe execution problems, GFMRAP was streamlined to concentrate primarily on the implementation of SPAN, managed by a single directorate general. The Tax Administration component was developed as a SIL (PINTAR), during the second year of implementation of GFMRAP. The Customs Administration decided to carry out its modernization separately with its own resources. It is recommended that reforms of revenue and expenditure agencies be executed separately, each supported by specialized Bank staff, particularly when complex reforms are envisaged.

80. Implementation Timing. Front-load procurement activities prior to Bank approval to speed up procurement implementation. Approval of bidding documents, including all technical specifications, for the core IT contract should be a condition for going to the board when the contract is planned for the first year of implementation. Funding for the preparation of the bidding documents could be financed either through the state budget or through an RETF set up to support the project. Average time to complete a World Bank FMIS projects is 7.9 years with the longest taking over 13 years. Phase I of GFMRAP was planned for 4½ years of implementation, but it

29

took that long just to procure SPAN. This prolonged procurement time was not anticipated. In addition to the need for ensuring accuracy and completeness of the technical requirements before bids are invited, it is also essential, particularly in the case of large and complex contracts such as SPAN, that a project procurement strategy be devised early in the project preparation process to form the basis for putting in place fit-for-purpose procurement arrangements to meet the project’s needs and objectives and help achieve value for money. The strategy would guide the most suitable market approach, selection method, and selection arrangement for the particular contract, based on the operating context, market analysis, the implementing agency’s procurement and contract management capacity, and risks. For example, in large and complex contracts, this could justify the use of non-traditional selection arrangements, such as Competitive Dialogue, where appropriate. Neither was the fact that in an environment where the business processes were in a state of change, it will take longer to reach an agreement on the technical requirements. While the procurement contract required both parties to the contract to agree on the “freeze,” but with a number of procedural and policy changes happening at once (performance-based budgeting, accrual accounting, electronic system of state revenue), the government might have been reluctant to agree on a deadline to stop system requirements. The extent of the customization of the COTS was not originally envisioned and the volume of changes contributed to the delays seen in the project. Once it was clear that evolving changes of business process improvements would take more time to implement, the length of the programming (or application development) should have been adjusted. Customizing an off-the-shelf solution to support the complex and evolving requirements of treasury and budget preparation of a large country like Indonesia takes a significant amount of time. Testing, piloting ensuring the accuracy of accounting, and finally deploying the system was equally a complex and lengthy process, taking almost two years. As a result, SPAN implementation took about 10½ years from bid preparation to deployment. It is recommended that future FMIS projects be realistically programmed in terms of time span and fully considerate of Bank experience.

81. Change Management. If the change management consultancy was deployed as planned to support the entire GFMRAP operation, relevant stakeholders would have been better prepared for the project. A change management consultancy was planned to support the entire GFMRAP and it was appropriately included as an effectiveness condition. Its goal was to establish a two-way flow of information throughout the project cycle. The initial consultancy for the entire three-phased program of GFMRAP was never procured and the conditionality waived. As a result, no change management initiative supported the early management of SPAN within the MOF. A consultancy was later brought onboard for a limited duration but was not extended when SPAN development and deployment was delayed. This consultancy was not well utilized and the contract was discontinued, which coincided with the delay of the development of software deployment. It is likely that delays in piloting, user readiness, and roll out of SPAN by DG Treasury could have been better mitigated if the activities were supported by experienced change management support. It is recommended that (i) the Bank makes a greater effort to convince governments of the need for a substantial investment in change management to support major institutional reforms; (ii) that Change Management includes the establishment of capacity and resources to implement and sustain the reforms; and (iii) that the Bank insists that Change Management is in place early on to provide cradle to grave support to the transformation process.

30

6.2 Implementation

82. Championing Modernization. Early establishment of a dedicated structural unit within the Directorate General of Treasury to champion the modernization effort was key to project success. The roles and responsibilities of government officials assigned to this unit were geared towards developing, testing, and rolling out the SPAN system. Unit staff had no other duties besides implementing SPAN, a departure from normal Bank projects managed by civil servants who are often part of the PIU but still have regular functional responsibilities. Unfortunately, this was not the same for other DG units. For example, the Directorate General of Budget did not establish such a unit dedicated to modernization, which could be a contributing factor for why the budget module of SPAN is still not fully functional.

83. Overcoming Knowledge Gaps. The establishment of the PFM MDTF provided the additional fund to bridge knowledge gaps. GFMRAP charged the MOF with significant responsibilities in the preparation of policies and regulations for the implementation of SPAN. MOF did not lack commitment to improving financial management or willingness to fulfill its responsibility, but rather, lacked knowledge and experience in implementing an integrated financial management. However, this knowledge gap was successfully overcome thanks to a trust fund created specifically to support the implementation of the GFMRAP. The PFM MDTF was established in 2007, more than two years after the GFMRAP loan was signed. It augmented the loan resources through grant funds to retain full-time senior experts and several overseas study tours specifically to gain FMIS knowledge to support the government in preparing the necessary policy options related to adopting an FMIS and developing the SPAN system itself. The PFM MDTF is still ongoing and has proven to be critical to GFMRAP’s success.

84. Bundling Application Software and Hardware. For SPAN it would have been better to sequence the delivery of hardware after the assessment phase of the application software was completed. GFMRAP procurement plan called for the procurement of SPAN in a single package that included the design, customization, development, installation, and implementation of a very complex application and delivery of the hardware infrastructure to support the application. The procurement document was based on requirements prepared in 2004 for functionality and transaction load. The MOF did its best to provide accurate information to the Bank but that information originated from weak legacy systems and staff with little knowledge of full-fledged FMIS. Nevertheless, the SPAN contract was designed as a turnkey solution in 2004 with the contractor providing hardware, the application software, and networking solutions. Once the SPAN contract was signed, hardware was delivered. By the time the application software was deployed four years later, the hardware was already obsolete. Moreover, the hardware infrastructure capacity had to be increased tenfold due to severe underestimation of the number and complexity of transactions during preparation of the technical requirements and the subsequent functionality creep-up. It is recommended that in the case of complex applications, which may take a year or more to develop or customize, and considering that hardware technical depreciation occurs in about four years, the delivery schedule of the contract should take into account the need for sequencing the supply of application software before hardware infrastructure so that a more accurate estimation of capacity and more up-to-date technical specifications can be used, thus minimizing total cost.

31

85. Avoiding Procurement Conflicts. Increase training of state auditors on Bank procurement to reduce risk of conflicting interpretations. Procurement policies of the Bank and the Borrower are often not aligned resulting in procurement gridlocks and project failures. GFMRAP design included a good procurement framework, but many times during implementation it conflicted with the country procurement system. On the single-sourced consultancy package in the procurement of a Case Management system for the Tax Court, auditors from the Finance and Development Supervisory Agency (BPKP) disregarded the Bank’s advice and no-objection, thus signaling that country procurement rules overrode Bank procurement guidelines. This component of the project was eventually cancelled due to this procurement failure. Throughout project implementation, all GFMRAP procurement evaluation committees were very conservative because of the personal liability of committee members on procurement decisions, which in turn led to significant delays in implementation, disqualification of potentially well qualified bidders, and re-do of the second stage of procurement. Procurement issues dominated the first five years of the GFMRAP and potentially could have resulted in the operation being cancelled had it not been for MOF top management intervention. The risks under such operating context could have been better recognized and addressed during project preparation if a project procurement strategy had been prepared. It is recommended that the Bank work proactively with the government and the auditors through training to help understand the full scope of Bank procurement guidelines to avoid delays and potential disqualification of qualified bidders as a result of differing interpretations. Over the last two years, the Indonesian team has been conducting a procurement capacity building program for national auditors.

86. Planning for Sustainability. All projects should build in an exit strategy at design stage to ensure project sustainability post implementation. GFMRAP PAD design for the section on “Change Management, Monitoring, Oversight and Sustainability” was silent on Sustainability Planning and no technical assistance was planned to assist the Borrower in the preparation of such plans. The PMQA consultancy developed a post-implementation report that included a sustainability plan. The plan confirmed findings of the ICR mission that the following is lacking: (i) information security management against hacking; (ii) human resource management plan to retain knowledge and experience; and (iii) the ability to sign multi-year contracts with key technology providers. It is recommended that in the case of complex applications the sustainability issue be well covered in the PAD and that resources be provided in the project for the preparation of a sustainability plan. Moreover, consideration should be given to include in the Loan Agreement conditions to make it feasible for the implementing agency to put in place said plan. It would have helped if the Loan Agreement would have required that the counterpart funding would include not only the maintenance (which it did), but also the operational support and system enhancement in place after delivery through multi-year funding support.

87. Adaptive Learning. Project teams should anticipate the need to be adaptable, and to redirect resources to address newly emerging threats to project outcomes. Changing political economy and client capacity to manage the project cannot always be accurately anticipated – especially years in advance. GFMRAP was able to achieve the project development objectives by cancelling portions that had limited support, and refocusing resources to areas that had stronger project champions. The Bank mobilized the PFM MDTF and built flexibility into the trust fund itself in order to leverage additional policy reform instruments such as DPLs and to offer just-in-time consulting services to plug knowledge gaps on the part of the Treasury.

32

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

88. The PSSU provided a full government completion report of the GFMRAP operation. This is included in Annex 8.

33

Annex 1. Project Costs and Financing (a) Project Cost by component

Component IBRD 4762 IDA 4026 PHRD TF053556

PFM MDTF TF09004724 Total

US$ US$ (equiv) US$ US$ US$ A. Public Financial Management A.1 Policy Capacity Development 757,320 851,487 1,608,807

A.2.a Budget Planning and Development (Bappenas)

98,561 91,404 187,764 377,729

A.2.b Budget Planning and Development (DG Budget)

138,521 58,438 226,708 423,666

A.3 Budget Execution and Treasury Modernization

51,608,637 1,639,761 4,445,103 1,002,110 58,695,611

A.4 Procurement Reforms 14,700 61,335 76,035

B. Revenue Administration B.1 Customs Modernization 11,651 42,251 53,902

B.2 Tax Revenue Administration 204,104 204,104

C. Governance and Accountability C.1 Legislative Oversight 153,458 344,993 498,451

C.2 Resolution of Tax Dispute 526,375 56,997 583,372

C.3 Internal Accountability 277,845 277,845

D. Project Governance and Implementation D.1 Project Governance and Accountability 893,676 1,420,875 544,820 2,859,370

Refund of Prefinancing Advances

616,095 616,095

Front-end Fee 275,000 275,000 Total Disbursements25 54,414,161 4,969,772 4,598,561 2,567,494 66,549,988

24 An amount of about US$1.2 million under TF090047, one of the PFM MDTF child trust funds (CTF), is not included in this table. This amount was allocated to support Pusintek, which was not part of the original scope for GFMRAP. Also not included here is an amount of US$1.06 million disbursed under TF091414, another one of the PFM MDTF CTFs. This CTF supported the establishment of a national single window that was implemented by the Coordinating Ministry for Economic Affairs, and was also not part of the original scope for GFMRAP. Both CTFs were administratively linked to the GFMRAP operation and were executed by the government. There were a number of other child trust funds under the PFM MDTF umbrella that were not directly linked to GFMRAP and were executed by both the government and the Bank. These funds provided additional support to the government’s broader PFM reform agenda beyond the scope of the GFMRAP. 25 The Bank disbursed a total of US$54,998,965 of the IBRD fund to the government’s special account. Actual spending by the government was US$54,414,161 as recorded in the table. The government is in the process of refunding the unspent balance to the Bank.

(b) Financing

Source of Funds Appraisal Estimate (US$ or equivalent)

Actual/Latest Estimate (US$)

Percentage of Appraisal

Borrower 15,000,000 30,000,000 200.00 IBRD 4762 55,000,000 54,414,161 98.93 IDA 4026 5,000,000 4,969,772 99.40 PHRD TF053556 5,000,000 4,598,561 91.97

PFM MDTF TF090047 TF not available at appraisal 2,567,494

Total 65,000,000 96,549,988 148.54

35

Annex 2. Outputs and Activities by Components There were 30 original intermediate outcome indicators26 listed in Annex 3 of the PAD but only 15 in the loan agreement (14 in Schedule 6 plus one implied in Schedule 2 as part of the project description). This ICR will address these 15 indicators listed in the loan agreement to assess the overall efficacy of the GFMRAP program. Intermediate Outcome Indicators

• Component A: Public Financial Management o A.1: Policy Capacity Development

Survey feedback (including from Minister) verifies the completion of at least four high-quality policy analysis products in the priority areas (tax policy, trade policy), published on the web site (silent in Schedule 6 of the loan agreement, though “Strengthening the policy research capacity of MOF was mentioned in Schedule 2 of the loan agreement)

o A.2: Budget Planning and Development First Medium Term Expenditure Framework endorsed by the

Government of Indonesia and forward estimates presented to the DPR

A comparative study of pilot and non-pilot line ministers (via circulars) verifies progress in reform agenda (silent in the loan agreement)

o A.3: Budget Implementation and Treasury Modernization SPAN Stage I (as such is described in the invitation for bids for the

contract for the procurement thereof) evaluated with user input with respect to content, timeliness and verifiability; specified periodic financial reports prepared for concerned entities

A Treasury Single Account (TSA) system functioning with acceptable parameters: all on-budget central government bank accounts and own-revenue bank accounts of line ministries are brought into the TSA system

Central Government financial statements meet Government Accounting Standards as promulgated by the MOF, and the consolidated management reports produce by the DG Treasury include financial assets and liabilities, and cash-based reports of extra-budgetary funds

A survey of end-users of SPAN verifies increasing knowledge (functional and technical) and satisfaction with the system (the system is being used to inform planning and decision-making). The system enables line ministries and other users of treasury resources

26 Schedule 6 of the loan agreement refers these Intermediate Outcome Indicators as Performance Indicators. All intermediate indicators in this section are listed with a remark if they are silent in the loan agreement. Some of the indicators listed are worded differently in the loan agreement and the PAD. This list retains the wordings as in the loan agreement.

36

to access and process financial information, and compile reports on their operations (silent in the loan agreement)

o A.4: Procurement Reform An evaluation of the MOF e-procurement pilot is completed and an

action plan is adopted by the MOF for wider rollout A survey of procurement prices relative to international benchmarks

is completed Percent of MOF procurement of common office supplies (by value)

through e-procurement systems (silent in the loan agreement)

• Component B: Revenue Administration o B.1: Customs Modernization

The DGCE has adopted a comprehensive and consultatively prepared modernization action plan with inputs from stakeholders

The DGCE has institutionalized an annual survey of stakeholders, and annually published a summary of the results

The DGCE has implemented Web-based filing for about 75 percent of trade transactions

Additional revenue from compliance activities (excluding additional revenues recovered of post clearance audits) expressed as percent of total revenue collected (silent in the loan agreement)

The DGCE has reviewed the consultants’ report on the integrated Customs information system, and adopted an action plan to implement the key recommendations arising from the report (silent in the loan agreement)

o B.2: Tax Administration The DG Tax has adopted a Results-oriented Modernization Strategy

and Plan with measurable performance indicators, aimed at the eventual redesign of processes along single taxpayer account lines and based on the results of the pilot re-engineering of one representative district office

Revenue growth from Large Taxpayer Office (silent in the loan agreement)

Survey instrument institutionalized and administered, and summary of the results published (silent in the loan agreement)

• Component C: Governance and Accountability

o C.1: House of Representatives (DPR) Survey verifies strengthened capacity of the Budget Committee of

the DPR Increased transparency of Budget Committee/Commissions’

functioning through telecast of proceedings and other broadcast programs – consistent increase in viewer numbers and satisfaction (silent in the loan agreement)

o C.2: Tax Court

37

Annual surveys of stakeholder satisfaction with Tax Court administration and case management have been established; an action plan has been adopted to address systems weaknesses (the second part was silent in the PAD)

Tax case decisions are accessible to the public through the Tax Court website

Reduction in time required to process and resolve revenue disputes (silent in the loan agreement)

A baseline survey of tax-payer satisfaction with tax court administration and case management is completed by December 2005 (silent in the loan agreement)

Percent of Tax Court decisions published online (silent in the loan agreement)

o C.3: Inspectorate General of MOF (silent in Schedule 6 of the loan agreement though strengthening “the Office of the Inspector General of MOF” was mentioned in Schedule 2 of the loan agreement) An investigation unit is established and fully operational in the IG

MOF, vested with the necessary legal powers to fully investigate allegations of corruption against MOF staff (moved to Component D in the loan agreement)

Number of investigations initiated per annum (silent in the loan agreement)

Number of investigations with a successful outcome (censure, administrative action, penalties or prosecution) as a proportion of the total number of investigations (silent in the loan agreement)

• Component D: Project Governance and Implementation

o Periodic reviews verify that: (i) Project governance and all project activities are consistent with the Governance and Accountability Action Plan (GAAP), and project procurement is consistent with the Procurement Plan; (ii) the Project Steering Committee (PSC) is strengthening project governance and oversight; (iii) an investigation unit is established and fully operational in the Office of the Inspector General of MOF, including the establishment of the necessary legal powers to investigate fully allegations of misconduct and/or corruption against MOF staff; and (iv) project monitoring and evaluation information (including stakeholder/user feedback survey results) is publicly available (this last part was silent in the loan agreement)

Achievements by PDO Indicators27 PI-1: National government policy priorities are reflected in a Medium Term Expenditure Framework and annual budgets

27 There are two parts to the numbering format of the PDO and intermediate indicators separated by a hyphen. The first part PI or II signifies PDO Indicator and Intermediate Indicator, respectively, and the second part is the indicator number.

38

In June 2009, DG Budget and Bappenas jointly launched a new manual on MTEF and PBB that provided guidance to line ministries on new formats for submission of work plans and annual budget documents. Since then, MTEF was implemented with temporary set ceilings. For FY2011, budget was formulated in the medium term revised program structure with measurable results and targets that were aligned with organizational structure in the National Medium Term Development Plan (RPJM) for 2010-2014. There has also been some progress on improving the transparency of the MTEF process. A table is shown in the financial note of the state budget setting out, at a high level, how the budget compares to the forward estimate from the prior year’s financial note. In the 2013 financial note there were significant details provided on both the methodology used to determine bottom up costings and medium-term projections by both ministries and functions. It would be beneficial, from both transparency and a budgetary control perspective, in providing more detail of the medium-term plan to show the multi-year costs of major policy changes that would be introduced in the annual budget and to re-introduce the level of details on methodology and departmental ceilings provided in 2013. MOF issued its regulation No. 143/2015, which will “obligate” line ministries to submit their 3-year forward estimates along with the annual budget proposals. This 3-year forward estimate is a rolling plan to be updated every year to fit with the current macro-economic assumptions and the latest national government policy priorities set in the annual government work plan (RKP). This new regulation will increase the compliance of the line ministries in submitting MTEF information to be included in the financial note documents. PI-2: Reduced leakage in expenditure flows to end-users as measured by Public Expenditure Tracking Surveys GOI completed the implementation of the Treasury Single Account towards achieving consolidation of all government bank accounts. The regulations issued in 2007-2009 authorized MOF to bring all accounts held by line ministries in various commercial banks into the TSA held at Bank Indonesia. TSA is now operational in all MOF Treasury offices, with all state receipts deposited in and all state expenditures withdrawn from the TSA, through electronic transfer to ensure reduction of leakage in expenditure flows to end-users. In April 2012, the World Bank and MOF jointly released a report from a study on public expenditure tracking survey (a DIPA tracking study). This study aimed to support the MoF in identifying new reform initiatives to improve budget execution, and to reduce leakage in expenditure flows to end-users. The study included surveys to gather information from key stakeholders, such as spending units, the local treasury offices (KPPNs), and contractors. Policy recommendations were formulated by drawing upon the analysis from the field surveys as well as broader analysis undertaken by the WB Jakarta. Based on the findings, the government issued two regulations: (i) Finance Minister Regulation 190/2012 requiring DG Treasury to process all requests for payment in one day at maximum; and (ii) Government Regulation 45/2013 requiring the payment of contractor’s invoices within a maximum of thirty (30) calendar days after the invoice supported by proof of delivery is received. At the end of 2015, DG Treasury signed an MOU for Technical Assistance with the US Treasury Office. The US Treasury OTA (in collaboration with the World Bank Jakarta team) will assist MoF on: (i) a study of electronic money (debt card/credit card/mobile phone account bank/virtual account) for government expenditure payments both to the suppliers and individual social aid

39

beneficiaries; and (ii) a review of the government’s policy, strategy, and progress with regards to implementing a Treasury Notional Pooling (TNP). The results of these planned studies are expected to (i) further reduce the leakage in government expenditure flows to the end-users by limiting “cash in hands” of the spending unit; and (ii) support a less cash society. PI-3: Automated treasury payment system enables accurate and timely financial reporting, reduces incidence and size of idle cash balances, and reduces corruption in payments This indicator represents the core of the GFMRAP operation. The completion of SPAN, an automated treasury payment and budget preparation system, signifies a major achievement, which also underscores the difficulties and complexities of this type of interventions and that it takes longer to complete than other WBG operations. The SPAN system is meant to cover not only transactions on the budget execution side, but the entire budget cycle from budget planning to reporting. SPAN, as a crosscutting reform initiative, is expected to increase the timeliness, reliability and transparency of budget disbursement and financial reporting as well as improve the capability of the government to efficiently manage public resources and reduce corruption in payments. The design and implementation of an automated treasury payment and budget preparation system (SPAN) by a turnkey contractor commenced in July 2009. Benefits of FMIS such as SPAN are well documented. They include: (i) improved business processes with fully automated operations to effectively control budget allocation, expenditure commitments, and spending limits; (ii) availability of centralized databases; (iii) reduced human errors and potential frauds through data capture at source; (iv) comprehensive reports on government financial transactions that are generated online, with real-time reporting and full accrual accounting; (v) improved cash flow forecasts and planning with a reliable cash management system; (vi) audit trail capability; and (vii) efficient submission and access of public finance information. The design and development of SPAN by a turnkey contractor commenced in July 2009. The development of the SPAN was completed, including user acceptance testing (UAT), at the end of 2013. Additional time was needed to pilot and gradually roll out the SPAN system to all users’ locations across Indonesia. Roll-out of SPAN was completed by the end of Feb 2015. SPAN was launched by the Indonesia’s President Joko Widodo in April 2015. Final payment for SPAN was made in June 2015. SPAN is now being used by the Ministry of Finance at DG Budget and DG Treasury headquarters, in addition to the 182 Treasury local services offices (KPPNs) and 33 Treasury regional offices (Kanwils) all over Indonesia serving over 24,000 spending units of the central government line ministries. SPAN is managing 100 percent of all government financial transactions, using electronic transfer as the method of payment with less than 1 percent payments made through petty cash for office operations. No other cash payment is processed. Achievements by Intermediate Indicators There were 30 original intermediate outcome indicators listed in Annex 3 (Results Matrix) of the PAD. Only 15 of these were included in the loan agreement dated Dec 22, 2004 (14 in Schedule 6, which refers these Intermediate Outcome Indicators as Performance Indicators, plus one implied in Schedule 2 as part of the project description). GFMRAP underwent two restructuring processes with the first a Level One Restructuring that was approved by the Board in May 2009 with new project closing date set to Dec 31, 2013. The second restructuring effort was a simpler Level Two Restructuring approved by the Board on December 20, 2013 primarily to extend the final closing

40

date to Dec 31, 2015. The scope of the GFMRAP was significantly reduced as the result of project restructuring. The entire Component B related to revenue administration was dropped as the result of the first restructuring in May 2009. Since negligible amount, less than US$54,000, was disbursed to Component B, the corresponding 4 indicators for Component B became irrelevant in the assessment of results of this GFMRAP project. Accordingly, only 11 of the 15 intermediate indicators will be assessed in this ICR. Component A: Public Financial Management Sub-Component A.1: Policy Capacity Development II-1: Increase policy development capacity within the Fiscal Policy Office as verified by the completion of several high-quality policy analysis products Formulation of this indicator is based on the Amendment dated May 22, 2009 for the first project restructuring. The original formulation, which is silent in the original loan agreement dated Dec 22, 2004, but included in the Results Matrix (Annex 3) of the PAD, states: "Survey feedback (including from Minister) verifies the completion of at least four high-quality policy analysis products in the priority areas (tax policy, trade policy) published on the web site." As the project progressed, GFMRAP funding for advisory technical assistance was shifted to support post-graduate scholarships. There have been indications that BKF through PhD and Masters scholarship programs funded by GFMRAP have contributed a lot in the policy development capacity within the BKF. Some staff who completed the scholarship programs were promoted into higher ranked positions, more research papers were produced with high quality of policy analysis. The MoF and other line ministries have utilized these papers for policy decision in strategic area. Each scholars was required to produce a quality policy analysis product at the end of their study. Accordingly, more than 4 targeted policy analysis products were achieved. Sub-Component A.2: Budget Planning and Development II-2: First Medium Term Expenditure Framework endorsed by the Government of Indonesia and forward estimates presented to the DPR This indicator was dropped per the first project restructuring in 2009. PEFA PI 12 is a proxy to measure overall budget information transparency and the use of MTEF. Since the 2011 PEFA review, there has been some progress on improving the transparency of the MTEF process. A table is now included in the financial note showing forward budget estimate from prior year including the methodology used to determine the bottom up costings and medium-term projections. There has not been a PEFA review since 2011. MTEF and PBB were introduced with forward estimates included for the first time in the FY2012 budget documents, decrees issued to streamline budget documents, and treatment of performance based budgeting’s new initiatives within the budget. However, there remains considerable development and refinement required over the next few years. Sub-Component A.3: Budget Implementation and Treasury Modernization II-3: SPAN Stage I (as such is described in the invitation for bids for the contract for the procurement thereof) evaluated with user input with respect to content, timeliness and verifiability; specified periodic financial reports prepared for concerned entities Rollout of SPAN was completed as of the end of February 2015 to all 222 locations, including 181 treasury branch offices, 33 provincial treasury offices, and 8 treasury directorate units across

41

Indonesia. SPAN now manages 100% of all financial transactions of over 24,000 government spending units across Indonesia. FOA was issued and SPAN contractor completely paid. On Nov 11, 2015, a SPAN conference was conducted to (i) heighten awareness of SPAN within and beyond MoF to maximize usage and system potential; (ii) provide a platform for knowledge exchange on Indonesia's FMIS implementation and its relationship to broader PFM reform; and (iii) exchange ideas on how to build on the SPAN to improve government effectiveness in service delivery. MoF-IG also conducted thorough administrative due diligence through the RTM (requirements traceability matrix) in which the actual deliverables from the supplier were compared with the initial requirements stated in the bid documents. With the final payment made in June 2015, which marked the complete system delivery, there is no other supplier’s obligation remained. II-4: A Treasury Single Account (TSA) system is functioning with acceptable parameters: all on-budget central government bank accounts and own-revenue bank accounts of line ministries and spending units are brought into the TSA system TSA enforced with consolidation of government funds with no material exceptions. TSA for expenditure has been in operation since October 2007. MoF has continued the expansion and extension of the TSA, and it now covers all expenditure accounts of all treasury offices in the country. Cash management reform is one of the PFM reform pillars in Indonesia. Cash balances are consolidated in a TSA. The result has been lowered financing costs and improved control of both revenue and expenditures. While significant progress has been made, DG Treasury is still facing several challenges, mainly in terms of how to optimize idle cash, integrate system environment to ensure comprehensiveness and accuracy of cash forecast, manage the electronic money products, and establish an institutional link between cash and debt management. II-5: Central Government financial statements meet Government Accounting Standards as promulgated by Government Regulation and the consolidated management reports produce by the DG Treasury include financial assets and liabilities, and cash-based reports of extra-budgetary funds Preparation for accrual accounting started in 2010, which included regulation issuance, capacity building, system development, and transitioning from cash to accrual based accounting processes. The MoF is in the process of implementing full accrual accounting in 2015, which is mandated by the state finance law (No. 17/2003). The Government Financial Report for 2015, which will be audited in mid-2016, will be full accrual-based accounting. Sub-Component A.4: Procurement Reform II-6: An evaluation of the MoF e-procurement pilot is completed and an action plan is adopted by the MoF for wider rollout This indicator was dropped with the corresponding subcomponent closed on Jun 30, 2009 as the result of the first project restructuring in 2009. Bappenas, which was the initial implementing agency for this subcomponent withdrew from GFMRAP in April 2008. Subsequent implementing agency became the newly established National Public Procurement Office (LKPP), the agency tasked to develop the regulatory framework, including e-procurement system, for public procurement in the country. After restructuring, support to this sub-component with LKPP as the counterpart/implementing agency has been under the PFM MDTF. At GFMRAP closing, an e-procurement system, which was developed by LKPP for mandatory use by all government

42

agencies, is being implemented for MoF. An Echelon 2 unit responsible for e-procurement was established under the MoF Secretary General. II-7: A survey of procurement prices relative to international benchmarks is completed This indicator was dropped with the corresponding subcomponent closed on Jun 30, 2009 as the result of the first project restructuring in 2009. Component B: Revenue Administration Sub-Component B.1: Customs Modernization Intermediate indicators II-8 to II-10 under this subcomponent were dropped as the result of the first project restructuring in 2009. Sub-Component B.2: Tax Administration II-11: The DG Tax has adopted a Results-oriented Modernization Strategy and Plan with measurable performance indicators, aimed at the eventual redesign of processes along single taxpayer account lines and based on the results of the pilot re-engineering of one representative district office This indicator was dropped with the corresponding component closed on Jun 30, 2009 as the result of the first project restructuring in 2009. This sub-component to support DG Tax spun off as a SIL operation named PINTAR that came on line in the last quarter of 2009. PINTAR was eventually cancelled in 2014 due to negligible disbursement. After GFMRAP was restructured, DG Tax continued to receive technical and capacity building support under the PFM MDTF towards its second wave of tax administration reforms focusing on improving taxpayer services, increasing taxpayer enrollment, providing support for ICT, and increasing taxpayer compliance as part of the overall implementation support to PINTAR. Tax administration reform remains a priority area for the government and the WBG continues to engage through the on-going PFM MDTF program and the programmatic fiscal DPL operation. Another investment project is in the pipeline as well. Component C: Governance and Accountability Sub-Component C.1: House of Representatives (DPR) II-12: Verification of strengthened basis for ICT development in the DPR This indicator was from the Amendment dated May 22, 2009 per the first project restructuring. The original indicator stated: "Survey verifies strengthened capacity of the Budget Committee of the DPR." This indicator was dropped on Dec 31, 2013 as a result of the second project restructuring. After a reprogramming of this subcomponent in 2007, most of the funding was shifted from the PHRD grant to the PFM MDTF. Focus of activities also shifted to building research capacity to support parliamentary budget review, capacity building in budget analysis, and strengthening of administrative support functions by the DPR Secretary General’s office. When GFMRAP was re-structured in June 2009, the remaining activities budgeted under the PHRD grant were further reprogrammed to focus on two small consultancies related to ICT development and two related to the strengthening of the administrative support function within the DPR Secretariat General. However, the two ICT related consultancy activities were eventually cancelled because the

43

procurement activities could not be completed in time to meet the closing date of the PHRD grant on December 31, 2010.

44

Annex 3. Assessment of Achievements by Objective

PDO outcome indicators Baseline Target Actual Comments Objective 1: To improve efficiency in the Borrower’s public financial management (Rating: Substantial) 1. Automated treasury payment system enables accurate and timely financial reporting, reduces size of idle cash balances, and reduces incidence of corruption in payments

1.1 Transparency and duration of payment processing time for supplier invoices 1.2 Payroll slips issued by system through direct payments 1.3 Cost savings from paperless printing, phone/fax communications, and courier fee

No time limit to process payment invoices Manual issuance of checks Manual checks and paper transactions

Less than one day (1-2 hours) to process payments in KPPN Direct deposit Direct deposit and electronic transactions

Achieved. • Various reforms under GFMRAP

have significantly contributed to timely payments, which has discontinued “illegal fees” that were given to expedite payments.

• SPAN introduces new workflow

processes (front/middle/back offices) so that once a payment request is processed, the status of the request can be known, thereby eliminating the justification for “fees” to expedite payments.

• $4 million worth of checks issued through system and suppliers’ checks made payable direct to bank accounts of 300,000 suppliers

• Salaries of 1.62 million government employees paid through direct electronic bank transfer

• $55 million saved in printing and communications costs. OM-SPAN is an online monitoring system for payment transactions that can be accessed to check on status on a real-time basis

2. Strengthened cash management

2.1 A Treasury Single Account (TSA) system is functioning with acceptable parameters: all on-budget central government bank accounts and own-revenue bank accounts of line ministries and spending units are brought into the TSA system

2.2 Consolidation of cash balances

A baseline of >18,000 government cash operations with no cash management standards IDR 3.6 trillion in daily cash balance held by KPPN and Kanwil

TSA operational, government funds consolidated with no material exceptions

TSA has been fully implemented since FY 2009 2% current account deposit rate on IDR 3.6 trillion saved

Achieved.

• Full interface that integrates expenditure (SPAN) and revenue collection (MPN) data from all 182 KPPNs provides full picture of the government’s cash position on a real time basis

• Off-budget accounts have been

brought into compliance • Existing miscellaneous accounts of

line ministries have been closed • New accounts can only be opened

after MOF authorization

45

PDO outcome indicators Baseline Target Actual Comments 2.3 Savings from TSA 2.4 Full picture of government’s cash position from central government treasury offices captured on real time basis

No TSA in 2008 Multiple sources consulted to obtain cash position

TSA established in 2009

$377 million USD saved as a result of income generated from TSA Full picture of cash position on real time basis

• $300 million of interest earned on consolidated cash balance centrally held in Treasury Single Account at the Central Bank

• $47 million saved from revenue daily sweep into Treasury Single Account to reduce idle floating cash previously held in commercial banks

• $30 million saved from consolidated cash accounts previously held in commercial banks

• TSA covers all expenditure accounts of all treasury offices in the country

• Daily cash balance of IDR80 trillion • Daily sweeping of revenue accounts

into TSA fully implemented • TSA for revenue developed (2010)

Objective 2: To improve governance in the Borrower’s public financial management (Rating: Modest) 3. Improved legislative scrutiny of the annual budget law

3.1 Improved capacity of members of Parliament in providing oversight over the budget formulation process 3.2 Enhanced analytical support during budget formulation process to members of Parliament

No legislative scrutiny of annual budget law

Strengthened legislative oversight of annual budget law

Achieved. • The review undertaken by

Parliament covers macroeconomic framework, main fiscal policies and expenditures and revenues. A detailed discussion of the annual work plans of line ministries and AGAs takes place directly with the relevant parliamentary sectoral budget commissions.

• The budget review is undertaken over a period of about seven months. Parliament has some 8-10 weeks (compared to 6-8 weeks before) to review the draft budget once it is tabled in mid-August until it is formally adopted in October

4. National government policy priorities are reflected in a Medium Term Expenditure Framework and annual budgets

4.1 Medium-term budget forecasting tools 4.2 Change in budget classification

No MTEF in place

MTEF in place

MTEF in place but poor reliability of forward estimates of line ministries

Partially Achieved. • The medium-term expenditure

framework was introduced in the 2011 budget and performance from previous year is taken as a baseline for the subsequent year.

• All line ministries prepare detailed forward estimates for two out-years, reflecting the government’s priority. However, the first forward estimate of last year’s MTEF is not yet relied on as the basis for calculating

46

PDO outcome indicators Baseline Target Actual Comments indicative ceilings of next year’s budget.

• DG Budget has set up a Medium Term Budget Framework at an aggregate level by economic classification but without detail by expenditure units/program/activity

• On performance-based budgeting, outputs are defined for each program area every year and progress against each output is reported

• Moving forward, further fine-tuning of the implementation of MTEF and PBB will be required, including ensuring that performance made on outputs can be used for decision making on budget allocations.

5. Increased policy development capacity within the Fiscal Policy Office as verified by the completion of several high-quality policy analysis products

0 4 Returned scholars produced reports as policy analysis products

Partially Achieved. • 37 officers trained from FPO: (a) 12

Masters degrees completed; (b) 7 PhD degrees completed; (c) 18 specialized training

• PhD and Masters scholarship programs funded by GFMRAP have contributed a lot in the policy development capacity within the FPO. Some staff who completed the scholarship programs were promoted into higher ranked positions, more research papers were produced with high quality of policy analysis. The MOF and other line ministries have utilized these papers for policy decision in strategic area.

6. Verification of effective project governance, external oversight, change management and implementation

No GAAP, PSC or investigation unit.

GAAP, Procurement Plan, PSC, PSSU, Investigation Unit in MOF-IG all in place and operational with positive results.

Project governance and activities are consistent with the GAAP; Project procurement has been consistent with procurement plan; PSC and PSSU and the investigation

Achieved. • Fully operational investigation unit

in the Office of the Inspector General of MOF established in November 2008, with adequate legal powers for investigating allegations of corruption against MOF staff, including tax officers.

• PSSU has been successful in ensuring project governance and activities are consistent with the GAAP; project procurement activities were also consistent with the Procurement Plan. The PSC and the SecGen of MOF have been consistent in carrying out project

47

PDO outcome indicators Baseline Target Actual Comments unit are all operational and functioning well.

governance and oversight with frequent and regular meetings.

Objective 3: To improve integrity and transparency in the Borrower’s public financial management (Rating: Substantial) 7. Reduced leakage in expenditure flows and effective corruption reducing measures related to accounts and budget control applied

7.1 Reduced payment error to unintended recipients due to inaccurate data 7.2 Manual checks replaced with regular system reporting 7.3 Increased in fiscal discipline over payments 7.4 Auditors perform audits using transaction data in an efficient manner

1% of payments paid to wrong accounts

Almost zero error or incidence in transfer payment

Virtually zero error or incidence in transfer payment with multi-layered screening (supplier database of SPAN and Real Time Gross Settlement of the banking system)

Achieved. • SPAN introduces new module

called supplier database to prevent payments to suppliers whose account information is not fully verified. Payment requests with inaccurate account information are blocked before transfer is attempted.

• Some payments are rejected due to new data entry requirements introduced with SPAN.

• Manual checks and reconciliation between regional and local Treasury offices now done electronically

• SPAN also has a commitment module which allows spending units to spend against these contracts

• Audit reports prepared using transaction data on real time basis

8. Strengthened commitment control

8.1 Annual budget ceilings established 8.2 Budget execution falls within annual ceilings

Weak commitment control by MOF, limited to checking the availability of funds left from budget only when processing requests for payments

Commitment control is applied by reserving the budget ceiling in the system for any signed contracts so that the budget ceiling that has been committed cannot be used for other purposes

Achieved.

• Annual budget ceiling is allotted and established for each of the 24,000 spending units and ceiling data uploaded to SPAN database. Once budget ceiling data is integrated with SPAN, spending units cannot disburse beyond this ceiling

• SPAN has a budget commitment management module, in which any signed contract is recorded as commitment that will reduce budget through an encumbrance journal

• Information on the payment schedule for the commitment and the amount from encumbrance journal are inputs for the cash plan and payment management

• Discrepancies in the recoding of certain transactions still exist (e.g. intergovernmental revenues, non-tax oil and gas revenue, and foreign exchange debt) but these are all

48

PDO outcome indicators Baseline Target Actual Comments 8.3 Coverage of timely internal audit (% of ministries/agencies)

within internal auditor’s materiality threshold of 0.5%

• Individual line ministries have their own internal auditors (Inspectorate General) as an Echelon 1 level position below the Minister. Inspectorate General as the Internal Auditor assists all units to comply with all prevailing regulations within ministries and agencies.

9. Improved quality and timeliness of financial reporting

9.1 Application of International Public Sector Accounting Standards

9.2 Processing and recording of government transactions in a single database

9.3 Quality of annual financial reports (extent of coverage and completeness and consistency with generally accepted accounting principles)

Cash-based accounting for revenues and expenditures, but accrual basis for assets, liabilities, and equity No single database for financial data

Full accrual based accounting

Full accrual-based accounting (compliant with IPSAS) being implemented since 2015 Single database to record all financial data

Mostly Achieved. • Accrual based accounting is

expected to ensure better cash forecasting and provide a full picture of activity costs

• Preparation for accrual accounting started in 2010. The MOF started to implement full accrual accounting in 2015, which is mandated by the state finance law (No. 17/2003). The Government Financial Report for 2015, which is being audited and will be published in mid-2016, will be full accrual-based accounting.

• Indonesia Government Accounting Standard adopted full accrual accounting of IPSAS with only small gaps

• SPAN applies a single database to reduce data inconsistencies previously occurring from extracting data from multiple sources

• Supreme Audit Institutions review of audited financial statements for FY2015 still on-going and expected to be released to the public in late June 2016

10. Improved budget and fiscal information that the government makes available to the public

10.1 SPAN Stage I (as such is described in the invitation for bids for the contract for the procurement thereof) evaluated with user input with respect to content, timeliness and verifiability; specified periodic

SPAN fully functional accordance with ITB specifications

Achieved. • MOF-IG conducted thorough

administrative due diligence through the RTM (requirements traceability matrix) in which the actual deliverables from the supplier were compared with the initial requirements stated in the bid documents. With the final payment made in June 2015, which marked the complete system delivery, no

49

PDO outcome indicators Baseline Target Actual Comments financial reports prepared for concerned entities 10.2 Timely disclosure of central government monthly budget realization data (in days or weeks) 10.3 Public access to key fiscal information

1-2 months Upon request only

2 weeks after the end of each month Key fiscal figures are available on MOF website

other supplier’s obligation remained.

• Reduction in the time taken for central government monthly budget realization data to be publicly available from 1 month to 2 weeks after the end of the month

• Monthly budget execution reports

generated on time. Reconciliation mechanism between regional and local Treasury offices now done electronically.

• Progress has been made with greater access to the semi-annual budget report and to contract awards above a threshold of IDR 50 million on the websites of agencies.

50

Annex 4. Economic and Financial Analysis Economic analysis confirms that GFMRAP generated positive returns of 50 percent over a 15-year period, applying a 15-percent discount rate, with positive economic NPV of US$632 million due to efficiency gains in the process of modernizing Indonesia’s public financial management. While direct attribution of economic and financial benefits of PFM reform is challenging and many of the interventions are difficult to quantify in monetary terms, there are some elements of the reforms that GFMRAP supported for which a financial assessment has been conducted. The assumptions for these calculations are the following:

• General economic and project parameters:

o Financial and economic analysis conducted in US dollars assuming US$ inflation rate of 2 percent, IDR inflation rate of 3 percent and nominal exchange rate tracking purchasing power parity over project life starting with a 2015 exchange rate of Rp13,500/US$

o Real economic growth rates follow IMF World Economic Outlook

estimates through 2019 and then 8 percent per annum thereafter

o Project costs are based on actual costs over implementation during 2005-2015 and projections for 2016-2019. A general real increase in costs of 2 percent per annum is included in cost projects, additional real increases in specific costs items are noted below and a 10 percent cost overrun contingency is provided for

o All Net Present Value and Internal Rate of Return estimates are based on

constant price net cash forecasts over a project life time of 15 years, from 2005-2019

• SPAN implementation

o Average base salary of Treasury staff is $5,600 per year (which does not

include stipends and other remuneration) and 40% of their staff time was previously spent to do manual verification and reconciliation of accounting data

o Costs are actual project expenditures provided by the PSSU. These have

been indicated at the relevant cost line

o Treasury staff capacity building, change management, training, sensitization, and other workshops stay constant after 2015

51

o Project Management, operation and maintenance, and Helpdesk stay constant after 2015

o Full opportunity cost of the delays incurred during implementation has not been included in this analysis. Technical specifications for COTS were developed almost a decade before the pilots were rolled out and during that time the volume of transactions that SPAN was intended to handle was underestimated significantly. This resulted in increased hardware costs which were later incorporated into the contract.

• GFMRAP implementation costs

o Average annual supervision costs, valued at approximately $2.3 million from 2005-2015

o PFM MDTF, both Bank-executed and Recipient-executed, in the amount of $12.3 million from 2007-2015 was used to support implementation

o Government co-financing of $30 million from 2005-2015, in addition to staff time of the Ministry of Finance Project Services and Support Unit and other PIUs

o Project preparation advance of $616,095 used to prepare TORs used for SPAN turnkey contract and for the IVV/PMQA consultancies

o Fixed front-end IBRD loan origination fee of $275,000

Three benefit streams that accrue both financially to the Government of Indonesia and to the economy as a whole:

• Cost savings from TSA: Based on a conservative estimate of interest earnings from consolidated cash balances now kept at the Central Bank through the Treasury Single Account with daily cash balance of IDR 80 trillion with a remuneration of 65% x 7% BI rate per year. This also includes opportunity savings from (tax and nontax) revenue daily sweep into TSA of about IDR 4.2 trillion, daily floated for 3 days at a cost of 7% BI as well as opportunity savings from consolidated cash accounts previously held by KPPN and Kanwil commercial banks, assuming a daily cash balance of IDF 3.6 trillion x 2% current account deposit rate per year. Finally, it also includes income from Treasury Notional Pooling of 24,000 spending unit’s petty cash accounts, assuming average annual balance is IDR 200 million remunerated at a 6% savings rate.

• Productivity savings: These have been estimated on the basis of efficiency gains

in administrative costs calculated from the introduction of the FMIS in local and regional Treasury offices in Indonesia, including savings from reduced printing as a result of paperless transactions and internal communication costs. These calculations assume a gradual increase in the rate of savings from reduced printing,

52

transport and communications costs. Full benefits are expected to be reached by 2018-2019 once regulations catch up with system capacity. It also includes efficiency gains from time saved of Treasury staff who previously spent 40% of their time to do manual verification and reconciliation of accounting data. Treasury spends $45 million per year on staff salaries, so this amounts to $18 million saved per year.

Based on these assumptions of the economic costs and benefits of the project, the real EIRR over a 15 year period is 50 percent and the project has a positive economic NPV of US$632 million (and at a 15 percent discount rate the NPV). It is noted that a number of potential economic benefits were not estimated. These include: (a) reductions in transaction costs of contractors and suppliers in using the e-procurement system and being reimbursed through the reformed FMIS; and (b) efficiency gains from increased tax collection. The table below presents different scenarios to illustrate how the NPV changes under different discount rates and using alternative assumptions related to costs and benefits. The IRR stays consistent. Scenario 1: IRR and NPV applying 50% of MDTF amount as part of the costs

15% productivity gains



IRR 50% 50% 51% NPV at 15% $635 million $644 million $658 million

Scenario 2: IRR and NPV applying 100% of MDTF amount as part of the costs




IRR 50% 50% 49% NPV at 15% $632 million $642 million $656 million

The added value arises mainly from the Bank’s technical input based on international experience, particularly on the FMIS implementation. The World Bank’s task team’s advice on the Terms of Reference and hands-on advice on the bidding documents for the FMIS with the internal best practices will provide potential cost and time savings.

53

Annex 5. Bank Lending and Implementation Support/Supervision Processes (a) Task Team Members

Name Title Unit Task Team Leaders Amitabha Mukherjee (2003-2005)

Lead Public Sector Specialist GGO15

William Wallace (2005-2006)

Lead Economist PREM

Jens Kromann Kristensen (2006-2010)

Lead Public Sector Specialist GGO19

Theo Thomas (2010-2013) Economic Adviser OPSPQ Bernard Myers (2013-2016) Senior Public Sector Specialist GGO14 Supervision/ICR Achmad Zacky Wasaraka Procurement Analyst GGO08 Ahsan Ali Lead Procurement Specialist GGO08 Carlos Ferreira Consultant GGO15 Corry Huntangadi Program Assistant EACIF Hannah Kim Young Professional GGOOS Hari Purnomo Senior Public Sector Specialist GGO14 Imad Saleh Operations Advisor OPSPQ James Sheppard Consultant PREM Lina Lo Consultant GGO14 Lynn Yeargin Senior Program Assistant GGO18 Mark Ahern Program Leader MNC04 Nina Herawati Program Assistant EACIF Ramesh Siva Lead Information Officer GTI09 Sandra Buana Sari Team Assistant EACIF Sarah Horrigan Consultant PREM Sebastian Eckardt Senior Economist GMF02 Suresh Gummalam Adviser GGO14 Tatong Permana Consultant GGODR Unggul Supriyatno Senior Financial Management Specialist GGO20 Vijay Ramachandran Senior Public Sector Specialist PREM Yash Gupta Senior Procurement Specialist GGO08 Yogana Prasta Operations Adviser EACIF

54

(b) Staff Time and Cost

Stage of Project Cycle Staff time and cost (Bank Budget only)

No. of Staff Weeks US$ (including travel and consultant costs)

Lending FY04 58.88 583,398.32 FY05 44.97 251,473.98

Subtotal 103.85 834,872.30 Supervision/ICR FY06 63.43 346,529.20 FY07 45.82 397,613.74 FY08 88.58 370,145.83 FY09 14.64 198,756.11 FY10 14.08 115,961.65 FY11 16.04 114,652.91 FY12 17.18 117,210.41 FY13 11.62 105,374.37 FY14 15.99 126,332.73 FY15 37.36 252,575.08 FY16 (as of April 2016) 6.90 104,309.35

Subtotal 331.64 2,249,461.38

Total28 435.49 3,084,333.68

28 This amount does not include TA and advisory services funded under the PFM MDTF since 2007 totaling an amount around US$12.3 million.

55

Annex 6. Beneficiary Survey Results The contents in this section were taken from User Satisfaction Level Survey of the Implementation of SPAN conducted by MOF in December 2015. The objective of the SPAN User Satisfaction Survey is to measure the level of satisfaction of the users of the SPAN application. There were 3,145 respondents were surveyed from local treasury offices (KPPN), Kanwil, and MOF headquarter staff from all over Indonesia. The SPAN User Satisfaction Survey was conducted using the SERVQUAL Methodology, a quality management framework first published in 1977 by Valarie Zeithaml, A. Parasuraman & Leonard Berry to measure quality in the service sector. It highlights the main components of high quality service in terms of five factors or dimensions, namely, tangibles, reliability, responsiveness, assurance, and empathy. A total of 22 variables were established for the SPAN User Satisfaction Survey in accordance with the SERVQUAL methodology. The 22 variables were grouped into the five service quality dimensions:

• Tangibles (6) - The appearance of physical facilities, equipment, personnel and communication materials

• Reliability (5) - The ability to perform the promised service dependably and accurately

• Responsiveness (4) - The willingness to help users and to provide prompt service

• Assurance (4) - The knowledge and courtesy of employees and their ability to convey trust and confidence

• Empathy (3) - Caring, individualized attention provided to the users The results from the various data analysis methods are as follows:

1. The average performance score that shows the highest level of user satisfaction is in the OM-SPAN application (3.91, good level of user satisfaction), followed by Custom Web (3.75, good) then SPAN (3.54, adequate), from which the overall performance score for all applications is 3.69 (good)

2. The service quality dimension with the highest performance score is Tangibles (3.81), followed by Responsiveness and Assurance (both 3.70), then Reliability (3.63) and Empathy (3.62)

3. The average importance score that shows the highest level of importance or expectation is in the SPAN application (4.47, high level of importance), followed by Custom Web (4.46, high) then OM-SPAN (4.41, high), from which the overall importance score for all applications is 4.45 (high), with Reliability (4.49, high) having the highest expectation

Based on the survey results, it is recommended that improvements be done in: - Data access speed of the application - Ability of the application to generate reports quickly - Promptness of the response of the service desk when complaints are submitted

56

- Consistency in the quality of the networks during normal conditions as well as during disruptions in the primary networks

- Clarity of information provided in problem/error messages - Promptness of the response of the service desk when complaints are submitted - Availability of adequate supporting infrastructure facilities (computers, printers, etc.) - Ease in identification of mistakes and performing correction in the applications.

57

Annex 7. Inputs from Consultations with Stakeholders How did change happen? Analysis shows that changes were triggered when reform champions took ownership in operationalizing regulatory reforms. Insights from stakeholders show that while a number of reforms were initiated in the early 2000s, it took time for these reforms to be fully incorporated into business process improvements. Pockets of resistance were identified upfront but building consensus among a large group of stakeholders took time. Change aspect Automated payment system Cash to accrual accounting Cash management Commitment control of

spending units Issues Changing business process Technical support High level support Setting up a system Reform challenge How to shift from repetitive

manual processes for payment and receipts to an automated process?

How to improve the comprehensiveness in the coverage of government accounting report?

How to get a consolidated view of government cash balances?

How to ensure spending is within annual budget ceilings and the progress of activity can be monitored?

How the reform was initiated

An automated treasury payment system was one of the main components of SPAN.

The 2003 state finance law and 2004 state treasury law called for the move from cash toward accrual to full accrual accounting.

The State Finance Law and the State Treasury Law provide for daily sweeping of government revenues into the TSA, zero based accounts for expenditure, and treasury notional pooling for petty cash accounts of over 24,000 spending units.

Introduction of encumbrance concept to record commitment by spending units. This was added by the shift of responsibility for the ratification and management of annual budget ceilings (DIPA) from DG Treasury to DG Budget so that a “single” budget office undertakes full cycle of the budget process from formulation and appropriation to allotment and virement.

Timeframe

The pilot was launched in 2014 and the full system was rolled out in 2015.

Accrual accounting was meant to be implemented by 2010, but it was then extended to 2015 given the recognition that more detailed accrual policies required to be developed and more training was needed to change the way line ministries recorded their transactions.

Implementation of TSA for revenue happened during 2007-2010 while for expenditure was during2009-2012. Between 2008 and 2011, 9,275 illegal or extra budgetary fund accounts were closed. While 40,248 bank accounts are still maintained by spending units, these were retained as operational accounts with very minimum balance kept in each account

DG Budget established as the single budget office in 2012. Budget ceiling for 24,000 spending units integrated into Custom Web module of SPAN in 2015 to ensure better control of budget execution. The commitment module is part of the SPAN that has been implemented since 2015.

Outputs SPAN is now used in 182 local Treasury offices and 33 regional Treasury offices serving 24,000 spending units.

Full accrual accounting applied for transactions of all 24,000 spending units of the central government as well as the local government agencies.

TSA is fully implemented with no other account kept outside of TSA in Central Bank of Indonesia

Budget execution of 24,000 spending units does not exceed individual spending ceilings.

Outcomes Enhanced accountability, timeliness, and efficiency of payments

Improved comprehensiveness of financial position

Better accountability through transparency in the management of state cash receipts and expenditures

Increased commitment control and fiscal discipline

Senior leader Automated payment system Cash to accrual accounting Treasury Single Account Commitment control of spending units

How did you manage the politics of the reform?

Strong commitment and ownership from the Minister of Finance created an opportunity for change. A steering committee of representatives from different parts of the Ministry of Finance helped convince members of the project’s benefit and why it should be delivered.

Accrual accounting was already built into the design of SPAN. There are still some aspects that are not fully recorded on an accrual basis, but increased training of new processing procedures is expected to improve quality of consolidated financial statements.

Strong commitment from the Minister of Finance. It is added by an MOU signed between MOF and Bank Indonesia to agree on the remuneration of the government’s cash balance kept in TSA.

The integration of budget ceilings with budget execution data has helped maintain spending within spending ceilings. Regional Treasury offices have also played an important oversight role to keep track of the spending units performance on disbursement.

Implementers Automated payment system Cash to accrual accounting Treasury Single Account Commitment control of spending units

What elements made your reform successful?

Demand-driven. All stakeholders were on board.

SPAN generated key accrual based accounting principles. Training and dissemination to all over 24,000 spending units of line ministries

Task was led by the Minister of Finance

The strong motivation to equally distribute the disbursement pattern within a year. Greater buy-in from spending units of the value of real-time budget execution data

Which implementation challenges did you need to address?

Regulations were not updated as business process improvement changed, and vice versa. Implementing a system when both of these aspects were in flux made the full roll out difficult.

Accrual based accounting still a fairly new concept and the practice of recording accrued tax and non-tax revenue for oil and gas as well as foreign currency remains a challenge

Consolidation of government bank accounts meant spending units would no longer have direct control of their financial position. Some people are felt of losing “power” since they are no longer able to control the cash in bank accounts

Keeping spending in check became a challenge when spending units knew in advance the budget ceiling for the next year would be reduced. Add a new requirement for the spending units to register and inform the commitment to MOF before the payment is made.

How did you collaborate across units within your

A steering committee with all the DGs within the Ministry of Finance was the main

The Public Sector Accounting Committee (KSAP) helped to develop full accrual accounting

A steering committee with relevant DGs within the Ministry of Finance was the

Commitment module is embodied in SPAN that will be required to be followed by the

59

agency or across multiple agencies?

mechanism used for coordination and to ensure buy in from all stakeholders.

policies and technical guidelines which is required to be followed by both central and local government’s agencies

main mechanism used for coordination and to ensure buy in from all stakeholders. It is added by a close coordination with Central Bank of Indonesia

spending unit when they submit request for payment

Beneficiaries Automated payment system Cash to accrual accounting Treasury Single Account Commitment control of spending units

How was communication during the reform period?

Socialization workshops were part of the change management strategy.

Deadline to implement accrual accounting was set for 2015 and this was the expectation. It is added by intensive training and socialization to all over 24,000 spending unit staff

State Treasury Law and government regulations on cash management set the framework needed to establish the TSA.

Spending units informed of annual budget ceilings from DG Budget that is used as reference before making any commitment that is recorded in SPAN.

What is the most valuable thing to come from the reform?

Real-time monitoring of budget execution helped ensure timely and accurate payments

Improved the comprehensiveness of government financial statement report

Cost savings from idle balances, improve interest earnings from consolidated accounts in TSA, and reduction of borrowing cost by having an accurate information on daily cash balances available in TSA

Smooth and consistent spending patterns through the year instead of bunching towards the end of the fiscal year

60

Annex 8. Borrower’s Completion Report and/or Comments on Draft ICR

A. Background

1. Indonesia was one of several Asian countries hardest hit by the Asian financial crisis of 1997-1998. Many in Indonesia considered it a “total crisis” which called for comprehensive and wide ranging reforms. Recognizing that the nation-wide call for reforms could become the trigger for broad-based reforms in the public sector, then Minister of Finance Boediono issued Ministerial Decree No 196/KMK.01/2001 establishing the Financial Management Reform Committee with the task to formulate the White Paper on the “Reform of Public Financial Management System in Indonesia” and to draft the laws on state finance, state treasury and state audit, thereby providing the legal basis for institutionalizing the reforms.

2. Following the finalization of the White Paper in 2002, the enactment of the State Finance Law No 14 on 5 April 2003, and the prospect of the near term enactment of the law on State Treasury, the Government issued an Economic Policy Package in September 2003 which called for significantly increasing efficiency, transparency and accountability in public financial management and resource use on one hand, and on resource mobilization through revenue administration on the other.

3. Recognizing that managing change of the magnitude envisioned in the finance laws would require high-level support and attention, dedicated human and financial resources, sustained technical support and a strategy that goes beyond the medium term, Minister Boediono approached the World Bank and the International Monetary Fund to get their support to ensure the realization of the envisioned reform goals. Thus, on 22 December 2004, the Ministry of Finance (MOF) signed with the World Bank a package of loan, credit and grant agreements for the Government Financial Management and Revenue Administration Project which was designed to lock-in the commitment of MOF and other central agencies to the reform agenda. The project was envisaged as a 3-phased Adaptable Program Loan (APL) covering (i) public financial management (PFM) reforms, (ii) customs administration reforms, and (iii) tax administration reforms. The loan, credit and grant package that was signed on December 22, 2004 only covered the first phase for the APL.

B. Project Funding

Source Ceiling in USD

Disbursed in USD

Original Closing Date

Final Closing Date

IBRD 4762-IND 55,000,000 54,414,161 June 30, 2009 Dec. 31, 2015 IDA 4026-IND 5,000,000 4,969,772 June 30, 2009 Dec. 31, 2015 TF 053556 - Japan Grant for GFMRAP 5,000,000 4,598,561 Dec. 31, 2008 Dec, 31, 2010 Total 65,000,000 63,982,494 Signing Date December 22, 2004 Effectiveness Date October 27, 2005

Executing Agency Directorate General of Treasury, MOF (2004-2009) Secretariat General, MOF (2009-2015)

C. Overall Project Management-Government of Indonesia (GOI) 4. GFMRAP was administered by the Project Support and Services Unit (PSSU), which was established initially under the Directorate General (DG) of Treasury, and subsequently under the Secretariat General, MOF as executing agency. Its functions included managing the allocation of GFMRAP funds, coordinating the planning and implementation of activities, as well as monitoring

63

and evaluating the activities of implementing agencies. It also served as liaison between the Project Implementation Units (PIUs) in each implementing agency and the World Bank. From 2005-2009, a Project Steering Committee was formed to oversee all project components, but it proved to be ineffective. Starting in 2009, a more focused Project Steering Committee composed of key Echelon I level MOF officials was established solely for the purpose of providing guidance and direction, and making major decisions in connection with the implementation of the Treasury and Budget System (Sistem Perbendaharaan dan Anggaran Negara or SPAN). This arrangement proved to be more effective.

5. To ensure the availability of dedicated full-time staff to manage and implement the SPAN, in 2008, a new Echelon II unit called the Directorate for Treasury Transformation was established at DG Treasury primarily to manage the treasury reforms under GFMRAP and to oversee all activities related to the development and implementation of SPAN. Furthermore, the governance structure for SPAN was developed with roles defined, which included the Steering Committee, the Project Director (Director of Treasury Transformation), a number of Technical Coordination teams, and a Change Control Board that evaluated all change order proposals before they are submitted to the Steering Committee for approval, and a PIU. It became a policy of the SPAN teams to hold weekly meetings among key project officials, consultants and other stakeholders to monitor project progress and address new as well as pending issues.

6. All subcomponent implementing agencies set up their own PIUs to administer their fund allocations and expenditure, and to manage agency project activities. The governance structure for each Implementing Agency included among others, an Echelon II level official-in-charge of the project supported by a technical working team responsible for the project output, an Echelon III level PIU Head in charge of project administration, a procurement team, a finance team, and a secretariat.

D. World Bank Performance

7. Over the duration of the GFMRAP, there were five (5) Task Team Leaders (TTLs) who worked with the project. The style of leadership of these TTLs have evolved over the years, with the first three TTLs being more deeply involved with project supervision, while the last two TTLs were not as involved, due mainly to the diminishing complexity of procurement issues, and the government project management teams becoming more mature and developed. This change in the Bank’s leadership style has not adversely affected the effectiveness and quality of Bank supervision over the project.

8. Although there had been strains in the relationship between the World Bank Team and their GOI Counterparts, mostly in connection with the procurement of SPAN and the Case Management and Court Administration System of the Tax Court, both sides have successfully found ways to resolve differences and improve cooperation as the project progressed. The provision of continuing Bank advisory support to the project made possible through the PFM MDTF has considerably facilitated better communication and understanding between the GOI project management and the Bank, and has helped speed up necessary approvals and agreements on critical issues, especially during the last few years of project implementation.

E. Project Components and Design 9. The table below provides the complete list of the project components and subcomponents, the dates when the subcomponents closed, and the total disbursement of each subcomponent.

64

Components/Sub-components Implementing Agency

Closing Date per LA

Disburse-ment in USD

Percent of Total

A. PUBLIC FINANCIAL MANAGEMENT A.1 Policy Capacity Development Fiscal Policy Office 31 Dec 2013 1,608,807 2.51% A.2.a Budget Planning and Development Nat’l Planning and Dev.

Agency (BAPPENAS) 30 June 2009 189,965 0.30% A.2.b Budget Planning and Development DG Budget (DG) MOF 31 Dec 2015 196,959 0.31% A.3 Budget Execution and Treasury

Modernization DG Treasury, MOF 30 June 2009 57,693,502 90.17% A.4 Procurement Reforms BAPPENAS 30 June 2009 76,035 0.12% B. REVENUE ADMINISTRATION B.1 Customs Modernization DG Customs, MOF 30 June 2009 53,902 0.08% B.2 Tax Revenue Administration DG Taxes, MOF 30 June 2009 - - C. GOVERNANCE AND ACCOUNTABILITY C.1 Legislative Oversight Secretariat General

DPR 31 Dec 2010 153,458 0.24% C.2 Resolution of Tax Dispute Tax Court, MOF 31 Dec 2013 526,375 0.82% C.3 Internal Accountability Inspectorate General,

MOF 30 June 2009 277,845 0.43% D. PROJECT GOVERNANCE AND IMPLEMENTATION D.1 Project Governance and

Accountability PSSU, Secretariat General, MOF 31 Dec 2015 2,314,551 3.62%

Refund of Prefinancing Advances 616,095 0.96% Front-end Fee 275,000 0.43% Total Disbursements

(Loan+Credit+Grant)

63,982,494 100.00%

10. The wide range of PFM reform initiatives covered in GFMRAP, and the participation of many implementing agencies (eleven in all) seem very ambitious, especially if viewed in retrospect. However, the Minister of Finance, senior MOF officials, World Bank and IMF officials who were involved with project planning when the project was originally designed, considered this design to be the best approach to guarantee an across-the-board participation in the reform effort by key MOF and central agencies performing PFM and revenue administration functions. Furthermore, since a new government was to be installed by October 2004, those involved in designing the project considered that packaging the GFMRAP program in this manner was one way of ensuring that PFM and revenue administration will be pursued in a comprehensively.

11. Notwithstanding the fact that the project covered many components and involved eleven (11) Echelon I level agencies, the project was designed and approved in what could be considered record time, which was just about one year. Due to the need for speed, many of the concepts and programs incorporated in the design where ideas that came from the World Bank team, with modest buy-in or limited understanding on the part of the GOI counterparts.

12. DG Customs and the DG Taxes eventually withdrew from GFMRAP. While the DG Tax component spun off to become the Project for Indonesia Tax Administration Reform (PINTAR). The PINTAR project was also ultimately cancelled before it could start implementation. It is possible that the cancellation of these two major components of GFMRAP was in part due to the fact that the White Paper and the laws that were enacted did not cover customs and tax revenue administration, thus reducing the pressure for institutional reforms in these agencies.

65

13. The subsequent withdrawal from the project of other sub-components before their goals could be achieved can be attributed to one or more of the following factors which are not necessarily related with project design:

a. Changes in the leadership in key Echelon I and Echelon II units involved in GFMRAP implementation has resulted in a decline in the degree of commitment of some agencies to the reform initiatives envisioned in GFMRAP.

b. For some agencies, the availability of grants from PFM MDTF became a more attractive source of funding for their activities compared with the GFMRAP loan and credit.

c. Rigorous procurement procedures and sometimes overly restrictive audit practices have resulted in implementation issues which discouraged some agencies from further pursuing involvement in the project.

14. The withdrawal by some subcomponents in the end proved to be a blessing in disguise since funds which were not used for these subcomponents were subsequently reallocated to finance the requirements of the most important PFM component, namely the Budget Execution and Treasury Modernization sub-component, whose funding allocation for its major activity, the State Treasury and Budget System (Sistem Perbendaharaan dan Anggaran Negara or SPAN) was significantly under estimated.

F. Budget Execution and Treasury Modernization – the Primary Project Subcomponent

15. Of the ten (10) project subcomponents, the Budget Execution and Treasury Modernization sub-component eventually received the biggest allocation which amounted to 90% of the total GFMRAP funding. This subcomponent encompassed the core of the reforms embodied in the State Treasury Law (Law 1/2004) and parts of the State Budget Law (Law 17/2004). The main activities under this subcomponent included:

a. Supply and installation of the SPAN, including recurrent cost b. Independent Verification and Validation (IVV) Consultancy Services c. Business Process Improvement Consultancy Services (for DG Treasury and DG Budget) d. Change Management and Communication Consultancy Services e. Project Management and Quality Assurance Consultancy Service (PMQA)

16. The key objectives of the SPAN project are fully articulated in the Project Appraisal Document (PAD) as summarized below:

• Establish a core financial management system for budget planning and budget execution of the Government of Indonesia

• Allow for increased transparency in all aspects of financial management • Improve the efficiency of business processes thus improving quality and efficiency of

budget planning and budget execution – improved payment cycles, reduced errors in payment, ease of operations for end users

• Assist in transformation of the budget planning and budget execution business units through increased structure and discipline of daily operations, improved compliance with existing standards and regulations and reduced operating cycles of activities and tasks

• Improve accuracy and reliability of financial reporting and ensure that budget execution was well aligned with budget plans

17. The SPAN system had been successfully implemented and is now fully operational at the MOF. All contract components have been delivered based on the revised project time frame. The major components included in the SPAN contract and the completion date of each component are as follows:

66

Component Operational Acceptance Date 1. DC/DRC Fit out 24-06-2011 2. Cable Installation 25-11-2011 3. WAN Installation 10-10-2011 4. CE (Collaboration Environment) Implementation 29-11-2011 5. COTS (Budget Execution and Budget Preparation

Solution) – Accepted in stages 13-01-2014 to 12-06-2015

6. Budget Preparation Custom Web Solution 22-02-2014 7. Server Scale up for EBS Solution 16-12-2014 8. Operations Support for ITSU (IT Support Unit) 31-12-2015 9. Final Operational Acceptance 20-06-2015

18. The performance indicators for the Budget Execution and Treasury Modernization subcomponent as stated in the Loan Agreement, and the implementation outcomes is shown in the Table below:

Performance Indicators Outputs/Outcomes 1. SPAN Stage I (as such is

described in the invitation for bids for the contract for the procurement thereof) evaluated with user input with respect to content, timeliness and verifiability; specified periodic financial reports prepared for concerned entities

An independent evaluation and administrative due diligence was done by the Inspectorate General, MOF to determine if the SPAN and other outputs delivered by the supplier (LG CNS) conformed to the requirements stated in the bid documents, and subsequent amendments thereto. The evaluation showed that all deliverables have been satisfactorily complied with by the supplier. Final Operational Acceptance was signed off on June 30, 2015.

SPAN was launched by the President of the Republic pf Indonesia on April 29, 2015. SPAN users, at the MOF head offices, the regional offices and the field treasury offices attest to the countless benefits they have enjoyed with the implementation of SPAN and the SPAN On-line monitoring (OM-SPAN). The Office of the President, Office of the Minister of Finance and line ministries, from the spending units to the Minister’s Office, can now access information regarding their accounts through the OM-SPAN.

SPAN has completely replaced the silo-type legacy systems previously implemented in the DG Treasury. On budget preparation at the DG Budget, the custom-web solution is now fully operational. However, the hyperion solution which is to be used for the state budget formulation continues to face challenges due to the frequent changes in government policies which impact on the system’s configuration.

2. A Treasury Single Account (TSA) system functioning with acceptable parameters: all on-budget central government bank accounts and own-revenue bank accounts of line ministries are brought into the TSA system

The TSA system for all bank accounts for expenditure under the MOF was implemented starting in 2007. The TSA system for bank accounts for revenue under the control of the MOF and of line ministries was implemented starting in 2009.

The Treasury Notional Pooling of the bank accounts of expenditure and revenue treasurer’s in all line ministries was implemented starting in 2011.

Balances of accounts deposited in Bank Indonesia earn interest at negotiated rates.

Modified in 2009: 3. Central Government

financial statements meet Government Accounting Standards as promulgated by Government Regulation and

Government financial statements using accrual-basis accounting standards promulgated by Government Regulation have been prepared using SPAN starting in 2015. With the implementation of the TSA, there are no longer extra-budgetary funds operating outside the State Budget. Challenges remain with regards to the accuracy of the SPAN generated financial statements, due mainly to the following reasons: (i) reservations

67

the consolidated management reports produce by the DG Treasury include financial assets and liabilities, and cash-based reports of extra-budgetary funds

on the accuracy of the beginning balances of balance sheet accounts carried forward from the legacy system, and (ii) discrepancies between the SPAN generated consolidated financial statements and the consolidated financial statements derived from consolidating the financial statements prepared by line ministries which still use the legacy agency accounting system. However, it is expected that these issues can be resolved over time with the implementation of a new agency financial management system (SAKTI) which is now being piloted in a number of DG Treasury spending units.

G. SPAN Implementation Challenges and Lessons Learned 19. In the course of the GFMRAP implementation, particularly of SPAN and SPAN related activities, numerous challenges were faced, not only by government teams but also by the SPAN supplier other contractors, as well as the World Bank. As mentioned in preceding sections, some of these challenges have resulted in the withdrawal of some sub-components from the GFMRAP. However, the strong commitment of the Project Sponsors and Project Owners, the hard work of MOF officials and team members, the continuing support of the World Bank, and the perseverance of the SPAN supplier and other project consultants all contributed to the successful completion and implementation of the SPAN. Some of the challenges and lessons learned during the SPAN implementation are described below.

G.1 Project Design

20. As stated in earlier, GFMRAP was designed to cover a wide range of reform initiatives involving multiple agencies. Furthermore, the planning of the project was accelerated to put in place the reform agenda with assured funding, before the change in government took place in 2004. Due to the need to finalize the loan agreement as soon as possible, activities planned for some components were firmed up without adequate understanding of the project’s goals, and with half-hearted agreement from key stakeholders. This could be one of the factors for the eventual withdrawal of some components before they have achieved their goals.

21. There is a need to strike a balance between the desire to “lock-in” a long term reform agenda and the need to ensure acceptance of project plans from stakeholders, especially from senior and middle management level officials. Without adequate “buy-in”, the commitment of stakeholders to project implementation diminishes over time, especially where there is frequent rotation of senior officials.

G.2 Project Leadership:

22. From the time planning for GFMRAP was initiated in 2003, until project closure in December 2015, MOF had been through six (6) Ministers, five (5) Director Generals (DirGen) of Treasury, and five (5) DirGens of Budget. Generally, the Ministers, as well as the DirGens had sufficient knowledge and appreciation of the project and were able to provide the leadership, guidance and motivation to those involved in the SPAN implementation. However, there were a few cases where the Ministers/DirGens did not give enough attention to the project, either because they had limited knowledge of the project and its objectives, or because there were other emergent and urgent concerns that they needed to focus attention on. During these times, there were perceived slow-down in the progress of the project implementation

23. As champion and sponsors of SPAN, it is crucial that the Minister and Director Generals provide the leadership needed to steer the project in the right direction and to promote acceptance by all stakeholders. One way of enhancing the commitment to the project of newly appointed Ministers and DGs is through the transfer of vision and knowledge sharing from a previous leader to the in-coming

68

leader. Another way of doing it is for the Project Manager/Director to ensure that the new leader is given adequate and timely briefing on the nature, objectives, goals and status of the project.

G.3 Project Management

24. The active involvement of the Project Steering Committee and some high ranking officers across echelon I units at MOF, including the IG had significant contibution towards effective project management. Likewise, the establishment of the Directorate for Treasury Transformation at the DG Treasury, with the task to manage the implementation of Budget Execution and the Treasury Modernization subcomponent is one of the key factors in the successful implementation of the project. Headed by an Echelon II level official and staffed with some of the most qualified young and energetic full time staff, the unit was given the mandate to manage the SPAN project, recommend the necessary changes in the Treasury’s business processes, draft the implementing regulations and oversee the work of all contractors and consultants involved with the project.

25. Three other factors that have influenced implementation of SPAN are: (i) The appointment by Minister of Finance Sri Mulyani of a full-time IT Expert Staff with the rank of an Echelon I official to oversee and monitor the project, provide guidance to the Project Director, project teams and consultants, and to maintain direct communication links with Echelon I and Echelon II officials who had a stake in the project; (ii) the appointment of an IT expert with extensive experience in the implementation of huge IT projects in the Indonesian banking sector as World Bank Advisor to the project, and (iii) the engagement of the PMQA Consultant which provided overall project management and coordination support as well as assistance in project documentation. Together, these three provided the experience, maturity and understanding of the complex nature of the project that members of the MOF team lacked, thereby facilitating better decisions and faster resolution of issues.

G.4 Procurement

26. The SPAN procurement could possibly be considered one of the longest procurement processes ever experienced in the Ministry of Finance. It took almost four (4) years to complete from the procurement, from the time the bidding document was approved until the contract was signed. Some of the reasons for the long delayed procurement are follows:

a. The SPAN project was a complex ICT project which called for the procurement of a commercial off-the-shelf (COTS) solution in the form of an ERP system, a completely new concept for which the MOF teams had no experience.

b. The SPAN procurement called for the use of the two-stage bidding process for which the procurement team had also very negligible knowledge and experience,

c. During the procurement process there were several disagreements between the MOF procurement team and the WB task team due to differences in the interpretation of the certain provisions in the bidding document and in the Bank guidelines. Resolution of these differences took a long time.

27. However, the involvement of the IVV consultant in the procurement process helped a great deal in providing guidance and advice to the MOF teams. Furthermore, the abovementioned challenges can be further mitigated through the following measures: (i) enable the MOF teams to observe/study systems implemented by other countries which make use of COTS solutions, before they start the procurement process; (ii) ensure that the procurement teams are supported by procurement experts with sufficient experience in the procurement method being used, and (iii) there should be closer face-to-face communication between the government teams and the WB teams to address differences in a timely manner.

69

G.5 Business Process Improvement (BPI)

28. The BPI Consultancy contract was finalized only two months before the SPAN contract was signed. The reason for this was because the BPI Consultancy was not part of the original project design. While urgent need for this activity was identified in the early years of the project implementation, funding was only available after the withdrawal by some GFMRAP subcomponents.

29. One of the limitations of the BPI process was the fact that the consultant’s team had insufficient knowledge and understanding of the existing business processes at MOF. That was also the case with the MOF counterpart, since many were staff who were newly assigned to the project with limited experience and understanding of what could be the future MOF business processes. As a result, even though the BPI Consultancy team members were experienced in the Oracle EBS solution, it was not always possible for the Consultant to recommend, or for the MOF team to agree on the ideal business processes which would minimize customization. Both teams had to go through a slow learning process.

G.6 Implementation

30. Some of the challenges faced during the implementation of the SPAN include the following:

a. As stipulated in the SPAN contract, the hardware and software were procured and deployed two (2) years before the SPAN was ready for piloting and rollout, resulting in some degree of obsolescence before the SPAN roll-out.

b. There was weak coordination and interaction between the BPI Consultant, Change Management Consultant and the SPAN Contractor during the early stages of the Consultancy and the SPAN development.

c. There was limited participation by the SPAN users during the business process improvement stage. Most decisions and dealings with consultants were done by the MOF technical teams.

d. Changes in the business processes continued even after systems requirements have been frozen.

e. The MOF technical teams as well as the Consultants/Contractors did not appear to have a good appreciation of the dependencies between the work they did and work other teams were doing.

f. Some members the SPAN contractor’s team did not have adequate knowledge and understanding of the functionalities of the Oracle EBS solution.

31. In due time, most of the abovementioned challenges have been identified and overcome by the project, while others can be considered as lessons learned for future consideration.

H. Plans to Ensure Continued SPAN Sustainability

32. The recommendations to sustain to the SPAN solution in the near term include the following, among others:

70

a. The need for continuous Operations Support for SPAN through (i) Contracting an external firm to provide operations support and maintenance; and (ii) Development of and sustaining the capabilities of ITSU and Management of Data References (PDR).

b. Implementation of the highest-priority Change Requests for the Budget Preparation and Budget Execution Solutions

c. Establishment of ongoing governance and controls to ensure that MOF is ready for and can sustain the necessary support and maintenance requirements to address changes in business processes and ongoing improvements.

d. Allocation of sufficient training budget to maintain the level of proficiency of SPAN users, and to familiarize new staff joining DG Budget and DG Treasury offices as a result of expansion or regular staff turnover.

e. Development of Business Continuity Plans

33. The recommendations to further enhance the SPAN Solution in the medium-term include the following, among others:

a. Budget Preparation Solution

i. Implementation of enhancements from duly prioritized Change Requests ii. Upgrade of the Budget Preparation Solution infrastructure

iii. Improvements of the analytical capabilities of Hyperion Planning and Custom Web and tightening the controls around the budget approvals

b. Budget Preparation and Budget Execution Solution

i. Implementation of enhancements from Change Requests duly prioritized based on business process needs

ii. Improvements in budget execution vis-à-vis the budget preparation business processes, and vice versa, to consistently and optimally use SPAN’s functionality for recording and reporting non-financial/statistical information to support performance-based budgeting

iii. Budget execution through Oracle EBS technology and Budget Preparation using the Hyperion and Custom Web systems should be seamlessly integrated and streamlined to support the end to end processes from DG Budget to DG Treasury

iv. Adopt a quality improvement program that promotes improvement of state budgeting at every stage of the cycle from planning to reporting through stronger monitoring and evaluation and increased transparency.

v. Development of the datawarehouse and implementation of the plan to link SPAN with national procurement system (LKPP), asset management and debt management systems.

71

I. Component Indicators, Outputs and Outcomes – Secondary Project Subcomponents

34. The performance indicators for the other GFMRAP subcomponents as stated in the Loan Agreement as amended, and the outputs and outcomes from these sub-components are as follows:

Sub-component Performance Indicators Output and Outcomes A.1 Policy Capacity Development

Increase policy development capacity within the Fiscal Policy Office as verified by the completion of several high-quality policy analysis products

The project succeeded in providing much needed education and training for 7 Fiscal Policy Office (BKF) staff who obtained PhD degrees, 12 staff who earned Master’s degrees, 16 staff who underwent training through twinning/on the job training in 4 countries, and 2 staff participating in short courses. A number of these graduates now occupy Echelon II positions and have produced high quality research papers which became the basis for a number of policy decisions in MOF.

A.2.a Budget Planning and Development (Closed on June 30, 2009)

First Medium Term Expenditure Framework endorsed by the Government of Indonesia and forward estimates presented to the DPR

GFMRAP funds were used mostly for benchmarking studies and training on MTEF and PBB, and limited technical assistance during the early stages of GFMRAP implementation. The knowledge gained by those who went on training provided valuable inputs in the development of the initial framework for MTEF and PBB implementation. MTEF was initially piloted in the line Ministries in 2008-2009. It formally became part of the budgeting process starting in 2011. The central government has started to implement PBB but operationalizing it still remains a challenge.

A.3 Budget Execution and Treasury Modernization Discussed in a separate section A.4 Procurement Reforms (Closed June 30, 2009)

An evaluation of the MOF e-procurement pilot is completed and an action plan is adopted by the MOF for wider rollout

MOF established the Center to Support Electronic Procurement (LPSE) in 2009. It now operates in all 33 provinces and supports electronic procurement not only in all MOF offices nation-wide but also in 25 other non-MOF central and local government agencies. The establishment and operations of the LPSE did not receive funding support from GFMRAP but it received support in the form of training from PFM MDTF.

B.1 Customs Modernization (Closed on June 30, 2009)

The DGCE has adopted a comprehensive and consultatively prepared modernization action plan with inputs from stakeholders

This subcomponent implemented only a few of the activities that it planned to undertake with very insignificant disbursements. The development of the modernization action plan was not initiated by the time it withdrew from GFMRAP in 2009.

B.2 Tax Revenue Administration (Closed on June 30, 2009)

The DG Tax has adopted a Results-oriented Modernization Strategy and Plan with measurable performance indicators, aimed at the eventual redesign of processes along single taxpayer account lines and based on the results of the pilot re-engineering of one representative district office

• DG Tax’s initial reform of its tax administration system (mainly the rollout of modernized tax offices and formulation of a strategic plan) was done without GFMRAP funding.

• With funding from a PHRD grant, DG Tax developed a transformation roadmap focused on streamlining business processes, modernizing IT systems, and re‐aligning human resources. Implementation of this plan was to be funded by the Project for Indonesian Tax

72

Sub-component Performance Indicators Output and Outcomes Administration Reform (PINTAR). This project was eventually cancelled

C.1 Legislative Oversight (Closed December 31, 2013)

From: Survey verifies strengthened capacity of the Budget Committee of the DPR To: Verification of strengthened basis for ICT development in the DPR

The activities of the Secretariat General DPR consisted mostly of study visits, staff training and consultancy. Not long after the indicator was revised in 2009, there was a change in the priorities of the Office of the Secretary General, DPR, from ICT development to the development of the capacity of DPR Secretariat staff to support DPR budgetary functions. Funds from the PHRD grant and PFM MDTF grant were used to provide a wide range of training and TA activities aimed at improving the staff’s competence in producing quality analytical reports for members of Parliament. The capacity of the Secretariat General’s staff to do analytical work to support DPR’s budgetary functions has improved significantly such that members of Parliament are now requesting the Secretariat General’s staff to do more studies for them.

C.2 Resolution of Tax Dispute (Closed December 31, 2013)

Annual surveys of stakeholder satisfaction with Tax Court administration and case management have been established; an action plan has been adopted to address systems weaknesses

Support for the Tax Court included the following: Training of Tax Court staff on taxation and customs.

This was deemed very useful by the Tax Court because it provided the staff with basic understanding of the tax and customs regulations and procedures.

Procurement of equipment used in piloting the Tax Court website.

Consultancy for the development of the Case Management and Court Administration System (CMCAS) enterprise architecture, technical specifications and bidding documents. (The contract for the development and implementation of the CMCAS did not materialize due to procurement issues with the Bank, and audit issues with BPKP, which eventually lead to Tax Court’s withdrawal from the project.)

The Tax Court proceeded to develop internally its ase Administration System, with the help of MOF’s own Center for Information and Technology (Pusintek), utilizing to a large extent the technical specifications recommended by the GFMRAP funded Consultancy.

C.3 Internal Accountability (Closed June 30, 2009) The indicator for this subcomponent was erroneously placed under sub-component D.1

An investigation unit is established and fully operational in the Office of the Inspector General of MOF, including the establishment of the necessary legal powers to investigate fully allegations of misconduct and/or corruption against MOF staff

When GFMRAP started, the Investigation Unit was newly established with inadequate furniture and equipment, and with staff who were not trained to do investigation work. By December 2007, the investigation unit has been operational and fully equipped with furniture, computers and office equipment provided by GFMRAP, and was manned by staff who had undergone foreign, as well as domestic training in the conduct of investigation work in all aspects of the Ministry’s operations.

The unit was given adequate legal powers for investigating allegations of corruption against staff from all MOF units.

73

Sub-component Performance Indicators Output and Outcomes D.1 Project Governance and Accountability

Periodic reviews verify that: • Project governance and all

Project activities are consistent with the Governance and Accountability Action Plan and project procurement is consistent with the Procurement Plan

• The Project Steering Committee (PSC) and the Secretary General of the Ministry of Finance are strengthening project governance and oversight

• Periodic reviews conducted by the World Bank supervisory teams showed that project activities are consistent with the GAAP and procurement is consistent with the Procurement Plan.

• The project Steering Committee established during the first five (5) years of the project was not very effective. Thus, in the second half of the project, from 2009 on-wards, the role of the Project Steering Committee was modified to provide guidance and oversight only over the SPAN implementation. This proved very effective in steering the SPAN project towards completion.

• Project implementation methodically followed the guidelines in the Project Management Manual prepared by the PSSU and PIUs, and approved by the World Bank.

74

Annex 9. Comments of Co-financiers and Other Partners/Stakeholders Not applicable.

75

Annex 10. List of Supporting Documents

• Project Implementation Plan • Project Appraisal Document • Aide Memoires, Back-to-Office Reports, and Implementation Status Reports. • Project Progress Reports. • Borrower's Evaluation Report dated May 2016

76

Annex 11. Online Monitoring SPAN Module The Online Monitoring SPAN (OM-SPAN) module is a web-based application developed by DG Treasury to monitor and present transactional information processed by SPAN. Information from OM-SPAN can be accessed by all spending units, based on their authorized access level, through the internet anytime, anywhere, using any electronic devices, including mobile phones, PCs, laptops, and tablets. Data available from OM-SPAN include: Flash Report Managerial (dashboard), budget allotment ceiling, status of payment request (any approval or rejection due to inadequate ceiling or other administrative non-compliance will be shown), budget realization report/absorption for national total and/or individual Ministry, Echelon 1 level, spending unit detailed by activity, authority, source of funds, and regional area of spending. OM-SPAN can also provide data on the funds availability left/unutilized budget from the contract that has been committed but not yet disbursed. Furthermore, in almost real time, OM-SPAN provides information on the balance of the state (tax and non tax) revenue collected. Below is a screen shot of the OM-SPAN:

77

Annex 12. Map of Indonesia