performance mongolia: customs evaluation modernization ... · currency unit–togrog (mnt) at...

EvaluationIndependent

Performance Evaluation

Report

Mongolia: Customs Modernization Project

Raising development impact through evaluation

Reference Number: PPE:MON 2016-12 Project Number: 35376 Loan Number: 2307 Grant Number: 0070 Independent Evaluation: PE-789

Performance Evaluation Report July 2016

Mongolia: Customs Modernization Project This document is being disclosed to the public in accordance with ADB’s Public Communications Policy 2011.

NOTES

(i) In this report, “$” refers to US dollars. (ii) The fiscal year (FY) of the government ends on 31 December. (iii) For an explanation of rating descriptions used in Asian Development

Bank evaluation reports, see Asian Development Bank. 2016. Guidelines for the Evaluation of Public Sector Operations. Manila.

In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department (IED) does not intend to make any judgments as to the legal or other status of any territory or area. The guidelines formally adopted by IED on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of the management of IED, there were no conflicts of interest of the persons preparing, reviewing, or approving this report.

Director General V. Thomas, Independent Evaluation Department (IED) Officer-in-Charge V. Salze-Lozac'h, Independent Evaluation Division 2, IED Team leader F. De Guzman, Senior Evaluation Officer, IED Team members M. Diza, Evaluation Officer, IED

E. Li-Mancenido, Evaluation Analyst, IED

Abbreviations

ADB – Asian Development Bank CAIS – customs automated information system DMF – design and monitoring framework DRS – disaster recovery system EIRR – economic internal rate of return GAMAS – Mongolian Customs Automated Data Processing System ICT – information and communication technology IT – information technology KOICA – Korea International Cooperation Agency MCGA – Mongolian Customs General Administration O&M – operation and maintenance PCR – project completion report PSC – project steering committee SEW – single electronic window TA – technical assistance

Currency Equivalents Currency Unit–togrog (MNT) At Appraisal At Project Completion At Evaluation (21 Aug 2006) (31 Aug 2011) (18 Mar 2016) MNT1.00 = $0.00086 $0.000801 $0.000503 $1.00 = MNT1,165.0 MNT1,249.0 MNT1,988.5

Contents

Page

Acknowledgements vii

Basic Data ix

Executive Summary xi

Chapter 1: Introduction 1

A. Evaluation Purpose and Process 1 B. Summary of Expected Impact, Outcome, and Outputs 1

Chapter 2: Design and Implementation 3

A. Rationale 3 B. Time, Cost, Financing, and Implementation Arrangements 6 C. Technical Assistance 7 D. Procurement, Construction, Consultants, and Scheduling 7 E. Safeguard Arrangements and Gender Action Plan 7 F. Design Changes 8 G. Loan Covenants, Monitoring, and Reporting Arrangements 8

Chapter 3: Performance Assessment 9

A. Relevance 9 B. Effectiveness 10 C. Efficiency 14 D. Sustainability 16

Chapter 4: Other Assessments 18

A. Development Impact 18 B. ADB Performance 19 C. Borrower and Executing Agency Performance 19

Chapter 5: Overall Assessment, Issues, and Lessons 20

A. Overall Assessment 20 B. Issues 21 C. Lessons 22 D. Follow-Up Actions 22 Appendixes 1. Design and Monitoring Framework 25 2. Appraisal and Actual Project Costs 27 3. Overview of Recent Information Systems in Mongolian Customs General

Administration 28 4. Economic Reevaluation 32

Acknowledgements A team of staff and consultants from the Independent Evaluation Department (IED) prepared this study. It was composed of F. De Guzman (team leader), M. Diza, and E. Li-Mancenido. J. Supangco and J. Ganbold were the consultants. Valuable inputs and comments at various stages were received from M. Gatti and B. Nguyen as peer reviewers. The report benefited from the guidance of V. Thomas, V. Salze-Lozac’h, and B. Finlayson. The team would like to thank ADB staff and the Mongolian Government, and various stakeholders who were interviewed, for their time and opinions. Also, the team would like to acknowledge comments on the draft provided by the East Asia Department and the Mongolia Resident Mission. IED retains full responsibility for this report.

Basic Data

Loan No. 2307 and Grant No. 0070-MON: Customs Modernization Project

Key Project Data ($ million)

Per ADB Loan Documents Actual

Total project cost 6.76 7.02 ADB loan (SDR equivalent) 5.00 5.20 Local currency cost 1.26 1.34 Other grant financing 0.50 0.48 Grant amount cancelled 0.0185 Key Dates Expected Actual Appraisal 21 August–1 September 2006 Loan and grant negotiations 22 November 2006 Board approval 20 December 2006 Financing agreement 6 February 2007 Loan and grant effectiveness 6 May 2007 21 May 2007 First disbursement 19 November 2007 Loan closing 31 August 2010 13 December 2011 Borrower Mongolia Executing Agency Ministry of Finance Mission Data Type of Mission No. of Missions No. of Person-Days Fact finding 1 108 Appraisal 1 120 Inception 1 15 Review 6 83 Project completion 1 20 Independent evaluation 1 10 ADB = Asian Development Bank.

Executive Summary This project performance evaluation report presents the findings from the evaluation of the Customs Modernization Project in Mongolia to assess its performance and highlight lessons. It provides input to the 2016 Country Partnership Strategy Validation for Mongolia. The Project The project was approved on 20 December 2006 and became effective on 21 May 2007. The expected impacts were increased volume and smooth flow of trade. The expected outcomes were improved efficiency, transparency, and sustainability of customs services and administration. The project included three outputs: (i) migration and upgrading of the automated data processing system to an internet-enabled environment and central database; (ii) infrastructure improvement at selected major border crossing posts, including provision of inspection equipment and laboratory apparatus; and (iii) institutional strengthening, consisting of business process reengineering, capacity building, interagency coordination, regional cooperation, and public–private partnership.

The project was prepared during a period of deteriorating economic performance with declining growth in gross domestic product, weakening of fiscal and balance-of-payments positions, and sharply decreasing foreign direct investments. These conditions provided the incentives for a political consensus to proceed with key reforms in trade, including accession to the Revised Kyoto Convention and enactment of a revised Customs Law. However, efforts were needed to spur customs reforms and modernization in order to enhance revenue collection and lower the incidence of corruption. The Mongolian Customs General Administration (MCGA) was using an outdated information system that hindered the widespread use of modern business processes. Also, staff were not trained for work in a modernized customs environment. The government sought to accelerate these reforms by implementing the project. A compelling basis for support by the Asian Development Bank (ADB) was the need to provide impetus for trade and revenue collection through the provision of modern information technology (IT) equipment and facilities and through capacity building.

Overall Assessment Overall, the project is assessed successful. Key factors for the project’s success were (i) the government’s strong ownership of the project, (ii) the decision to set up an entirely new information and communication technology (ICT) system instead of just upgrading the existing one, and (iii) a strong partnership with the private sector.

The project is assessed relevant. Envisaged activities generally reflected conditions related to trade facilitation, especially in customs administration. The project was also in line with the development strategies of the government and ADB. The initial project design was significantly improved with the introduction of an entirely new Customs Automated Information System (CAIS).

xii Mongolia: Customs Modernization Project

The project is assessed effective. The intended project outcomes were achieved. There have been marked improvements in customs operations. Envisaged outputs have likewise been achieved. The project is assessed efficient in the use of resources. The economic viability indicator was re-estimated at 16.4%. Physical completion of the project was delayed by almost a year. The project is assessed most likely sustainable since it is technically sound, and sufficient maintenance and operating procedures and budget are in place. A recently approved project will include upgrading of the CAIS, including the integration of new applications and an increase in capacity.

Key Issues

No detailed and thorough feasibility study was undertaken to clarify the alternative designs for the project. From these could have evolved the proposed set of technically feasible options, especially for the automated data processing system to be used. This could have led to better project preparation, especially related to the possible future phasing and sequencing of infrastructure investments in border post crossings not covered under the project. A disaster recovery system should have been made a prerequisite component in the original project design. It was eventually included near the end of project completion or almost 6 months after the new ICT system was officially accepted by the government. However, this unduly exposed the new system to possible security risks.

Key Lessons

A customs modernization project, aimed at improving efficiency and transparency to reduce corruption, needs a multidimensional approach. In this case, it required three interrelated components: (i) IT infrastructure, (ii) laws and procedures, and (iii) coordination among concerned agencies. One component could not be tackled alone. For instance, provision of IT infrastructure would not be sufficient to improve customs business processes and procedures. Reforms of customs laws and operational regulations should be established initially so that related components can ensure their implementation. Moreover, trade facilitation and customs modernization is not the responsibility of customs administrations alone. Close collaboration with other stakeholders is essential to address constraints to trade.

The fast-changing nature of the IT environment requires that a project’s IT infrastructure (e.g., operating systems, core data processing applications, network, and telecommunications technologies) have enough flexibility in terms of compatibility, connectivity, and modularity to integrate with technology development. Hence, planning for a customs modernization project must take into account the possible obsolescence of IT infrastructure in a shorter time period after a project’s completion than originally envisaged. Also, in designing projects of this nature, a disaster recovery system should be included.

Partnership between customs and the private sector should be a major thrust of a customs modernization project. Extra effort must be devoted to ensuring proactive engagement with the private sector, particularly with the trading community. This can help ensure buy-in, improve the user-friendliness of the IT system, and facilitate public–private partnership in customs administration. It can also assist in identifying issues and problems, and in facilitating acceptance of reengineered business processes for the newly introduced IT system.

Executive Summary xiii

The collection of baseline data should be given high priority. Efforts could have been exerted to ensure that baseline data were established during appraisal stage. In the absence of measurable baseline values set up at the appraisal stage, meaningful indicators should be carefully formulated to enable continuous monitoring for updates throughout project implementation. The project’s independent evaluation was hampered by the absence of these baseline indicators. Follow-Up Actions

On the basis of the evaluation findings, two follow-up actions are proposed: ADB should remain engaged in supporting and monitoring developments pertaining to the move toward a single electronic window and how interconnection among the concerned agencies’ IT systems, especially those in the border crossing posts, will be made and their eventual link-up with the CAIS. In particular, ADB could provide support to the government through capacity-building technical assistance (TA) projects, taking into consideration ongoing related operations in the areas of trade facilitation and ICT. This TA should also help in incorporating best practices and should advance customs practices in customs operations.

ADB could provide support to the government in strengthening benefit and information monitoring, especially in improving data gathering, monitoring and evaluation of economic benefits, and social impacts of trade facilitation initiatives. This support may be undertaken through a capacity-building TA. TA resources may be devoted to improving systematic monitoring of indicators related to customs services such as the number of documents processed per hour and average clearance time. This data gathering and monitoring should also include information related to trade flows.

CHAPTER 1

Introduction

A. Evaluation Purpose and Process

1. The Customs Modernization Project was designed to support the Government of Mongolia in its efforts to promote trade and improve the investment climate through customs reforms and modernization. It was also aimed at lowering the incidence of corruption and smuggling, enhancing revenue collection, providing a platform for customs information and communications technology (ICT), and reducing trade barriers. 2. The project was included in the 2016 work program of the Independent Evaluation Department (IED) in order to provide inputs to the final review validation for the country partnership strategy for Mongolia. This project performance evaluation report, undertaken 2 years after the project completion report (PCR) in 2014, allows sufficient time for project outputs and outcomes to be reassessed and the impact to become apparent, although attribution solely to the project is difficult. Following ADB’s evaluation guidelines, 1 the report assesses the performance of the project and highlights lessons. 3. The PCR rated the project highly successful, highly relevant, highly effective, highly efficient, and most likely sustainable.2 The PCR noted that the project achieved more than the output targets and fully achieved the outcomes within the original budget. It indicated that the government made genuine efforts to reform its customs system by adopting a new Customs Law and reengineering the business process. 4. The PCR was validated in February 2015.3 The project was rated successful with criteria ratings of relevant, effective, efficient, and likely sustainable. For the relevance criterion, the project design was found deficient in terms of specifications of procurement packages. For the effectiveness criterion, the PCR did not report performance as per the design and monitoring framework (DMF) outcome indicators. On the efficiency criterion, the project incurred delays and the economic reevaluation methodology was inadequate. On the sustainability criterion, the operation and maintenance (O&M) plan was not completed at project completion.

B. Summary of Expected Impact, Outcome, and Outputs

5. According to the project’s DMF in the report and recommendation of the President, the expected impact was increased volume and smooth flow of trade.4 The

1 ADB. 2016. Guidelines for the Evaluation of Public Sector Operations. Manila. 2 ADB. 2014. Completion Report: Customs Modernization Project in Mongolia. Manila. 3 IED. 2015. Validation Report: Customs Modernization Project in Mongolia. Manila: ADB. 4 ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Loan and

Grant Administration to Mongolia for the Customs Modernization Project. Manila.

2 Mongolia: Customs Modernization Project

expected outcomes were improved efficiency, transparency, and sustainability of customs services and administration. 6. The intended outputs were migration and upgrading of the automated customs data processing system to an internet-enabled environment and a central database; infrastructure improvement at selected major customs houses and border posts, including provision of inspection equipment and laboratory apparatuses; and institutional strengthening, to include business process reengineering, capacity building, interagency coordination, regional cooperation, and public–private partnership. The DMF shows the targeted and achieved impact, outcome, and output indicators (Appendix 1).

CHAPTER 2

Design and Implementation

A. Rationale

7. After years of strong economic performance, Mongolia’s economy showed signs of vulnerability at the time of project preparation in 2006. The budget position started to weaken due to wage increases in the civil service, social spending, tax cuts, and low copper prices. Government revenues and expenditures were on a downward trend.5 The trade balance achieved a small surplus in 2006 after 2 consecutive years of deficit although it was expected to be in the negative once the project began operations the following year. The current account surplus declined in 2007, although the overall balance-of-payments position and foreign direct investment (FDI) remained strong.6 8. Efficient revenue collection was considered critical to the economy to finance various social and development programs. During project formulation in 2006, fiscal revenue averaged about 33% of gross domestic product. A governance assessment indicated that the import taxes collected fell short of the potential because of exemptions, collection inefficiencies, and corruption. A number of measures were proposed to improve tax administration practices and efficiency, such as enhanced computerization, improved staff training, and revised inspector manuals.7 9. Since its accession to the World Trade Organization in 1997, Mongolia has decreased its tariff level from 15% to 5% for most imported goods. However, much still needed to be done to increase the volume and ease the flow of trade. As a landlocked country between two large economies—the Russian Federation and the People’s Republic of China—Mongolia has huge opportunities to spur trade with these countries. Bilateral and regional customs cooperation could be further enhanced through the harmonization of customs procedures for cross-border trade. 10. Efforts to spur customs reforms and modernization were needed to enhance revenue collection and at the same time, reduce leakages from corrupt practices, smuggling, and diversion of tax and duty payments. The Mongolian Customs General Administration (MCGA) was using an outdated information system which stymied the widespread use of modern business processes, including risk management (Box) and post-entry audit. 8 Customs officers and staff had not been trained for work in a modernized customs environment.

5 International Monetary Fund. 2007. Mongolia: 2006 Article IV Consultation. Washington, DC. 6 International Monetary Fund. 2008. Mongolia: 2008 Article IV Consultation. Washington, DC. 7 ADB. 2004. Governance: Progress and Challenges in Mongolia. Manila. 8 Post-entry audit involves scrutinizing a shipment after its entry has been cleared, especially those identified

as coming from high-risk importers.


11. Customs inspection equipment and detection facilities in key customs posts and offices were either non-existent or insufficient to undertake customs control and anti-smuggling functions. In addition, there was a perceived need for cooperation with private sector users of the new customs information system, to facilitate broader acceptance and usage. The government understood that modernization of its customs services and administration was the correct path toward improving trade and revenue collection performances; hence its request for ADB support.

1. Customs’ Automated Data Processing System 12. The automated data processing system for customs (called GAMAS)9 in use at the time of project preparation lacked a central database system and was manually operated, with weak communications and network linkages.10 Its server’s capacity was low, and the system could no longer meet international standards for customs clearance. Risk management, post-clearance audits, and electronic submission of trade documents could not be undertaken without an internet-enabled system and a centralized database. Also, GAMAS could not be linked with private companies and traders, resulting in limited public accessibility. 13. These problems impeded customs services and administration, which could result in delays, as well as in abuse and corrupt practices. In line with Mongolia’s anticorruption law of 2006, there was a need to address the issue of governance through improved business processes and equipping the MCGA with state-of-the-art technology and facilities. The migration from GAMAS to a more advanced system would meet the increasing demand and requirements of customs administration. A compelling basis for ADB’s

9 It was first developed for a Microsoft Windows–based environment to capture customs data from import

and export declarations submitted by the trading community. However, it did not interface with a web-based environment. Because it used a programming language that had been overtaken by more advanced ones, such as Java, new technologies including data warehousing and security features (e.g., digital signatures) could not be put to use in GAMAS.

10 GAMAS required intensive labor, with traders moving from one customs processing point to another to seek official stamps and signatures. Human intervention in this process caused delays in customs clearance and was vulnerable to corrupt practices. This led to a lack of transparency.

Risk Management in Customs Operations Risk management and the use of risk-based selectivity (through red, orange, and green channels) allow Customs to allocate its scarce resources in detecting and examining “high-risk” cargos and travelers while increasing the efficiency of the clearance process for low-risk shipments. In some countries, Customs still applies a 100% physical inspection regime, i.e., every shipment is stopped and physically examined (partially or completely), causing significant delays at border crossings, and other customs inspections sites. Such an inspection regime hinders trade and creates an environment for corrupt practices to speed up the process. Under the risk management module, traders are classified into three categories based on selectivity criteria, profiling, and risk scenarios based on their history: (i) red channel as high risk, (ii) orange channel as medium risk, and (iii) green channel as low risk. The risk category determines the level of inspection each cargo receives. The red channel requires manual inspection of the cargo and paperwork, orange requires a paperwork checkup, and green requires a few fields of the customs declaration to be checked and thus allows rapid customs clearance. With this system, only the most high-risk cargo is physically inspected, saving time and money for Customs and traders.

Source: Asian Development Bank, Independent Evaluation Department.

Design and Implementation 5

support for customs ICT initiatives in Mongolia was the obsolescence of the existing automated data processing system and the government’s initiative to move toward a single electronic window SEW to support delivery of trade-related services (para. 55).

2. Infrastructure Improvements 14. Mongolia was beset by infrastructure bottlenecks, especially at border crossing posts. Technical equipment, including anti-smuggling detection facilities, was either inadequate or not in place. The bottlenecks hindered the smooth flow of goods and the conduct of proper examinations of goods. These, in turn, led to higher transaction costs for traders, e.g., higher storage fees and spoilage of perishable commodities. There was a need for a dedicated cargo clearance checkpoint on heavily trafficked key border crossing posts and exclusive terminals for passengers. Travelers and vehicles were mostly confined to a limited space which made it difficult to conduct proper examinations. Moreover, the MCGA had only one Central Customs Laboratory, located in Ulaanbaatar, where samples of traded goods from border crossings had to be brought for examination. Transporting these samples took 5–7 days, which resulted in longer customs clearance times. In addition, a properly equipped laboratory can help prevent misdeclaration of goods. ADB support was therefore needed to facilitate the movement of goods and people by addressing the shortcomings of customs inspections, and the shortage of surveillance and scanning equipment and facilities, including laboratory equipment.

3. Institutional Capacity and Linkages

15. In the Mongolian context, capacity building and the use of state-of-the-art customs ICT could not be tackled in isolation. Initiatives in both these areas would have more chance of succeeding if undertaken simultaneously. For fully effective, modern customs business processes and practices, an internet-enabled system and a centralized database were required. The situation was compounded by the weak human resource capacity in ICT and the lack of awareness of modern customs processes. In addition, there was a need to improve interagency and stakeholder coordination and cooperation among traders, banks, the MCGA, customs brokers, and other government agencies working in the cross-border stations. Against this backdrop, an opportunity was present to improve customs administration and processes, especially the transparency and efficiency of the customs clearance system, while helping the government improve trade and investment opportunities. 16. In general, the project design rationale was consistent with the situation in the MCGA. ADB assistance—through the provision of a cutting-edge, internet-enabled data processing system, infrastructure facilities and equipment, and institutional strengthening measures—was appropriate in positioning Mongolia for future growth in trade, strengthening governance, and improving the investment climate through efficient, transparent, consistent, and predictable customs procedures and processes. 17. Recent economic developments continue to validate the importance of the project. Mongolia’s narrow economic base makes the country highly vulnerable to shocks. The economy is dominated by the mining sector. This lack of diversification made the economy prone to boom-bust cycles. Gross domestic product grew by 17.3% in 2011 but had been on the decline. Growth rates in gross domestic product were 7.9% in 2014 and 2.3% in 2015. Inflation rose substantially in 2014, to 12.8%, due to currency depreciation and policy stimulus, and although it had eased by 2015, it


remained high at 6.6%.11 The economy has faced sharp declines in FDI the past few years. The current account balance continued to be in deficit from 2011 to 2014, although the deficit tapered off to -5.0% in 2015. Growth in both exports and imports are expected to pick up in the coming years, although this will depend on growth in the economies of Mongolia’s major trading partners. It is therefore essential that custom reforms and modernization efforts continue to accommodate expected growth in trade.

B. Time, Cost, Financing, and Implementation Arrangements

18. At appraisal, the estimated total project cost, including taxes, duties, and interest during project life, was about $6.8 million. ADB provided a loan of SDR3.4 million ($5.0 million equivalent), accounting for 74% of the total cost.12 A grant of $0.5 million from the e-Asia and Knowledge Partnership Fund and counterpart financing from the government of $1.3 million were provided. The actual project cost of $7.0 million was slightly higher (4.0%) than the appraisal estimate of $6.9 million due to the increase in the dollar equivalent of the special drawing rights for the ADB loan. 19. The project was prepared during an appraisal mission on 21 August–1 September 2006, and the loan was negotiated with the government on 22 November 2006. The project was approved on 20 December 2006 under a “no objection” procedure and became effective on 21 May 2007. It was completed within budget.13 The loan was disbursed on time in four installments, with a total disbursement of SDR3.35 million, or a utilization rate of 99.2%. The unutilized loan amount of SDR28,416.00 was cancelled. From the $0.50 million grant, the total amount disbursed was $0.48 million. The remaining grant balance was cancelled. After loan approval, the Korea International Cooperation Agency (KOICA) provided a grant of $2.50 million in parallel financing to the ADB loan to support Mongolia’s customs modernization (Appendix 2).14 20. The implementation period was envisaged at 3 years. The original loan closing date was 31 August 2010, but this was extended to 13 December 201115 to cover start-up delays due to the need to reach agreement on parallel financing, the complex procurement process, setting up of the disaster recovery system (DRS), and technical glitches during field testing. Implementing institutions were those planned. The Ministry of Finance was the executing agency and had overall responsibility for the project. The MCGA was the implementing agency for all project activities. A project management office was established, headed by a project director who was assisted by technical and administrative staff. A project steering committee (PSC) was created and coordinated project implementation.

11 ADB. 2015. Asian Development Outlook Update. Manila. 12 The loan had a repayment period of 32 years, including a grace period of 8 years, with an interest rate of

1.0% per annum during the grace period and 1.5% per annum thereafter. 13 Some minor adjustments between subcomponents’ costs were made, but the project was completed

within budget. 14 In February 2008, ADB, KOICA, and the MCGA signed a tripartite memorandum of understanding,

committing to delineating financing of subcomponents. 15 This refers to the project’s financial closing, although all project activities were completed in June 2011.

Design and Implementation 7

C. Technical Assistance

21. There was no preparatory technical assistance (TA) for the project. However, it was developed through a number of TA grants addressing customs modernization, trade facilitation, and regional cooperation. In 2005, a customs needs assessment was carried out under an ADB TA project. 16 This was followed by the preparation of Mongolia’s ICT-supported e-Customs master plan, financed by the Government of the Republic of Korea.17 Under the Customs Cooperation Committee, attempts were made to simplify and harmonize the customs procedures in Central Asia Regional Economic Cooperation while adopting ICT for customs modernization. Two ADB regional TA projects—one to pilot a joint customs control between Mongolia and the People's Republic of China, and the other to provide training to MCGA customs officials—complemented the project’s capacity-building component.18

D. Procurement, Construction, Consultants, and Scheduling

22. About 4.5 person-months of international and 44.5 person-months of national consulting services were to be provided under the grant portion for business processes, ICT implementation, interagency coordination, and project implementation and monitoring. The budgeted international and national consulting services were fully utilized. 19 The bidding process for a few procurement packages took longer than expected as a result of lack of interest from international and local firms due to the small contract sizes, limited number of qualified bidders, unrealistically high bidder qualification and technological criteria, and lack of experience of local vendors in bidding for internationally financed contracts. The national competitive bidding was changed to international competitive bidding as there were no suitable potential suppliers in the local market. All procurement activities complied fully with ADB’s guidelines.

E. Safeguard Arrangements and Gender Action Plan

23. The project was category C for potential environmental impacts and risks, requiring no environmental management plan or resettlement action. The independent evaluation mission (IEM) did not find any project-related environmental issues as minor civil works such as installation of ICT and equipment were carried out within the customs premises. The IEM did not find any involuntary resettlement issue or impact on indigenous peoples. There were no indications that women were disproportionately affected.

16 ADB. 2005. Technical Assistance Consultant’s Report: Trade Facilitation and Customs Modernization in

Mongolia. Manila. 17 Completed in July 2006. The Republic of Korea consultant team submitted to the MCGA the final report of

the ICT master plan. 18 ADB. 2006. Technical Assistance for the Development of Regional Cooperation Programs for Mongolia and

the People’s Republic of China. Manila; ADB. 2008. Technical Assistance for Capacity Development for Regional Cooperation in the People’s Republic of China and Mongolia. Manila.

19 Two additional lump-sum contracts were signed with the international business process and ICT specialist to prepare a price benchmarking study on procurement and a reorganizational plan for the IT center.


F. Design Changes

24. The original design did not include a DRS20 because of the limited resource envelope for the project. In January 2011, ADB approved the MCGA proposal for a change in scope to include the development of a DRS that would protect sensitive customs information and transactions in the event of system anomaly or failure. The DRS provided a backup server and data storage capacity, funded from loan savings. The additional KOICA funding (para. 19) enabled the development of a new automated customs information system (the CAIS), instead of merely upgrading GAMAS, as originally envisaged (para. 40). ADB’s loan financing allowed the development of both external portal and integrated customs information systems and the acquisition of ICT equipment for CAIS. A change in procurement method from national competitive bidding to international competitive bidding was necessary as a result of the complexity of contract packaging of ICT-related projects (para. 22).

G. Loan Covenants, Monitoring, and Reporting Arrangements

25. The loan agreement included 24 covenants; 22 were fully complied with. The two covenants not fully complied with were on the PSC and on the O&M of project facilities. The PSC was established late—not within 1 month, as intended, but 3.5 months after the loan effectiveness date. The MCGA was not able to complete the O&M plan at project completion. As envisaged in the project design, a project performance monitoring system based on the DMF was developed by the MCGA and the project management office. Quarterly progress reports were prepared, and key information on the status of implementation was fed into the system.

20 Procedures and facilities for use when essential systems are not available for a period long enough to have

a significant impact on the business caused by nature, hardware, power failure, component failure, software failure, data corruption or loss, human error, communications, computer network equipment, security failure, or computer viruses. This would include back-up hardware at a secured site including back-up of software and data (alternative data center).

CHAPTER 3

Performance Assessment 26. The evaluation criteria focused on the project’s performance and not on the performance of the executing agency and ADB. The IED’s guidelines identify four core criteria: (i) relevance of the project to the government and ADB development strategies and relevance of the design to achieving project objectives, (ii) effectiveness of the project outputs and outcomes, (iii) efficiency of the project’s utilization of resources, and (iv) sustainability of the project outputs and outcomes (footnote 1). Non-core assessments were undertaken on the project’s development impact and the performance of ADB and the borrower.

A. Relevance

27. The project is assessed relevant. It was relevant at the time of project appraisal as well as at completion. The project’s envisaged impact, outcome, and outputs were consistent with government development strategies and ADB country strategies and programs. However, the quality of the DMF was poor. The link between performance indicators and the statements on outputs, outcomes, and impacts was not clear (para. 31). 28. The project was consistent with the government’s National Development Strategy, 2008–2021, with its key priorities of developing an export-oriented, high-technology, knowledge-based economy and an accountable system free from corruption and red tape. 21 The government has placed increasing emphasis on private sector development, finance, and regional cooperation, as articulated in the Government Platform, 2012–2016.22 Thus, the project remains consistent with the current strategies and priorities of Mongolia. 29. The project was in line with ADB’s various strategies. ADB’s strategy for Mongolia at project approval was based on the country strategy and program for 2006–2008, which aimed to support stable, broad-based economic growth and inclusive social development, including improvements in the quality of public services.23 The project supported ADB’s strategy through customs modernization to facilitate trade. It also remains aligned with the strategic pillar of ADB’s Country Partnership Strategy for 2012–2016 and with its update, 24 which emphasized competitive, sustainable, and regionally integrated growth.25 ADB’s Central Asia Regional Economic Cooperation program on regional trade facilitation and customs cooperation is being coordinated by the Customs Cooperation Committee, of which Mongolia is an active participant. It is aimed at promoting concerted customs reforms and modernization, and supporting interagency cooperation and partnership with the private sector. 21 Government of Mongolia. 2008. Millennium Development Goals-Based Comprehensive National

Development Strategy of Mongolia. Ulaanbaatar. 22 Government of Mongolia. 2012. The Government Platform (2012–2016). Ulaanbaatar. 23 ADB. 2005. Country Strategy and Program: Mongolia, 2006–2008. Manila. 24 ADB. 2014. Interim Country Strategy: Mongolia, 2014–2016. Manila. 25 ADB. 2012. Country Partnership Strategy: Mongolia, 2012–2016. Manila.


30. The project was designed during a period of deteriorating economic performance and structural weaknesses (i.e., small economic base dominated by a single sector, paras. 7 and 17). These conditions provided the incentives for a political consensus to proceed with key reforms on trade. Various changes addressing these issues were put in place, including Mongolia’s accession to the Revised Kyoto Convention 26 and its enactment of the revised Customs Law in 2008. The government sought to accelerate these reforms by implementing the project. Government ownership of the project and its place in the national agenda were well established. Given the need to provide impetus for trade after these reforms, project investment lending was the appropriate instrument to use to address these problems. 31. However, project design could have been better. Some of the performance indicators during appraisal could have been more realistically formulated, taking into careful consideration the direct relevance of these performance targets to impact, outcome, and outputs, including establishment of baseline data. For example, it is not clear how outcome indicators on the reduction in the number of cases of customs infringement and improvement of the MCGA’s ranking on the perception index of corruption survey are indicative of improvements in efficiency, transparency, and sustainability of customs services and administration. Data from indicated sources are not regularly produced. 32. Nonetheless, the original project design was improved with the inclusion of a DRS, which helped strengthen data security in the CAIS. The IEM is of the view that a DRS should be made an integral part of original design in any customs modernization project. Similarly, the government requested KOICA in May 2007 for support to enhance the functionality of GAMAS, which was approved on 14 December 2007. KOICA’s parallel funding for CAIS, an entirely new system to GAMAS, proved to be fortuitous since the envisaged ADB project design involved only migration and upgrading of GAMAS. Nonetheless, ADB financed three of five subcomponents, which contributed to the completion of CAIS (para. 24 and Appendix 3, para. 5).

B. Effectiveness

33. The project is assessed effective. The intended outcome of improved efficiency, transparency, and sustainability of customs services and administration has been achieved. Given the absence of periodic data from envisaged source and weaknesses observed in the DMF (para. 31), alternative data sources were used to better measure improvements in efficiency, transparency, and sustainability in customs operations. The IEM conducted comparative analysis of the CAIS and utilized results from Doing Business studies to reasonably capture the actual achievements of outcomes and outputs. On the whole, there have been marked improvements in customs operations as a result of the project. The outcome targets for customs infringement cases and improvement in the perception survey on corruption have been achieved. 34. On the extent of customs offenses being committed, apprehended violations increased by 4.3% on average, during the implementation of the CAIS. About 97%–99% of these violations have been administrative, such as undervaluation, misclassification, and misdeclaration. However, comparing these violations with the total number of declarations reflects a declining pattern—from 2.1% of total declarations to 1.2% of total declarations (Table 1).

26 Mongolia deposited its instrument of accession on 1 July 2006.

Performance Assessment 11

Table 1: Comparative Analysis of Mongolian Customs General Administration

Indicators “without” CAIS,

2006–2009 “with” CAIS, 2010–2014 Difference

Increase in customs declarations (%) 4.9 28.8 23.9 Increase in exports declarations (%) 0.3 66 65.7 Increase in imports declarations (%) 7.8 17.4 9.6 Number of declaration per staff 140 210 70 Revenue per staff (MNT) 413.7 1,018.6 605.9 Revenue per budget (MNT) 28.5 61.9 33.4 Growth in customs revenue (%) 16.8 28.7 11.9 Increase in number of violations (%) 3.7 4.3 0.6 Violation as % of total declaration 2.10 1.20 (0.83) ( ) = negative, CAIS = Customs Automated Information System, MNT = Mongolian togrog. Sources: Government of Mongolia, Mongolian Customs General Administration; Asian Development Bank, Independent Evaluation Department.

35. In terms of customs operations’ efficiency, the number of declarations processed by the MCGA increased by 24%, with the introduction of the CAIS. From 2006 to 2009, the total number of declarations grew by 4.9%, compared with 29% in the 2010–2014 period. Significant increases were also noticeable in the export and import declarations. As a result of the project, the number of customs declarations processed per staff rose from 140 to 210. The increase in the number of declarations also resulted in greater revenue collection, from a 16.8% increase during 2006–2009 to 28.7% in the 2010–2014 period, or an increase of 11.9%. Revenue collection as a ratio of the togrog budget also registered more than a two-fold increase, from MNT28.5 to MNT61.9 (Table 1). 36. The Asia Foundation’s Annual Survey of Perceptions and Knowledge of Corruption (SPEAK) shows that the nationwide perception of corruption in the MCGA has been improving through the years relative to other government institutions in Mongolia. 27 From being perceived as the second most corrupt agency in 2006 and consistently making the top five list until 2009, the MCGA ranking improved to seventh in 2015 among the 16 government institutions included in the survey (Figure).

27 The SPEAK surveyed 1,360 households in eight districts of Ulaanbaatar and in 22 soums of six aimags. The

survey presents nationwide perceptions of corruption at the household level; it captures data on citizens’ perceptions of corruption, and their actual experience of corruption at the household and national levels. Asia Foundation and Sant-Maral Foundation. 2015. Annual Survey of Perceptions and Knowledge of Corruption (SPEAK): Strengthening Transparency in Mongolia Project. http://asiafoundation.org/ publications/pdf/1513

Rank of Mongolia’s Customs Office in Public Perception Surveys

Source: Asia Foundation and Sant-Maral Foundation. 2015. Annual Survey of Perceptions and Knowledge of Corruption (SPEAK): Strengthening Transparency in Mongolia Project. http://asiafoundation.org/publications/pdf/1513

012345678

Mar

-…

Sep-

06

Mar

-…

Sep-

07

Mar

-…

Sep-

08

Mar

-…

Sep-

09

Mar

-…

Sep-

10

Apr

-11

Nov

-…

Mar

-…

Sep-

13

Mar

-…

Apr

-15

Rank

http://asiafoundation.org/


37. The rating of Mongolia’s burden of customs procedures suggests that business executives’ perception of the efficiency of customs procedures is improving.28 From a rating of 2.62 in 2007, it steadily rose over time to 3.30 by 2014. The lead time to export and import has been continuously decreasing, though the time to process documentary requirements remains—11 for export and 12 for imports.29 The average time to clear exports through customs decreased from 19 days in 2009 to 10 days in 2013 (Table 2).

Table 2: Doing Business Indicator and Logistic Performance Index for Mongolia, 2007–2014

Indicator 2007 2008 2009 2010 2011 2012 2013 2014 Burden of customs procedure 2.62 2.75 3.12 3.28 3.30 3.30 3.10 3.30 Lead time to export, median case (days) … … … 14 … 5 … 1 Lead time to import, median case (days) … … … 12 … 4 … 1 Documents to export (number) 11 11 11 11 11 11 11 11 Documents to import (number) 12 12 12 12 12 12 12 12 Average time to clear exports through customs (days) … … 19 … … … 10 …

Cost to export ($ per container) 1,807 2,131 2,131 2,131 2,265 2,555 2,745 2,745 Cost to import ($ per container) 2,274 2,274 2,274 2,274 2,400 2,710 2,950 2,950 Time to export (days) 44 44 41 41 41 44 44 44 Time to import (days) 44 44 42 42 42 45 45 45 Logistics performance index (LPI) 2.08 … … 2.25 … 2.25 … 2.36 Customs score (LPI) 2.0 … … 1.81 … 1.98 … 2.20 Distance to frontier 13.23 18.02 18.80 25.74 26.18 29.07 27.08 27.84 … = no data collected during the particular year. Sources: World Bank Doing Business Survey and World Development Indicators, World Economic Forum, Global Competiveness Report.

38. In comparison with other countries’ performance, the World Bank’s logistics performance index ranked Mongolia 135th out of 160 economies in 2014.30 In terms of the efficiency of the clearance process and the speed, simplicity, and predictability of formalities by customs and border control agencies, Mongolia has ranked 132 among 160 countries, a notch lower than its rank in 2007. Its score has increased slightly from 2.0 in 2007 to 2.2 in 2014. These indicate that Mongolia’s performance, relative to other countries, was not exceptional.

39. The logistics performance index also provides the distance to a “frontier” score to assess the absolute level of regulatory performance and how it improves over time.31 28 “Burden of customs procedure” measures business executives' perceptions of their country's efficiency of

customs procedures. The rating ranges from 1 to 7, with a higher score indicating greater efficiency. Data are from the World Economic Forum's Executive Opinion Survey, conducted for 30 years in collaboration with 150 partner institutes. The 2009 round included more than 13,000 respondents from 133 countries. Sampling follows a dual stratification based on company size and the sector of activity. http://data.worldbank.org/indicator/IQ.WEF.CUST.XQ

29 Lead time to export is the median time (the value for 50% of shipments) from shipment point to port of loading. Lead time to import is the median time (the value for 50% of shipments) from port of discharge to arrival at the consignee. Data are from the Logistics Performance Index survey. http://www.worldbank.org/lpi

30 The World Bank’s logistics performance index is a benchmarking tool to help countries identify the challenges and opportunities they face in their performance on trade logistics and what they can do improve their performance. The score is the weighted average of the country scores on six key dimensions: customs, infrastructure, international shipments, logistics quality and competence, tracking and tracing, and timeliness. The scorecards demonstrate comparative performance—the dimensions show on a scale (lowest to highest score) from 1 to 5, relevant to the possible comparison groups—of all countries (world), region, and income groups.

31 An economy’s distance to the frontier is reflected on a scale from 0 to 100, where 0 represents the lowest performance and 100 represent the frontier. For example, a score of 75 in Doing Business 2015 means an economy was 25 percentage points away from the frontier constructed from the best performances across


It shows the distance of each economy to the “frontier,” which represents the best performance observed on each of the indicators across all economies in the Doing Business sample since 2005. Results showed that Mongolia’s score started low in 2007 and started to pick up in 2011. Still, Mongolia is about 70 percentage points away from the “frontier.” 40. Expected outputs have been achieved. From the original plan of GAMAS’ migration and upgrading, a new system (CAIS) was developed. The CAIS is able to meet the need for cutting-edge technology and features as well as the functionalities and interfaces that could not be achieved with the mere updating of GAMAS. As a result, all trading transactions for filing customs declarations are now being undertaken online (Appendix 3). The manual declaration process has been removed, and customs clearing is centralized nationwide at MCGA headquarters. These changes could help to reduce opportunities for corruption. Only one template is being used for all the customs agencies either at the office or at the border crossing. In an aid-for-trade case study by the Organisation for Economic Co-operation and Development, the time consumed by import clearance is only 23 minutes with the CAIS as compared with 3 hours and 6 minutes with GAMAS, while that consumed by export clearance is 13 minutes as compared with 2 hours and 20 minutes under GAMAS.32 The project also established a central data warehouse in July 2009 which became fully operational a year later. 41. The project also supported the improvement of infrastructure at selected major customs houses and customs border posts, including the provision of inspection equipment and laboratory apparatus. The provision of testing and analysis equipment to the MCGA’s Central Customs Laboratory resulted in more accurate valuation of goods, reduction of tax loss caused by misclassification of goods, minimization of errors in classification at different custom houses, proper assessment and collection of customs duties and other taxes, detection of customs offenses, and enhanced customs control of prohibited and restricted goods within a shorter period of time. The project provided testing equipment in regional laboratories at Zamyn-Uud, Altanbulag, Bayan-Ulgii, Dornod, Bichigt, and Choir. Test results are transmitted through the CAIS to the relevant customs border crossing office, which reduced customs clearing time by 2–3 days and lowered concomitant expenses by 1.5–2.0 times.33 42. The project also provided advanced cargo surveillance equipment, including seven sets of x-ray cargo inspection machines at the Ulaanbaatar Customs House, and at Khavirga and Zamyn-Uud. Infrared detection instruments were deployed in six customs posts, while surveillance equipment, including closed-circuit televisions, was provided to eight customs posts. Likewise, the road cargo terminal at Zamyn-Uud was expanded. 43. The project also strengthened the MCGA through business process reengineering, capacity building, interagency coordination, regional cooperation, and public–private partnership. Training was provided to 2,771 customs officers on various subjects including customs legislation, classification, intellectual property rights, authorized economic operators, customs valuation, risk management, and use of the CAIS. Manuals were printed and distributed to customs officers and relevant

all economies and across time. A score of 80 in Doing Business 2016 would indicate the economy is improving. In this way the distance to the frontier measure complements the annual ease of doing business ranking, which compares economies with one another at a point in time.

32 Organisation for Economic Co-operation and Development. 2011. Aid for Trade: Case Study for the ADB's Mongolian Customs Modernization Project. Paris.

33 MCGA. 2011. Central Customs Laboratory Annual Report. Ulaanbaatar.


stakeholders. Bilateral talks have become regular between Mongolia and the People’s Republic of China after the harmonized cargo manifest between the Zamyn-Uud Custom House and Erlian Customs was launched on 15 December 2009 through a regional TA project. 34 Since then, both countries have agreed to further strengthen their cooperation. The partnership of the MCGA with the Mongolia National Chamber of Commerce and Industry through a permanent consultative committee has provided room for regular consultative meetings, exchange of information of mutual concern, establishment of channels for consultations, and sharing of best business practices.

C. Efficiency

44. The project is assessed efficient. The economic reevaluation resulted in an economic internal rate of return (EIRR) of 16.4% (Appendix 4, Table A4.6). The recomputed EIRR was lower than what was estimated at appraisal (19.6%) and at completion (37.2%) but still higher than the social opportunity cost of capital of 12.0%. The details on the assumptions and methodology used in the economic reevaluation, particularly the recomputation of the EIRR, are in Appendix 4. In terms of process efficiency, the project did not experience cost overruns but incurred a delay in physical completion of about a year (paras. 19–20). 45. With more than 19 border crossing posts in Mongolia, customs transactions were becoming complex and time-consuming, and required optimizing the use of the latest technology in data processing and storage, business transactions, and communications among various stakeholders, including the MCGA’s central and border posts. As the project’s focal component, operationalization of the CAIS enabled the MCGA to improve the efficiency of customs inspectors, to reduce customs clearance and processing transaction time, and to create a centralized repository of information collected for easier retrieval. 46. Both the report and recommendation of the President and the PCR estimated economic benefits in terms of (i) efficiency gains, through reduction in customs clearance time and better use of transport, and (ii) trade expansion. The estimation of savings in clearance time was based on the salaries of customs personnel while the savings in transport was derived from efficiency in the use of trucks. The IEM notes a few methodological issues in these estimations. First, savings in trucking costs were not the project’s intended benefits. Not all imports or exports move by trucks; they are also transported by rail and air. Also, customs clearance processing time is subject to wide variation, ranging from a few minutes to several days, which makes it difficult to estimate trucking cost savings with reasonable accuracy. Second, improved trade competitiveness resulting in increased trade volumes, while a valid benefit, could not be wholly attributed to the project. 47. Third, the appraisal and completion reports overlooked the rapid technological obsolescence to which computer systems and equipment are subject, such as (i) decreased capacity to process and store the increased transactions and additional applications integrated into the system, (ii) more frequent equipment breakdowns caused by heavy use, and (iii) non-availability of services and spare parts for equipment, as these may no longer be maintained by the suppliers and manufacturers.

34 ADB. 2006. Development of Regional Cooperation Programs for Mongolia and the People’s Republic of

China. Manila.


48. As an alternative, the IEM’s economic reevaluation utilized a different approach. Recent studies have provided estimates of the ranges of benefit values as a percentage of cargo value for different improvements undertaken by border management agencies including customs. These benefits occur in four areas (Table 3). The project’s economic benefits are calculated only for two types of benefits from the process improvements effected by the CAIS—reduction in customs administration costs engendered by the improved productivity of customs inspectors and the reduced costs as a result of savings in customs clearance time for the traders.

Table 3: Estimated Benefits for Various Improvements at Border Management Agencies

Type of Improvement Benefits Reasonable Range for

Estimated Benefits Reducing administrative costs Reduced costs for

government and traders –0.1% to 0.5% of cargo value

Reducing clearance times Reduced traders’ costs –0.5% to 0.8% of cargo value Reducing the variability of

clearance time Reduced inventory levels for

traders, leading to reduced traders’ costs

For each 50% reduction in the standard deviation, –0.25% of cargo value

Increasing competitiveness Increased export growth +1 percentage point Source: World Bank. 2011. Border Management Modernization. Washington, DC. 49. The efficiency in customs clearance processing using the CAIS is the result of two basic capabilities of the system—risk management, and information and transactions processing. The risk management capability classifies goods and persons or entities involved in the transaction using the red, orange, and green channels for customs control (Box). With time and experience, a high percentage of imports and exports should be categorized as green or orange, which would allow a more efficient flow of goods through customs. Customs inspectors would have more time available for the inspection of high-risk goods (red channel). Thus, more customs clearances will be processed than would have been possible under the old system. With import and export trade growing, resulting in an increase in customs clearance volume, the CAIS has contributed substantially to the MCGA’s capability in handling the increased transactions without having to hire additional staff. 50. Information and transactions processing also enable accredited brokers to input information into the system, submit it for processing, and pay the required duties, taxes, and fees. An e-payment capability has recently been installed in the CAIS (Appendix 3). The MCGA’s Central Customs Laboratory can transmit tests results on goods immediately to the border crossing where the goods are being held. These improvements have resulted in savings in overall customs clearance time for traders, estimated at about 8 hours.35 Other indirect economic benefits, such as reduction in informal payments, lower inventory cost, and lower depreciation or spoilage of perishable goods being transported, were not estimated although these could contribute to trade expansion. 51. For project cost, the IEM differed from the appraisal and completion reports with the inclusion of the KOICA grant of $2.5 million and the decrease in maintenance cost to $0.24 million. The IEM finds that the grant funded major elements of the CAIS and that, if it is excluded, the calculation ignores a major cost item that unduly increases the return on ADB funding of the project. In addition, the inseparability of the CAIS benefits requires that both ADB and KOICA funding be treated jointly. For the O&M costs, recent data provided by the MCGA showed a lower cost than used in the appraisal and completion

35 MCGA. 2015. Mongolian Customs Time Release Study Presentation. Ulaanbaatar.


reports. The IEM used a shorter project analysis period of 10 years compared with the 25 years used for both appraisal and project completion.

D. Sustainability

52. The project is assessed most likely sustainable. The IEM’s inspection found that the main project outputs are in generally good working condition and have been extensively used by the MCGA and its stakeholders. In terms of technical sustainability, the project is technically sound with sufficient O&M procedures in place. Since the completion of the project, several upgrades have been implemented in the CAIS to enhance its capabilities. In 2011, four modules and 30–40 submodules were rolled out and evolved to five modules and 60–70 submodules in 2015. The MCGA IT center has a staff complement of 17 fully trained IT engineers in its headquarters and about 40 IT personnel spread across Mongolia’s border posts. These well-qualified engineers have been trained for the O&M of the CAIS and have been in the forefront of designing and implementing system upgrades and routine O&M. Also, the MCGA has a career development program and special incentives in place to retain its IT engineers. Although intermittent system breakdowns are not uncommon in IT systems, in view of network and server overload and the usual wear and tear of physical hardware, the IT center has a turnaround time of 10–15 minutes to restore the CAIS. 53. In terms of institutional sustainability, the merger of the MCGA with the General Department of Taxation (now called General Administration of Customs and Taxation) in January 2016 was aimed at monitoring tax payments from businesses, including the importers, wholesalers, retailers, and end-users. All functions of the MCGA’s IT Center are now being handled by the Information Technology Center of Customs and Taxation under the General Administration of Customs and Tax. However, this is not likely to impede customs operational processes since the utilization of the CAIS in particular, has been fully embedded in Mongolia’s trading community.36 Also, the project implementation unit of the Regional Improvement of Border Services project, approved in April 2016, will be staffed with a dedicated ICT specialist to support the GACT’s IT department. Close collaboration is expected to ensure transfer of knowledge and build the capacity of government IT specialists, tasked with the maintenance of the system after project completion.37 54. The MCGA’s budgetary allocation from the government, in nominal terms, had increased over the years. In 2010, the government allocated MNT18,744.6 million and had increased the amount to MNT25,506.0 million in 2015. O&M expenditure likewise increased from MNT18,838.0 million in 2012 to MNT25,156.0 million in 2015. Excluding salaries, about 2% of the total expenditure was spent by the IT center in 2010 on IT-related services, the usage fee for the communication line, and routine operating expenses. This figure had increased to 4% in 2014. With the merger, the projected GACT expenditure is expected to increase by 24%, from MNT55,077.0 million in 2016 to MNT68,049.0 million in 2017. Budgetary expenditures for the succeeding years are expected to increase by 10% yearly. The government has given assurances

36 The results of the parliamentary election in June raise the possibility that the merger may not be fully

implemented and could be reversed by the succeeding administration if it so desires. The issues of interoperability and interconnectivity of the ICT systems of the two agencies are challenging and require support. The recently approved Regional Improvement of Border Services project will provide advisory support to develop and address the compatibility issues of the two ICT systems. This will be undertaken together with a business process integration strategy to inform the ICT systems interconnection process.

37 ADB. 2016. Proposed Loan to Mongolia for the Regional Improvement of Border Services in Mongolia. Manila.


that it will allocate adequate budgetary resources to cover recurrent costs associated with maintenance of the IT system and other facilities (footnote 37). 55. The government’s commitment and ownership strengthened the credibility of the project. It is committed to advance trade facilitation and customs modernization together with development partners. Under the Regional Improvement of Border Services project, the CAIS will be upgraded to provide added functionalities, will be made single-window ready, and will gain capacity for anticipated higher volumes of use. Operating and processing capacities are to be increased through system software installations and hardware upgrades, together with network upgrades. Customer support centers are to be established, as well as help desk and customer support software. Advisory support is expected to help in developing the interconnectivity and interoperability of information systems at the customs and taxation authorities. A business process integration strategy is to be formulated, which will help inform the ICT systems' interconnection process. A few civil works are planned for improvements at selected border crossing points, as well as preparatory works for the SEW.

CHAPTER 4

Other Assessments

A. Development Impact

56. The DMF indicated that the targeted impact was increased volume and smooth flow of trade. The performance indicator was a 10%–15% annual increase in external trade. Exports increased in value from $4.4 billion in 2012 to $4.7 billion in 2015. The value of imports, a year after completion, in 2012, was $6.7 billion but then gradually decreased to $3.8 billion in 2015 (Table 4). Given the importance of mining in trade, brought about by favorable mineral prices and FDI inflows in the sector, it is difficult to determine the project’s contribution to the increase in trade.

Table 4: Exports and Imports

($ million)

Item 2011 2012 2013 2014 2015 Exports 4,817 4,385 4,269 5,775 4,669 Imports 6,598 6,738 6,358 5,237 3,797 Trade balance (1,781) (2,353) (2,089) 538 872

Source: Government of Mongolia, Mongolia Statistical Information Service. 2016.

57. However, the project contributed to the creation of an institutional environment toward a SEW that would be based on the CAIS. The SEW will be an integrated electronic platform that will enable the parties involved in trade and logistics to lodge standardized information and documents with a single point of entry and to fulfill regulatory requirements through one-time submission of individual pieces of data. The project has facilitated trade flows by reducing delays for traders, which has encouraged micro, small, and medium enterprises and women entrepreneurs to participate actively in economic activities. 58. Given the landlocked topography of Mongolia, the project has helped facilitate transit trade between its large neighboring countries. Cargos in transit are entered and monitored under the CAIS to ensure that the transport of goods is unimpeded and that cargos are not diverted.38 The project is part of regional efforts to enhance the flow of goods along the Central Asia Regional Economic Cooperation corridor. In particular, the improved facilitation of goods and traders at border crossing points at Zamyn-Uud and Altanbulag has boosted regional trade. The project also built capacity for implementation, especially at the MCGA, and fostered learning. On the whole, the development impact is assessed satisfactory.

38 Transit arrangements may be treated like a "green" lane, where only documents need to be presented at

the border crossing points (both entry and exit).

Other Assessments 19

B. ADB Performance

59. In general, ADB’s performance is assessed satisfactory, for its diligent conduct of discussions with government agencies, particularly the Ministry of Finance and the MCGA, and support to the project management office in various areas of project implementation. ADB carried out six review missions to resolve implementation issues and ensure that ADB policies and procedures were complied with. In particular, ADB assisted the MCGA and the project management office in resolving difficulties related to the procurement process for some packages. It also demonstrated flexibility in approving the MCGA’s request for inclusion of a DRS funded through loan savings and the contingencies. Of significance was ADB’s approval of the additional parallel funding participation by KOICA to assist in the development of the new CAIS in lieu of the mere upgrade of GAMAS. ADB was directly involved from the very beginning in discussions with KOICA that finally led to signing of the parallel financing agreement. The stakeholders interviewed indicated that they were satisfied with ADB’s performance and appreciated the support provided during project implementation. However, some indicators in the DMF were poorly defined and difficult to monitor. These indicators were not adjusted during project implementation. The PCR prepared by the MCGA indicated the need to simplify ADB’s processing procedures in terms of procurement. Design changes were initiated by the government and subsequently approved by ADB (para. 24).39

C. Borrower and Executing Agency Performance

60. The performance of the borrower and of the executing agency are assessed satisfactory. Both coordinated well with ADB staff in reviewing the project’s progress. The implementation arrangements that were set up, including the PSC and various task forces, enabled the project to move forward.40 In particular, the establishment of the working group during the installation of the CAIS played a key role in operationalizing the system, given the limited time available to smooth out technical glitches. Recognizing the limitations of an upgraded GAMAS, both the borrower and the executing agency were quick to decide on the development of a new IT system. There was no major change in implementation arrangements during the project years.

39 MCGA. 2011. Customs Modernization Project Completion Report. Ulaanbaatar. 40 The PSC consisted of representatives from Ministry of Finance, MCGA, General Department of National

Taxation, State Specialized Inspection Agency, Mongolia Agency for Standardization and Metrology, and Mongolian National Chamber of Commerce and Industry.

CHAPTER 5

Overall Assessment, Issues, and Lessons A. Overall Assessment

61. Overall, the project is assessed successful, the second highest tier in a four-scale rating system (Table 5). The project is assessed relevant. Targeted activities in the project design generally reflected logistical constraints in trade facilitation, especially in the area of customs administration. Various project components had a common thread and interrelationships (paras. 12−17). The project design was significantly improved with the introduction of the CAIS, which was a new ICT system different from what was originally envisaged. This improvement was made possible through parallel financing with KOICA; ADB had no direct hand in its implementation. However, ADB’s financing of three subcomponents led to the completion of the CAIS. The project was also in line with the development strategies of the government and ADB. However, the DMF was of poor quality. It lacked clarity in terms of linking performance targets and the corresponding outputs, outcome, and impact statements. Some of these performance indicators did not have valid assumptions and did not come out of the application of the internal logic and analytical basis of the design underlying the project.

62. The project is assessed effective. The intended project outcome has been achieved. There have been noticeable improvements in customs operations. Targeted outputs have been achieved as well. The project is assessed efficient in the use of resources to achieve its intended outcome and outputs. Economic benefits were reduced administrative and traders’ costs, with the EIRR at 16.4%. The project was completed within budget, although physical completion was delayed by almost a year. The project is assessed most likely sustainable as sufficient O&M are in place. The CAIS has been upgraded from time to time and will undergo major upgrading once the Regional Improvement of Border Services project has been implemented. 63. Key factors for the successful rating were (i) the government’s strong ownership of the project, (ii) the decision to set up an entirely new IT system instead of just upgrading the existing one, and (iii) the strong government and private sector partnership.

Overall Assessment, Issues, and Lessons 21

Table 5: Overall Assessment of Project Performance

Evaluation Criteria PCR PPER Key Reasons for Disagreements

and Comments Relevance Highly relevant Relevant The most viable technical option was not

considered in the initial project design. Also, some of the performance indicators could have been more realistically formulated. A DRS should have been incorporated in the original design.

Effectiveness Highly effective Effective Envisaged outcome indicators have been achieved, but Mongolia’s performance relative to other countries is not exceptional in these areas.

Efficiency Highly efficient Efficient The economic viability indicator exceeds the social discount rate, but its value is not that high.

Sustainability Most likely sustainable

Most likely sustainable

Overall assessment Highly successful

Successful

Preliminary assessment of impact

Positive Satisfactory

Borrower and executing agency

Satisfactory Satisfactory

Performance of ADB Highly satisfactory

Satisfactory Some DMF indicators were poorly defined and difficult to monitor. These could have been adjusted during project implementation.

ADB = Asian Development Bank, DMF = design and monitoring framework, DRS = disaster recovery system, PCR = project completion report, PPER = project performance evaluation report. Source: ADB Independent Evaluation Department.

B. Issues

64. In retrospect, a detailed and thorough feasibility study could have clarified the alternative designs for the project. From these could have evolved the proposed set of technically feasible options, especially for the automated data processing system to be used by the MCGA. However, the most viable technical option that could meet the MCGA’s medium-term needs was not considered or firmed up in the initial project design. It was assumed that the increasing demands of customs control and administration could be met by merely migrating, adding, and upgrading the programming tools, functionalities and communications infrastructure of GAMAS. Although an ICT master plan was developed, the blueprint and standards for software components and technological infrastructure were still to be based on the planned GAMAS modifications and upgrade of its features. Also, in view of limited resources, infrastructure improvements had to be done on selected key customs houses and border posts. A project preparatory TA could have been considered to better prepare the project, especially the possible future phasing and sequencing of infrastructure investments in other border post crossings not covered under the project (para. 21). 65. A DRS is essential in preventing loss of data in case of system breakdown or anomaly as a result of natural or manmade disasters. Thus, it should have been made a prerequisite component in the original project design. A DRS was eventually included near project completion in January 2011, almost 6 months after the government


officially accepted the CAIS. However, this unduly exposed the CAIS to possible security risks.

C. Lessons

66. A customs modernization project, aimed at reducing corruption, needs a holistic approach that combines technical, policy, and capacity-building elements. In this case, it is an integration of three interrelated components: (i) IT infrastructure, (ii) laws and procedures, and (iii) coordination among concerned agencies. The IT infrastructure is important in reducing direct, face-to-face interactions that provide opportunities for corruption. Customs laws and procedures, in line with international standards, could provide the institutional framework for a modernized business environment. Likewise, close cooperation is needed among government agencies involved in examining and issuing licenses, in terms of aligning and lessening differences in cross border procedures and systems. One component could not be tackled without the other components. For instance, the provision of IT infrastructure alone would not be sufficient to improve the MCGA’s business processes and procedures. Reforms of customs laws and operational regulations should be established initially so that other related components can ensure their implementation. Moreover, trade facilitation and customs modernization is not the responsibility of the MCGA alone. Close collaboration with other stakeholders is essential to address constraints to trade. 67. The fast-changing nature of the IT environment requires that a project’s IT infrastructure (e.g., operating systems, core data processing applications, network, and telecommunications technologies) have enough flexibility in terms of compatibility, connectivity, and modularity to integrate with technology development. Hence, a customs modernization project must be planned so as to take into account the possible obsolescence of IT infrastructure in a shorter time period after a project’s completion than originally envisaged. 68. A partnership between customs and the private sector should be a major thrust of a customs modernization project. Extra effort must be devoted to ensure proactive engagement with the private sector, particularly with the trading community. This could help ensure buy-in, improve the user-friendliness of the IT system, and facilitate public–private partnership in customs administration. Establishing an ongoing platform for regular interaction between the private sector and customs officials would also assist in identifying issues, problems, and proposed improvements and in facilitating acceptance of reengineered business processes for the newly introduced IT system. 69. The collection of baseline data should be given high priority. Efforts could have been exerted to ensure that baseline data were established during appraisal stage. Without measurable baseline values set up at appraisal stage, meaningful indicators should be carefully formulated to allow continuous monitoring for updates throughout project implementation. The project’s independent evaluation was hampered by the absence of these baseline indicators.

D. Follow-Up Actions

70. Based on the evaluation findings, two follow-up actions are proposed: ADB should remain engaged in supporting and monitoring the developments pertaining to the move toward a SEW and how interconnection among the concerned agencies’ IT systems, especially those in the border crossing posts, would be made as well as their

Overall Assessment, Issues, and Lessons 23

eventual link-up with the CAIS. In particular, ADB could provide support to the government through capacity-building TA projects, taking into consideration ongoing related operations in the areas of trade facilitation and ICT. This TA should also help in incorporating best practices and should advance customs practices in the customs operations.

71. Also, ADB could provide support to the government in strengthening benefit and information monitoring, especially in improving data gathering, monitoring and evaluation of economic benefits, and social impacts of trade facilitation initiatives. This support may be undertaken through a capacity-building TA. TA resources may be devoted to improving systematic monitoring of indicators related to customs services such as the number of documents processed per hour and average clearance time. This data gathering and monitoring should also include information related to trade flows.

Appendixes

APPENDIX 1: DESIGN AND MONITORING FRAMEWORK

Design Summary Performance

Targets/Indicators

Assessment Project Achievements Impact Increased volume and smooth flow of trade

10%–15% annual increase in external trade (starting from the first year after project completion as compared with the previous year)

Achievable Trade turnover increased in value by 27% on average from 2010 to 2014. Exports increased annually by 28.7% and imports by 26.9%, during the same period.

Outcome Improved efficiency, transparency, and sustainability of customs services and administration

Number of cases of customs infringement to be reduced by 10% annually during the first 5 years after project completion

Achieved

The number of customs offenses, both administrative and civil, increased annually by 4.3% from 2010 to 2014. However, as a percentage of the total declaration, the offenses decreased by 16.5% annually during the same period.

Ranking of the MCGA in assessment or survey of perception index of corruption to improve above the average among peer agencies or ministries in Mongolia (starting from the second year after project completion)

Achieved

The public perception of the MCGA relative to other government agencies in Mongolia has been improving through the years—from being perceived as the second most corrupt agency in 2006 and consistently making the top 5 list until 2009, the public perception of the MCGA improved to seventh in 2015.a

Outputs Migration and upgrading of the GAMAS system to an internet-enabled environment and central database

Number of online customs declarations increased from 50% before project completion to 90 % by project completion

Achieved With the new CAIS in place in 2010, all trading transactions for filing customs declarations are now undertaken online.

Establishment of central data warehouse by project completion

Achieved Trial operation of the data center commenced in July 2009. It became fully operational a year after.

Improvement of infrastructure at selected major customs houses and customs border posts, including provision of inspection equipment and laboratory apparatus

50% reduction in the time needed for customs inspection and clearance (compared with before the infrastructure improvements) by project completion

Achieved Time consumed for import clearance is 23 minutes compared with 3 hours and 6 minutes under GAMAS. Export clearance time is 13 minutes compared with 2 hours and 20 minutes under GAMAS.b There is significant variation in clearance processing time, depending on the channel classification of each transaction under the risk management system.

60% reduction in the amount of time needed for customs laboratory analysis results (including cross-city transport) by project completion (compared with before the project)

Achieved Customs clearing time was reduced by 2–3 days and savings from expenses are lower by 1.5–2 times. Certain tests can be done within 1 day depending on the type of commodity, with the results communicated through the CAIS. However, samples of goods to be tested that are outside the testing capability of cross-border laboratories have to be brought to the Central Customs Laboratory in Ulaanbaatar for testing.

Institutional strengthening, comprising business process reengineering,

100 customs officers trained with modernized customs environment

Achieved

Training was provided to 2,771 customs officers and on a limited scale to border-post customs officers and business users, on various subjects including customs legislation, classification, intellectual property

26 Appendix 1

Design Summary Performance

Targets/Indicators

Assessment Project Achievements capacity building, interagency coordination, regional cooperation, and public–private partnership

rights, authorized economic operators, customs valuation, risk management and use of the CAIS.

Client satisfaction to exceed 80% by project completion

Insufficient evidence to assess achievement

Users’ perception of the CAIS has been generally positive, on the basis of the IEM interviews.

Manual of reengineered business processes printed and circulated to customs officers

Achieved Manuals were printed and distributed to customs officers and relevant stakeholders.

Coordination with SSIA and MASM improved

Insufficient evidence to assess achievement

Efforts have been made in improving business processes under the 2008 Customs Laws and related customs regulations but the extent to which this has translated to improved coordination between these two agencies is not clear.

Procedures harmonized and data exchanged in customs cooperation with neighboring countries

Achieved A harmonized cargo manifest between the Zamyn-Uud Custom House of Mongolia and Erlian Customs in the PRC was launched in 15 December 2009. Since then, bilateral talks between Mongolia and the PRC have been regular and both parties have agreed to further strengthen existing cooperation.

Participation by the private sector in project implementation and expansion of the customs–business partnership

Achieved The MCGA had a business partnership with the MNCCI through a permanent consultative committee. The partnership provides a framework for regular consultative meetings for representatives from customs administrations and business associations. The agreement also includes exchanging information of mutual concern, establishing official channels for consultations, and sharing best practices on customs warehouses, bonded exhibition sites, and customs special zones. Both the MCGA and the MNCCI have involved in performance surveys of customs services and dialogue about customs improvements.

ADB = Asian Development Bank, CAIS = customs automated information system, IEM = independent evaluation mission, MASM = Mongolia Agency for Standardization and Metrology, MCGA = Mongolian Customs General Administration, MNCCI = Mongolian National Chamber of Commerce and Industry, PRC = People's Republic of China, SSIA = State Specialized Inspection Agency. a Asian Foundation. 2015 Annual Survey of Perceptions and Knowledge of Corruption (SPEAK). http://asiafoundation.org/

publications/pdf/1513 b Organisation for Economic Co-operation and Development. 2011. Aid for Trade: Case Study: The Mongolian Customs

Modernization Project. http://www.oecd.org/aidfortrade/48292480.pdf Sources: ADB. 2006. Report and Recommendation of the President to the Board of Directors: Proposed Loan and Grant Administration to Mongolia for the Customs Modernization Project. Manila; ADB. 2014. Completion Report: Customs Modernization Project in Mongolia. Manila; ADB Independent Evaluation Department.

http://asiafoundation.org/publications/pdf/1513

http://asiafoundation.org/publications/pdf/1513

http://www.oecd.org/aidfortrade/48292480.pdf

APPENDIX 2: APPRAISAL AND ACTUAL PROJECT COSTS

Table A2.1: Total Project Cost by Component

Item Appraisal Actual A. ADB Loan

Component 1 (SDR) ICT hardware 275,000 444,446 ICT network 779,000 789,625 Workstation and computer 188,000 169,959 ICT software 495,000 446,371 Application development 417,000 610,034 Component 2 (SDR) Supervision and inspection 378,000 195,899 Laboratory and analytical equipment 293,000 525,759 PMO furniture 8,000 6,515 PMO salaries 134,000 117,976 Interest charges 40,000 40,000 Unallocated/contingencies 368,000

Total SDR 3,375,000 3,346,584 Total Equivalent (Components 1 and 2, $) 5,000,000 5,196,161 B. KOICA Parallel Funding ($)

Provision of facilities and equipment 450,000 … System development 1,821,000 … Dispatching of specialist 134,000 … Trainings in Korea 63,000 … Miscellaneous 32,000 …

Subtotal 2,500,000 … Total ($) 7,500,000

... = data not available, ICT = information and communication technology, KOICA = Korea International Cooperation Agency, PMO = project management office, SDR = special drawing right. Sources: Asian Development Bank. 2014. Completion Report: Customs Modernization Project in Mongolia. Manila; KOICA. 2012. Ex-Post Evaluation Report on the Two E-Government Projects in Mongolia. Gyeonggi-do.

Table A2.2: Total Project Cost by Financing

… = data not available, ADB = Asian Development Bank, IDC = interest during construction, KOICA = Korea International Cooperation Agency. Sources: ADB. 2014. Completion Report: Customs Modernization Project in Mongolia. Manila; KOICA. 2012. Ex-Post Evaluation Report on the Two E-Government Projects in Mongolia. Gyeonggi-do.

Costs Appraisal Actual Implementation costs Borrower 1.26 1.34 ADB 5.00 5.20 e-Asia and Knowledge Partnership Fund 0.50 0.48 KOICA parallel financing 2.50 … Total 9.26 IDC Costs Borrower 0.00 0.00 ADB 0.60 0.61

APPENDIX 3: OVERVIEW OF RECENT INFORMATION SYSTEMS IN MONGOLIAN CUSTOMS GENERAL ADMINISTRATION A. Customs Automated Data Processing System 1. Mongolian information technology (IT) programmers developed the customs automated data processing system (GAMAS) in order to capture customs data from the export and import declarations submitted by the trading community. It was introduced in 2003 to cover various functions such as customs registration, duties, declaration, and statistics. The introduction commenced in the major customs offices at Ulaanbaatar, Selenge, and Zamyn-Uud. The second stage, introducing GAMAS version 2.0, started in 2004. This version covered remote declaration of freight through the information network and the introduction of online banking. Two hundred seventy computers and equipment were procured for the use of GAMAS. 2. GAMAS consisted of three main shells—software, information network, and hardware. The software included a complex information program for customs registration and control, a general intranet for the customs, and a website. A particular feature of GAMAS was the information network, which covered all customs services in Mongolia using fiber-optic cable, a very small aperture terminal, and a modem. The major customs offices were linked to the central customs administration through a very small aperture terminal. The information system in the main custom office at Ulaanbaatar functioned in online mode; it covered 65% of all customs registration and accounted for 90% of the customs duties. However, GAMAS was written using Visual Basic, a programming language, and ran in an SQL (standard computer language for assessing and manipulating databases) environment. These were outdated tools, which were good for their time during the 1990s. They have since been overtaken by more advance tools such as Java. Also, GAMAS did not utilize advanced technologies such as data warehousing, data mining, a business rule engine, a workflow engine, and security features such as public key infrastructure, digital signature, and watermarking. 3. Given the absence of these features, GAMAS was primarily people driven, i.e., human intervention was required in the processing and clearance of goods. Often, this resulted in a lack of transparency and unpredictability of service. For instance, the trading community had to submit flat-files on diskettes for extraction by the Mongolian Customs General Administration (MCGA). In most cases, data were transmitted only on a weekly basis from customs offices and border posts where the volume of transactions was low. Data from border points without telephone lines was sent to the headquarters on floppy disks.1 The data transmission speed was slow, often at only 28.8 kilobytes per second or lower for telephone line dial-up, and was usually unreliable. The improvements in GAMAS had minimal benefits. Manual submission of the customs declaration forms required traders to move from one customs processing point to another with the accompanying requirement of official stamping and signatures. Whenever there was a need for system upgrades, to fix bugs or update and change the codes, the information was transmitted to the various servers over the network or using compact discs, floppy disks, or flash drives. The updating and synchronizing of the system usually took several days to complete. The unreliability of this transmission mechanism could lead to loss of data, which may not be easily detected. The time lag in transmission of data to the central database did not allow real-time compilation of reports and statistics, which is an essential tool to support decision making. In addition, GAMAS did not have enough server capacity.

1 Transmission of data was subject to the availability, reliability, and stability of the network, which comprised a mix of optical

fibers for Ulaanbaatar, Zamyn-Uud, and Sukhbaatar customs houses; very small aperture terminal satellite links at Altanbulag and Bor Undur; a microwave connection to the Buyan-Ukhaa international airport; and telephone line dial-up for the rest of the border posts.

Overview of Recent Information Systems in Mongolian Customs General Administration 29

B. Customs Automated Information System 4. In 2005, Asian Development Bank (ADB) supported a study to identify the deficiencies in customs operation that constrained the adoption of trade facilitation measures. 2 The study recommended addressing legislative amendments, physical infrastructure development, capacity development, and business process reengineering. One of the recommendations pertained to improvements in the information and communication technology (ICT) infrastructure within and outside the MCGA. As use of GAMAS continued to expand, there was a need for more development work and utilization of better technology. The MCGA recognized that outdated technology might need to be upgraded to improve the efficiency and transparency of customs services. 5. During implementation of the Customs Modernization Project, minor changes were made which included migrating and updating GAMAS to a new system called the Customs Automated Information System (CAIS).3 This was in response to up-to-date technical requirements and features as well as the functionalities and user interface, which posed complications even for just updating GAMAS. The collaboration of the MCGA with the Korea IT Industry Promotion Agency resulted in discussions with a Korean company—KTNET, which specializes in customs automation and single electronic window (SEW) projects. KTNET also eventually became the contractor for the CAIS after a minor change in scope, changing the procurement method from national competitive bidding to international competitive bidding. With the additional parallel financing from the Government of Korea through the Korea International Cooperation Agency (KOICA), the MCGA proceeded with the CAIS. The partnership was formalized in 2008 with a tripartite memorandum of understanding with the Government of Mongolia through the MCGA, ADB and KOICA; that set the scope of collaboration. KOICA agreed to provide $2.5 million in the form of parallel financing. ADB financed three of five subcomponents of the CAIS, including an external portal system, integrated customs information systems, and ICT hardware and related equipment. KOICA financed the customs administration information system and external connection (Figure A3.1).

6. The data center, which is the core of the CAIS, was set up and opened for trial operation on 2 July 2009. During this period, extensive testing of the operation of software and hardware components was undertaken to identify defects and work out remedial measures. A working group of senior officials, from various departments of the MCGA and the three major customs houses, and an acceptance committee were formed to oversee the final testing and acceptance. Comments and suggestions were collected from various users, including MCGA headquarters, key customs houses across the country, and business users such as customs brokers and freight forwarders. More than 95% of these comments had been addressed through the changes and improvements in the system. Training was also provided to the MCGA’s IT staff and on a limited scale to customs officers in selected border customs posts and business users. The CAIS component of the contract was delivered and physically completed by KTNET on 28 February 2010. KTNET also provided extra efforts after the turnover date to further test and improve the system. 7. The CAIS was officially launched and accepted on 5 July 2010 after a year of pilot operation. GAMAS has ceased operation since then. Initial response from the customs staff, customs brokers, and traders had been less than satisfactory, with the usual start-up problems of network overload, interconnection issues with border crossings, and inadequate technical user training. This was also exacerbated by the resistance of MCGA staff to change and problems of energy supply during that time. The number of complaints has eventually gone down as business users such as traders and

2 ADB. 2005. Mongolia: Trade Facilitation and Customs Modernization Project. Technical Assistance Consultant’s Report. Manila. 3 ADB, East Asia Department. 2009. Memorandum of Understanding between ADB and MCGA and Department of Development

Financing and Cooperation for Loan 2307-MON: Customs Modernization Project Review Mission, 7–10 December (internal).

30 Appendix 3

customs brokers became more familiar with CAIS and after updates and changes were made to the system.4

8. The CAIS has a centralized database and server. Users anywhere can access the CAIS through the internet, and the application system and database runs on a central server. Unlike GAMAS, which operated in a decentralized environment, information are transmitted to a local server for each customs house and border post and then re-transmitted to MCGA main database (Figure A3.2). The CAIS has met the increasing demands of customs control and administration, and has continuously been upgraded with other applications and business processes such as risk management, self-assessment, and post-clearance audits. Through electronic submission of trade documents, better coordination with border crossing points has been made. 9. With the CAIS, the MCGA can accept, process, and issue export and import clearances through entirely online submissions. The CAIS eliminates the manual declaration process. Customs clearing is centralized nationwide at MCGA headquarters. A case study undertaken by the Organisation for Economic Development and Co-operation indicated that the time consumed for import clearance is only 23 minutes with CAIS as compared with 3 hours and 6 minutes with GAMAS while that of export clearance is at 13 minutes as compared with 2 hours and 20 minutes with GAMAS. 5 One of the modules of the CAIS is the risk management system, whereby traders experience faster and easier

4 ADB, East Asia Department. 2011. Back-to-Office Report for Loan 2307: Customs Modernization Project Review Mission.

7 October (internal). 5 Organisation for Economic Development and Co-operation. 2011. Aid for Trade: Case Study for the Mongolian Customs

Modernization Project. Paris.

Figure A3.1: Scope of ADB and KOICA Project under the Customs Automated Information System

ADB = Asian Development Bank, CCI = Mongolia National Chamber of Commerce and Industry, EDI = electronic data interchange, GDNT = General Department of National Taxation, ICT = information and communication technology, KOICA = Korea International Cooperation Agency, MASM = Mongolian Agency for Standardization and Metrology, OGA = other government agencies, SGA = Statistics General Agency, SSIA = State Specialized Inspection Agency, XML = extensible markup language. Source: Mongolian Customs General Administration. 2010. Mongolian Customs Modernization Project. Presentation during the Ninth Customs Cooperation Committee Meeting, Tokyo. 6–10 September. http://www.carecprogram.org/uploads/events/ 2010/9th-CCC/Day1-Plenary2-Modernization-MON.pdf

Overview of Recent Information Systems in Mongolian Customs General Administration 31

customs clearance. With improved risk management based on risk selectivity, the MCGA has been able to allocate its scarce resources to high-risk cargos, while improving the clearance process for low-risk cargos.

10. The CAIS also makes possible an e-payment system, which eliminates errors and omissions in customs-related payments made through bank tellers. Multiple payment modalities are available using barcode and invoice numbers for bank teller transactions, internet banking, and mobile payments. It also provides real-time payment information to traders and customs officials, making the clearance process faster and eliminating one required document. The system links the MCGA with eight commercial banks through the Mongol Bank interbank clearance system. As of 31 July 2015, it had handled more than 52,000 transactions, representing 90% of the total number of customs payments, totaling $58 million.6 11. The CAIS has much capacity to accommodate future changes. In its original framework, the CAIS had about 4 general modules and 30–40 submodules. In 2015, there were 5 general modules and about 60–70 modules. This system provides a whole new level of opportunities for the usage and application of the CAIS, including interoperability with other systems as well as the planned SEW environment. On an annual basis, the MCGA undertakes a formal review of the CAIS, during which it considers suggestions and comments from staff as well as traders and customs brokers, and includes necessary adjustments in the annual work plan of the IT department. 12. The CAIS is considered one of the most advanced IT systems in Mongolia among government institutions. It is tailored for Mongolia customs and uses world-class technology, including its operating system and software language. The CAIS has been the platform for customs ICT, which could allow integration with SEW and e-government systems. With the CAIS, it is possible to exchange data at both domestic and international levels with information security in place. The CAIS also is ready for information and data sharing with the international community as a 10-gigabyte fiber-optic cable passes through Mongolia, connecting Thailand to London. The implementation of customs control techniques such as risk management, self-assessment, supply chain management, coordinated border management, and joint customs control can be readily undertaken from the IT system side.

6 Chemonics International Inc. 2015. Mongolia Business Plus Initiative, Final Report. Washington, DC: United States Agency for

International Development.

Figure A3.2: Comparison of the Customs Automated Information System and GAMAS

CAIS = customs automated information system, DB = database, MCGA = Mongolian Customs General Administration. Source: MCGA. 2010. Mongolian Customs Modernization Project. Presentation during the Ninth Customs Cooperation Committee Meeting, Tokyo. 6–10 September. http://www.carecprogram.org/uploads/events/2010/9th-CCC/Day1-Plenary2-Modernization-MON.pdf

APPENDIX 4: ECONOMIC REEVALUATION 1. At project appraisal, the economic internal rate of return (EIRR) estimate was 19.6%, and at completion, 37.2%. The methodology used was similar for both project appraisal and completion. The independent evaluation mission (IEM) refined the methodology on the basis of more current approaches on, for example, the identification and valuation of economic benefits from customs modernization projects, specifically relating to information and communication technology improvements. A. Methodology 2. This economic reevaluation used the following assumptions and factors in the estimation of the project’s economic costs and benefits:

(i) All values are expressed in US dollars. (ii) The analysis covers a 10-year period, including the 4-year project implementation period.

The appraisal and completion reports used the 25-year project periods. This was adjudged too long and did not consider the technological obsolescence to which information technology projects are subject to.

(iii) The financial capital costs and subsequent conversion into economic costs are the same as in the completion report and excluded price contingencies.

(iv) Economic costs were calculated on the basis of the annual project costdisbursements, with annual recurrent costs estimated based on actual 2015 costs to the Mongolian Customs General Administration (MCGA) for information technology operation and maintenance, adjusted to 2006 prices. Capital costs and operation and maintenance costs were adjusted using a shadow exchange rate factor of 1.10. Tradable goods are about 55% of project costs. The KOICA grant of $2.5 million, which funded a major component of the Customs Automated Information System (CAIS) development, was included in the capital cost for a more realistic recomputed EIRR.

3. The differences in the appraisal and completion reports and the post-evaluation approaches were in the estimate of capital and of operation and maintenance costs and the calculated benefits. The benefits calculated at appraisal and completion were on efficiency and trade gains as a result of the project. The shorter time for customs processing of imports and exports generates efficiency gains in the form of lower transportation costs and customs clearance costs (savings). The decrease in customs processing waiting time would redound to more trips made by trucks and hence lower trucking costs, assuming savings are passed on to the traders. The improvement in customs processing was assumed to generate increased trade due to the more predictable environment in imports and export processing at customs. 4. The IEM determined that the delays in import and export processing are not entirely attributable to customs clearance processing. The complexities of international trade require permits, licenses, and other documentation from different government agencies. The MCGA validates these trade documents as part of documentary validation procedures to prevent spurious declarations and entry of illegal goods into the country. In certain cases, the classification of import and export goods requires validation, and tests are undertaken by the Customs Central Laboratory (CCL) in Ulaanbaatar or customs laboratories in selected border crossings. The latter have limited capacity, and samples have to be transported and tested at the CCL or its certified laboratories and the results relayed to the customs border station using the CAIS. This was not possible under the GAMAS. For contested results, the shipment is held until the issue is resolved. Thus, attribution of delays in clearing goods shipments to

Economic Reevaluation 33

the MCGA alone is not correct. Customs inspection of imports and exports are made at the closest customs house in the import’s final destination as indicated in its documents.

5. The IEM determined that savings in transport costs fell outside of the project’s intended benefits and have unresolved computation issues. Not all imports or exports are being moved by trucks. Railways and air transport are also being used (Table A4.1). For example, the movement of freight between Mongolia and the People's Republic of China uses rail and road transport or both, whereas transit freight movements between the Russian Federation and the People's Republic of China use rail.

Table A4.1: Volume of Freight Carried by Mode of Transport, 2000–2015

Year Total

(1,000 tons) Mode (1,000 tons) Percentage of Total (%)

Railway Road Air Railway Road Air 2000 10,643.4 9,158.5 1,480.4 2.9 86.00 13.90 0.03 2001 11,810.5 10,147.7 1,658.2 2.9 85.90 14.00 0.02 2002 13,529.9 11,637.0 1,888.7 2.4 86.00 14.00 0.02 2003 17,622.8 12,284.7 5,335.9 2.2 69.70 30.30 0.01 2004 21,596.0 14,031.8 7,561.9 2.3 65.00 35.00 0.01 2005 23,670.5 15,586.3 8,081.7 2.0 65.80 34.10 0.01 2006 23,971.4 14,779.8 9,189.4 2.2 61.70 38.30 0.01 2007 23,281.6 14,072.6 9,207.1 1.9 60.40 39.50 0.01 2008 23,904.4 14,646.9 9,255.7 1.8 61.30 38.70 0.01 2009 24,736.7 14,171.5 10,563.8 1.4 57.30 42.70 0.01 2010 29,415.9 16,804.0 12,610.2 1.6 57.10 42.90 0.01 2011 44,086.0 18,447.7 25,635.3 2.9 41.80 58.10 0.01 2012 53,348.1 20,445.2 32,898.9 4.0 38.30 61.70 0.01 2013 42,557.3 21,035.5 21,517.7 4.1 49.40 50.60 0.01 2014 44,636.2 21,118.6 23,514.2 3.4 47.30 52.70 0.01 2015 35,828.9 19,143.9 16,682.2 2.8 53.40 46.60 0.01

Source: Government of Mongolia, Mongolia National Statistics Office. 2016. 6. The efficiency in customs clearance processing using the CAIS depends on two basic capabilities of the system—information exchange and risk management. For the information exchange, accredited brokers input information into the system, submit it for processing, and pay the required duties, taxes, and fees. An e-payment capability has been recently integrated into the CAIS. From the CCL, test results are transmitted to the border crossing where the import or export is held. For risk management, import and export goods are classified based on type, propensity for misdeclaration, and the customs transactions records of the importer, exporter, and broker. Customs uses the red, orange, and green channels for customs control: (i) the red channel requires mandatory physical and documentary inspection of the goods, (ii) the orange channel requires mandatory check of documents only, and (iii) the green channel requires only a few fields of the customs declaration to be checked. Through time and experience, a high percentage of imports and exports should be categorized “green or orange,” allowing more efficient flow of these goods through customs. Customs inspectors could then focus their time and effort in the thorough inspection of red channel goods.

7. The project enabled customs to process more clearances than would have been possible under the old system. With import and export trade growing and the attendant increase in customs clearance volume requirements, the project has contributed to the MCGA’s capability to handle the increased transactions, according to customs officials and customs brokers alike. Table A4.2 shows the trade value, customs clearance volume, customs revenue to the State, and number of apprehensions for administrative and criminal violations from 2005 to 2014. By 2011, when the CAIS was brought online, there had been a 48.8% increase in the customs clearance volume. The capability of the CAIS to process

34 Appendix 4

transactions extends to the border crossings, especially in Zamyn-Uud, the main border for the People's Republic of China trade.

Table A4.2: Performance of the Mongolian Customs General Administration, 2005–2014

Year

Foreign Trade Value ($ million) Customs Clearance Volume Customs Violation Customs

Revenue Exports Imports Total Exports Imports Total Admin Criminal Total 2005 1,063.9 1,177.3 2,241.2 34,922 86,186 121,108 2,916 66 2,982 291,000 2006 1,542.0 1,435.0 2,977.0 37,541 106,503 144,044 2,551 91 2,642 360,500 2007 1,947.5 2,061.8 4,009.3 30,390 127,634 158,024 2,963 61 3,024 418,700 2008 2,534.5 3,244.5 5,779.0 30,802 165,510 196,312 4,030 69 4,099 587,200 2009 1,885.4 2,137.7 4,023.1 34,288 96,494 130,782 3,006 112 3,118 511,100 2010 2,908.5 3,200.1 6,108.6 43,769 143,638 187,407 2,875 85 2,960 877,600 2011 4,817.5 6,598.4 11,415.9 64,636 214,187 278,823 3,254 62 3,316 1,405,000 2012 4,384.7 6,738.4 11,123.0 67,428 208,015 275,443 3,312 49 3,361 1,444,000 2013 4,269.1 6,357.8 10,626.9 62,871 201,486 264,357 3,020 117 3,137 1,612,026 2014 5,774.6 5,236.6 11,011.2 224,588 191,133 415,721 3,666 94 3,760 1,566,298

Source: Government of Mongolia, General Customs Administration. 2016. 8. Border management projects usually include benefits such as (i) predictions of reduced costs for traders in preparing customs documents and other clearance documents for border management agencies, together with reduced facilitation payments; and (ii) predictions that cargo will be cleared faster and more predictably—allowing traders to maintain smaller inventories—and that international trade will expand as the economy becomes more competitive. An Organisation for Economic Co-operation and Development study in 2003 suggested that clearance costs range from 3.5% to 15% of the value of imported cargo.1 9. The benefits from the project are to be achieved over time, as features are introduced and as the MCGA and its partner-stakeholders adapt to the change. The expected benefits as identified by the evaluation are (i) increased productivity by customs officers in the processing of customs clearances, resulting in decreased administration costs and considered as savings to the government; and (ii) improvement in customs clearance times, generating savings that accrue to the trader. To estimate the benefits of improved customs clearance times, the evaluation utilized the values as given in Table A4.3 as references and adjusted for application to the estimate of project benefits. Only two types of improvement at border agencies were utilized: reducing administrative costs and reducing clearance times. The other identified benefits—reducing the variability of clearance time, and increasing competitiveness—were not considered, as data to estimate variability of clearance time were not available and estimating increased export growth would not have contributed much to the accuracy and reliability of the EIRR and net present value recomputation. It is further noted that the project did not address all problems at border management agencies, but only specifically those concerning the MCGA. Thus, there are still unresolved problems, some of which will be addressed by the recently approved Asian Development Bank loan to Mongolia on the Regional Improvement of Border Services.2

1 World Bank. 2011. Border Management Modernization. Washington, DC. p. 90. 2 ADB. 2016. Report and Recommendation of the President to the Board of Directors: Proposed Loan to Mongolia for the

Regional Improvement of Border Services Project. Manila (Loan 3387).


Table A4.3: Estimated Benefits for Various Improvements at Border Management Agencies

Type of Improvement Benefits Reasonable Range for

Estimated Benefits Reducing administrative costs Reduced costs for government and traders –0.1% to 0.5% of cargo value Reducing clearance times Reduced traders’ costs –0.5% to 0.8% of cargo value Reducing the variability of clearance time

Reduced inventory levels for traders, leading to reduced traders’ costs

For each 50% reduction in the standard deviation, –0.2% of cargo value

Increasing competitiveness Increased export growth +1 percentage point Source: World Bank. 2011. Border Management Modernization. Washington, DC.

B. Results 10. In calculating the benefits of traders from improved customs clearance time, the following were considered: (i) customs clearance times without the project, (ii) forecast of imports and exports over the project period,3 (iii) reduction in customs clearance time with the project, (iv) benefits per day of savings in customs clearance time in dollars, and (v) proportion of benefits per day of savings in customs clearance time attributable to the project. The economic benefits from reduced clearance times are estimated at 0.5% of cargo value, the lowest value in the range given in Table A4.3, for each day clearance time is lowered. However, the project reduced only some problems at the MCGA pertaining to the GAMAS; other problems remain unresolved and are outside the project’s scope. It was conservatively assumed that the estimated benefits from the project in reduction of traders’ cost was only 20% of 0.5% of the cargo value per day of reduction or an estimated hourly savings of 0.00016 of the cargo value.4 Based on the time-release study conducted by the MCGA, customs clearance time from 2012 to 2014 improved by 20% for road transport and 33% for rail transport, or 26% on average.5 This translates to a decrease of about 8.0 hours in customs clearance time, on average. The computation of the benefits from improvement in customs clearance time is given in Table A4.4.

Table A4.4: Estimates of Traders’ Benefits due to Improvement in Customs Clearance Time

Year Total Trade

Customs Clearances

under Orange/Green Channels (%)

Estimated Trade Value Classified under Orange/

Green Channelsa ($ million)

Estimated Benefits/Hour of

Customs Clearance Time Savedb

Estimated No. of Hours Savedc

Traders’ Benefits due to Reduction

in Customs Clearance

Processing Time ($ million) (% of trade value) ($ million)

2011 11,416 10.0 1,141.60 0.013 8.00 1.142 2012 11,123 13.8 1,534.97 0.013 8.00 1.535 2013 10,627 31.3 3,326.25 0.013 8.00 3.326 2014 11,011 32.8 3,611.61 0.013 8.00 3.612 2015 8,466 36.1 3,056.23 0.013 8.00 3.056 2016 9,840 39.7 3,906.48 0.013 8.00 3.906 2017 9,389 43.7 4,102.99 0.013 8.00 4.103

a Estimated based on the percent of clearances using red, orange, and green channels. b Estimated traders’ benefits in customs clearance time reduction (estimated at 0.013% of cargo value per hour saved). c Estimated number of hours saved for customs clearance using time-release study data. Sources: World Bank. 2011. Border Management Modernization. Washington, DC. p. 91; Mongolian Customs General Administration. 2015. Mongolian Customs: Time Release Study. Presentation for Customs Cooperation Committee and Private Sector Dialogue: Central Asia Regional Economic Cooperation Program. Ulaanbaatar. 20–21 August; International Monetary Fund. 2015. Mongolia: 2015 Article IV Consultation. Washington, DC.

11. The improvement in border administration depends on, among others, the successful implementation of the risk management system and requires accurate inputs by customs inspectors on

3 International Monetary Fund forecast data were used. 4 An 8-hour work day at the MCGA was assumed. The 0.5% of cargo value from the 2011 World Bank study was used. 5 United States Agency for International Development. 2013. Time Release Study. http://pdf.usaid.gov/pdf_docs/PA00KQ95.pdf

36 Appendix 4

the required transaction criteria for the sustained updating of the channel classification of import and export goods, and broker and trader transactions history. The available estimate of the administrative cost of transactions per channel classification are (i) red channel takes 2.0 hours at $5, (ii) orange channel takes 0.5 hours at $3, and (iii) green channel takes 0.1 hours at $1. On average, it is estimated that savings of $3 in administration cost is generated for each reclassification from red to orange/green. However, depending on the assessment by the customs inspector of the latest transactions of a trader or broker, the channel classification could either be improve (red to green/orange), remain (red), or regress (green/orange to red). Table A4.5 gives the estimate of the savings in customs inspector times in terms of the monetary value of the processing time saved for goods classified under the orange/green channel.

Table A4.5: Estimated Savings in Customs Inspection Time

Year

No. of Customs

Clearances

Customs Clearances under Orange/Green

Channels (%)

Average Cost Savings for Orange/Green Channel

Processinga

Benefits due to Savings in Customs Inspector

Timeb ($ per clearance) ($ million)

2011 278,823 10.0 3.0 0.084 2012 275,443 13.8 3.0 0.114 2013 264,357 31.3 3.0 0.248 2014 415,721 32.8 3.0 0.409 2015 246,646 36.1 3.0 0.267 2016 366,791 39.7 3.0 0.430 2017 393,591 43.7 3.0 0.516

a Computed as the cost of clearance processing for red channel minus average for orange/green channels. b Independent evaluation mission estimate. Source: T. Davaa. 2014. Improvement of Risk Management Implementation Process of Mongolian Customs. Paper presented in the International Network of Customs Universities Global Conference 2014 for Trade facilitation Post-Bali: Putting Policy into Practice. Baku.

12. The economic reevaluation of the project resulted in an EIRR of 16.4% (Table A4.6). The recalculated EIRR was lower than the appraisal and project completion report estimates (para. 1). Table A4.7 shows a comparison of the computed EIRRs and net present values at appraisal, completion, and PPER.

Table A4.6: Results of the Project Economic Internal Rate of Return Recalculation

Year

Project Cost

Maintenance Cost Total Cost

Savings in Administration

Cost

Savings in Customs

Clearance Cost Total

Benefits Net

Benefits ($ million) ($ million) ($ million) ($ million) ($ million) ($ million) ($ million)

2008 2.386 2.386 0.000 (2.386) 2009 2.727 2.727 0.000 (2.727) 2010 3.079 3.079 0.000 (3.079) 2011 1.375 0.242 1.617 0.084 1.142 1.226 (0.391) 2012 0.242 0.242 0.114 1.535 1.649 1.407 2013 0.242 0.242 0.248 3.326 3.574 3.332 2014 0.242 0.242 0.409 3.612 4.021 3.779 2015 0.242 0.242 0.267 3.056 3.323 3.081 2016 0.242 0.242 0.430 3.906 4.336 4.094 2017 0.242 0.242 0.516 4.103 4.619 4.377 NPV@12% $1.58 EIRR 16.4% ( ) = negative, EIRR = economic internal rate of return, NPV = net present value. Source: Asian Development Bank Independent Evaluation Department.


Table A4.7: Computed Economic Internal Rate of Returns and Net Present Values at Appraisal, Completion, and Project Performance Evaluation Report

Period EIRR NPV at 12% (%) ($’000)

Appraisal (report and recommendation of the President) 19.63 4,485.17 Project completion report 37.20 20,241.76 Project performance evaluation report 16.40 1,580.00 EIRR = economic internal rate of return, NPV = net present value. Source: Asian Development Bank Independent Evaluation Department.