I

Project Completion Report

PCR: UZB 29664

Basic Education Textbook Development Project (Loans 1594-UZB and 1595-UZB[SF]) in Uzbekistan June 2005

2

CURRENCY EQUIVALENTS

Currency Unit – sum (SUM)

At Appraisal At Project Completion (14 November 1997) (14 May 2005)

SUM1.00 = $0.013 $0.00092 $1.00 = SUM77.24 SUM1087.38

ABBREVIATIONS

ADF – Asian Development Fund AFER – Agency for Foreign Economic Relations AFS – audited financial statement BETDP – Basic Education Textbook Development Project CBU – Central Bank of Uzbekistan DP – direct purchase DFI – development finance institution ICB – international competitive bidding IS – international shopping LCB – local competitive bidding MOF – Ministry of Finance MOPE – Ministry of Public Education NBU – National Bank for Foreign Economic Activities of Uzbekistan NPPT – National Program for Personnel Training OCR – ordinary capital resources PCR – project completion report PIU – project implementation unit PPTA – project preparatory technical assistance RRP – Report and Recommendations of the President RTRFF – republican textbook revolving fund foundation SCL – student-centered learning SDR – special drawing right SES – state education standards STDP – Second Textbook Development Project SY – school year TA – technical assistance TRS – textbook rental scheme

NOTES

In this report, "$" refers to US dollars.

CONTENTS Page BASIC DATA i

MAP vii

I. PROJECT DESCRIPTION 1

II. EVALUATION OF DESIGN AND IMPLEMENTATION 2 A. Relevance of Design and Formulation 2 B. Project Outputs 3 C. Project Costs 6 D. Disbursements 7 E. Project Schedule 7 F. Implementation Arrangements 8 G. Conditions and Covenants 9 H. Related Technical Assistance 9 I. Consultant Recruitment and Procurement 10 J. Performance of Consultants, Contractors, Suppliers, and Subborrowers 10 K. Performance of the Borrower and the Executing Agency 11 L. Performance of the Asian Development Bank 11

III. EVALUATION OF PERFORMANCE 12 A. Relevance 12 B. Efficacy in Achievement of Purpose 12 C. Efficiency in Achievement of Outputs and Purpose 13 D. Preliminary Assessment of Sustainability 13 E. Environmental, Sociocultural, and Other Impacts 14

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 14 A. Overall Assessment 14 B. Lessons Learned 15 C. Recommendations 15

APPENDIXES 1. Output Targets and Achievements 17 2. Textbooks and Learning Materials 23 3. Appraisal and Actual Project Costs 24 4. Disbursements 26 5. Implementation Schedule 28 6. Compliance with Loan Covenants for Loan 1594-UZB 29 7. Compliance with Loan Covenants for Loan 1595-UZB(SF) 34 8. Completion Report of TA 2947-UZB 39 9. Completion Report of TA 2948-UZB 41 10. Consulting Services 43 11. Staff Development 44 12. Equipment Procurement 46 13. Project Performance Rating Assessment 47

BASIC DATA A. Loan Identification 1. Country 2. Loan Numbers 3. Project Title 4. Borrower 5. Executing Agencies 6. Amount of Loans a. Development Finance Institution (DFI) Loan b. Project Loan 7. Project Completion Report Number

Republic of Uzbekistan 1594-UZB and 1595-UZB(SF) Basic Education Textbook Development Project Government of Uzbekistan 1594: National Bank for Foreign Economic Activity of Uzbekistan 1595: Ministry of Public Education US$20 million SDR14.455 million ($20 million equivalent at appraisal) PCR:UZB 891

B. Loan Data 1. Appraisal – Date Started – Date Completed 2. Loan Negotiations – Date Started – Date Completed 3. Date of Board Approval 4. Date of Loan Agreement 5. Date of Loan Effectiveness – In Loan Agreement – Actual – Number of Extensions

6. Closing Date a. DFI Loan

– In Loan Agreement – Actual – Number of Extensions

b. Project Loan – In Loan Agreement – Actual – Number of Extensions 7. Terms of Loan

a. DFI Loan - Interest rate

i. As of 14 August 2002, ($18,182,259.72); ii. $1,817,740.28

- Maturity (number of years) - Grace period (number of years)

27 Aug 1997 9 Sep 1997 4 Nov 1997 7 Nov 1997 17 Dec 1997 23 Jun 1998 21 Sep 1998 14 Aug 1998 0 30 Jun 2003 30 Jun 2003 0 30 Jun 2003 19 Nov 2004 2 Pool-based variable lending rate for US dollar loan London interbank offered rate (LIBOR)-based loan 25 5

ii

b. Project Loan

- Service Charge - Maturity (number of years) - Grace Period (number of years) 8. Terms of Relending (if any)

a. DFI Loan - Interest Rate - Maturity (number of years) - Grace Period (number of years) b. Onlending rate - Interest rate - Maturity Paper Equipment

1% per annum 35 10 25–35%, revised from time to time by Central Bank of Uzbekistan plus 4% by the Ministry of Finance 20 5 25–41.5% 18 months 7 years

9. Disbursements a. Dates (i) DFI Loan Initial Disbursement

16 Apr 2001

Final Disbursement

30 Jun 2003

Time Interval

26 months

Effective Date

14 Aug 1998

Original Closing Date

30 Jun 2003

Time Interval

58 months

(ii) Project Loan Initial Disbursement

2 Nov 1998

Final Disbursement

19 Nov 2004

Time Interval

72 months

Effective Date

14 Aug 1998

Revised Closing Date

30 Sep 2004

Time Interval

73 months

b. Amount

(i) DFI loan ($ million) Subloan

Original

Allocation

Last Revised

Allocation

Amount

Canceled

Net Amount

Available

Amount

Disbursed

Undisbursed

Balance Equipment 9,000,000 — — — 8,592,370 407,630 Paper 11,000,000 — — — 10,918,941 81,059 Total

20,000,000 — — — 19,511,311 488,689a a Undisbursed balance was cancelled effective 30 June 2003.

iii

(ii) Project Loan (SDR million)

Category

Original

Allocation

Last Revised

Allocation

Amount

Canceled

Net Amount Available

Amount

Disbursed

Undisbursed

Balance Equipment 466,000 466,000 0 466,000 460,674 5,326 Textbook and Materials

11,539,000 12,052,500 513,500 12,052,500 12,057,786 (5,286)

In-service Training

630,000 10,234 (619,766) 10,234 10,234 0

External Training

519,000 314,000 (205,000) 314,000 284,783 29,217

Seminars and Workshops

112,000 32,000 (80,000) 32,000 27,971 4,029

Research Studies

117,000 6,835 (110,165) 6,835 6,835 0

Consulting Services

921,000 1,251,431 330,431 1,251,431 1,153,333 98,098

Project Administration

151,000 151,000 0 151,000 103,363 47,637

Total 14,455,000 14,284,000 (171,000) 14,284,000 14,104,979 179,021a

a Undisbursed balance was cancelled on 19 November 2004.

10. Local Costs (Financed) - Amount ($) 0.384 - Percent of Local Costs 0.53% - Percent of Total Cost 0.35% C. Project Data

1. Project Cost ($ million) Cost Appraisal Estimate Actual

Foreign Exchange Cost 40.2 38.0 Local Currency Cost 66.7 73.1 Total

106.9 111.1

2. Financing Plan ($ million) Cost Appraisal Estimate Actual

Foreign Local Total Foreign Local Total Implementation Costs ADB-Financed 13.081 6.919 20.000 18.533 0.384 18.916 NBU 20.000 0.000 20.000 19.511 0.000 19.511 MOPE 0.000 59.851 59.851 33.748 38.964 72.712 Total 33.081 66.770 99.851 71.792 39.348 111.140IDC Costs NBU 6.663 0.00 6.663 0.000 0.000 0.000 MOPE 0.430 0.00 0.430 0.000 0.000 0.000 Total 7.093 0.00 7.093 0.000 0.000 0.000 ADB = Asian Development Bank, NBU= National Bank for Foreign Economic Activity of Uzbekistan, MOPE= Ministry of Public Education, IDC = interest during construction.

iv

3. Cost Breakdown by Project Component ($ million)

Implementation Costs Appraisal Estimate Actual

Foreign

Currency Local Currency

Total

Foreign Currency

Local Currency

Total

A. Staff Development 1. In-service Training 0.000 1.244 1.244 0.000 2.054 2.054 2. External Training 0.718 0.000 0.718 0.370 0.000 0.370 3. In-country Seminars/Workshops 0.000 0.156 0.156 0.000 0.039 0.039 Subtotal (A) 0.718 1.400 2.118 0.370 2.093 2.463 B. Consulting Services 1. International 1.077 0.081 1.158 1.351 0.000 1.351 2. Domestic 0.000 0.116 0.116 0.000 0.177 0.177 Subtotal (B) 1.077 0.198 1.274 1.351 0.177 1.528 C. Research Studies 0.004 0.158 0.162 0.000 0.009 0.009 D. Revolving Credit Fund 1. Equipment Procurement 9.000 0.000 9.000 8.592 0.000 8.592 2. Paper Procurement 11.000 0.000 11.000 10.919 0.000 10.919 Subtotal 20.000 0.000 20.000 19.511 0.000 19.511 E. Textbook and Materials 1. Textbook Production and Distribution 8.534 62.584 71.118 48.889 32.591 81.480 2. Library Materials and Instructional Kits 2.168 1.628 3.796 1.051 0.701 1.752 Subtotal 10.702 64.211 74.914 49.940 33.292 83.232 F. Equipment 1. Educational Equipment 0.309 0.035 0.344 0.408 0.000 0.408 2. Office Equipment 0.271 0.030 0.301 0.212 0.000 0.212 Subtotal 0.580 0.065 0.645 0.620 0.000 0.620 G. Incremental Recurrent Cost 1. PIU Staff salaries 0.000 0.141 0.141 0.000 0.177 0.177 2. Incremental Operating Expenses 0.000 0.205 0.205 0.000 0.064 0.064 Subtotal 0.000 0.346 0.346 0.000 0.241 0.241 H. Taxes and Duties 0.000 0.391 0.391 0.000 3.536 3.536 Subtotal 0.000 0.391 0.391 0.000 3.536 3.536 D. Interest and Charges 1. Ordinary Capital Resources 6.663 0.000 6.663 0.000 0.000 0.000 2. Asian Development Fund 0.430 0.000 0.430 0.000 0.000 0.000 Total 40.174 66.770 106.944 71.792 39.348 111.140

v

4. Project Schedule

Item Appraisal Estimate Actual Date of Contract with Consultanta Start Date IV Qtr 1998 II Qtr 2000 Completion Date IV Qtr 2002 IV Qtr 2002 Equipment Procurement First Procurement IV Qtr 1998 II Qtr 1999 Last Procurement IV Qtr 2000 III Qtr 2004 Textbook and Learning Materials Procurement First Procurement IV Qtr 1998 III Qtr 2000 Last Procurement IV Qtr 2002 III Qtr 2004 Staff Development Start Date IV Qtr 1998 III Qtr 2000 Completion Date IV Qtr 2001 III Qtr 2004

Qtr=quarter

5. DFI Loan - Implementation Data A. Number of Subloans From the Loan 19 Cumulative (including Revolving Fund as of Dec 2004 ) 24 B. Distribution of Subloans from the Loan

Number Amount ($ million) Equipment 7 8.5 Paper 12 11.0 Total 19 19.5

C. Size of Subloans (actual) ($19,511,311)

Range ($) Number of Subloans Aggregate Amount Up to 1,000,000 8 2,630,530 From 1,000,000 to 2,000,000 10 13,643,781 Over 3,000,000 1 3,237,000 Total 19 19,511,311

D. Subloan Above Free Limit Subloan Aggregate Number Amount Equipment 1 $ 3,237,000

vi

6. Project Performance Report Ratings

Ratings Implementation Period

Development Objectives

Implementation Progress

A. DFI Loan From Jan 99 to Nov 99 S S From Dec 99 Jun 00 PS PS July 00 S S From Aug 00 to Feb 01 S PS From Mar 01 to May 01 S HS From Jun 01 to Jul 01 S S Aug 01 S PS From Sep 01 to May 02 S S From Jun 02 to Dec 03 S U

B. Project Loan From May 00 to Jun 00 PS PS Jul 00 S S From Aug 00 to Dec 00 S PS From Jan 01 to Feb 01 PS HS From Mar 01 to Jul 01 S S Aug 01 S PS From Sep 01 to Dec 04 S S

D. Data on Asian Development Bank Missions

Name of Mission

Date

No. of Persons

No. of Person-Days

Specialization of Membersa

Fact-finding 14 May–3 Jun 1997 2 40 a, b Appraisalb 27 Aug–9 Sep 1997 4 52 a,b,c,d, Inception 6–23 Oct 1998 3 50 a,e,f Review 1 9–20 Mar 1999 2 18 e,g Review 2 22 Nov–3 Dec 1999 1 11 e Review 3 4–17 July 2000 2 28 e,g Midterm review 11–26 Sep 2001 1 16 e Review 4 15–25 Jan 2003 2 15 e,g Project Completion Review c 15–28 Jan 2005 3 37 g,h,i, a a = senior education specialist, b = senior program officer, c = counsel, d = financial analyst-staff consultant, e = project economist, f = project implementation consultant, g = assistant project analyst, h = senior social sector

economist, i = staff consultant. b In conjunction with fact-finding for project preparatory technical assistance. c The project completion report was prepared by L. Wu, senior social sector economist; C. Nazario, assistant project

analyst; and J. Latini, staff consultant.

Aral Sea

Amudarya R.

Syrdarya R.

Gazh

Muynak

NukusKhodzheyli

Urgench Turtkul

SukhJordan

TASHKENTChirchik

Gulistan

Djizzak

Samarkand

Navoi

Bukhara

OklyaKarshi

Denau

Termez

Uchkuduk

Zarafshan

Namaganh

Kokand

Andijan

Fergana

Kungrad

Angren

NAVOI

SAMARKAND

DJIZZAK

KARAKALPAKSTAN

NAMAGANHANDIJAN

FERGANA

TASHKENT

SYRDARYA

SURKHANDARYA

KASHKADARYA

BUKHARA

KHOREZM

T U R K M E N I S T A N

AFGHANISTAN

TAJIKISTAN

KYRGYZREPUBLIC

KAZAKHSTAN

KAZAKHSTAN

National CapitalProvincial CapitalCity/TownMain RoadRailwayRiverProvincial BoundaryInternational Boundary

Boundaries are not necessarily authoritative.

UZBEKISTANBASIC EDUCATION TEXTBOOK DEVELOPMENT PROJECT

(as completed)

0 50 100 150

Kilometers

N

05-2494 HR vii

69 00'Eo

69 00'Eo60 00'Eo

60 00'Eo

39 00'No 39 00'No

45 00'No45 00'No

I. PROJECT DESCRIPTION

1. Uzbekistan inherited a well-developed educational system upon independence from the former Soviet Union in 1991. In basic education there was almost full enrolment. However, the system was inefficient, expensive, and based on Soviet values and labor requirements of a centrally planned economy, which were ill-suited to post-independence needs of a nation in transition. The economic downturn ensuing after the breakup of the former Soviet Union impacted negatively on the education system and on most budget-supported sectors. As a result, basic education was underfunded and underinvested. Lacking essential inputs and operating basically within the old system in terms of curricula, syllabi, and teaching methodologies, the quality of, relevance, and access to basic education became major concerns of the Government. 2. Before the Basic Education Textbook Development Project (the BETDP), textbooks were outdated, unattractively designed, and did not meet emerging needs of a transitional economy, reducing learning achievements. The textbook production system followed a central planning approach that was neither efficient nor cost-effective. In 1995, the Government had decided to change from Cyrillic to Latin script, requiring the comprehensive rewriting and reprinting of all textbooks. Improving education quality by providing better designed, more durable, and more widely available textbooks produced more efficiently became a priority for the reform of the education sector. 3. The BETDP, comprising (i) a development finance institution (DFI) loan (ordinary capital resources [OCR]) of $20 million to be relent to publishers and printers for upgrading equipment and procuring high-quality paper and cover board; and (ii) a project loan (Asian Development Fund [ADF]) of SDR14.455 million ($20 million equivalent) to be used for other project components, including capacity building and purchase of selected textbooks, was approved on 17 December 1997. The BETDP aimed to improve the quality and relevance of basic education by increasing the quality, durability, and availability of textbooks and other instructional materials, and enhancing the efficiency of the textbook production system. These improvements were to be accomplished by (i) developing capacity in curriculum development, textbook writing, and publishing; (ii) providing new Latin-script textbooks in key subject areas; (iii) training classroom teachers to use student-centered teaching methods; (iv) reviewing and strengthening the textbook production system; (v) upgrading equipment for publishing and printing; and (vi) providing high-quality paper and overboard for more durable textbooks. At appraisal, it was anticipated that the Loan 1595-UZB(SF) would provide about 7.1 million textbooks,1 255,800 teacher guides, 1.68 million library books, and 15,000 kits of classroom teaching aids, and would directly benefit about 3.2 million rural students and 12,000 senior rural teachers. Additionally, the BETDP included two associated advisory technical assistance (TA) projects to strengthen the monitoring of educational reforms and improve the cost-effectiveness and efficiency of the education system.

1 Quantitative outputs of textbooks and printed learning materials given in Appendix 8 of the Report and

Recommendation to the President (RRP) match with the allocation for textbooks and materials. Reference to a textbook target of 15.5 million given in the project framework (Appendix 1) does not match the RRP text and is not supported by budgetary allocations.

2

II. EVALUATION OF DESIGN AND IMPLEMENTATION A. Relevance of Design and Formulation 4. The BETDP addressed key concerns of the Government in basic education by (i) strengthening textbook publishing and printing capacity; and (ii) providing more durable textbooks with modern pedagogical content in core education subjects as well as teacher guides, library books, and teaching aids. The BETDP development objective was consistent with the Government’s education reform program and consistent with the Asian Development Bank (ADB) country operational strategy at that time. Project outputs, inputs, activities, and implementation arrangements were clearly articulated and documented. Overall, the BETDP design was relevant and its formulation sound, and improved availability, use, and quality of learning materials has had a positive impact on learning and teaching. In retrospect, project design could have included a more rigorous monitoring and evaluation framework. 5. Soon after loan approval the Government announced the enactment of three laws affecting the education system, which, in turn, required amendments in the implementation arrangements of the newly approved Loan 1595-UZB(SF): (i) the introduction of the National Program for Personnel Training (NPPT) in August 1997 to guide long-term educational reforms in training of teachers and extending compulsory education to 12 years; (ii) the introduction of state education standards (SES) in August 1999; and (iii) the abolition of free state provision of textbooks in January 1998, which resulted in parents being asked to pay for textbooks from grades 2 to 11. The modifications to the project design (i) strengthened the effectiveness of teacher education; (ii) supported curriculum development and introduced the concept of content integration (learning objectives, teaching styles, and assessment system); and (iii) were instrumental in creating the textbook rental scheme (TRS), which provided for sustainable provision of textbooks for basic education. 6. The TRS allows parents to pay a fraction of textbook purchase price as rental fees. The scheme is based on three principles: (i) improving affordability—amortizing the cost of textbooks over 4 or more years, thus keeping the rental fees low; (ii) strengthening sustainability—fees collected are designated for buying replacement copies of textbooks when due; and (iii) enhancing cost-effectiveness and physical durability by incorporating quality measures in textbook procurement. A pilot TRS was implemented with success in school year (SY) 2001-2002, supplying 1.6 million textbooks and teacher guides to 575 schools (about 6% of total number of schools) in all 14 regions. As a result of the successful pilot implementation, the Cabinet of Ministers approved phased extension of the TRS to the entire country starting from September 2002. 7. At completion, the BETDP reached and surpassed most of the expected outputs anticipated at appraisal. It (i) achieved an average ratio of 10.4 textbooks per pupil for primary and lower secondary students;2 (ii) improved the quality, durability, and availability of textbooks and other instructional materials; and (iii) enhanced the efficiency of the textbook production system. A comparison of targets listed in the project framework with achievements is in Appendix 1. 8. The rationale for the DFI loan administered by the National Bank for Foreign Economic Activities of Uzbekistan (NBU) was to provide $20 million of revolving credit facilities to

2 Total textbooks distributed for grades 1-9 was 58.2 million and the student population in grades 1-9 was 5.6 million

(Ministry of Economy, 2004).

3

domestic publishers and printers—$9 million to procure equipment and $11 to procure designated paper and printing consumables. This loan addressed two critical issues in textbook development: (i) limited durability attributable to poor-quality paper and binding, which limited the classroom life of textbooks to about 1 year and led to textbook shortages; and (ii) inadequate access to foreign exchange and cost inflation, which prevented publishers and printers from upgrading equipment and purchasing better-quality paper. The annual interest charged to sub-borrowers varied from 35.5% (2002 and 2003) to 21.5% (2004), based on the refinancing rate determined by the Central Bank of Uzbekistan (30% to 16% per annum) plus an annual exchange risk premium of 4% specified by the Ministry of Finance (MOF) and 1.5% profit margin for NBU.3 9. The two associated TAs supported sector reforms and complemented the project objectives by strengthening data collection and policy analysis capacities of the Social Complex of the Cabinet of Ministers, and identifying strategies for improving cost-effectiveness of the education system. B. Project Outputs 10. The BETDP was implemented from August 1998 to September 2004. The outputs under the six principal components are reviewed below. Achievements are compared with output targets in Appendix 1. 1. Capacity Building in Curriculum Development 11. The first component aimed to strengthen local curriculum development capacity in six key subject areas and implement the SES by providing in-country and overseas training to curriculum designers, carrying out research studies, improving syllabi for six subjects, and providing new textbooks and teaching materials. 12. Eighteen senior curriculum and textbook developers, and authors in six subject areas took part in a study tour to the United Kingdom and upon return carried out six research studies on curriculum reforms. Training of 151 district and raion curriculum designers was provided on curriculum design, integration of subjects, redesigning of the curriculum, and preparation of teacher guides. New student books and teacher guides, based on SES, were developed in six subject areas. Student books included a strong cross-curricular dimension, promoted critical thinking and were systematically piloted before publication. Books distributed to schools were well received. In addition, four desktop computers equipped with publishing software were procured and delivered to the Authors’ Center within the Republican Education Center. 13. All output targets were achieved and targets on the number of new textbooks provided to schools and staff trained were surpassed (Appendix 1). Appendix 2 indicates the outputs reached in the production of textbooks, teacher guides, library books, and materials. 2. Improving the Pedagogical Quality of Textbooks 14. The objective of the second component was to enhance the learning impact of education by improving the capacity of the Ministry of Public Education (MOPE) to design basic education textbooks. Significant outputs included (i) research studies on gender 3 Complete information on relending arrangements is in Appendix 10 of the RRP.

4

presentation in textbooks, textbook design baseline, and authors’ pay; (ii) capacity building in manuscript development, editing, and evaluation for 122 persons; (iii) training of 22 authors at primary and 40 at secondary levels; and (iv) procurement of desktop publishing equipment and resource materials for school libraries. Targeted outputs were reached, with two exceptions: (i) the number of library books delivered to schools was 0.56 million against the appraisal estimate of 6.68 million, as MOPE considered textbooks a higher priority than library books (para. 21); and (ii) three short-term overseas working attachments for local publishers and printers anticipated at appraisal were cancelled. In both cases, MOPE decided to concentrate the financial resources available to provide one textbook per subject to all pupils in grades 1 to 9 (para. 21) and supply the balance of reference materials later. This proved to be justified since the quantity of textbooks envisaged at appraisal was not sufficient to provide one textbook per pupil in grades 1 to 9 in rural and urban schools (Appendix 2). 15. These activities enabled MOPE to assess quality changes achieved by the Project, set a framework for revising publishers’ attitudes to authors’ pay, and provided the publishing industry with a practical guide to textbook design functions. At the end of the Project, MOPE had a cadre of 62 authors and 122 specialists trained in desktop publishing, manuscript evaluation and editing, and gender issues in textbooks. 16. A sampling of views of teachers and parents from 12 schools conducted by the project completion review (PCR) Mission provided anecdotal evidence of improvements in the pedagogical quality of new instruction materials. Those singled out by teachers focused more on the process of learning; took creative approaches to real-life situations; developed a climate of openness, trust, and concern for others; and expected the learner to actively seek information. Teachers linked improved learning outcome with activity-based learning and increased teachers’ effectiveness. 3. Improving the Use of Learning Materials 17. This component was intended to ensure that high-quality learning materials were effectively used in the classroom. Two research studies were conducted to survey teaching methodologies and learning skills, and assess needs for in-service training. Based on the findings, the component (i) developed training materials for student-centered learning (SCL) in six subject areas, (ii) trained a cadre of trainers of trainers, (iii) provided in-service training to teachers, and (iv) procured training materials and training equipment for in-service training institutes. All quantitative outputs set at appraisal were reached and some were exceeded (Appendix 1, item 3.3). 18. The major impact of the activities was the introduction of SCL to the teaching force, and the knowledge and confidence passed on to teachers to carry out SCL. The training materials produced and the cascade system for delivering the training were also important resources for capacity building. Numerous requests for further training in other subject areas emerged from these activities. 4. Improving the Quality and Efficiency of Textbook Production 19. The fourth component supported improvements in the textbook production system by (i) developing strategies for improving textbook production, and (ii) capacity building for local publishers and educators to implement them. Three research studies on the analysis of textbook sales and provision formed the basis to develop the MOPE textbook policy, which

5

led to the development and adoption of the TRS and a draft charter for establishing a republican textbook revolving fund foundation (RTRFF). A study tour to publishing houses in the United Kingdom for 12 directors and publishing managers of local publishing of printing houses and 6 officials from MOPE and other relevant agencies funded under the project, and in-country training of 29 participants from the printing industry was ground-breaking and strengthened the capacity for textbook production and management. Training in bid evaluation for 75 practicing teachers in various subject areas provided under this component created a pool of trained evaluators for MOPE. A total of 17,482 librarians, teachers, and school directors were trained in support of the TRS. The short-term overseas working attachments for two local publishers and printers planned at appraisal were not implemented due to a lack of qualified candidates with sufficient knowledge of English. 5. Increasing the Supply of Textbook and Learning Materials 20. This component aimed to increase the supply of Latin-script textbooks and supplementary learning materials for basic education, especially in rural schools, by providing new Latin-script textbooks and teacher guides, Latin-script Uzbek-language reading materials for school libraries, selected basic education textbooks for non-Uzbek-language schools, and kits of supplementary teaching aids. 21. At appraisal, project funding allocated for textbooks was to provide at most 6 books per student (34.7 million books for 5.8 million pupils in grades 1 to 9). During implementation, MOPE and ADB agreed to provide one textbook per student in each subject in grades 1 to 9, and the accompanying teacher guides. MOPE requested revisions in the textbook plan and lists of textbooks and materials. This led to a minor change in scope approved by ADB in October 1999. Accordingly, funds originally planned for library books were reallocated so as to produce more textbooks. Consequently, the Project procured 58.22 million textbooks (an average of 10.4 textbooks per student for 5.59 million pupils) and 1.30 million teacher guides, against the appraisal estimates of 34.69 million textbooks and 525,000 teacher guides.4 In addition, 755,000 library books were produced against the appraisal estimates of 1.6 million,5 and 6,600 textbooks for blind children, and 49,000 visual kits of supplementary materials for schools were provided. Textbooks and teacher guides procured under ADB financing are listed in Appendix 2. 22. This component also included a seminar on textbook marketing attended by 17 local publishers. The seminar was particularly relevant to publishers because in the past they had no need to market their books and therefore had no specialized staff with marketing skills. To further strengthen cooperation among education institutions and publishers, clear and transparent criteria for selecting books against specified technical criteria6 and commercial terms were developed and implemented. 23. This component also brought about three unanticipated benefits. The first was the launching of successful competitive bidding for textbook production which opened the way to liberalization of reform of authorship, educational publishing and printing, and has since been adopted by MOPE as the standard textbook procurement mode. The second benefit was the selection of more authors of textbooks and teacher guides from oblast and local 4 Figures include ADB and government funding. 5 ADB. 2004. Second Textbook Development Project. Manila (Loan 2093-UZB) will provide more library books and

supplementary materials for all grade schools. 6 The selection criteria were: factual accuracy, conformity to syllabus, number and size of illustrations, content

presentation, and methodology.

6

levels, compared with the former preference for authors from cities. The third was the heightened awareness and recognition by headmasters of the importance of helping teachers overcome the anxiety of moving from traditional to more interactive teaching. 6. Improving Textbook Durability 24. The sixth component aimed to increase the usable life of textbooks to at least four years by upgrading domestic capacity for producing more durable textbooks through the use of high-quality paper and better bookbinding techniques. Funding to modernize and upgrade printing and binding equipment and to purchase high-quality paper was provided through two revolving funds, one for on-lending for importing paper, and the other for printing equipment, established under the DFI loan and managed by NBU. 25. Twelve subloans to procure paper totaling $11.0 million and seven subloans for procurement of pre-press and printing equipment for $8.5 million were approved. The two revolving funds provided vital support to printers and publishers to purchase printing equipment and high-quality paper.7 The attractive lending terms allowed the printers and publishers to receive financing in foreign currency and repay in local currency. 26. Activities financed through the project loan under this component were only part of the contribution made by the BETDP to improving textbook durability. Noteworthy was the introduction of TRS, with its requirement for durable textbooks and accompanying competitive bidding procedures, which laid the foundation for institutionalizing improvements to physical specifications and production of textbooks. A seminar was conducted on textbook production for 30 publishing directors, printers, and production managers, building upon the experience of the study tour. Other activities completed were a seminar on alternative textbook provision policies, and training of 225 district staff members in book repair. 27. At appraisal, the provision of 250 book-repair kits for raion offices and the training of district staff and librarians in book repairs were planned. After training 225 district staff in book repair and providing repair kits to six pilot schools, MOPE reassessed the cost-effectiveness and sustainability of book repair applied to new textbooks, which were made with high-quality paper and had modern binding, and concluded that it was not justified. After consultation with ADB, this activity was dropped in 2001. The first three years of use of new textbooks suggested that the usable life of new textbooks would be four years or more. C. Project Costs 28. At appraisal, the total project cost, including taxes, duties, and service charges, was estimated at $106.9 million equivalent, of which $40.2 million equivalent was the foreign exchange cost and $66.7 million equivalent was the local currency cost. ADB provided two loans totaling $40 million equivalent, comprising (i) a DFI loan (OCR) of $20 million to be relent to publishers and printers for upgrading equipment and procuring high-quality paper and cover board; and (ii) a project loan (ADF) of SDR14.455 million ($20 million equivalent) to be used for other project components, including capacity building and purchase of selected textbooks. The two loans were to finance 38% of total project cost. The Government committed to financing the

7 When the PCR mission was carried out, five further subloans for paper in the amount of $4.8 million had been

granted, after the closing of Loan 1594-UZB, utilizing funds received from repayment of subloans.

7

foreign exchange cost of interest and service charges for $7.1 million and local cost of $59.8 million. 29. The actual project cost was $111.1 million equivalent, comprising $71.8 million in foreign exchange cost, and $39.3 million in local currency cost. Under the project loan, the Government contribution amounted to $72.7 million and incurred no interest or service charges. The Government contributed $2.0 million (82.8%) of the cost for staff development, compared with the appraisal estimate of $0.37 million (17.6%). Appendix 3 compares appraisal cost estimates and actual costs by category and financing sources. Loan savings under staff development, research studies and incremental recurrent cost of the project loan were reallocated to cover (i) the higher than anticipated cost of consulting services resulting from the implementation of the TRS, and (ii) the printing of larger quantities of textbooks than planned at appraisal. D. Disbursements 30. DFI Loan. Disbursements amounted to $19.5 million against $20 million at appraisal. Initial disbursements were slow due to delays in startup activities including establishing appropriate management structures at NBU (para. 33), late delivery of equipment from suppliers, and an initial lack of demand for subloans. In June 2003, after unsuccessful reminders to NBU to submit the outstanding audited financial statements (AFSs) for 2000, 2001 and 2002, the Government’s request for loan extension was not approved by ADB,8 and the undisbursed balance of the loan was cancelled on 30 June 2003, the original loan closing date. The time interval for utilizing the loan was 58 months. Disbursement is in Appendix 4 (Table 1). 31. Project Loan. Disbursements totaled the equivalent of $18.91 million, or 94.5% of the loan amount of $20 million equivalent at the time of appraisal. Actual disbursements during the first two years were low, mainly because of the late recruitment of project implementation unit (PIU) staff, delays in the engagement of consultants, and delayed procurement of new textbooks. Other factors were the adjustments to the curriculum necessitated by the introduction of new SES and changes in Government policy to abolish free state provision of textbooks in January 1998. Anticipated savings of $0.250 million were initially cancelled on 30 June 2004. The unutilized loan balance of 0.277 million was cancelled on 19 November 2004. Disbursement is in Appendix 4 (Table 2). 32. The Project was provided with an imprest account ceiling of $1 million, but this was not fully utilized because of low turnover. In 2001-2003, turnover was low because the imprest account was used mainly for PIU operational expenses, local workshops, auditing services, and individual international consultants. In 2004, turnover increased significantly when the imprest account was used to pay for textbooks in advance. E. Project Schedule 33. Both loans were approved on 17 December 1997 and signed on 23 June 1998. They became effective on 14 August 1998. Certification of availability of $20 million equivalent ADF resources for the project loan was issued only in June 1998 and caused a six-month gap between the approval and effectiveness of both loans. DFI loan was to be implemented over five years, with physical completion on 31 December 2002, and loan closing by 30 June 2003. 8 ADB letter dated 7 July 2003 to the minister of finance.

8

Initial project activities were delayed due to slippage in the recruitment of the project manager, limited capacity in the management of the two revolving funds to prepare appraisal reports, and business plans of subloans, and a lack of demand for subloans. Utilization of the two revolving funds commenced after an international individual consultant, a printing and publishing specialist, was recruited to help NBU assess appraisal reports and business plans, manage revolving funds, and procure paper and printing equipment in accordance with ADB guidelines. The first subloan was awarded in March 2000. Peak disbursement for the two revolving funds was in 2001. A total of 7 subloans were granted for equipment and 12 for paper. 34. The project loan was to be implemented over five years, with the closing of the loan account by 30 June 2003. The Project was initially delayed by slow recruitment of consultants, and the adoption by the Government of the new SES and the abolition of free textbook provision (para. 5). As a result, the first procurement of new textbooks and materials was deferred to 2000. Subsequently, project implementation proceeded smoothly. Two extensions of loan closing date for a total of 15 months were approved to allow for the introduction of a new generation of textbooks and learning materials over 4 consecutive school years. Appendix 5 compares the appraisal and actual implementation schedule. F. Implementation Arrangements 35. DFI Loan. NBU was the executing agency for this loan. The Government relent the loan to NBU under a subsidiary loan agreement, and NBU was responsible for on-lending to textbook publishers and printers. NBU managed two revolving funds, one to finance the procurement of paper ($11 million) and the other for printing equipment ($9 million), to be relent to financially sustainable, technically, and managerially capable publishing and printing firms under the agreed-on relending terms.9 Within NBU, two units managed the two revolving funds separately. High staff turnover contributed to inadequate communications between the two units in reviewing subloan business plans and appraisal reports, affecting decisions to approve subloans. NBU lacked qualified staff to compile financial statements acceptable to ADB. The external auditor gave a qualified opinion of financial statements of the project accounts for 2000, 2001, and 2002, as it was unable to verify the use of funds transferred to MOF.10 NBU did not calculate its debt service coverage ratio as of 31 December 2000 and 2001 as required by the Loan Agreement. However, the less-than-efficient administration of the revolving funds had no significant negative impact on the production and delivery of education materials. In spite of the problems affecting the management of revolving funds, NBU utilized 97.5% of the loan for paper and equipment. 36. Project Loan. MOPE was the executing agency for this loan. Implementation arrangements were established in line with those envisaged at appraisal. A project steering committee, consisting of heads of relevant ministries and agencies, was established for overall coordination and guidance. The project director, who was responsible for internal coordination 9 Under the subsidiary loan agreement, NBU paid to MOF interest at a rate based on (i) an annual refinancing rate

established by the Central Bank of Uzbekistan, which included a variable foreign exchange risk premium; and (ii) a 4% interest rate to MOF. NBU also charged sub-borrowers an additional 1.5% premium for its services. Sub-borrowers obtained subloans denominated in hard currency and repaid the principal and interest in local currency. Principal and interest amounts on subloans were collected by NBU and transferred to MOF for conversion to US dollars. If sub-borrowers delayed payments for any reason, NBU was obliged to transfer funds from its own resources by 10 January and 10 July of the year in question. The maturity period for paper subloans was 18 months and for equipment 7 years.

10 There was a 6-month period in which MOF withheld repayments from NBU and did not return them to the revolving funds. Consequently, the amount of funds available from the revolving funds was negatively affected. The issue was resolved with ADB intervention.

9

within the MOPE and for reporting to the committee, changed three times during project implementation. From April 2003, when the second director resigned, the first deputy minister of MOPE coordinated project activities through to project completion in November 2004. The PIU became operational in June 1999, 6 months behind schedule, due to delays in staff recruitment. To guide PIU staff members in ADB procedures and guidelines and on-the-job training in project management, the Project provided an international project implementation specialist for 6 person-months. Overall the PIU functioned effectively. G. Conditions and Covenants 37. DFI Loan. Among the 19 major covenants listed in the Loan Agreement and the Project Agreement directly related to project implementation, 15 were fully complied with and 4 partially complied with. Covenants with partial compliance include those related to the regular reporting of funds transferred to MOF, reporting to ADB on operations of the revolving funds including conditions that affect subloan performance, subproject performance monitoring, and maintenance of debt–equity ratio. AFSs for 2003 are yet to be submitted. Compliance with covenants is in Appendix 6. 38. Project Loan. Eleven out of 12 main loan covenants were complied with on time. There was a two-month delay in establishing the textbook development working group, causing delayed compliance with the covenant concerned. Quarterly progress reports were submitted on time and regularly with the exception of the first one covering October 1998 to March 1999. Submission of AFS was generally on time, except for 1998 when submission was delayed by 1 year because there were no transactions during the year. The financial statements for 1998 were audited together with those for 1999. No covenants were modified, suspended or waived. Compliance with covenants is in Appendix 7. H. Related Technical Assistance 39. Two advisory TA projects,11 were associated with the BETDP. TA 2947-UZB developed a functional education reform monitoring system that provided relevant and timely reports to policy makers, a database containing updated information on key indicators, and conducted a series of seminars to examine the status of reforms. Main lessons learned include the importance of providing long-term support to achieve institutional change and lasting impact, including changes pertaining to information sharing and transparency. The TA was rated as partly successful. The TA completion report is in Appendix 8. 40. TA 2948-UZB identified strategies for improving the cost-effectiveness and efficiency of the education system. It carried out a review of public spending on education, assessed the financial sustainability of the NPPT, recommended funding strategies and contributed to strengthening the financial management and policy analysis skills of key personnel. Capacity-building activities included the introduction of economic concepts used in modern education planning, training a core of national experts in methodologies and techniques for data processing, and analysis and provision of international exposure to senior officials in education reform processes. The TA was rated as successful. The TA completion report is in Appendix 9.

11 ADB. 1997. Monitoring the Implementation of Education Reform. Manila; ADB. 1997. Capacity Building in

Education Finance. Manila

10

I. Consultant Recruitment and Procurement 41. Consultant Recruitment. Consulting services were provided for all six components of the BETDP. Selection and engagement of the consulting firm and individual consultants was carried out in accordance with ADB’s Guidelines on the Use of Consultants. The value of the consulting contracts exceeded the budget allocated at appraisal and required reallocation of funds. In February 2002, following a Government decision to implement the TRS and refocus expert services toward fewer subjects, variations were introduced to the main contract. A total of 160.9 person-months of consulting services, comprising 56.9 person-months of international consultant and 104.0 person-months of domestic experts, were utilized compared with the 116.0 person-months planned at appraisal. The increase was mainly due to the introduction and implementation of TRS, which was not anticipated at appraisal. 42. One short-term individual consultant, a specialist in project management, and monitoring and evaluation (6 person-months), was recruited under the project loan to assist the PIU. Anticipated and actual consultant services are compared in Appendix 10. 43. Training. The BETDP envisaged training of 12,777 teachers, school principals, curriculum designers, authors, publishers and printers, school librarians, and district staff members. Three times as many (38,831) were trained. Quantitative outputs for in-service teacher training were 160% the appraisal estimates. Staff development activities are in Appendix 11. 44. Procurement of Goods. Procurement of goods under DFI loan was done in compliance with schedule 2 of the Loan Agreement, which covers, among other things, the eligibility of goods, reasonableness of prices, and reliability of goods. All procurement contracts were awarded through transparent competitive bidding, and approved by a procurement tender committee as was the case for Loan 1595-UZB (SF). 45. Procurement under the project loan was carried out in accordance with the appraisal plan and ADB’s Guidelines for Procurement. While the Loan Agreement stipulates local competitive bidding for textbook procurement, MOPE took significant steps to introduce a broader competitive textbook procurement system by adopting international competitive bidding procedures. Measures introduced by MOPE include (i) the introduction of royalties and fee negotiations of authors’ remuneration, including for example a fixed sum for manuscript preparation; (ii) open competition between publishers for manuscript preparation; and (iii) adoption of competitive bidding for printing and publishing work. Appendix 12 compares the appraisal plan with actual procurement. Inefficient Government approval procedures and inadequate interagency coordination resulted in substantial delays in preparing bidding documents and bid evaluations. There were no other significant problems with procurement during implementation. J. Performance of Consultants, Contractors, Suppliers, and Subborrowers 46. Consultant performance under the DFI loan was satisfactory. Consultants fulfilled their terms of reference in helping NBU assess appraisal reports and business plans, manage revolving funds, and procure paper and printing equipment in accordance with ADB guidelines. 47. Subloan performance was generally satisfactory. To date, one of the seven subloans funded by ADB under the revolving fund for equipment is in delinquency. The total overdue payments, including interest and principal, amounted to $1.93 million against total loans of $8.5

11

million. Of the 12 subloans funded by ADB under the revolving fund for paper totaling $11 million, one is in delinquency, with $0.27 million in overdue interest and principal payments. Reasons for late repayment of subloans include (i) subborrowers’ mismanagement, (ii) increasing competition in the printing industry, and (iii) late delivery of equipment by the supplier. 48. The performance of consultants under the project loan was highly satisfactory, particularly with respect to their work organization, timeliness of delivery, quality of outputs, and work relationship with counterpart staff and MOPE. The support to introduce the TRS was greatly appreciated by the Government, and facilitated the effective technical transfer of expertise to MOPE staff members responsible for the TRS. 49. The performance of printers and publishers of textbooks as well as suppliers of paper and printing equipment was considered satisfactory. In some instances, late delivery of learning materials was penalized. The funds recovered were reinvested in additional learning materials. K. Performance of the Borrower and the Executing Agency 50. DFI Loan. The Borrower substantially met its financial obligations. NBU took longer to reach the expected level of performance. In spite of shortcomings in project management practices, staff retention, preparation of financial reports, and monitoring of financial covenants, both revolving funds were almost fully utilized and provided vital and intended support to printers and publishers through subloans. At the closing of the loan in June 2003, NBU had granted 7 subloans for equipment ($8.5 million) and 12 for paper ($11.0 million). The revolving fund was kept operational and up to date, and five more subloans for paper were granted totaling $4.8 million, using the funds paid back from sub-borrowers. The overall performance of NBU was considered partly satisfactory. 51. Project Loan. The Borrower implemented the project loan on time despite initial delays. MOPE’s performance was highly satisfactory as exemplified by its innovative introduction of the TRS in response to a changing policy environment. Compliance with loan covenants, including submission of progress reports and AFS, was satisfactory. Throughout implementation, the Borrower, MOPE, the PIU, and consultants coordinated closely with each other. Considering their initial lack of familiarity with ADB guidelines and procedures, and the complexities of the BETDP, PIU staff members, with the support of consultants, performed beyond original expectations. MOPE, particularly toward the second half of project implementation, demonstrated strong commitment to the project purpose and objectives, as well as leadership in implementation. The delivery of textbooks and learning materials and the training of teachers exceeded appraisal targets. L. Performance of the Asian Development Bank 52. ADB’s performance was satisfactory. During project implementation, from October 1998 to September 2004, ADB fielded six review missions, including inception and midterm review missions. The focus of ADB supervision was on implementation progress, identification and resolution of problems, and general project management issues. Toward the end of the Project, there was a gap of 19 months in field supervision due to staffing constraints. During this period

12

more intensive support was provided through telephone and electronic mail. The assistance provided by ADB missions and the Uzbekistan Resident Mission, especially during the recruitment of consultants, approval of the TRS, and resolution of implementation issues, contributed significantly to successful implementation. ADB maintained a cordial working relationship with MOPE and NBU, which provided a good basis for policy dialogue on capacity building and sustainability issues.

III. EVALUATION OF PERFORMANCE A. Relevance 53. The objectives of the BETDP were consistent with the education sector’s most urgent priorities and with ADB’s assistance strategy for Uzbekistan and the ADB’s education policy August 2002. The Project has significantly improved textbook provision and paved the ground for establishing a sustainable and affordable system of textbook provision, thus contributing to effective schooling from the perspective of teachers, pupils, and parents, and the Government’s reform priorities. The BETDP’s design proved to be robust, in that it was able to accommodate, without major changes in design and implementation arrangements, changes necessitated by the Government’s abolition of free textbook provisions. However, the project design could have emphasized monitoring and evaluation more. In retrospect, MOPE’s decision to implement the TRS, and to provide additional resources to the Project was appropriate and enhanced the relevance of project design. The DFI loan was also designed to address urgent issues facing the publishing and printing industry. Based on these considerations, the BETDP is rated as highly relevant. B. Efficacy in Achievement of Purpose 54. The BETDP is rated efficacious. It addressed major concerns of basic education and attained its goals. By project completion, new textbooks and better teachers’ guides were made available for the core curriculum in sufficient numbers and at affordable cost to teachers and pupils in all schools. Through the TRS and the proposed creation of a foundation to handle its administration, the BETDP established a financially and institutionally sustainable basis for a system for provision of affordable textbooks. The introduction of a transparent and competitive textbook procurement process stimulated the development of local textbook publishing capacity, and the market conditions for continuing development of new and better textbooks. This capacity to get learning material to schools in a timely, comprehensive and efficient manner will substantially facilitate ongoing efforts to improve education quality, including implementation of continuing curriculum reforms. 55. In-service training under the BETDP equipped teachers with a new didactic form of teaching and gave them confidence by providing new syllabi for six subjects, new textbooks, and teacher guides. There is now widespread recognition of the need for changing approaches to education, not only in terms of structures and regulations, but more fundamentally, concerning what goes on in schools and classrooms. Moreover, through exposure to international practices and its own experimentation, MOPE has greatly increased its evaluation capacity. The BETDP helped MOPE learn and acquire considerably better knowledge and skills to steer and manage the ongoing sector reforms, and MOPE has a better understanding of the need for strategic thinking and monitoring of implemented policies. Finally, training provided under the Project helped broaden the pool of experts within the education system who are available for education reform work and advising schools on how to cope with the new expectations.

13

56. MOPE especially benefited from the knowledge and skills relating to its advisory and regulatory function of textbook authoring, publishing, and distribution and professional development of teachers. Project activities, including those relating to curriculum improvement and new teaching methodologies, combined to foster a gradual but visible change in attitudes, which is of long-term significance. At project completion many teachers supported, and had an understanding of the need for, curriculum development, and MOPE officials refer to curriculum reform as a key means to transform education to meet the needs of a modern society. These combined are expected to have had a positive impact on learning and teaching outcomes, although specific statistics on these outcomes are not available. Institutionalized competitive bidding helps to ensure that textbook production and quality meet emerging demands. The DFI loan helped publishers and printers produce quality textbooks and other learning and teaching materials. C. Efficiency in Achievement of Outputs and Purpose 57. The BETDP is rated as efficient in attaining its intended outputs and purpose cost-effectively. This was accomplished through substantive improvements in the quality, durability, and availability of textbooks, and efficiency in textbook production and supply system, which considerably reduced costs. Other tangible outcomes were strengthened policy and institutional frameworks and knowledge through experimentation in developing a responsive education system. The project scope, outputs, and activities remained unchanged during implementation, with the exception of support for the introduction of the TRS, which was not anticipated at appraisal. 58. Financial and human resources were efficiently utilized for capacity building in curriculum development, improving the availability and quality of textbooks, using learning materials, improving the quality and efficiency of textbook production, and increasing the supply of textbooks. NBU almost fully utilized the DFI loan, although its administration of revolving funds did not show adequate fund management capacity. The Project significantly increased textbook supplies, averaging 10.4 textbooks per pupil nationwide at project completion, as compared with two or three students sharing one textbook before the Project. Similarly, printing and distribution costs for textbooks were considerably reduced. At appraisal, the estimated unit cost of a printed book and the distribution cost as a percentage of production cost were $1.63 per copy, and 11%, respectively. In SY2004 the average unit cost of textbooks for grades 1–9 dropped to $1.35 and the distribution cost was reduced to 8.4% of production cost. The Government provided, out of its own resources, an additional $8.09 million for textbooks and learning materials and $1.66 million for staff development. D. Preliminary Assessment of Sustainability 59. The BETDP is rated most likely sustainable for the following reasons. 60. Financial Sustainability and Fiscal Impact. The introduction of TRS under the Project laid a solid foundation for sustainable provision of affordable textbooks to all students in basic education. Experience indicates that the basic design of TRS is financially sound, which makes rental fees affordable to most parents. This is reflected in the high collection rate of rental fees. On average, about 15% of students are unable to pay the rental fees. These students are financed by the Government from its budget. Four years of Government and donor investment in start-up textbook rental stock will create the fee income needed to maintain the TRS on a sustainable basis. This is estimated to save the Government around $230 million in total at current costs over the period of loan repayment. The subsidy to the 15% poor students at about $2 million a

14

year amounts to less than 0.3% of total Government expenditure in the education sector, with no significant fiscal impact. The financial sustainability established laid the foundation for, and will be further strengthened by, the Second Textbook Development Project (STDP) supported by ADB (footnote 5), which has been under implementation since November 2004. Financially, the DFI loan is most likely sustainable due to its revolving nature. 61. Policy Reform. The policy reform process initiated with the BETDP is considered sustainable. The policy measures envisaged at appraisal were achieved, and the BETDP has been an important link in the chain of education reforms in the country. Among the key achievements are ensuring the availability of affordable textbooks, and the introduction of transparent competitive bidding for education materials, which was subsequently institutionalized. TRS innovations provided a firm foundation for institutional changes that affect education officials at all levels, parents, pupils, publishers, authors, and distributors. The knowledge acquired by MOPE through experimentation with new models and ideas in developing the education system considerably strengthened the education sector reform and its institutional framework. The STDP is benefiting directly from these improvements and will carry them forward. With regard to the DFI loan, policy reforms were slower and further progress in liberalizing the publishing and printing industry would be desirable. 62. Institutional Arrangements. The BETDP has laid the foundation for a sustainable, institutionalized system of nationwide provision of textbooks. The system includes textbook production based on competitive bidding, wider sourcing of authors, and a pool of trained textbook evaluators in most subjects. At the core of the system is the TRS that has proved to be institutionally appropriate, effectively managed and financially sound. A draft charter for the RTRFF has been prepared and will be registered upon issuance of a Government resolution on establishing the foundation. The RTRFF will significantly strengthen TRS as an institution based on transparency, accountability, participation (especially by schools and parents), and financial prudence. A fully functional RTRFF under these principles constitutes the cornerstone of the STDP, and will be further strengthened by it. E. Environmental, Sociocultural, and Other Impacts 63. The impact of the BETDP is rated as substantial. The BETDP had no adverse impact on the environment. On the contrary, the BETDP contributed to capacity development in environmental education through workshops for teachers and authors, including holding a national water-saving competition for schools to raise awareness among teachers of environmental education. The BETDP contributed to the education of ethnic minorities by making available a total of about 0.7 million new textbooks in Kazakh, Kyrgyz, Tajik, Turkmen, Russian, and Karakalpak minority languages, which is about three textbooks for each minority language pupil in grades 1 to 4 nationwide. Outside the school environment the Project promoted market-oriented authoring, publishing, and distribution operations.

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS A. Overall Assessment 64. The BETDP is rated highly successful (Appendix 13). It was well conceived, comprehensive in its approach, successfully implemented as designed, and supported the successful introduction of the TRS in response to a changed policy environment. Project implementation followed the concepts developed at appraisal and improved on them. It produced a satisfactory impact in the education sector which is likely to sustain and supported ongoing reforms. Virtually all quantitative targets (teachers trained, new curriculum developed,

15

number of textbooks, teacher guides) were achieved, and some exceeded. The Project introduced new thinking and practices in MOPE, which has continued to benefit ADB operations in basic education as well as contributing to cumulative progress in the education sector reforms. B. Lessons Learned 65. A particular strength of the Project was that the objectives were firmly based on Borrower’s priorities and on stakeholder assessment conducted during the preparatory phase and considerable direct consultation with beneficiary groups. Related to this was the demonstrated strong Government commitment to the purpose and objectives of the BETDP. Project ownership was evident among the senior management of MOPE and MOF and helped resolve problems on implementation and ensure its success. 66. The BETDP did not provide for the systematic gathering and analysis of data on the project impact, including effectiveness of teaching and learning in schools before and after the intervention. For future projects, a system to collect and maintain data, and to monitor and analyze quantitative and qualitative outputs and impacts should be built into project design. 67. The introduction of SCL is a paradigm shift in Uzbekistan. Given the former rigid educational environment, which focused on learning facts rather than acquiring skills, the understandable skepticism among teachers and trainers about SCL and the relative inexperience of counterparts with hands-on experience on SCL are changing. Ideas of interactive learning methods discussed during in-service training sessions were far more often referred to by teachers than the actual improvement of selected syllabi as primary outcomes of the Project. 68. At appraisal it was not foreseen that a large amount of paper would need to be imported using foreign exchange. In reality, about 50% of total costs of textbook production was attributable to the cost of imported paper. As a result, the Government shared significant project funding in foreign currency (Appendix 3). Future project design should ensure accurate estimates of the need for foreign exchange and avoid unnecessary burdening of the Government with foreign currency financing. C. Recommendations 69. DFI Loan. The Government should be encouraged to continue the revolving fund modality instituted under the Project to benefit printers and publishers of textbooks and learning materials in the procurement of paper and equipment. Considerable macroeconomic changes, including the unification of the exchange rates, have taken place over the past few years. It is necessary to review the design of the funds to reinvigorate sub-borrowers’ interest, and make on-lending competitive and sustainable, as well as to incorporate lessons learned in strengthening fund management. 70. It is recommended that ADB (i) closely monitor the submission to ADB of AFSs for 2003, which were outstanding at the time of the PCR mission, and which NBU committed to submit by July 2005; (ii) closely monitor the functioning of the revolving funds, given that the effective functioning of the revolving funds is essential to the successful development of the textbook sector; and (iii) incorporate the review of the revolving funds as part of the midterm review of the STDP, currently under implementation.

16

71. Project Loan. MOPE should prepare indicators for monitoring the qualitative impact on learning and teaching effectiveness and update them regularly with agreed-on intermediate and process benchmarks. These indicators should be used for evaluating the effectiveness of the new textbooks and other training materials. 72. To further reduce the costs of textbooks and learning materials, MOPE should introduce unrestricted and transparent competitive bidding for delivery of education materials to schools as a matter of priority. 73. MOPE should give continuous support to train talented practicing teachers in design and content of new teaching syllabi and manuscript development, editing, and evaluation. 74. General. To support the Government’s institutionalization of competitive bidding for education materials, ADB should provide periodic training to members of various procurement committees in design of bid documents, bid evaluation criteria, bidding and evaluation of bids, and the management of the competitive bidding process. 75. To further improve quality and reduce costs of education materials production, targeted actions are required by the Government to update legislation regarding copyrights, and further liberalize publishing, printing, and distribution industries.

Appendix 1 17

OUTPUT TARGETS AND ACHIEVEMENTS1

Design Summary Output Targets (Verifiable Indicators) Achievements2

A. Sector/Area Goals Improvement of the quality of basic education to enhance equity and serve the needs of transition to a market economy

- Widely available and affordable

textbooks are a key determinant of student performance.

- Improved textbooks and capacity building in methodologies promoted student centered learning in classrooms; there also emerged wide recognition of the need for changing education in terms of structures and what goes on in schools and classrooms.

- MOPE has considerably better knowledge and skills today to steer and manage the education reform.

- Affordable textbooks under TRS benefit the poor who would otherwise need to pay full purchase price of textbooks.

- The PCR Mission noted that teacher and pupils had gained a lot from in-service training and new curriculum materials that were provided to the project.

B. Objective/Purpose To improve the quality, quantity and durability of textbooks and learning support materials for basic education

- One textbook per student in each

subject in primary and lower secondary levels using revised content and methods

- One Latin script Uzbek language

book per student in minority language schools

- Teacher guides in accordance with

the needs - Average textbook life increased to

four years

- The project produced and made available

60.2 million textbooks and teacher guides for Grades 1 to 9; this is equivalent to an average of 10.4 books per pupil. Timely production and delivery to some schools has been a challenge.

- Target substantially reached in Dec 2004. - Target largely achieved: 2.0 million

teacher guides distributed - Expert opinion confirmed that 4 years

useful life of new textbooks will be reached and even exceeded

1 Due to discrepancies between the Project Framework (RRP, Appendix 1) and the Staff Development Overview

(RRP, Appendix 18), the latter is taken as reference for the purpose of listing staff development activities under Output Targets.

2 Complete information for each project activity is provided in relevant RRP appendixes.

18 Appendix 1

Project Component Output Targets Achievements

C. Project Components and Outputs 1. Capacity Building

in Curriculum Development

- Seminars on strategies in curriculum

development on six main subject areas: 150 curriculum designers x 21 days;

- Study tours for 18 curriculum designers x 30 days; - New textbooks, teachers' guides, and

workbooks produced for basic economics (Grades 9-11) and environmental studies (Grades 1-5);

- Redesigned curriculum for 6 subjects; - Six research studies on curriculum; - Four computer for desktop publishing; - Six international consultants for 20

person-months; - Domestic experts for 20 person-

months; - Resource materials in six subject

areas;

- Hands-on seminars curriculum

designers on Curriculum Development strategies conducted on:

Economics: 20 persons x 10 days Mathematics: 20 persons x 10 days Environment: 25 persons x 10 days Informatics: 16 persons x 5 days Physics, Chemistry, Biology: 30 persons x 10 days English: 30 persons x 40 days - Overseas study tour curriculum

designers conducted: 18 persons x 22 days

- 60.2 million textbooks, teachers’

guides, and workbooks distributed for a total value of $81.4 million, against appraisal estimate of 45.6 million books ($57.7million).

- Curriculum redesigned for: Subject Grade - Mathematics: 1-9 - Science: 6-9 - Informatics: 8-9 - English: 5-9 - Economics: 8-9 - Environmental education: 5-9 - Six studies completed and approved -.Four computers procured, delivered

and commissioned - Six international consultants recruited

for a total of 10.8 person-month - Six domestic experts recruited for a

total of 14.9 person-months - Sixty eight titles of resource materials

in six subject areas completed and distributed

Additional Activities - Seminar on integration of

environmental education conducted; 19 participants x 1 day

- Seminar on teacher guides conducted; 12 participants x 5 days

Appendix 1 19

Project Component Output Targets Achievements 2. Improving the

Pedagogical Quality of Textbooks

- Authorship training in new approaches:

60 persons x 14 days - One seminar on manuscript evaluation

20 participants x 14 days - One seminar on gender presentation in

textbooks:20 participants x 10 days - One workshop on manuscript editing 20 participants x 14 days - One workshops on on-screen editing 15

participants x 14 days - One workshops on desktop publishing

15 participants x 14 days - 4 desktop publishing units, photocopy

and fax machines, purchased; - Judgments by experts that quality of

manuscripts has improved; - More positive assessments by

teachers; - Three study attachments for printers

and publishers; - Five person-months of international

consulting services; - Eight person-months of domestic

consulting services; - Resource materials

- Authorship training provided on

student-centered learning provided on: Humanities: 24 persons x 10 days;

Natural Sciences: 20 persons x 10 days;

Primary Education: 22 persons x 10 days

- Seminar conducted : 31 persons x 10

days - Seminar conducted : 25 persons x 10

days - W/shop conducted : 31 persons x 10

days - W/shop conducted : 18 persons x 8

days - W/shop conducted : 17 persons x 8

days - All equipment procured, delivered and

commissioned - Independent survey of teachers, pupils

and parents confirmed that quality of manuscripts and textbooks has improved

- Independent survey by PCR mission

confirmed the positive assessment of teachers

- Deleted from program - Four international consultants were

recruited for a total of 4.5 person-months

- Two domestic experts were recruited for 2.2 person-months

- 560,000 resource/library books

provided against 1.7 million anticipated to be funded by ADB

Additional activities carried out: - Research studies on: Gender

Presentation in Textbooks; Textbook design; and Author Pay conducted.

3. Improving the

Use of Learning Materials

- Seminar on Student-centered Learning: 50 persons x 14 days

- Seminars on Training of trainers in 6

subject areas : 60 persons x 14 days

- One seminar conducted: 15 persons x 5 days

- One seminar conducted: 34 persons for 9 days

- Seminars conducted on classroom use of textbooks on:

20 Appendix 1

Project Component Output Targets Achievements - Training of in-service teachers on

classroom use of new textbooks: 12,000 persons x 7 days

- Overseas study tour in design/

management of in service training for staff of in-service institutes: 15 persons x 30 days

- 280,000 teacher guides produced; - In-service training materials produced

for 6 subjects; - 4 person-months of international

consulting services; - 25 person months of domestic

consulting Services - 15 teacher in-service training institutes

linked with e-mail and internet services system;

- Research report on in-service training

system; - 15 sets of basic equipment for in-

service institutes(Computer, TV, VCR, Scanner)

Natural Sciences: 103 persons x 1 day Informatics: 23 persons x 5 days Economics: 33 persons x 10 days Environment: 42 persons x 3 days English 1: 28 persons x 14 days English 2: 21 persons x 3 days French: 55 persons x 3 days - Training conducted: 20,000 persons X 7 days - Study tour conducted for 17 persons

chosen among staff of in-service training institutes: 17 persons x 21 days

- 2.0 million teacher guides distributed - Training material produced for 6 core

subjects - One international consultants recruited

for a total of 3.8 person-months - Five domestic experts recruited for a

total of 10.6 person-months - All 15 teacher in-service training

institutes linked with e-mail and internet

- Report completed and available - All items procured, installed and

commissioned Additional Activities: - Introductory Training of Trainers on

student centered learning: 10 persons x 2 days

- Training on Interactive Learning 17 persons x 2 days

- Two research Studies on Student-Centered Teaching Methods and In-Service Training System conducted

4. Improving the

Quality and Efficiency of Textbooks

- A report with recommendations on future improvement in textbook supply system;

- Study tours on Textbook Production

Management: 15 persons x 30 days

- Report completed on 21 Oct 2002, available on request

- Two study tours conducted for:

publishers and ministry officials: 18 persons x 11 days

- Work attachments for two persons; - 3 person-months of international

consulting services and 8 person-

- Deleted from program - Two international consultants recruited

for a total of 10.1 person-months;

Appendix 1 21

Project Component Output Targets Achievements months of domestic consulting services

- Seminar on Textbook Production: 42

persons x 7 days

four domestic experts recruited for a total of 12.8 person-months

- One seminar on textbook production:

17 persons x 5 days - One seminar on textbook production

for participants of study tour 12 participants x 3 days Additional Activities: - Training in evaluation of bids for

procurement of textbooks: 75 persons x 5 days - A series of seminars on textbook

rental scheme for 17,842 participants (librarians, teachers and school directors)

- Research Studies conducted on: Analysis of Textbooks; Textbook

production systems; and Evaluation of Textbook Rental Scheme

5. Increasing the

Supply of Textbooks and Learning Materials

- All schools using the main language of instruction receive Uzbek language Latin script texts as required

- 15.6 million3 improved Latin-script

textbooks received by schools for grades 6 to 9

- 15,000 kits of supplementary

materials distributed - 28,000 copies of each of 60 new

Latin-script supplementary library books printed and distributed

- Selection of authors at Oblast level for

methodology guides;

- Over 55 million Latin-script textbooks for grades 1 to 9 distributed to schools.

- More than 29 million textbooks

inclusive of 6,600 books for the blind distributed to schools for grades 6 to 9.

- 49,000 kits of supplementary materials

distributed - 560,000 copies of supplementary

library books, comprising 28 titles distributed

- Authors for methodology guides

selected at Oblast level

- Selection of authors at national and local levels for new library materials;

- One foreign expert for three weeks; - One seminar for on book marketing; 20 persons x 30 days

- Authors for new library materials selected at national and local levels

- Two international consultants for 1.9

person-month recruited - One seminar on textbook marketing: 17 persons x 14 days Additional Activities: - Training of experts on manuscript

3 RRP Appendix 8 specifies 7.31 million textbooks to be financed under the loan for Grades 1–9. Reference to 15.6

million textbooks for Grades 6–9 quoted in Project Framework is not available in the RRP text.

22 Appendix 1

Project Component Output Targets Achievements evaluation: 74 persons x 14 days

- Training of expert on Bid Evaluation

for textbooks: 229 participants x 3 days

6. Improving the

durability of Textbooks

- Revolving fund used to purchase modern publishing and printing equipment and high quality paper;

- 250 districts receive book repair kits; - New textbooks last 4 years on

average; - 1.0 person-month of domestic

consultant; - 4.0 person-months international

consultant; - Seminar on Textbook Production

Methods 20 person x 28 days - Seminar on Textbook Production

Specifications: 10 person x 10 days - Seminar on Production Management

and Finance: 15 person x 10 days - Seminar on revolving fund operation: 10 persons x 7 days - Workshop on Use of Computers in

Planning and Management of publishing:

20 persons x 14 days - Training of trainers in book repairs: 15 persons x 10 days - Training of district staff on book repair: 210 persons x 10 days

- Implemented by NBU - Only 6 pilot schools provided with

repair kits; this activity was subsequently dropped from the program with ADB approval.

- Expert opinion confirms the useful life

of new textbooks will be at least 4 years

- No domestic experts recruited - Four international consultants

recruited for 8.2 person-months - Seminar conducted: 30 persons x 14

days. - Seminar cancelled - Seminar cancelled - Seminar cancelled - Workshop cancelled - Training conducted: 15 persons x 10

days. - Training conducted: 225 persons x 10

days. Additional Activities: One seminar on Textbook Provision

Policy for 16 participants x 5 days

Appendix 2

TEXTBOOKS AND LEARNING MATERIALS

($’000)

Appraisal ActualSubject Textbooks Teacher Guides Textbooks Teacher Guides

Number Cost Number Cost Total cost Number Cost Number Cost Total CostA. Textbook and Teacher's Guides

1. Uzbek Language 6,363 9,449 221 187 9,636 13,728 13,229 497 457 13,686

2. Latin-script Uzbek Language for OtherNational Schools (Russian, Kazakh,Turkmen, Kyrgyz, Karakalpak) 744 1,548 24.8 21 1,569 35 29 - - 29

Textbooks and Teachers Guides for Non-Uzbek National Languages (Russian, Kazakh, Tajik, Turkmen, Kyrgyz, Karakalpak) 205 513 10 13 526 637 609 86 102 712

B. Library books 1,680 2,520 - - 2,520 755 762 - - 762

C. Teaching Aid Kits 15 750 - - 750 49 927 - - 927

Grand total cost (A+B+C) 15,000 16,116

Appendix 2 23

24 Appendix 3

APPRAISAL AND ACTUAL PROJECT COSTS

Foreign Local Total Foreign Local Total Foreign Local Total Foreign Local TotalА. Staff Development

1. In Country - 1,244.0 1,244.0 - - - - 871.0 871.0 - 373.0 373.0 2. External 718.0 - 718.0 - - - 718.0 - 718.0 - - - 3. In-country Seminars - 156.0 156.0 - - - - 156.0 156.0 - - -

Subtotal 718.0 1,400.0 2,118.0 - - - 718.0 1,027.0 1,745.0 - 373.0 373.0

B. Consultants Services1. International 1,077.0 81.0 1,158.0 - - - 1,077.0 81.0 1,158.0 - - - 2. Domestic - 116.0 116.0 - - - - 116.0 116.0 - - -

Subtotal 1,077.0 197.0 1,274.0 - - - 1,077.0 197.0 1,274.0 - - -

C. Research Studies 4.0 158.0 162.0 - - - 4.0 158.0 162.0 - - -

D. Revolving Fund1. Equipment Procurement 9,000.0 - 9,000.0 9,000.0 9,000.0 - - - - - - 2. Paper Procurement 11,000.0 - 11,000.0 11,000.0 - 11,000.0 - - - - - -

Subtotal 20,000.0 - 20,000.0 20,000.0 - 20,000.0 - - - - - -

E. Textbooks and Materials1. Textbook Production and Distribution

8,534.0 62,584.0 71,118.0 - - - 8,534.0 3,636.0 12,170.0 - 58,948.0 58,948.0 2. Library Materials and Instructional 2,168.0 1,628.0 3,796.0 - - - 2,168.0 1,628.0 3,796.0 - - -

Subtotal 10,702.0 64,212.0 74,914.0 - - - 10,702.0 5,264.0 15,966.0 - 58,948.0 58,948.0

F. Equipment1. Educational Equipment 309.0 35.0 344.0 - - - 309.0 35.0 344.0 - - - 2. Office Equipment 271.0 30.0 301.0 - - - 271.0 30.0 301.0 - - -

Subtotal 580.0 65.0 645.0 - - - 580.0 65.0 645.0 - - -

G. Incremental Recurrent Cost1. PIU Staff Salaries - 142.0 142.0 - - - - 85.0 85.0 - 57.0 57.0 2. Incremental Operating Expenses - 205.0 205.0 - - - - 123.0 123.0 - 82.0 82.0

Subtotal - 347.0 347.0 - - - - 208.0 208.0 - 139.0 139.0

H. Taxes and Duties- 391.0 391.0 - - - - - - - 391.0 391.0

Total Base Cost 33,081.0 66,770.0 99,851.0 20,000.0 - 20,000.0 13,081.0 6,919.0 20,000.0 - 59,851.0 59,851.0

Interest and Services ChargesA. Ordinary Capital Resources 6,663.0 - 6,663.0 - - - - - - 6,663.0 - 6,663.0 B. Asian Development Fund 430.0 - 430.0 - - - - - - 430.0 - 430.0

Subtotal 7,093.0 - 7,093.0 - - - - - - 7,093.0 - 7,093.0

Grand Total 40,174.0 66,770.0 106,944.0 20,000.0 - 20,000.0 13,081.0 6,919.0 20,000.0 7,093.0 59,851.0 66,944.0 37.6% 62.4% 100.0% 18.7% 18.7% 12.2% 6.5% 18.7% 6.6% 56.0% 62.6%

Source: RRP: UZB 29664

Table 1. Appraisal Cost Estimates ( $ '000)B39

Category Total Cost ADB Financing-OCR (DFI) ADB Financing-ADF Government Financing-Total

Foreign Local Total Foreign Local Total Foreign Local Total Foreign Local Total

А. Staff Development 1. In Service Country 0 2,054 2,054 0 0 0 0 14 14 0 2,040 2,040 2. External Training 370 0 370 0 0 0 370 0 370 0 0 0 3. In-country Seminars/Workshops 0 39 39 0 0 0 0 39 39 0 0 0

Subtotal 370 2,093 2,463 0 0 0 370 53 423 0 2,040 2,040

B. Consulting Services 1. International 1,351 0 1,351 0 0 0 1,351 0 1,351 0 0 0 2. Domestic 0 177 177 0 0 0 0 177 177 0 0 0

Subtotal 1,351 177 1,528 0 0 0 1,351 177 1,528 0 0 0

C. Research Studies 0 9 9 0 0 0 0 9 9 0 0 0

D. Revolving Credit Fund 1. Equipment Procurement 8,592 0 8,592 8,592 0 8,592 0 0 0 0 0 0 2. Paper Procurement 10,919 0 10,919 10,919 0 10,919 0 0 0 0 0 0

Subtotal 19,511 0 19,511 19,511 0 19,511 0 0 0 0 0 0

E. Textbooks and Materials 1. Textbook Production and Distribution 48,889 32,591 81,480 0 0 0 15,859 0 15,859 33,030 32,591 65,621 2. Library Materials and Instructional Kits 1,051 701 1,752 0 0 0 333 0 333 718 701 1,419

Subtotal 49,940 33,292 83,232 0 0 0 16,192 0 16,192 33,748 33,292 67,040

F. Equipment 1. Educational Equipment 408 0 408 0 0 0 408 0 408 0 0 0 2. Office Equipment 212 0 212 0 0 0 212 0 212 0 0 0

Subtotal 620 0 620 0 0 0 620 0 620 0 0 0

G. Incremental Recurrent Cost 1. PIU Staff Salaries 0 177 177 0 0 0 0 100 100 0 76 76 2. Incremental Operating Expenses 0 64 64 0 0 0 0 44 44 0 20 20

Subtotal 0 241 241 0 0 0 0 144 144 0 96 96

H. Taxes and Duties 0 3,536 3,536 0 0 0 0 0 0 0 3,536 3,536

Total Project Cost 71,792 39,348 111,140 19,511 0 19,511 18,533 384 18,916 33,748 38,964 72,712

Percentage of Project Cost 64.6% 35.4% 100.0% 17.6% - 17.6% 16.7% 0.3% 17.0% 30.4% 35.1% 65.4%

Interest and Service Charges 0 0

Source: ADB AND MOPE

Table 2. Project Financing Plan - Actual( $ ' 000)

Project ComponentsTotal Project Cost ADB Financing (ADF) Government FinancingADB Financing OCR (DFI)

Appendix 3 25

26 Appendix 4

DISBURSEMENTS

Table 1. L1594-UZB

Year Quarter Quarterly Disbursement

($ million)

Cumulative Disbursement

($ million)

Percent of

Loan

1998 III 0.000 0.000 0.00 IV 0.000 0.000 0.00

1999 I 0.000 0.000 0.00

II 0.000 0.000 0.00 III 0.000 0.000 0.00 IV 0.000 0.000 0.00

2000 I 0.000 0.000 0.00 II 0.000 0.000 0.00 III 0.730 0.730 3.74 IV 0.560 1.290 6.61

2001 I 7.771 9.061 46.44 II 2.257 11.318 58.01 III 0.597 11.915 61.07 IV 3.372 15.287 78.35

2002 I 1.471 16.758 85.89 II 1.424 18.182 93.19 III 0.000 18.182 93.19 IV 0.239 18.421 94.41

2003 I 0.095 18.516 94.90 II 0.995 19.511 100.00 Total 19.511

Appendix 4 27

Table 2. L1595-UZB

Year Quarter Quarterly Disbursement

($ million)

Cumulative Disbursement

($ million)

Percent of

Loan

1998 I 0.000 0.000 0.00 II 0.000 0.000 0.00 III 0.000 0.000 0.00 IV 0.460 0.460 2.43

1999 I 0.000 0.460 2.43

II 0.000 0.460 2.43 III 0.042 0.502 2.65 IV 0.047 0.549 2.90

2000 I 0.011 0.560 2.96 II 0.009 0.569 3.01 III 0.659 0.228 6.49 IV 0.519 1.747 9.24

2001 I 0.481 2.228 11.78 II 0.757 2.985 15.78 III 1.033 4.018 21.24 IV 2.186 6.204 32.80

2002 I 1.282 7.486 39.58 II 0.318 7.804 41.26 III 2.573 10.377 54.86 IV 2.583 12.960 68.52

2003 I 0.294 13.254 70.07 II 0.236 13.490 71.32 III 0.863 14.353 75.88 IV 1.953 16.306 86.21

2004 I 1.114 17.420 92.10 II 1.220 18.640 98.55 III 0.322 18.962 100.25 IV -0.047a 18.915 100.00 Total 18.915

a Refund of remaining advances to imprest account.

28 Appendix 5

IMPLEMENTATION SCHEDULE

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3

1. Capacity Building in Curriculum Development

2. Improving the Pedagogical Quality of Textbooks

3. Improving the Use of Learning Materials

4. Improving the Quality and Efficiency of Textbooks

5. Increasing Supply of Textbooks and Learning Materials

6. Improving the Durability of Textbook

1998 1999 2000 2001 2002 2003 2004Year 5 Year 6 Year 7Year 1 Year 2 Year 3 Year 4

Establishment of PIU PIU staff training PIU Study Tour1

1.1 Seminars on International Approaches 1.2 Study Tours 1.3 Curriculum Studies 1.4 Development of Basic Economics 1.5 Development of Environmental Studies

2.1 Upgrading the Textbook Unit of REC 2.2 Establishment of Author Resource Center 2.3 Authors Seminars 2.4 Textbook Development Seminars 2.5 Overseas Assignments 2.6 Study on Authors Pay 2.7 Study on Gender Issues 2.8 Baseline Study on Textbook Format

4.1 Seminars on Book Chain 4.2 Study Tours2

3.1 Seminars and Development Study on Teacher Math. 3.2 Study Tours 3.3 Training of Trainers 3.4 Teacher Training

6.5 Book Repair Equipment4

5.2 Financing of Latin-character Textbooks 5.3 Financing of Learning Support Materials 5.4 Financing of Library Materials

6.1 Revolving Fund for Equipment

ACTIVITY

6.2 Revolving Fund for Paper 6.3 Seminars 6.4 Book Repair Campaign

4.3 Overseas Assignments2

4.4 Study of Supply System/Book Chain3

4.5. Study of Book Sale3

5.1 Seminar on Marketing

3.5 Printing of Methodological Guides 3.6 Teacher Training Network

1 PIU staff participated at training seminars conducted by ADB in Uzbekistan. 2 The overseas working attachments were cancelled, but the number of participants of study tours was increased. 3 The Studies 1.3 and 4.4 were amalgamated into a single one. 4 The special equipment for repair of the books was deleted from the program with ADB consent. Legend Appraisal Actual Actual revolving fund using repayments

Appendix 6 29

COMPLIANCE WITH LOAN COVENANTS FOR LOAN 1594-UZB

Covenant

Reference in Loan Agreement

Compliance

A. Loan Agreement 1. The Borrower shall make available to NBU, promptly as needed and on terms and conditions acceptable to the Bank, the funds, facilities, services and other resources which are required in addition to the proceeds of the Loan.

Loan Agreement, Section 4.02

Partially complied with. Reporting of fund transfer to MOF for the purpose of the Project was inadequate. This will be followed up with NBU in future discussions.

2. The Borrower shall cause NBU to (i) maintain separate accounts for the Part F(i) of the Project; (i) have such accounts and financial statements audited annually, in accordance with sound international auditing standards by independent auditors whose qualifications, experience and terms of reference are acceptable to the Bank; (iii) furnish to the Bank as soon as available but in any event not later than six (6) months after the end of such related fiscal year, certified copies of such audited accounts and financial statements and the report of the auditors relating thereto (including the auditor’s opinion on the use of Loan proceeds and compliance with the covenants of the Loan Agreement), all in the English language; and (iv) furnish to the Bank such other information concerning such accounts and financial statements and the audit thereof as the Bank shall from time to time reasonably request.

Loan Agreement, Section 4.03. (d)

Complied with substantial delay. Audited financial statements for 2000, 2001 and 2002 submitted in 2003. Audited financial statements for 2003 not yet submitted. NBU committed to submit these by July 2005.

B. Project Agreement 3. (a) NBU shall maintain records and accounts adequate to record the progress of the Project and to reflect, in accordance with consistently maintained sound accounting principles, the operations and financial conditions of NBU.

Project Agreement, Section 3.02. (a)

Complied with.

(b) NBU shall cause each sub-borrower to maintain records and accounts adequate to record the progress of its Subproject (including cost thereof) and to reflect, in accordance with consistently maintained sound accounting

Loan Agreement, Section 3.02.(b)

Complied with

30 Appendix 7

Covenant

Reference in Loan Agreement

Compliance

principles, the operations and financial condition of such sub-borrower. 4. NBU shall promptly inform the Bank of any condition which interferes with, or threatens to with, the progress of the Project, performance of its respective obligations under this Project Agreement or the Subsidiary Loan Agreement, or the accomplishment of the purposes of the Loan.

Project Agreement, Section 3.03. (b)

Limited compliance due to weak capacity in portfolio analysis and high staff turnover.

(a) NBU shall furnish to the Bank all such reports and information as the Bank shall reasonably request concerning the (i) the Loan and expenditure of the proceeds thereof; (ii) the Project; (iii) Subborrowers, Subprojects and the Subloans; (iv) administration, operations and financial condition of NBU; and (v) any other matters relating to the purposes of the Loan.

Project Agreement, Section 3.04. (a)

Complied with.

(b) NBU shall furnish to the Bank quarterly reports on the execution of the Project and on the operation and management of NBU within 60 days after the end of the relevant quarter.

Project Agreement, Section 3.04. (b)

Complied with. Quarterly reports submitted regularly.

(c) Promptly after the closing date for withdrawal from Loan account, but in any event not later than 3 months after the closing date or such later date as the Bank may agree for this purpose, NBU shall prepare and furnish to the Bank a Project completion report on the utilization of the loan, the execution of the subprojects, their costs, performance by NBU of its obligations under the Project Agreement and accomplishment of the purposes of the Loan.

Project Agreement, Section 3.04. (c)

Complied with. PCR received in Dec 2004.

(d) NBU shall have its accounts and financial statements (balance sheet and statement of sources and applications of funds) audited annually, in accordance with appropriate auditing standards consistently applied, by independent auditors whose qualifications, experience and terms of reference are acceptable to the Bank, and shall, promptly after their preparation but in any event not later than six (6) months after

Project Agreement, Section 3.04. (d)

Complied with. Audited financial statements of NBU were regularly submitted.

Appendix 6 31

Covenant

Reference in Loan Agreement

Compliance

the close of its fiscal year to which they relate, furnish to the Bank (i) certified copies of such audited accounts and financial statements and (ii) the long form audit report of the auditors relating to the supplementary information on NBU’s accounts (including the auditors’ assessment on the quality of NBU’s long term loans and adequacy of NBU’s provision for doubtful, distressed and marginal loans, and the auditor’s opinion on the use of Loan proceeds and compliance with the covenants of the Ordinary Operations Loan Agreement and this Project Agreement), all in the English language. NBU shall furnish to the Bank such further information concerning such accounts and financial statements and the audit thereof as the Bank shall from time to time reasonably request.

5. NBU shall enable the Bank’s representatives to inspect any Sub-borrower, any subproject, the goods financed out of the proceeds of the Loan, and any relevant records and documents.

Project Agreement Section 3.05

Complied with. Some sub-borrowers were visited during the midterm review mission in Sep 2001.

C. Execution of the Project 1. Selection and Appraisal of Subprojects 6. NBU shall appraise all projects in accordance with a standard appraisal report format agreed upon between NBU and the Bank. The appraisal reports shall include a business plan and projected financial statements (including income statements, balance sheets and cash flow statements) and an explanation of the assumptions on which such statements are based.

Project Agreement Schedule para. 2

Complied with. All subloans submitted to ADB were supported with appraisal reports and business plans.

7. NBU shall appraise proposed Sub-borrowers on the basis of Sub-borrower’s financial, managerial, technical competence, marketing and procurement requirements, as well as the Subborrower’s financial viability, employment and environmental impact when appraising and selecting the sub-borrowers.

Project Agreement, Schedule, para. 3

Complied with

8. NBU shall obtain the Bank’s prior approval with regards to (i) the first four Subloans to be processed by NBU; and (ii)

Project Agreement, Schedule, para. 6

Complied with. All subloans granted had been approved by ADB.

32 Appendix 7

Covenant

Reference in Loan Agreement

Compliance

any Subloan which exceeds the agreed maximum amount for procurement of either paper or equipment. 2. Selection Criteria for the Subborrowers

9. NBU shall provide the Subloans only to Sub-borrowers which met the selection criteria.

Project Agreement, Schedule, para. 8

Complied with. All sub-borrowers met the criteria.

3. Maximum Amount of Subloans 10. Except as the Bank may otherwise agree, NBU shall not make any Subloan (i) for paper, in an amount exceeding the equivalent of $3,000,000; and (ii) for equipment, in an amount exceeding the equivalent of $3,000,000 or in an amount which would exceed such limit when added to the outstanding principal amount of all other Subloans made for the same Subproject.

Complied with. One subloan for equipment exceeded the $3 million limit, but with ADB approval.

4. Monitoring of Subprojects 11. NBU shall monitor the financial and economic performance, and socioeconomic and environmental benefits of the Subprojects. NBU shall assist the Bank to monitor and evaluate the long-term benefits of a random sample of five Subprojects.

Project Agreement, Schedule, para. 10

Partially complied with. Inadequate evaluation of long-term benefits accrued by sub-projects. This will be followed up with NBU in future discussions.

5. Revolving Funds 12. Subject to making the required payments of principal and interest under the Subsidiary Loan Agreement, NBU shall use funds received from the repayment of principal of the subloans as revolving funds to make additional loans in accordance with its Operational Policies.

Project Agreement, Schedule, para. 11

Complied with. While the revolving fund for paper had processed 5 more loans by end-2004, actual utilization of the equipment fund has been stagnant due to a lack of demand for further subloans.

B. Institutional Matters 1. Financial requirements for NBU 13. Except as the Bank may otherwise agree, and subject to any applicable banking

Project Agreement, Schedule, para. 12

Partially complied with. Revolving fund

Appendix 6 33

Covenant

Reference in Loan Agreement

Compliance

laws to the country, NBU shall at all times during the term of the Project Agreement maintain the specific financial requirements specified in Schedule para. 12, (a-i).

management affected by lack of unified and poor monitoring of financial requirements. This will be followed up with NBU in future discussions.

2. Interest Rates 14. NBU shall from time to time, at the request of either the Borrower or the Bank, exchange views with both parties concerning the interest rates to be charged by NBU, taking into consideration the refinancing rate on the Central Bank of Uzbekistan, profitability and interest rates charged by other banks and financial institutions in Uzbekistan.

Project Agreement, Schedule, para. 13

Complied with. NBU consulted MOF on interest rates and related issues.

34 Appendix 7

COMPLIANCE WITH COVENANTS FOR LOAN 1595-UZB[SF]

Covenant

Reference in Loan Agreement

Compliance

1. (a) The Borrower shall maintain, or cause to be maintained, records and accounts adequate to identify the goods and services and other items of expenditure financed out of the proceeds of the Loan, to disclose the use thereof in the Project, to record the progress of the Project (including the cost thereof) and to reflect, in accordance with consistently maintained sound accounting principles, the operations and financial condition of the agencies of the Borrower responsible for the carrying out of the Project and operation of the Project facilities, or any part thereof.

Loan Agreement, Section 4.06. (a)

Complied with.

2. (b) The Borrower shall (i) maintain, or cause to be maintained, separate accounts for the Project; (ii) have such accounts and related financial statements audited annually, in accordance with appropriate auditing standards consistently applied, by independent auditors whose qualifications, experience and terms of reference are acceptable to the Bank; (iii) furnish to the Bank, as soon as available but in any event not later than six (6) months after the end of each related fiscal year, certified copies of such audited accounts and financial statements and the report of the auditors relating thereto (including the auditors' opinion on the use of the Loan proceeds and compliance with the covenants of this Loan Agreement as well as on the use of the procedures for imprest account/ statement of expenditures), all in the English language; and (iv) furnish to the Bank such other information concerning such accounts and financial statements and the audit thereof as the Bank shall from time to time reasonably request.

Loan Agreement, Section 4.06. (b)

Complied with. Audited financial statements were regularly submitted.

3. (c) The Borrower shall enable the Bank, upon the Bank's request, to discuss the Borrower's financial statements for the Project and its financial affairs related to the Project from time to time with the Borrower's auditors,

Loan Agreement, Section 4.06.(c)

Complied with.

Appendix 7 35

Covenant

Reference in Loan Agreement

Compliance

and shall authorize and require any representative of such auditors to participate in any such discussions requested by the Bank, provided that any such discussion shall be conducted only in the presence of an authorized officer of the Borrower unless the Borrower shall otherwise agree.

4. (b) Without limiting the generality of the foregoing, the Borrower shall furnish, or cause to be furnished, to the Bank quarterly reports on the carrying out of the Project and on the operation and management of the Project facilities. Such reports shall be submitted in such form and in such detail and within such a period as the Bank shall reasonably request, and shall indicate, among other things, progress made and problems encountered during the quarter under review, steps taken or proposed to be taken to remedy these problems, and proposed program of activities and expected progress during the following quarter.

Loan Agreement, Section 4.07.(b)

Complied with. Quarterly reports were regularly submitted to ADB.

5. (c) Promptly after physical completion of the Project, but in any event not later than three (3) months thereafter or such later date as may be agreed for this purpose between the Borrower and the Bank, the Borrower shall prepare and furnish to the Bank a report, in such form and in such detail as the Bank shall reasonably request, on the execution and initial operation of the Project, including its cost, the performance by the Borrower of its obligations under this Loan Agreement and the accomplishment of the purposes of the Loan.

Loan Agreement, Section 4.07.(c)

Complied with, Borrower’s PCR was submitted in Dec 2004.

A. Implementation Arrangements 6. MOPE, as the Project Executing Agency for Parts A, B, C, D and F(ii) -(iii) of the Project, shall have overall responsibility for the planning, organization, management, supervision and monitoring of the entire Project.

Loan Agreement, Schedule 6, para. 1

Complied with.

36 Appendix 7

Covenant

Reference in Loan Agreement

Compliance

7. MOPE shall establish the Project Implementation Unit (PIU) prior to the Effective Date within REC. The PIU shall manage the day-to-day implementation of Parts A,B,C,D,E and F (ii)-(iii) of the Project. The PIU shall be responsible for ensuring (i) detailed Project planning and scheduling, (ii) procurement of all goods and services, (iii) recruitment and supervision of all international and domestic consultants, (iv) Project accounting, including arranging necessary audits, (v) disbursement of funds, including timely submission of withdrawal applications, (vi) managements of the impress account, and (vii) reporting to the Bank on Project progress based on systematic monitoring. The PIU shall be heated by a full-time Project Manager, assisted by an adequate number of appropriate professional staff who shall be given training in their areas of responsibility. The PIU shall contain sections: (i) finance and accounting,(ii) procurement, (iii) training, and (iv) monitoring and evaluation.

Loan Agreement, Schedule 6, para. 2

Complied with. MOPE established the PIU within REC. Seven permanent staff were assigned to PIU.

8. The Project Steering Committee (PSC), which has already been established by the Borrower, shall guide the PIU and ensure consistency with the overall educational policies and plans of the Borrower, The PSC shall be responsible for (i) coordinating Project activities among the agencies of the Borrower and other organizations involved, (ii) approving annual implementation plans, (iii) monitoring the progress of the annual implementation plan, (iv) identifying constraints to Project Implementation and arranging suitable methods for resolving these, and (v) advising the PIU as necessary. The PSC shall be chaired by a high-level official and include representatives of MOPE, Cabinet of Ministers, MOF, NBU, State Institute for In-service Training, SPC and state representative of oblast education directors. The PSC shall meet regularly to review progress and provide guidance to the Project Manager.

Loan Agreement, Schedule 6, para. 3

Complied with. Project Steering Committee (PSC), appointed by the Cabinet of Ministers, provided guidance to the PIU, supervised Project activities and monitored Project implementation.

Appendix 7 37

Covenant

Reference in Loan Agreement

Compliance

9. MOPE has designated a Project Director who shall (i) coordinate the Project activities within MOPE for Parts A-E and F(ii)-(iii) of the project, (ii) report on such parts of the Project to the PSC, and (iii) provide guidance, as required, to the Project

Loan Agreement, Schedule 6, para. 4

Complied with. The Project Director was appointed in October, 1997 before the signing of the Loan Agreement. However, the Project director was replaced three times. After the resignation of second Project Director in April 2003, the first Deputy Minister coordinated the all project activities.

10. The Borrower shall establish a Textbook Development Working Group (TDWG) within two month from the Effective Date, comprising senior officials from the ministries and agencies concerned to undertake a comprehensive review of the textbook production system and make recommendations to the Borrower for improving system performance and efficiency. The Borrower shall implement these recommendations in accordance with a detailed plan to be developed by TDWG.

Loan Agreement, Schedule 6, para. 5

Delayed compliance. TDWG was formed in December 1998, comprising officials from COM, MOPE, SPC, MOF, REC, State Institute for In-service Training, and publishing houses.

11. The Borrower shall ensure that the textbooks, teacher guides, library books, and supplementary learning materials financed by the Bank under the Project are distributed primarily to the rural school.

Loan Agreement, Schedule 6, para. 6

Complied with

B. Other Matters 1. Monitoring and Evaluation 12. The PIU shall establish, within twelve months from the Effective Date, a Project Performance Management System (PPMS) to provide accurate and systematic information and feedback on the quantitative aspects of the Project for effective monitoring data according to agreed indicators and targets. The collected monitoring data shall serve as the basis for assessing the progress of the Project on a quarterly and annual basis. In addition to maintaining a systematic data base on Project monitoring indicators, the PIU shall

Loan Agreement, Schedule 6, para. 7

Complied with.

38 Appendix 7

Covenant

Reference in Loan Agreement

Compliance

Arrange for selected evaluation studies to be undertaken, with the assistance of local consultants to be recruited under the Project and by using a research design and methodology acceptable to the Bank, to establish baseline data for the Project. The results of the evaluation studies shall be made available to the Bank in English.

2. Midterm Review 13. The Borrower and the Bank shall jointly undertake annual reviews of the Project to assess progress identify constraints and agree on strategies for resolving constraints, In addition, the Borrower and the Bank shall undertake a midterm review of the Project in the Third year of Project implementation. The midterm review shall (i) review the scope design, and implementation arrangement of the Project; (ii) identify changes needed since the time of the Project appraisal; (iii) assess implementation performance against establish compliance with the Loan covenants; (iv) review and establish compliance with the Loan covenants; (v) identify problems and constraints; and, (vi) if necessary, recommend changes in the design or implementation arrangements of the Project.

Loan Agreement, Schedule 6, para. 8

Complied with. Midterm review was fielded in Sep 2001.

Appendix 9 39

COMPLETION REPORT OF TA 2947-UZB Division: ECSS

Amount Approved: $900,000 TA No. and Name: 2947-UZB: Monitoring Implementation of Education Reform Revised Amount: n/a Executing Agency: Social Sector Department of the Cabinet of Ministers (COM)

Source of Funding: TASF TA Amount Undisbursed16

$293,515

TA Amount Utilized $606,485

Date Completion Date Original 30 Sep 2003

Actual 31 May 2004

Closing Date

Approval 17 Dec 1997

Signing 20 Jan 1998

Fielding of Consultants 18 Oct 1998

Original 30 Jun 2004

Actual 1

Description Early after independence, Uzbekistan realized the need to adapt its education system to the changing social and economic environment. ADB has supported Uzbekistan’s education development since it started operations in the country in 1996. In August 1997, the Parliament of Uzbekistan endorsed the National Program for Personnel Training aimed at providing guidance and directions for reform in the education sector. The Social Sector Department of the Cabinet of Ministers, (COM) requested technical assistance (TA) to strengthen central-level capacity to monitor the implementation of education reforms and to provide advice on strategies for coordination of external assistance. Objectives and Scope The general aim of the TA was to facilitate the implementation of education reform by strengthening data collection and policy analysis capacities of the Social Sector Department of the COM. The specific objectives were to (i) identify the key monitoring indicators for education reform, (ii) design a system for collecting and analyzing the necessary data, (iii) develop a means of reporting the status of education reform in a manner and at frequency required by policymakers, and (iv) strengthen the capacity of the Social Sector Department of COM to coordinate external assistance in education. The TA anticipated the establishment of an Education Reform Implementation Monitoring Unit (ERIMU) tasked with (i) collecting data on key indicators, (ii) reporting on reform implementation to policy-makers, (iii) delivering training, data processing and analysis at national and local levels, and (iv) undertaking special evaluation studies on sector reform issues and recommending actions to policymakers. Evaluation of Inputs Initial TA inputs comprised (i) 17 person-months of international consultancy (three experts), (ii) 120 personmonths of domestic consultancy (two long-term positions of 5 years), (iii) two study visits for 6 persons each, and (iv) funds for research, seminars and surveys. Seven contract variations were processed to adjust TA inputs to achieve the desired outcomes, including splitting a domestic consultant position into several specialized consultancies of shorter duration. Problems in initial input evaluation pertained to the need for specialized skills to design and manage a comprehensive information system, difficulties in obtaining government counterpart funding to the TA, and inappropriate budgeting (several budget categories were missing). The external assistance component was dropped because of changing views of the COM on the priority to be given to this component. The TA implementation period, originally estimated at 5 years, was shortened to 4 years during contract negotiations at the request of the Government, and re-extended to the initial 5 years during the implementation. The TA was extended mainly to assist COM in monitoring the implementation of the Education Sector Development Program (Loans 1960/61-UZB approved in December 2002 for $108.5 million) and transferring knowledge to ERIMU. However, delays in the start of the ESDP did not allow the TA team to fully support the COM in this respect. The consultant's services and level of inputs exceeded the terms of reference, primarily because of the extensive work conducted at the local level, in particular the constitution and training of regional working groups. The TA team, throughout the five years of implementation, also played a critical liaison role on education policy matters with the Government and supported all ADB loan processing missions. Thanks to this role played by the TA, the supervision and support role exercised by ADB went far beyond the requirements of the usual project administration missions.

16 To be determined after ADB’s receipt and payment of the final claim of the consultant.

40 Appendix 9

Evaluation of Outputs The TA produced outputs as planned under all components, except external assistance which was dropped at an early stage. TA-supported achievements include (i) establishment and staffing of the ERIMU (albeit much later than expected), (ii) constitution of an education database, Information System for Monitoring Education Reform (ISMER), comprising key indicators related to reform decrees, (iii) training of staff, both at the central and local levels through the constitution of regional working groups at a much higher scale than initially envisaged, (iv) in-depth analytical reports on five reform areas (teachers, students, textbooks, facilities, and expenditure). The quality of outputs and reports was generally good and clearly helped to prompt positive policy changes. In particular, the TA team was closely associated with the preparation of the ESDP (initiated in May 2000) and effectively contributed to facilitate sector policy dialogue between the ADB and the Government. The ISMER however, is probably an excessively sophisticated tool with insufficient policy relevance as its main purpose for the COM was to monitor the implementation of a series of Government decrees and resolutions. Overall Assessment and Rating The TA contributed to changing the attitude of senior officials regarding sector planning and monitoring. The need for reliable data for sound analysis and policy-making is acknowledged. In practice, the TA contributed to an improved formulation of government decrees (relevance to identified policy issues, specification of responsibilities and resources required, etc.). However, It is doubtful that a strong education policy advisory capacity was built through the TA, in part because of the late establishment of the ERIMU and the continued lack of familiarity with modern planning techniques and policy development processes. The ERIMU will also need to demonstrate its capacity to maintain the sophisticated information systems established with the TA support. Staff resources provided by the Government are clearly sufficient, as the responsibilities of the unit were recently expanded to include monitoring of other government programs in the social sectors. It is also the view of the TA team that regional working groups (tasked with data collection and processing at the local level) should undergo regular training/retraining as assigned members change frequently. Transparency of information and analysis remains a major challenge that could not be fully addressed through the TA. Confidentiality of data produced by the monitoring system has been a constraint impeding dissemination of information, policy advocacy, and donor coordination. The TA is rated as partly successful. Major Lessons Learned Institutional constraints, particularly those pertaining to information sharing and transparency, are difficult to address at the sector level. The TA effectively highlighted deficiencies in existing information systems, including lack of information sharing and cooperation between government agencies, input- rather than output- and outcome-orientation of data, weak analyses of information collected, excessive workload associated with data collection and processing tasks, monopolies in the production of ‘official statistics’, restricted access to information, etc. However, no decisive actions could be taken at the sector level to fundamentally reform information collection and dissemination processes. It is also important to acknowledge that Government resources are scarce, and that commitments to support TA implementation may not result in staff or budget allocations at the anticipated time and volume. Long-term support of the ADB is required to achieve institutional change and lasting impact. Limited TA resources, which provide a necessary platform for policy dialogue and capacity development, constrain ADB’s lead policy advisory in the education sector. Recommendations and Follow-Up Actions Several actions can be recommended to consolidate TA achievement, including the redefinition of the terms of reference of the ERIMU, follow-up discussions with the State Committee for Statistics regarding the dissemination of ISMER data, further training of regional working groups, and additional support to monitor ESDP implementation. Additional support is necessary to follow-up on such actions and assist in developing a full-fledged education management information system. Prepared by

Marc Cohen

Designation

Senior Poverty Reduction Specialist, RSPR

Appendix 10 41

COMPLETION REPORT OF TA 2948-UZB Division: AEEH

Amount Approved: $500,000 TA No. and Name: 2948-UZB: Capacity Building in Education Finance Revised Amount: $500,000 Executing Agency: Ministry of Finance, Department for the Financing of the Social Sphere

Source of Funding: TA Amount Undisbursed $9,946

TA Amount Utilized $490,054

Date Closing Date Approval 17 Dec 1997

Signing 20 Jan 1998

Fielding of Consultants 8 March 1999

Original Dec 1999

Actual April 2000

TA Description • In August 1997, Uzbekistan adopted an education sector reform program (the National Program for Personnel

Training or NPPT) for implementation in the period 1998-2007. The NPPT is an ambitious plan to reform the contents, methods and structure of education, making them more relevant to the needs of a market economy. Its key feature, in terms of investment needs, is a new education structure, adding three years of compulsory education at senior secondary level. The Government was aware that much of the cost of the reform would have to be met through improved cost-effectiveness and efficiency of the education system and requested technical assistance (TA) to undertake a comprehensive study of the costs and financing of education.

TA Objectives and Scope • The main purpose of the TA was to identify strategies for improving cost-effectiveness and efficiency of the

education system. The TA scope combined consulting services, research studies and staff development activities aimed at (i) carrying out a review of public spending on education, (ii) assessing the financial sustainability of NPPT and recommending funding strategies, and (iii) contributing to strengthening financial management and policy analysis skills of key personnel at the central and local levels of administration.

• The TA was considered very relevant in its objectives and scope. As this was the first advisory TA in the education sector, work constraints were probably underestimated in the original formulation. Consequently, the workplan and the input mix had to be slightly adjusted.

• During contract negotiations, the Government requested a change of executing agency, designating the Ministry of Finance (MOF) rather than the Social Complex of the Cabinet of Ministers (COM). (The absorptive capacity of the COM was considered inadequate, a view shared by ADB).

Evaluation of Inputs • Overall, good cooperation was enjoyed from MOF who fulfilled its principal commitments (provision of office

space and facilities, logistical and organizational support, access to information). A TA Steering Group was established, composed of senior officials (typically Deputy Ministers) of the main counterpart agencies (COM, MOF, Ministry of Public Education, Ministry of Higher and Secondary Specialized Education, Ministry of Macroeconomics and Statistics). However, lengthy working procedures, lack of time availability of MOF senior staff, and frequent changes in counterpart staff have affected work efficiency. Rating of the performance of MOF is thus considered less than satisfactory.

• The TA was implemented over a period of twelve months (instead of nine). Three ADB missions were fielded for inception (27 April to 07 May 1999), review (14 to 27 September 1999), and final review (01 to 14 February 2000), allowing close supervision of the TA team. ADB performance is considered satisfactory.

• A number of local constraints tended to slow down the implementation of the TA, including (i) cumbersome local procedures for releasing statistics and budget data that slowed down the research work; (ii) lack of reliability and consistency of data increasing the risk of misinterpretation and wrong analysis; (iii) limited time availability of counterpart staff in Ministries resulting in cancellation, postponement or shortening of important meetings; (iv) need for considerable translation / interpretation services; and (v) general lack of experience and familiarity of officials with work procedures of international agencies.

• Several contract variations were processed in the course of the TA to adjust inputs to actual needs and more specifically to (i) increase the person-months inputs of the two international consultants; (ii) terminate the services of one domestic consultant considered unsatisfactory; (iii) increase the budget for office operations and translation due to the extended implementation period. TA funds were adequate.

42 Appendix 10

TA Outputs Evaluation • The following reports were submitted within acceptable timeframe by the consultants: (i) Inception report, (ii)

Phase 1 report, (iii) Interim report, (iv) Draft final report, (v) Final report. The contents and scope of the reports were in accordance with the terms of reference (although the budget process was not investigated). A major benefit of the TA work has been to highlight critical sector reform issues and to facilitate dialogue both between ADB and the Government and among Government agencies. Of major importance was the increased realization among policy makers of the need to assess the feasibility of reform measures, and to work out implementation strategies and monitoring mechanisms.

• In the course of the TA, the capacity building approach was better delineated, moving away from the originally envisaged ‘training workshop’ approach (not practical in view of the numerous engagements of the main counterparts) to a combination of short working sessions and guidance for conducting focused policy-research studies. The capacity building impact of the TA can be summarized as follows: (i) new terminology was introduced and economic notions and concepts commonly used in modern educational planning were clarified and disseminated; (ii) a core group of national experts gained knowledge about methodologies and techniques for data collection, processing and analysis; (iii) senior officials (including some TA Steering Group members) gained awareness of educational reform processes in other countries through a study tour to UNESCO's International Institute for Educational Planning (Paris, France) and Fontys University, Netherlands.

• Client satisfaction with the TA outputs can be assessed in several ways. On the one hand, Government officials initially tended to reject policy recommendations that appeared critical of NPPT, especially on grounds of financial sustainability. On the other hand, the TA reports generated substantial interest and debate, especially at the local level. The final TA seminar was a major forum for openly discussing concerns raised by the NPPT.

TA Overall Assessment and Rating • The TA can be rated as successful in achieving its objectives in terms of depth of sector work and national capacity

development (albeit at a very small scale). It was also very instrumental in fostering the sector policy dialogue between ADB and the Government. This dialogue served as the main foundation for the formulation of the Education Sector Development Program (ESDP) whose processing started in June 2000. The TA largely contributed to the revision, in mid-2001, of NPPT along the lines of some TA recommendations (expanding the implementation time frame, revising quantitative targets in line with funding potential, achieving cost-efficiency gains by reviewing resource allocation norms). It thus had a lasting and sustainable impact on policy developments in the education sector.

Major Lessons Learned • As a result of the local constraints highlighted above, a substantial amount of the international consultants’ time

tended to be devoted to ‘non productive’ or organizational tasks, reducing the time available to carry out substantive work for which their expertise was required.

• Inputs of young local support staff (translators / interpreters, secretaries, research assistants) with good English skills, recruited on an ‘ad hoc’ basis by the consultants, have proven critical for the success of the TA. Their versatility and willingness to gain hands-on experience allow them to provide qualified professional services. In fact, the distinction between domestic consultants, identified by the consulting firms or recommended by the Government, and support staff in terms of actual functions and tasks performed often tended to disappear.

• Some simple recommendations can be made to improve the design of similar TAs in the country, including (i) ensuring understanding of the TA terms of reference through clarification of economic concepts often poorly comprehended, translated or interpreted; (ii) setting more realistic timeframes and phasing of activities, allowing sufficient time for translation of reports and in-depth review by national counterparts; (iii) agreeing with the executing agency, at an early stage, on mechanisms and procedures for accessing readily available information; (iv) seeking the assistance of the resident mission in screening domestic consultants; (v) allowing the flexible use of local bilingual personnel who are easily trainable and highly productive.

Recommendations and Follow-Up Actions • Follow-up actions are being discussed in the framework of the ESDP preparation. These include in-depth studies

on staff deployment (organizational audit of the education administration, school mapping exercise), further revisions of expansion plans of the senior secondary education (rationalization of boarding facilities), and pro-poor budget reallocations (targeting of scholarships at the tertiary level, supplementary transfers of resources to the poorest areas).

Prepared by

Marc Cohen

Designation

Project Economist, AEEH

Appendix 11 43

CONSULTING SERVICES Appraisal Actual Description Person

Months

Cost Person Months

Cost

A. International Consultants 1. Capacity Building in Curriculum Development 20.0 472.0 10.8 243.8 2. Improving the Pedagogical Quality of Textbooks 5.1 118.0 4.6 93.9 3. Improving the Use of Learning Materials 4.0 94.0 3.8 50.5 4. Improving the Quality and Efficiency of Textbook Production

3.0 71.0 10.1 224.1

5. Increasing the Supply of Textbooks and Learning Materials

0.8 18.0 1.9 40.2

6. Improving Textbook Durability 6.0 148.0 8.2 154.2 7. Project Implementation Unit 6.2 142.0 17.5 310.6 Subtotal A 45.1 1,063.0 56.9 1,117.3 B. Domestic Consultants 1. Capacity Building in Curriculum Development 24.0 36.0 14.8 10.0 2. Improving the Pedagogical Quality of Textbooks 8.0 12.0 2.2 1.5 3. Improving the Use of Learning Materials 25.0 38.0 10.6 7.3 4. Improving the Quality and Efficiency of Textbook Production

8.0 12.0 12.8 8.8

5. Improving Textbook Durability 1.0 2.0 0 0 6. Project Implementation Unit 5.0 8.0 63.6 37.9 Subtotal B 71.0 108.0 104.0 65.5 C. Out of Pocket Expenses 344.8 D. Contingencies 103.0 0.6 Total 116.10 1,274.0 160.9 1,528.20

44 Appendix 12

STAFF DEVELOPMENT

In Country External Training Topics of Training Appraisal Actual Topics of Training Appraisal Actual

Activities Duration No. of Duration No. of Activities Duration No. of Duration No. of Day Participants Day Participants Day Participants Day Participants

A. Capacity Building in Curriculum Development

1. Strategies in Curriculum development

21 150 75 151 1. Study tours in Curriculum Dev.

30 18 22 18

2. Integration of environmental education within other subjects

-

-

1

19

3. Writing of teacher’s guides on humanities

-

-

5

12

B. Improving the Pedagogical Quality of Textbooks

1. Authorship (3) 14 60 30 66 1. Assignment in Textbook Publishing

30 3 - -

2. Manuscript evaluation 14 20 10 31 3. Manuscript editing

14

20

10

15

4. Onscreen editing

4

15

5. Desktop publishing

14

15

8

35

6. Gender presentation in textbook

10

20

10

25

C. Improving the Use of Learning Materials

1. Student-centered Learning

28

50

14

49

1. Study tour in management/design of in- service training

30

15

21

17

2. Training of Trainers on Student- centered Learning

14

60

4

27

3. Training of in-service on classroom use of new textbooks

7

12,000

5

20,000

In Country External Training Topics of Training Appraisal Actual Topics of Training Appraisal Actual

Activities Duration No. of Duration No. of Activities Duration No. of Duration No. of Day Participants Day Participants Day Participants Day Participants

D. Improving the quality and Effectiveness of Textbooks

1. Textbook Production 2. Implementation of the Textbook Rental Scheme

7

-

42

-

6

1

19

17,482

1. Study tours in textbook production management 2. Assignment in textbook production management

30

30

15

2

11

-

18

-

3. Evaluation of bids for textbook procurement

- - 5 75

E. Improving Textbooks and Learning materials supply 1. Marketing 2. Manuscript Evaluation 3. Bid Evaluation for Textbooks

30 - -

20 - -

14 14 3

17 74

229

1. Assignment in publishing Management

30

3

-

-

F. Improving Textbook Durability 1. Textbook Production Methods 2. Textbook Production Specs 3. Production Management and Finance

4. Use of Computers in Planning and Management of Publishing 5. Textbook Program Budgeting 6. Book Repair 7. Revolving Funds Operations 8. Textbook Provision Policy

28

10 10

14

10 24 7 -

20

10 15

20

10 225 10

-

14

- -

-

- 20

- 5

30

- -

-

- 240

- 16

G. PIU

1. Study tours on project management

21

5

-

-

Appendix 11 45

46 Appendix 13

EQUIPMENT PROCUREMENT

Appraisal Actual Cost

($’000) Mode of

Procurement Cost

($’000) Mode of

Procurement A. Textbooks and Materials 1. Latin Script Textbooks for Selected Subjects, Grades 4-9 a. Publishing 2,877 LCB 13,848 ICB/LCB/DP b. Printing 8,633 LCB 795 ICB/LCB/DP 2. Teachers’ Guide for Selected Subjects, Grades 4-9 220 LCB 621 LCB/DP 3. Library Books 2,520 LCB 927 LCB 4. Instructional Aids Kits 750 LCB - - 5. Resource Materials, (Curriculum Materials, Reference Books for REC)

150 IS - -

B. Educational Equipment 1. VCR, Television Sets 27 IS/DP - - (In-Service Institute) 2. Overhead Projectors 15 IS/DP - - (In-Service Institute) 3. Book Repair Kits (districts) 50 IS/DP - - 4. Libraries of Eight Pilot Schools - - 66 IS C. Computer Printers, Scanners, Modems

1. Republican Education Center 37 IS/DP 34 IS 2. In-Service Institutes 191 IS/DP 240 IS 3. PIU 12 IS/DP 16 DP 4. 14 Regional and District - - 61 IS Departments of Education and 1 Central (TRS) D. Office Equipment 1. Republican Education Center 42 IS/DP 20 IS 2. In-Service Institute 210 IS 108 IS 3. PIU 16 IS/DP 54 DP

DP=direct purchase; ICB=international competitive bidding; IS=international shopping; LCB=local competitive bidding; PIU=project implementation unit

Appendix 14

47

PROJECT PERFORMANCE RATING ASSESSMENTa

Criterion

(a) Weight

(b) Assessment

(c) Rating Value

(d) Weighted Ratio

(b x d)

A. Project Outcome Assessment

1. Relevance 20% Highly relevant 3.0 0.60

2. Efficacy 25% Efficacious 2.0 0.50

3. Efficiency 20% Efficient 2.0 0.40 B. Sustainability 20% Most likely 3.0 0.60

C. Institutional Development and Other Impacts

15% Significant 2.0

0.30

Overall Rating Highly

Successful

2.40

a Rating follows ADB Guidelines for the preparation of Project Performance Audit Reports (September 2000).