document of the world...

Document of The World Bank

Report No:ICR000038

IMPLEMENTATION COMPLETION AND RESULTS REPORT ( IBRD-70010, JPN-26137, SIDA-20307 )

ON A

LOAN IN THE AMOUNT OF US$21.86 MILLION

TO THE

REPUBLIC OF LITHUANIA

FOR A

HEALTH PROJECT

June 28, 2007

Human Development Sector Unit Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 28, 2007)

Currency Unit = Lithuanian Lita (LTL) LTL 1.00 = US$ 0.39 US$ 1.00 = LTL 2.56

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

ALOS Average Length of Stay M&E Monitoring and Evaluation BBP Basic Benefit Package MOH Ministry of Health BOR Bed Occupancy Rate NPV Net Present Value CAS Country Assistance Strategy PAD Project Appraisal Document CHIF Compulsory Health Insurance Fund PCU Project Implementation Unit EMS Emergency Medical Services PDO Project Development Objective EU European Union PEH Public Expenditure on Health FRR Financial Rate of Return PIA Project Implementation Agreement FSU Former Soviet Union PHC Primary Health Care GDP Gross Domestic Product PPP Public-Private Partnership GOL Government of Lithuania PSR Project Status Report GP General Practitioner PTL Program Team Leader HMIS Health Management Information System QAG Quality Assurance Group HR Human Resources SIDA Swedish International Development

Agency HSR Health Service Restructuring SPF State Patient Fund ICR Implementation Completion and Results

Report TA Technical Assistance

IRR Internal Rate of Return TOR Terms of Reference ISR Implementation Status and Results

Report TPF Territorial Patient Fund

LHIC Lithuanian Health Information Center TTL Task Team Leader LHP Lithuania Health Project WB World Bank MTR Mid-Term Review WHO World Health Organization MOF Ministry of Finance

Vice President: Shigeo Katsu Acting Country Director: Suman Mehra

Sector Manager: Armin H. Fidler Project Team Leader: Pia Helene Schneider

ICR Team Leader: Pia Helene Schneider ICR Primary Author: Panagiota Panopoulou

COUNTRY Project Name

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design............................................... 12. Key Factors Affecting Implementation and Outcomes .............................................. 63. Assessment of Outcomes .......................................................................................... 114. Assessment of Risk to Development Outcome......................................................... 225. Assessment of Bank and Borrower Performance ..................................................... 236. Lessons Learned ....................................................................................................... 257. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 25Annex 1. Project Costs and Financing.......................................................................... 27Annex 2. Outputs by Component ................................................................................. 29Annex 3. Economic and Financial Analysis................................................................. 31Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 33Annex 5. Beneficiary Survey Results ........................................................................... 35Annex 6. Stakeholder Workshop Report and Results................................................... 37Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR..................... 38Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders....................... 42Annex 9. List of Supporting Documents ...................................................................... 43

MAP

i

A. Basic Information Country: Lithuania Project Name: Health Project

Project ID: P035780 L/C/TF Number(s): IBRD-70010,JPN-26137,SIDA-20307

ICR Date: 06/29/2007 ICR Type: Core ICR

Lending Instrument: SIL Borrower: REPUBLIC OF LITHUANIA

Original Total Commitment:

USD 21.2M Disbursed Amount: USD 19.5M

Environmental Category: C Implementing Agencies: Ministry of Health Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 04/17/1996 Effectiveness: 05/17/2000 05/17/2000 Appraisal: 03/21/1999 Restructuring(s): Approval: 11/30/1999 Mid-term Review: 06/03/2002 Closing: 09/30/2004 09/30/2006 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project No Quality at Entry None

ii

at any time (Yes/No): (QEA): Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

Moderately Unsatisfactory

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 12 12 Health 88 88

Theme Code (Primary/Secondary) Health system performance Primary Primary Injuries and non-communicable diseases Secondary Secondary Other communicable diseases Secondary Secondary Participation and civic engagement Secondary Secondary E. Bank Staff

Positions At ICR At Approval Vice President: Shigeo Katsu Johannes F. Linn Country Director: Suman Mehra Basil G. Kavalsky Sector Manager: Armin H. Fidler Annette Dixon Project Team Leader: Pia Helene Schneider Toomas Palu ICR Team Leader: Pia Helene Schneider ICR Primary Author: Panagiota Panopoulou F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project’s development objective was to improve the quality, efficiency, equity and access of the Lithuania health care system. Key performance indicators measured: (i) improved equity of resource allocation among health regions (apskritis), (ii) cost-containment through effective contracting between the State Patient Funds and health care providers; (iii) efficiency gains through hospital services consolidation and restructuring in four pilot regions; and (iv) improved access to General Practitioner services in four pilot regions. Revised Project Development Objectives (as approved by original approving authority) The project development objective did not change during the project life.

iii

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Standard health service efficiency indicators improve yearly over the life of the project.

Value quantitative or Qualitative)

ALOS (national level): 7.8 days; BOR (national level): 25.1%.

Decrease in ALOS; increase in BOR.

ALOS (national level): 6.7 days; BOR (national level): 29%.

Date achieved 12/31/2000 09/30/2004 09/30/2006 Comments (incl. % achievement)

Indicator 2 : 90% of health care providers stay within year-end, predefined fixed price-volume budgets.


No data is available.

Increase in the number of health care providers that stay within year-end, predefined fixed price-volume budgets.

92.5%


Indicator 3 : 50% of population in project areas is covered by certified GPs providing comprehensive services by end of project.


24.7%. 50%. 61.8%.


Indicator 4 : Referrals and self-referrals to ambulatory care specialists and hospitals are reduced by 20% in pilot areas by end of project.


Ambulatory specialist visits per capita Alytus: 1.34; Kaunas: 2.84; Utena: 1.48; Vilnius: 2.85. Hospital admissions per

20% decrease.

Ambulatory specialist visits per capita Alytus: 1.5 (11.9%); Kaunas: 3.2 (12.7%); Utena: 1.34 (-

iv

1,000 Alytus: 176.0; Kaunas: 262.9; Utena: 175.3; Vilnius: 250.4.

9.5%); Vilnius: 2.64 (-7.4%). Hospital admissions per 1,000 Alytus: 181.1 (2.9%); Kaunas: 285.6 (8.6%); Utena: 174.1 (-0.7%); Vilnius: 257 (2.6%).


There is no data on referral and self-referrals to ambulatory care specialists and hospitals. For the evaluation of this indicator ambulatory specialist visits per capita and admissions to hospital inpatient care per 1,000 population are used.

Indicator 5 : Patient satisfaction with the services they get from their primary care physicians is improved in pilot areas by end of project.


Project group Polyclinics: 3.73; Ambulatories: 4.01; All: 3.82.

Increase in population satisfaction.

Project group Polyclinics: 3.67; Ambulatories: 4.39;All: 3.85.


Respondents were asked to qualify services using a scale from 1 (absolutely dissatisfied) to 5 (very satisfied).

Indicator 6 : Policy framework for health service planning and restructuring in place and used by health administrators by end of project.


No policy framework in place.

Policy framewokr in place and used by health administrators.

The Strategy for the Restructuring of Health Care Institutions was approved on 03/18/2003 by GOL Resolution No. 335. The second stage of the Strategy was approved on 06/29/2006 by the GOL Resolution No. 647.

Date achieved 12/31/2000 09/30/2004 09/30/2006 Comments (incl. %

v

achievement)

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : 80% of funds allocated to regions according to population and needs-based formula by mid-term evaluation (June 2002) and 100% of health funds by end of project.

Value (quantitative or Qualitative)

28%. 80% by June 2002;100% by end of project.

54% in June 2003. 75.5% by end of project.


The delay in the adoption of the revised formula was mainly the result of the reorganization of the Territorial Patient Funds (which were reduced from 10 in 2002 to 5 in 2003).

Indicator 2 : A revised hospital reimbursement schedule based on standard costing study is in place by 2003.


Old hospital reimbursement schedule is in place.

Revised hospital reimbursement schedule based on a costing study is in place by 2003.

A revised reimbursement schedule was introduced in February 2003 but it was not based on a national costing exercise and is expected to be revised again in the near future.


Indicator 3 : Government guidelines for appraisal, allocation, monitoring and financing of health sector investments are developed and in use by Year 3 of project.


No government guidelines exist.

Government guidelines are developed and in use by Year 3 of project.

Government guidelines were approved on 02/21/2002 by MOH Order No. 91 and are still in use.


Indicator 4 : National and regional needs-based health service plans are developed in at least 7 counties (out of 10).

vi


No needs-based health service plans.

Needs-based health service plans are developed in at least 7 counties.

Although no separate needs-based health service plans were developed at the county level, the ‘Strategy for the Restructuring of Health Care Institutions’ was implicitly based on a needs assessment and service planning.


Indicator 5 : Numbers of health institution managers are trained in management


No managers are trained. Increase in number of trained managers.

There were no project activities related to this indicator.


Indicator 6 : Ambulance service review report is developed and disseminated by 2001.


No ambulance service review report.

Ambulance service review report is developed and disseminated by 2001.

Ambulance service development plans were prepared and adopted in 2003 in Kaunas and Utena counties. No national ambulance service review report prepared by the GOL exists.


Indicator 7 : National Health Report is published regularly.


No National Health Report is published.

National Health Report is published regularly.

National Health Report is published yearly by the LHIC.

Date achieved 12/31/2000 09/30/2004 09/30/2004 Comments

vii

(incl. % achievement)

Indicator 8 : 55% of the Alytus County population is covered by qualified GPs by end of project.


27%. 55%. 62%.


Indicator 9 : Total number of hospital beds in the Alytus County Hospital is reduced by 20%. Value (quantitative or Qualitative)

583 beds. 20% decrease. 431 beds (26% decrease).


Indicator 10 : Not less than 40% of all operations are performed in the day surgery of the Alytus County Hospital.


No day surgery center.

Not less than 40% of all operations are performed in the day surgery center.

31%.


The lower level of day surgery operations was to due to the fact that the center started functioning in 2005.

Indicator 11 : Average length of stay in the Alytus County Hospital is reduced to 9 days. Value (quantitative or Qualitative)

9 days. 9 days. 6.9 days.


Indicator 12 : 55% of the Kaunas County population is covered by qualified GPs by end of project.


14.5%. 55%. 76.6%.


Indicator 13 : The number of surgery beds in Kaunas Clinical Hospitals No.2 and No. 3 is

viii

reduced by 30%.


Kaunas Clinical Hospital No. 2: 150 beds. Kaunas Clinical Hospital No. 3: 140 beds

30% reduction.

Kaunas Clinical Hospital No. 2: 79 beds (38% reduction). Kaunas Clinical Hospital No. 3: 87 beds (35% reduction).


Indicator 14 : Not less than 4,000 operations (each) are performed in the day surgery centers of Kaunas Clinical Hospitals No. 2 and 3.


No day surgery centers.

Not less than 4,000 operations (each) in day surgery centers of Kaunas Clinical Hospitals No. 2 and 3.

1,391 operations in 2006 in both centers.


The lower number of operations is due to the fact that day surgery centers started operating in Kaunas Clinical Hospitals No. 2 and No. 3 in March 2005.

Indicator 15 : 20% reduction in the number of referrals to specialists in Utena county. Value (quantitative or Qualitative)

5 visits per capita. 20% reduction. 5.6 visits per capita (11% increase).


The increase in the no. of specialist visits appears to be closely related to the decrease in hospitalizations for the same period (substitution effect). In addition, there was an increase of 15.5% in the no. of ambulatory non-specialist visits.

Indicator 16 : 20% reduction in the number of direct arrangements to see specialists in Utena county.


Date achieved Comments (incl. % achievement)

This indicator cannot be assessed due to lack of data.

Indicator 17 : 30% reduction in the number of emergency calls in Utena County. Value (quantitative or Qualitative)

31,008 calls per year. 30% reduction. 47,126 calls per year (51% increase).

ix


This increase was mainly due to calls for transportation of pregnant women after the closing of the obstetric departments in Ignalina, Moletai and Zarasai hospitals.

Indicator 18 : 5% reduction in the number of hospitalization cases in Utena county. Value (quantitative or Qualitative)

32,000. 5% reduction. 30,000 (6.25% reduction).


Indicator 19 : Provision of general surgical, gynecological and pediatric services is introduced without increasing total number of beds in Visaginas Town Hospital (Utena county).


180 births per year. Increase in the number of services provided.

338 births per year (88% increase).


Indicator 20 : Regular monitoring of progress towards achievement of key performance indicators and agreed development objectives.


No monitoring in place. Regular monitoring in place.

Incomplete monitoring of key performance indicators.


Monitoring of key performance indicators took place ex post with the assistance of the PCU.

Indicator 21 : Timely contracting of goods, works, and services with quality outputs.


Beginning of contracting of goods, works, and services with quality outputs.

Timely contracting of goods, works, and services with quality outputs.

Timely contracting of goods, works, and services with quality outputs.


Indicator 22 : Efficiently managed PCU, with adequate staff and resources.


The PCU is created including: Director, Administrator, Accountant (financed by the MoH), Project Implementation Officer,

Efficiently managed PCU, with adequate staff and resources.

Efficiently managed PCU, with adequate staff and resources.

x

Procurement Specialist and Chief Financial Officer (financed by SIDA grant). PMU equipped from the WB Loan budget.


After the closing of SIDA grant (March 30, 2005) the PCU staff was reduced to a minimum (Director, Administrator and Accountant) and finance by the MOH.

Indicator 23 : PCU promotes an effective dialogue among key project actors and stakeholders, in particular in the four pilot regions.


Beginning of PCU operation.

PCU promotes an effective dialogue among key project actors and stakeholders, in particular in the four pilot counties.

PCU promotes an effective dialogue among key project actors and stakeholders, in particular in the four pilot counties.


Indicator 24 : Satisfactory project accounting systems and annual audits.



Satisfactory project accounting systems and annual audits.

The accounting system “HANSA Financial” was used during project implementation. Audits are performed yearly and are unqualified.


Indicator 25 : Annual PCU staff performance evaluations and training programs.



Annual PCU staff performance evaluations and training programs.

Project Steering Committee evaluates PCU staff performance annually. PCU key staff undertakes classes in international courses.

Date achieved 12/31/2000 09/30/2004 09/30/2006 Comments

xi

(incl. % achievement)

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 12/27/1999 Satisfactory Satisfactory 0.00 2 06/04/2000 Satisfactory Satisfactory 0.21 3 12/22/2000 Satisfactory Satisfactory 0.91 4 06/25/2001 Satisfactory Satisfactory 0.91 5 12/12/2001 Satisfactory Satisfactory 0.91 6 06/27/2002 Satisfactory Satisfactory 2.22 7 12/27/2002 Satisfactory Satisfactory 4.67 8 06/25/2003 Satisfactory Satisfactory 9.47 9 12/04/2003 Satisfactory Satisfactory 9.94

10 06/28/2004 Satisfactory Satisfactory 13.10 11 12/22/2004 Satisfactory Satisfactory 17.04 12 05/04/2005 Satisfactory Satisfactory 18.93 13 12/23/2005 Satisfactory Satisfactory 19.16 14 07/27/2006 Satisfactory Satisfactory 19.48

H. Restructuring (if any) Not Applicable

xii

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative)

1.1 Context at Appraisal (brief summary of country and sector background, rationale for Bank assistance) Country macroeconomic background Lithuania had made good progress in macroeconomic stabilization and the transition to a market economy during the 1990s. The consolidated fiscal deficit had fallen from 4.4% of GDP in 1996 to 1.7% in 1997, while the currency board-backed exchange rate remained fixed since April 1994. In 1997, accelerated privatization supported the growth of Foreign Direct Investment to record levels, while most other structural reforms proceeded apace. As a result, the GDP growth rate increased from 3.3% in 1995 to a 7.3% in 1998. The main objectives of the Country Assistance Strategy (CAS) for Lithuania for 2000-2002 were to: (i) help improve macroeconomic and financial stability; (ii) support the reform agenda and investments needed for EU accession; and (iii) support reforms, institution building and investments in social assistance and the health and education sectors. Country and sector background In the late 1990s, Lithuania faced problems in the health sector similar to other CEE/FSU countries. Aggregate health indicators had deteriorated compared to the pre-transition period (pre-1991), leaving Lithuania with a significant gap in health status in comparison with the EU. The health system did not proactively address the root causes of ill health because of lack of effective public health policies and programs. The health system was also not able to cope with the increased burden of ill health because of an inefficient health care delivery and health financing system. Declining public funding for health services and poor maintenance of investments in health care infrastructure exacerbated the situation. In 1995, Lithuania adopted a Primary Health Care Reform Strategy to restructure its health sector and adjust to changing socio-economic, epidemiologic, and demographic circumstances. There was a lack of effective public health policies and programs to address an emerging wave of non-communicable diseases. Historically oriented towards infectious diseases and environmental health, the public health system had little capacity and leadership to comprehensively address non-communicable diseases such as circulatory diseases, external injuries and malignant tumors that were the most frequent mortality causes. This situation was further aggravated by deteriorating socio-economic conditions during transition and life-style related risk factors. The health care delivery system was characterized by inefficient primary, secondary and tertiary level institutions, and work force. The system had an excessive number of poorly organized and low quality hospitals; absence of first level and family care services; and an inadequate skill mix in the work force. As a result of the imbalance between primary, and secondary and tertiary health institutions, there was an over-reliance on inpatient treatment. At the same time, there was a need for less acute care beds and more nursing and support beds where frail elderly and chronically ill individuals could be better taken care of at lower cost within the framework of community-based health services. Human resources, especially physicians, were in excess compared to the EU and Lithuania’s Baltic neighbors. There were also significant regional imbalances ranging from 1.3 physicians per 1,000 population in some predominantly rural counties to 6.2 in big cities.

2

Although Lithuania had taken considerable steps in reforming its health financing system towards separation of purchasing (through the State Patient Fund, SPF) and provision of services, it still faced challenges posed by a supply driven resource allocation formula, inefficient management of investment resources, inefficient purchasing practices, and payment systems that encouraged over-referral and hospital admission. Patient fund allocations were determined by the level of services provided by secondary and tertiary facilities within the territorial boundaries, which in turn resulted in large cross-territorial variations in per capita allocation. Investment decisions were taken by the Ministry of Health (MOH) and the municipal authorities without SPF participation. Service prices did not incorporate the cost of buildings and equipment. Consequently, investment decisions were encouraging ineffective resource use as health care institutions were receiving investment funding on the basis of criteria other than the number of services rendered. Territorial Patient Funds (TPF) distributed their budget on the basis of historical patterns and not actual needs, while the combination of capitation for primary care and activity-based payments for secondary care created incentives for under-utilization of primary health care (PHC) services and high level of referrals to specialists and the hospital sector. The health care benefit package was unaffordable. Officially all services were covered by the SPF unless there was a specific exclusion. The remaining benefit package was still too extensive to be sustained by country resources and rationing occurred in a number of different implicit ways. There was a need to strengthen institutional capacity and better define the institutional framework for public accountability. The MOH and SPF were the key institutions implementing health reforms. During project preparation, many stakeholders viewed the lack of capacity in the areas of reform planning, implementation and monitoring as one of the main reform obstacles. The MOH had important strengths that it could build upon, but also needed to strengthen institutional capacity for program management, monitoring and evaluation (M&E), communication, and needs and technology assessment. The change of health care facilities from budget organizations to public institutions in 1997 led to substantial autonomy and flexibility for these institutions, but it did not provide clear rules for public accountability. In reality, hospital owners (central government, health regions, and municipalities) played a passive role in oversight, leaving the task of financial monitoring largely to the SPF/TPFs, although the latter did not have the necessary power to address poor business hospital performance. Rationale for Bank Assistance The Lithuania Health Project supported the FY00-02 CAS objectives to “design cost-effective, financially viable social safety net and human development programs” and to help reorient public services and infrastructure in order to provide adequate and cost-effective social services. In the health sector, this translated into reorientation of medical services towards a general practice-based primary health care system, optimization and improvement in quality of hospital services, and introduction of appropriate incentives and efficient management into the health financing system. Support to the Government’s reform health program. The project supported the Government’s health reform policy agenda at a time that Lithuania was in need of external donor support. The project was financed by a World Bank Loan, a Grant from the Swedish International Development Agency (SIDA) and a Grant from the Government of Japan. Competitive selection of pilot counties based on the quality of proposals and consistency with the national health care

3

reforms was perceived to contribute to successful project implementation. Tangible positive results from the first successful activities were believed to facilitate nation-wide implementation of health sector reforms.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The project development objective was to improve the quality, efficiency, equity and access of the Lithuania health care system. Key performance indicators measured: (i) improved equity of resource allocation among health regions (apskritis), (ii) cost-containment through effective contracting between the SPFs and health care providers; (iii) efficiency gains through hospital services consolidation and restructuring in four pilot regions; and (iv) improved access to General Practitioner services in four pilot regions.1 Key Indicators The Lithuania Health Project (LHP) had the following sector related indicators from the CAS:

Improved efficiency of the health care system; Improved population satisfaction with national health services;

and the following indicators for PDO:

Standard health services efficiency indicators improve yearly over the life of the project; 90% of health care providers stay within year-end predefined fixed price-volume

budgets; 50% of population in project areas is covered by certified General Practitioners (GPs)

providing comprehensive services by end of project; Referrals and self-referrals to ambulatory care specialists and hospitals beds are reduced

by 20% in pilot areas by end of project; Patient satisfaction with the services they get from their primary care physicians is

improved in pilot areas by end of project; Policy framework for health service planning and restructuring in place, and used by

health administrators by end of project. In addition, the LHP had 25 indicators related to component outputs/outcomes. For a detailed list of the component-related indicators see the Results Framework Analysis.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The project development objective and key indicators did not change during the project life.

1.4 Main Beneficiaries, (original and revised, briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project) The PAD identified seven groups of project beneficiaries:

1 For the purposes of this ICR we will refer to the 10 apskritis of Lithuania as counties. The term region will be used when referring to the five regions of the TPFs.

4

(i) Society at large benefited from improved efficiency of public expenditure on health, better standards of care; and improved equity as a result of a needs-based resource allocation;

(ii) Population in pilot counties benefited from better access to care; better quality of care; reduction in urban-rural inequities; increased community participation in health and health care issues; and community nursing services (the latter mainly benefited the elderly and the chronically ill);

(iii) The Government benefited from improved decision making capacity through institutional and process improvements, and greater data availability from an improved information management; and better communication with the population and system stakeholders;

(iv) SPF and TPFs benefited from strengthened institutional capacity; improved job satisfaction through skill development activities and clear job descriptions; automation of data gathering, aggregation, and analysis function supporting statistics, policy development, and resource allocation; and reduction of fraud in the health insurance system;

(v) Health policy decision makers and opinion leaders, the Government, the Patient Funds, the Parliament and influential stakeholders benefited from improved information about reform processes; improved transparency of health care funding that would allow for better planning and accountability; strategic planning for the improvement of the health of the population through policy and other broad-based approaches; and improved detailed and aggregated data on health from the Health Management Information System (HMIS);

(vi) Health administration managers benefited from better management skills; (vii) GPs benefited from improved skills and incentives for good performance; improved

professional prestige; better control and flexibility over professional lives; and improved working conditions.

1.5 Original Components (as approved) The LHP had three components: Component A. Support to Health Reform (estimated total cost US$8.46 million). This component consisted of three sub-components, namely: A.1. Policy Development, A.2. Strengthening Capacity of National Health Institutions, and A.3. Information Management. Each sub-component was further divided into sub-components. Sub-component A1. Policy Development (estimated total cost US$1.16 million). This sub-component aimed at supporting the development of a regional resource allocation formula, a mechanism for the allocation of investment funding, provider reimbursement and contracting mechanisms, needs assessment and service planning, a basic package of services and clinical protocols, and health care service restructuring. Sub-component A1 included six sub-components. Sub-component A2. Strengthening Capacity of National Health Institutions (estimated total cost US$1.47 million). The objective of this sub-component was to strengthen institutional capacity to implement policy reforms. Institutional strengthening included staff skill development, provision of information, tools, and methodologies, and building capacity to effectively communicate with the consumers of the health care system and the main stakeholders. The beneficiaries of the component activities were the Medical Library, the Lithuanian Health Information Centre, the MOH, Patient Funds, and the National Health Board. Sub-component A2 included five sub-components.

5

Sub-component A3. Information Management (estimated total cost US$5.83 million). This sub-component aimed at strengthening the Lithuanian Health Information Center (LHIC) and developing a hospital and PHC information system. The Health Information System envisaged the development of an open-ended information system in a modular fashion that would be a flexible and cost-effective tool for patient management, administration and documentation. The sub-component also supported MIS development in four pilot counties and was linked to the facilities supported through Component B. Sub-component A3 included two sub-components. Component B: Health Services Restructuring (estimated total cost US$24.13 million). This component was designed to support health service restructuring in four pilot counties: Alytus, Kaunas, Utena and Vilnius. Lithuania has 10 counties in total. The pilot counties were selected on a competitive basis out of nine applications. Following selection, the four selected counties developed detailed regional health sector restructuring and development programs supported by international technical assistance. Sub-component B1. Alytus Pilot Project (estimated total cost US$5.26 million) included a Regional PHC Development Program and a Hospital Restructuring Program. Sub-component B2. Kaunas Pilot Project (estimated total cost US$7.01 million) included a Health Promotion and Primary Prevention Program, a Regional PHC Development Program, a Hospital Restructuring Program, a Community Mental Health Service Program, and an Ambulance and Emergency Services Development Program. Sub-component B3. Utena Pilot Project (estimated total cost US$4.38 million) included a Regional PHC Development Program, a Hospital Restructuring Program, and an Ambulance Service Development Program. Sub-component B4. Vilnius Pilot Project (estimated total cost US$7.48 million) included a Vilnius Apskrtitis PHC Service Restructuring Program and a Hospital Restructuring Program. Component C: Project Management (estimated total cost US$1.35 million). This component supported the operation of a Project Coordination Unit (PCU), staffed by project management and technical staff in the MOH. To oversee the restructuring of health services in the four pilot counties and facilitate implementation at the local level, a core team of at least two staff was placed at each of the four apskritis participating in the project.

1.6 Revised Components Component A. Sub-component A2. There was no substantial restructuring of this component during project implementation. However, the activities of sub-components A2.2. Management and Development Adviser to the MOH and A2.3. Management Training to be financed under the EU/PHARE project were implemented earlier and separate from the LHP activities because of delays in the negotiation of the World Bank (WB) loan. The implementation of these sub-components was primarily the responsibility of the Division of Foreign Affairs of the MOH. The LHP PCU did not participate in their implementation and therefore did not have any information on these activities. Efforts were made to obtain relevant information from the Division of Foreign Affairs of the MOH but because of changes in the Ministry’s personnel, the individual(s) responsible for the EU/PHARE project could not be located and interviewed and information could not be retrieved.

6

Due to changes in the activities financed under Sub-component A2 and a change in the exchange rate between the Lithunian Lita (LTL) and the US dollar (US$) which left activities under Sub-component A1 under-funded, Sub-component A2 funds were re-allocated to Sub-component A1 activities. During the re-allocation process, the Government decided to use funds under sub-component A2.5 for study tours for the National Health Board to finance activities under sub-component A1. Consequently, activities under sub-component A2.5 were cancelled.

1.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) Extensions of project closing date. The original closing date of the LHP was September 30, 2004. The closing date was extended twice and the project closed on September 30, 2006. The Government requested the first extension of the project's closing date (from October 1, 2004 to March 30, 2006) in order to: (i) complete activities under sub-component A3.2. Development of Hospital and Primary Health Care Information System or HMIS; and (ii) finalize payments for the day-surgery centers’ equipment, ambulances for the Emergency Medical Services (EMS) and civil works in the Vilnius Railway Hospital.

Following the first extension, the second extension from April 1, 2006 to September 30, 2006 was granted to finalize activities under the HMIS. The six-month extension request was based on the understanding between the Bank and the Borrower that a minimum of nine months was needed to ensure effective procurement and implementation of the HMIS.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) Quality at entry is rated moderately satisfactory by this ICR. The project was based on a sound and detailed background analysis. The project preparation period lasted for more than two years, from April 1996 (Concept Review) to November 1999 (Board Approval). This was due to a number of reasons including, the complexity of the proposed reform activities, both at the level of policy development and health service restructuring (HSR); the involvement of pilot areas with varying health sector characteristics and thus different needs; and structural changes in the MOH (during preparation, the Ministry leadership changed three times). To address the complexity of health services restructuring, pilot counties were selected on a competitive basis. Subsequently, the four selected counties developed detailed regional health sector restructuring and development programs supported by international technical assistance. The demand driven approach for competitive selection of pilot regions strengthened the project’s local ownership and assisted the process of addressing the complex political context of health sector restructuring. The Project Appraisal Document (PAD) highlighted a number of lessons learned from other projects in the region: for example, health sector reforms in the ECA region were a lengthy and politicized process; expectations from the reform process had been too optimistic for both the World Bank and the client countries; and projects per se were complex.

7

Unfortunately, the Bank team did not apply all these lessons when designing the LHP. The LHP was a complex project with three components and a total of 17 sub-components; 13 sub-components under Component A and 4 county sub-components under Component B; county sub-components were further divided in programs/other sub-components. In the case of Component A, the area of Policy Development aimed to address issues of resource allocation, investment funding, provider reimbursement and contracting, needs assessment and service planning, the basic package of services, and health sector restructuring. These issues are all inter-related and it could be argued that they have to be addressed simultaneously in order for a reform process to be comprehensive and effective. However, their inclusion under one project umbrella in an environment of weak institutional capacity, frequent political changes and a relatively low level of political commitment posed serious risks for project implementation. As is argued in Section 3. Assessment of Outcomes, while the majority of these issues were put on the table and discussed by the sector’s stakeholders, few political decisions were eventually taken in these areas. The latter was probably due to (i) the high political cost that these decisions involved, and (ii) the need for stronger institutional capacity in the MOH. A smaller number of sub-components could have benefited project implementation, and M&E. This is particularly true in the case of Component A that consisted of three sub-components and where each sub-component was further divided to multiple sub-components. In retrospect, the project could have followed a more modest approach and focused on a smaller number of policy issues. This would have allowed for a faster and easier implementation process –as discussed in section 2.2, the project faced various problems during the first years of implementation. At the same time, it would have allowed for the M&E arrangements to focus on a smaller number of key performance indicators, which under the present design reached approximately 30 indicators (see also discussion in section 2.3).

2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) Up to Mid-Term Review (MTR), which took place in June 2002, the implementation process of both Components A and B was slow. As far as Component A was concerned, few political decisions were taken, although working groups had been formed and the issues had been discussed in multiple fora and with the participation of main stakeholders. Project activities on the HMIS started only two years after project effectiveness. There were various reasons for this delay: i) there was lack of clarity in the Government and the health sector in general regarding the content of the HMIS, ii) non-technical issues such as issues of confidentiality and legal needed to be addressed in advance of any project activities, iii) there were various institutions that demanded the control of primary data and iv) there was lack of institutional capacity to fully develop the HMIS. Regarding Component B, during the first year of implementation there were considerable delays in signing the implementation agreements between the MOH, and the pilot counties and municipalities due to local elections which took place during that year. Eventually, three out of four agreements were signed by the end of 2000 (Alytus, Utena, and Vilnius), while it took until March 2001 for the Kaunas implementation agreement to be signed. The length of time and difficulty in the signing of the agreements was also due to the fact that the restructuring programs in some pilot counties were challenging, involving the merging of hospital facilities and the closing of hospital departments. Project implementation was affected by frequent changes in the MOH leadership and in the composition of the Bank team. During the project life (from May 2000 when it became effective until September 2006 when it closed) there were six Ministers of Health. On average, a

8

minister’s tenure was less than 18 months. The frequent changes of the MOH leadership caused delays in project implementation, especially in Component A, as each incoming Minister needed time to get acquainted with the project and provide support and leadership to the program. The existence of a Project Steering Committee, which was created during preparation and tasked to offer policy guidance and donor coordination, might have assisted in faster project implementation despite ministerial changes. However, the Chair of the Steering Committee appeared reluctant to cooperate with World Bank (WB) colleagues during the first three years of the Project period, thus posing additional difficulties for project implementation. At the same time, the Bank team also experienced frequent changes of task team leaders (TTL) and program team leaders (PTL). In total, there were three TTLs and four PTLs during the project life. Continuity in the relationships between the MOH and the Bank was affected by frequent changes of team members from both sides. A well-performing Project Coordination Unit (PCU) had a highly positive impact in project implementation and outcomes. The LPH benefited from a strong PCU, both in the MOH and the pilot counties. The PCU worked in a professional and dedicated manner in order to guarantee smooth implementation of a complex project in an unstable political environment. This had a highly positive impact on project implementation and it was acknowledged, unanimously, by Bank staff and project beneficiaries (e.g., health care providers involved in the project, employees of national health institutions, etc.) in project documents and interviews carried out during the preparation of this ICR. Activities under sub-components A2.2 Management and Development Adviser to the MOH and A2.3. Management Training were financed by the EU/PHARE project and supervised by the Division of Foreign Affairs of the MOH before the LHP became effective (see Section 1.6). Activities under sub-component A2.5 were cancelled as the Government decided to re-allocate funds from this sub-component to the financing of activities under subcomponent A1. A Quality Assessment Group (QAG) assessment carried out in October 2002 rated the quality of supervision as marginal (overall rating 3).2 The assessment stated that the project had an overly ambitious policy and institutional reform agenda in a local context characterized by continuous changes of sectoral authorities and insufficient buy-in of reforms by the national government. As a result, the project got off to a very slow start with serious implementation problems in the area of support to health reform (Component A). The QAG panel recognized the commitment and steady efforts of the Bank team to supervise a project with design problems from the outset causing slow and difficult implementation and commented that the management could have made stronger efforts to find solutions and provide strategic guidance to the team. The QAG panel observed that while the diagnosis of the problems was satisfactory, it was not translated into agreement with the Government on revising and possibly restructuring the scope and content of the project. The assessment concluded that the task team should revisit the MTR findings and move from problem diagnosis to discussion and agreement with national authorities on concrete remedial measures, including project restructuring. The panel also advocated the

2 The assessment rating for QAG is: 1=Highly Satisfactory, 2=Satisfactory, 3=Marginal, 4=Unsatisfactory, NA=Not Applicable. The ratings available in the ICR Portal (Section C in the Datasheet) are not compatible with QAG's ratings. In particular, they do not have an option for a marginal rating. Therefore, the rating for quality of supervision in Section C.3 of the Datasheet is different from QAG's rating shown in the main text.

9

active involvement of the country unit that could help raise project related issues to the appropriate senior levels of Government as part of a broader country portfolio review.

10

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

The M&E design identified an ambitious set of indicators to monitor progress towards the PDO and component outcomes. As discussed in section 1.2, the project had approximately 30 indicators for measuring progress against CAS related goals, the PDO and project components. In the case of Component B, the Results Framework Analysis stated that individual targets for each project apskritis were to be defined in Project Implementation Agreements (PIAs) and it made reference to examples of indicators that could be included in these agreements (e.g., reductions in hospital beds, increases in population coverage by certified GPs, etc.). The PAD also identified the collection methods for these indicators which included routine project monitoring by the PCU and the Bank team, reports of sectoral authorities such as the MOH, the SPF and regional authorities, and population surveys. The latter were expected to provide data on population satisfaction with PHC health services before and after implementation of project activities in the pilot counties using two types of population groups; individuals who visited PHC facilities that participated in the project (project group) and individuals who visited PHC facilities that did not participate in the project (control group).

The implementation of M&E arrangements was not done in a systematic way during project life. During project implementation, data on key performance indicators were collected in a sporadic way and not for all indicators. Furthermore, there was an absence of documentation of the M&E process in Project Status Reports (PSR) and Implementation Status and Results (ISR) reports. In interviews carried out during the preparation of this ICR, Bank team members and PCU staff mentioned that efforts were made to gather an additional/alternative set of national and regional data with the assistance of a Bank consultant (including data on number of hospital admissions, number of inpatient days, number of hospital beds, utilization of ambulatory services and utilization of ambulance and emergency services). Unfortunately, these efforts were not completed and the above mentioned data were not collected. As a result, when the project closed there were no baseline data or other type of data against which the project could be evaluated. The only exception was the baseline and evaluation surveys on patient satisfaction that were carried out in 2001 and 2005, respectively. During ICR preparation, data on key performance indicators were collected. In November 2006, the ICR author requested the collection of relevant project data ex post. An agreement was reached between the MOH (PCU) and the SPF that data in their disposal would be provided to the ICR author for the years 2000 (baseline), 2002 (MTR) and 2006 (actual). At the same time, the LHIC completed the data put together by the M&E Bank consultant for the period 2000-2005. A complete set of data was provided to the ICR author by March 2007. The fact that the Lithuanian authorities were able to provide detailed data ex post is evidence of their M&E institutional capacity. However, it is not clear why this effort was not made earlier in the project and if it was made, why it was not documented accordingly. One possible reason could be that the large number of key performance indicators had an adverse effect on the M&E process. The PCU members who were primarily responsible for M&E stated that they did provide data to the Bank mission sporadically. From the Bank side, the need for data collection for M&E purposes was raised by the Bank team in aide-memoires as well as in management’s comments on the PSRs and ISRs, and the QAG assessment.

2.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable)

11

Overall financial management is rated satisfactory. The PCU’s financial management systems during project implementation, including project accounting and financial reporting, internal controls, staffing, disbursements, and auditing arrangements were considered adequate. Starting 2001, the PCU used the automated accounting system “HANSA Financial,” while a revised, improved version of this system was used from 2003 onwards. Procurement throughout the project period proceeded with appropriate speed for goods and renovations/works for hospitals, day surgery centers and GP offices. However, considerable delays were observed in procurement of services that needed political decisions by the MOH. Audits were carried out annually and were unqualified.

2.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable) Following project closure, the Government has been using EU structural funds to finance the implementation of the remaining phases of the HMIS. Under the project, the Concept (Core system) of the HMIS or E-Health project was prepared and four functions –the basic electronic patient record, patient registration with PHC providers, referrals to secondary and tertiary level services and laboratories, and records of results- were developed. Development of the remaining 11 functions and roll-out of the system in three regions is currently financed through EU Structural Funds. The MOH has maintained the PCU in order to coordinate activities under the new EU “Pilot E-Health System Development Project” which became effective in February 2007 and is expected to close in September 2008. The use of EU structural funds for the completion of the HMIS provides clear evidence of the importance that the Government assigns to the development of E-Health in Lithuania and its commitment to complete project activities. At the same time, the MOH takes full advantage of in-house capacity developed under the LHP in order to co-ordinate the implementation of the EU-financed activities. The Government continues HSR through the approval of the second stage of the ‘Strategy for Restructuring of Health Care Institutions’ in June 2006. The Government approved the first stage of the ‘Strategy for Restructuring of Health Care Institutions’ in March 2003 (see also section 3.2). Implementation of this Strategy resulted in a 19% reduction in the number of hospital beds between 2000 and 2005. 43% of this reduction took place in non-pilot counties showing the commitment of the Government to HSR. The second stage of this Strategy was approved in June 2006 and contemplates continuation of the restructuring efforts. The Lithuanian Government and the World Bank have explored the possibility of future collaboration in various health-related topics. Three areas for future collaboration have been identified: developing a Health Human Resources strategy, supporting the MOH in implementing its Mental Health Policy 2005-2010 and exploring options for Public-Private Partnership (PPP) in delivery of health care services. Forms of collaboration discussed with the MOH included: (i) ad-hoc short-term TA, and cross-country TA; and (ii) longer-term TA on a cost-sharing basis with the Government for priority areas subject to the limits of Bank support to graduated countries, should the Government be interested.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy)

12

The objectives and implementation of the LHP remained relevant and consistent with Lithuania's current development priorities in the health sector and with current Bank country and sectoral assistance strategies and corporate goals. The 2006 ‘Implementation Strategy of Health Care Reform’s Aims and Objectives’ prepared by the MOH recognized ‘equity in health as the main principle of the national health policy’ and emphasized the importance of the primary health care sector and HSR for the Health Care Reform program. HSR was also supported through the Government’s ‘Strategy for Restructuring of Health Care Institutions.’

3.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) The project has achieved its development objectives satisfactorily as demonstrated through the achievement of most targets set out in the outcome and output indicators in the Results Framework Analysis. The project has contributed to policy development, strengthening capacity of national health institutions, developing a national HMIS and carrying out HSR in four pilot counties. By providing support to health reform, the project achieved a more equitable allocation of health resources among regions based on population size and other characteristics of the pool of beneficiaries and led to the development and adoption of health specific investment guidelines. One of the main accomplishments of the project was the development of a Master Plan for health facilities which covered the whole country and was used for HSR not only in the four pilot apskritis but in the rest of the counties as well. During the project period, issues such as provider reimbursement and contracting, needs assessment and service planning, ambulance and emergency services and the content of the basic benefits package (BBP) were widely discussed in the framework of national workshops and conferences and strategies on these issues were developed by health authorities. The project supported training of MOH officials from the Communication Unit and the Medical Library and provided equipment and furniture for the LHIC, thus increasing institutional capacity. The project also assisted with the development of the Core System and four functions of the HMIS and set the basis for the development of E-Health in the country. Finally, by investing in civil works, training, equipment and vehicles in PHC facilities, hospitals, and ambulance and emergency services, the project laid the foundation for the provision of more and better quality health care services to the population of pilot counties, improved efficiency of the health care system and increased patient satisfaction. As mentioned earlier, in the area of policy development, a stronger and more committed leadership could have led to some additional policy decisions and reform actions. The Results Framework of the LHP identified the following sets of indicators: (i) sector related-CAS goal related indicators; (ii) PDO indicators; and (iii) Component indicators. Achievement of the project development objectives is measured by progress made against each project indicator, as follows: Sector related-CAS goal indicators Improved efficiency of health care system. This indicator was fulfilled in a satisfactory way. Efficiency of the health care system was evaluated using two standard health service efficiency indicators: average length of stay (ALOS) in hospitals and bed occupancy rate (BOR). Both indicators have improved during the period of project implementation (Figure 1). In particular, ALOS decreased from 7.8 days in 2001 to 6.7 days in 2005 (14% decrease), while BOR increased

13

from 25.1% in 2000 to 29% in 2005 (15.5% increase). The increase in BOR is considered a step towards the right direction, although its value is still low by international standards. Figures 1 and 2. Improved efficiency of the health care system

7.8

7.4

6.7

6

6.5

7

7.5

8

No.

of d

ays

2001 2002 2005

Average Length of Stay (ALOS)

25.1

25.9

29

23242526272829

Perc

enta

ge

2000 2002 2005

Bed Occupancy Rate (BOR)

Source: SPF. Improved population satisfaction with national health services. Results related to this indicator are mixed. Patient satisfaction with service provision in PHC ambulatories increased by 9.5% between the baseline and evaluation periods (Table 1). However, no change in the level of patient satisfaction was observed for services provided in polyclinics between the two periods. When taking into account both ambulatories and polyclinics, patient satisfaction increased by 1% for the project group (individuals who visited facilities participating in the project). An increase in patient satisfaction for services provided in PHC ambulatories was also registered for the control group but it was lower compared to the one of the main group (8.7%). In the case of polyclinics, the control group registered a decrease of 5% in patient satisfaction. The main reasons for patient dissatisfaction in both project and control groups were long queues, doctors with low level of qualifications and inability to carry all tests in the facility. Table 1. Increased satisfaction of project group with provision of PHC services Level of satisfaction 2001 2005 % change Project group Polyclinics 3.73 3.67 -1.60 Ambulatories 4.01 4.39 9.50 All 3.82 3.85 0.80 Control group Polyclinics 3.65 3.46 -5.20 Ambulatories 3.80 4.13 8.70 All 3.70 3.64 -1.60 Note: 1. Respondents were asked to qualify services using a scale from 1 (absolutely dissatisfied) to 5 (very satisfied). 2. All changes are statistically significant with the exception of the project group for polyclinics. Source: Satisfaction Surveys, 2001 and 2005. PDO indicators Standard health service efficiency indicators improve yearly over the life of the project. This indicator was fulfilled in a satisfactory way. Improvements during the project life were observed both in ALOS and BOR. There was an approximately 14% decrease in ALOS for hospitals and a 15% increase in BOR, albeit from a very low baseline value of 25% (See also Figures 1 and 2). 90% of health care providers stay within year-end predefined fixed price-volume budgets. This indicator was fulfilled in a satisfactory way. According to data provided by the SPF, more than

14

90% of health care providers (92.5%) stayed within year-end predefined fixed price-volume budgets by the end of the project period. Unfortunately, no data is available for the baseline and MTR periods. 50% of population in project areas is covered by certified GPs providing comprehensive services by end of project. This indicator was fulfilled in a highly satisfactory way. One of the main objectives of Component B was the development of a PHC reform strategy promoting the provision of services by GPs) in order to improve access to quality PHC services. For this reason, regional PHC development programs were implemented in all pilot counties providing training and retraining of GPs and nurses, refurbishing of family practice offices and health centers, provision of medical and office equipment and limited number of vehicles for GPs in isolated rural areas. As a result of project activities, population coverage in pilot areas increased from 24.7% in 2000 to 61.8% in 2006. In terms of actual number of physicians, GPs increased from 692 in 2000 to 1,730 in 2006 (61% of all GPs in the country) in pilot counties. Referrals and self-referrals to ambulatory care specialists and hospitals are reduced by 20% in pilot areas by end of project. This indicator was fulfilled in a moderately satisfactory way. There is no data on referral and self-referrals to ambulatory care specialists and hospitals. For the evaluation of this indicator, ambulatory specialist visits per capita and admissions to hospital inpatient care per 1,000 population are used. Between 2000 and 2005, ambulatory specialist visits per capita increased in Alytus and Kaunas counties by around 12%, while they decreased in Utena and Vilnius counties by 9% and 7%, respectively (Table 2). In the case of hospital admissions, all counties showed an increase of services with the exception of Utena where hospital admissions remained approximately the same. Likewise, the number of ambulatory visits per capita, excluding specialist visits, increased in all four counties between 10% (Kaunas) and 21% (Alytus and Vilnius). The fact that there was a greater percentage increase in ambulatory visits to non-specialists as compared to specialists and hospitals provides evidence of the strengthening and higher utilization of the former services. In addition, a higher level of visits to ambulatory care specialists and hospital admissions might have been caused by a higher burden of disease between different periods in time and does not necessarily reflect an under-utilization of ambulatory non-specialist services. Table 2. Ambulatory visits (to specialists and non-specialists) and hospital admissions

2000

2002

2005 % change

(2000-2005) Ambulatory specialist visits per capita Alytus 1.34 1.41 1.50 11.90 Kaunas 2.84 3.11 3.20 12.70 Utena 1.48 1.44 1.34 -9.50 Vilnius 2.85 2.77 2.64 -7.40 Admissions to hospital inpatient care/1,000 population Alytus 176.00 185.00 181.10 2.90 Kaunas 262.90 267.50 285.60 8.60 Utena 175.30 169.50 174.10 -0.70 Vilnius 250.40 257.40 257.00 2.60 Ambulatory visits (excluding specialist visits) per capita Alytus 4.56 4.89 5.53 21.27 Kaunas 4.36 4.39 4.80 10.09 Utena 4.02 4.36 4.64 15.42 Vilnius 4.25 4.73 5.14 20.94 Source: LHIC.

15

Patient satisfaction with the services they get from their primary care physicians is improved in pilot areas by end of project. Results related to this indicator are mixed. Patient satisfaction with service provision in PHC ambulatories increased by 9.5% between the baseline and evaluation periods, however there was no change in patient satisfaction for services provided in polyclinics. Please see also discussion under indicator Improved population satisfaction with national health services and Table 1. Policy framework for health service planning and restructuring in place, and used by health administrators by end of project. This indicator was fulfilled in a highly satisfactory way. By May 2002, a “National Hospital Restructuring Strategy,” a “Strategy Implementation Plan” and “Proposals for Needs-Based Health Care Service Plans in at least Five Counties” were prepared by an international consultant under the project. These deliverables served as the basis for the preparation of the “Strategy for the Restructuring of Health Care Institutions” approved in March 2003 by the GOL. The government’s Strategy resulted in a 19% reduction in the number of hospital beds between 2000 and 2005 (Figure 3). Fifty-seven percent of this reduction took place in project pilot counties. More importantly, 43% took place in the rest of the country showing the government’s commitment to HSR. The Government approved the second stage of the Strategy in June 2006. Figure 3. Reductions in the number of hospital Figure 4. Reduction in average length of beds, 2000 and 2005. stay

3414527727

2028516580 13860

11147

05000

100001500020000250003000035000

All Project counties Non-projectcounties

No. of hospital beds

20002005

11.439.74

11.7110.21 10.15

8.97

11.4810.45

0.00

2.00

4.00

6.00

8.00

10.00

12.00

No.

of d

ays

Alytus Kaunas Utena Vilnius

Average length of stay

20002005

Source: LHIC. Component A 80% of health funds allocated to regions according to population and needs-based formula by mid-term evaluation (2002) and 100% of health funds by end of project. This indicator was fulfilled in a satisfactory way. In 2000, 28% of funds were allocated to regions based on a population and needs-based formula; PHC and ambulance services, nursing, and rehabilitation were financed through formula funds based on a capitation payment system. In the period between 2000 and 2002 (MTR) international TA assisted the SPF to assess the existing system of resource allocation and provided recommendations on further development of the formula. The adoption of a revised formula that included outpatient specialist services and hospital services took place in June 2003 and increased the level of funds allocated through the formula to 54% of total funds. The delay in the adoption of the revised formula was mainly the result of the reorganization of the TPFs (which were reduced from 10 in 2002 to 5 in 2003). In July 2006, a further revision took place which led to the inclusion of pharmaceuticals in the resource allocation formula and an increase of the funds to be allocated through the formula to 75.5%. Currently, the only interventions that are not included in the formula are high-cost procedures. Although Lithuanian health authorities and the SPF, in particular, have made a great effort in

16

revising and employing the resource allocation formula in the last five years, differences in public expenditure on health among regions remain relatively high. Under the assumption that the final goal of the adoption of the population and needs-based formula should be a more equitable distribution of public expenditure on health, Lithuania authorities should continue working toward this goal through refinement of the formula mechanism. A revised hospital reimbursement schedule based on standard costing study is in place by 2003. This indicator was fulfilled in a moderately satisfactory way. A revised reimbursement schedule was introduced in February 2003 but it was not based on a national costing exercise and is expected to be revised again in the near future. In 2003, the MOH revised the hospital reimbursement schedule by reducing the number of DRG-type (Diagnostically Related Group) of profiles it was using up to that date. Nevertheless, prices were not based on a Lithuanian costing exercise and in recent discussions (during the preparation of this ICR), government officials signaled the national authorities’ intention to carry out a new revision of the system where they will define upper limits for specific services and stop using group profiles. Government officials stated that the provision of project TA in this area was a valuable input to the sector dialogue, but at the same time they acknowledged that hospital reimbursement is still a topic under discussion. Government guidelines for appraisal, allocation, monitoring and financing of health sector investments are developed and in use by Year 3 of project. This indicator was fulfilled in a satisfactory way. In the period between 2000 and 2002 (MTR) international TA assisted the MOH to assess the existing system of investment allocation and provided recommendations on its future development. The MOH adopted health-specific investment guidelines in 2002 which are still in use. National and regional need-based health services plans are developed in at least 7 counties (out of 10). This indicator was fulfilled in a moderately satisfactory way. Although no separate needs-based health service plans were developed at the county level, the “Strategy for the Restructuring of Health Care Institutions” was implicitly based on a needs assessment and service planning. One of the reasons for which no separate plans were prepared was due to resource constraints. As a result, the MOH decided to concentrate more on sub-component A1.6 as compared to sub-component A1.4. Numbers of health institution managers are trained in management. This indicator was not fulfilled. There were no project activities related to this indicator. Ambulance services review report is developed and disseminated by 2001. This indicator was fulfilled in a moderately satisfactory way. Ambulance service development plans were prepared and adopted in 2003 in Kaunas and Utena counties in accordance with their PIAs. At the national level, TA was provided under the project on the “Development of an Ambulance Service Strategy” and “Implementation Plan,” but no final political decisions were made given resistance from the municipalities to transfer responsibilities of ambulance services to the county level. National Health Report is published regularly. This indicator was fulfilled in a satisfactory way. A National Health Report is prepared and published yearly by the LHIC (www.lsic.lt). Component B The implementation of Component B. HSR is rated as highly satisfactory. HSR in all four pilot counties was implemented in a smooth and highly effective way once PIAs were signed.

17

One of the most impressive outputs of this Component was related to the superior quality of civil works. This was an impression shared by the WB team, Lithuanian counterparts at the central level and the author of this ICR. On monitoring indicators, Component B performed equally well with reductions in the number of beds (up to 33% in Alytus Country) (Table 3) and ALOS (Figure 4); increases in population coverage by certified GPs (Figure 5); increases in utilization of PHC non-specialist services higher than increases in utilization of PHC specialist services and hospitalizations (Table 2); increases in outpatient surgery; and introduction of hospital finance management and personnel/patient administration systems. In addition, in accordance with the PIAs, Compulsory Health Insurance Fund (CHIF) resources were not used for the maintenance of premises becoming vacant upon completion of hospital restructuring activities. Component B had a great spill-over effect, as restructuring took place not only in the four pilot counties but in the rest of the country, as well. This resulted in a reduction in the number of beds by 19.5% in non-project counties (Figure 3). Furthermore, the GOL approved the second stage of the “Strategy for the Restructuring of Health Care Institutions” in June 2006, providing clear evidence of its intentions to continue the restructuring efforts. Table 3. Reduction in number of hospitals beds

No. of hospital beds County 2000 2005 % change Alytus 1,646 1,104 -32.9 Kaunas 7,986 6,556 -17.9 Utena 1,316 1,045 -20.6 Vilnius 9,337 7,875 -15.7

Source: LHIC . A detailed list of country-specific performance indicators/targets included in PIAs is provided below.3 Performance Indicators of the PIA in Alytus County. 55% of the Alytus County population is covered by qualified GPs by end of project. This indicator was fulfilled in a highly satisfactory way. 62% of Alytus County population was covered by qualified GPs by the end of the project (Figure 5). The total number of hospital beds in the Alytus County Hospital is reduced by 20%. This indicator was fulfilled in a highly satisfactory way. There was a 26% reduction in the number of hospital beds by the end of the project (from 583 beds in 2000 to 431 in 2006). Not less than 40% of all operations are performed in the day surgery center of the Alytus County Hospital. This indicator was fulfilled in a moderately satisfactory way. 31% of all operations in the Alytus County Hospital were performed in the day surgery center. The lower level of day surgery operations was to due to the fact that the center started functioning in 2005.

3 There are no indicators for Vilnius County. The Vilnius PIA included activities to be carried out under the Vilnius Apskritis PHC Service Restructuring Program and the Hospital Restructuring Program, but no performance indicators similar to the ones of the rest of the counties.

18

Average Length of Stay in the Alytus County Hospital is reduced to 9 days. This indicator was fulfilled in a highly satisfactory way. ALOS in Alytus County Hospital reached a level of 6.9 days in 2006. Figure 5. Increased population coverage by GPs in pilot counties.

27

5462

14.5

48.3

76.6

0

20

40

60

80

Percen

tage

Alytus Kaunas

% population coverage by GPs in pilot counties

2000

2002

2006

Source: PCU and regional data. Performance Indicators of the PIA in Kaunas County. 55% of the Kaunas County population is covered by qualified GPs by end of project. This indicator was fulfilled in a highly satisfactory way. 76.6% of Kaunas County population was covered by qualified GPs by the end of the project (Figure 5). The number of surgery beds in Kaunas Clinical Hospitals No.2 and No. 3 is reduced by 30%. This indicator was fulfilled in a highly satisfactory way. By the end of project, there was a 38% reduction in the number of surgery beds in Kaunas Clinical Hospital No. 2 (from 150 beds in 2000 to 97 in 2006) and a 35% reduction in Kaunas Clinical Hospital No. 3 (from 140 beds in 2000 to 87 in 2006). Not less than 4,000 operations (each) are performed in the day surgery centers of Kaunas Clinical Hospitals No. 2 and 3. This indicator was fulfilled in a moderately satisfactory way. Day surgery centers started operating in Kaunas Clinical Hospitals No. 2 and No. 3 in March 2005. During 2006, 1,391 operations took place in the day surgery centers. Performance Indicators of the PIA in Utena County. 20% reduction in the number of referrals to specialists. This indicator was not fulfilled. There was an 11% increase in the number of visits to specialists by the end of the project (from 5 visits per capita in 2000 to 5.6 visits per capita in 2006). Increases in the number of visits to specialists are probably closely related to decreases in the number of hospitalizations, with visits to specialists operating as a substitute for hospitalizations. Although it was envisaged for visits to specialists to be reduced because of a higher demand for ambulatory non-specialist visits, the restructuring of the county hospital system reduced the number of hospitalizations and patients sought specialist care. It is also worth noting that the highest increase in terms of health care services was recorded for ambulatory non-specialist visits (with a per capita increase of 15.5%). 20% reduction in the number of direct arrangements to see specialists. This indicator cannot be assessed due to lack of data.

19

30% reduction in the number of emergency calls. This indicator was not fulfilled. There was a 51% increase in the number of emergency calls by the end of the project (from 31,008 in 2000 to 47,126 in 2006). Nevertheless, this increase mainly involved calls for transportation of pregnant women after the closing of the obstetric departments in Ignalina, Moletai and Zarasai hospitals; based on the organizational system of the county, pregnant women are expected to call emergency services when in need of transportation to the hospital. Unfortunately, there is no available data on the number of emergency calls excluding the calls made by pregnant women that would allow a more accurate assessment of this indicator. 5% reduction in the number of hospitalization cases. This indicator was fulfilled in a highly satisfactory way. There was a 6.25% reduction in the number of hospitalizations by the end of the project (from 32,000 in 2000 to 30,000 in 2006). Provision of general surgical, gynecological and pediatric services is introduced without increasing total number of beds in Visaginas Town Hospital. This indicator was fulfilled in a highly satisfactory way. Number of births in Visaginas Town Hospital increased from 180 per year in 2000 to 338 in 2005 (88% increase). Component C Performance indicators related to project management were fulfilled in a highly satisfactory way with the exception of regular monitoring of progress towards achievement of key performance indicators and agreed development objectives which is rated as moderately satisfactory. Performance indicators of project management related to timely contracting of goods, works, and services with quality outputs; efficiently managed PCU with adequate staff and resources; satisfactory accounting systems, and annual audits; promotion of an efficient dialogue among key project actors, and annual staff performance evaluations, and training programs were rated as highly satisfactory. As mentioned before, the professionalism and dedication of the PCU to project implementation constituted one of the main factors for the project’s success. The only exception was the documentation of progress towards achievement of key performance indicators and agreed development objectives, which was rated as moderately satisfactory as there was lack of complete data on M&E at the end of the project. Nevertheless, it needs to be acknowledged that without the PCU involvement, the probability of collecting this data ex post would have been very limited.

3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return) The LHP was subject to an economic and financial analysis at preparation. Economic Analysis. The costs of the project and the expected benefits were estimated in three main areas: (i) policy development and support to national health institutions; (ii) HSR in the four apskritis that support the PHC reform and hospital capacity optimization; and (iii) total costs and benefits of the project including synergistic effects and indirect financial benefits. The costs and benefits of the restructuring component were related to: (i) changes in referral rates to hospitals and specialists; (ii) estimated benefits from expected changes in skills mix and structure of medical staff, costs and benefits of pilot PHC practices with a special focus on the shift of treatments from hospitals to PHC (including indirect benefits like reduced time-costs and travel costs); (iii) savings due to expanding day care surgery and concentration of some care and

20

avoided duplications, benefits in terms of shorter ALOS and cuts in hospital beds; (iv) estimated changes in direct costs and benefits for restructuring mental health by the expansion of out-patient treatment; and (v) estimated costs and productivity gains due to investment in the ambulance sector (including analyzing the volume of different services provided and organizational changes). The summary project costs included also the cost for project management as well as an additional benefit from the demonstration effect and synergistic effect of coordination and implementation process (equal to 1% of public expenditure on health starting from midterm of the project). Data for the analysis was derived from domestic reports, and from ad hoc data collection and research (questionnaires, telephone interviews, etc.). The economic analysis concluded that the project as a whole would yield an estimated internal rate of return (IRR) of 41% if the indirect financial benefits were included. The indirect benefits represented 14% of total project benefits, driven primarily by shorter length-of stay in hospitals due to expansion of day-care surgery and ambulatory care. The total benefits over ten years were nearly US$94 million and the net present value (NPV) over US$49 million. Financial Analysis. The financial project analysis estimated recurrent costs linked to investments, and analyzed how this would affect total public expenditure on health. The fiscal effects were to a large extent equal to the economic costs and benefits, excluding indirect financial benefits. The financial analysis concluded that the project would yield a financial rate of return (FRR) equal to 32% and a NPV of US$34.7 million. A re-estimation of the IRR, FRR and NPV for the economic and financial analysis was not possible due to the unavailability of the original project files. As stated in the PAD, details of the economic and financial evaluations were available in project files. However, during ICR preparation, it was not possible to locate these files either in the IRIS system or in the personal files of the TTL at preparation time and the local consultants responsible for the collection of data for the analysis. Some assumptions and data were included in the PAD, but they were not sufficient for the re-estimation of the IRR and FRR and NPV. Despite the fact that re-estimation of the project’s IRR, FRR and NPV was not possible, the achievement of performance indicators indicates that project benefits identified during preparation were realized First, there was an increase in the utilization of PHC services as compared to ambulatory specialist services and hospitalizations; this implied a reduction in the cost of treatment for patients who were treated by GPs and otherwise would have been treated by a specialist or at the hospital. Second, the creation of day surgery centers led to an increase in the number of day surgeries; saving resources that otherwise would have been used to cover full hospitalizations. Third, project activities resulted in a decrease in ALOS and an increase in BOR, both clear measures of efficiency. Fourth, the project reduced the cost of utilities of health care facilities by reducing excess building capacity. Finally, as demonstrated through the baseline and evaluation surveys, there were improvements in patient satisfaction in project health care facilities which were greater than the ones observed in control facilities.

3.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency) Rating: Satisfactory The overall outcome rating of the project is satisfactory. The objectives and implementation of the LHP remain relevant and consistent with Lithuania's current development priorities in the health sector as presented in the 2006 ‘Implementation Strategy of Health Care Reform’s Aims and Objectives’ and the ‘Strategy for the Restructuring of Health Care Institutions.’ The project

21

has achieved its development objectives in a satisfactory way as demonstrated through the accomplishment of intended outcomes and outputs for most project indicators (Table 4). The project contributed to the restructuring of PHC and the hospital sector in four pilot counties; the strengthening of the capacity of national health institutions; and the development of a HMIS. The project also assisted the process of policy development, albeit to a lesser extent, as political decisions were not taken for all project reform issues. One of the most important contributions of the project to the development of the health sector in Lithuania was paving the way for the continuation and deepening of HSR in the whole country. Furthermore, the project served as a catalyst for the development of the HMIS, as it addressed politically sensitive issues of data confidentiality and end-users. Efficiency wise, the project resulted in reductions in ALOS and number of hospital beds, increases in BOR and day surgeries and a re-focus towards PHC services, amongst other things. Population baseline and evaluation surveys also demonstrated an increase in patient satisfaction with PHC services in pilot counties. Table 4. Ratings of project performance indicators HS S MS NF M CBA Total PDO 2 2 1 - 1 - 6 Comp. A - 3 3 1 - - 7 Comp. B 7 - 2 2 - 1 12 Comp. C 5 - 1 - - - 6 Total 15 5 7 3 1 1 31 Notes: 1. Sector related-CAS goal indicators are not included separately in this table as they are also PDO indicators. 2. HS denotes Highly Satisfactory; S Satisfactory, MS Moderately Satisfactory, NF Not Fulfilled, M Mixed, and CBA Cannot Be Assessed.

3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) The project had a substantial impact on institutional development in Lithuania's health sector. Outcome indicators showing increased utilization of GP services and improvements in patients' satisfaction indicate increased knowledge and capacity of physicians and nurses trained under the project. Similarly, project activities assisted in increasing the understanding and raising the awareness on reform issues of staff of national health institutions such as the LHIC, SPF, TPFs and MOH. The continuation of the reform efforts through the second stage of the ‘Strategy for the Restructuring of Health Care Institutions’ provided further evidence of the impact of project activities on the institutional capacity of national health authorities. At the county level, implementation of project activities in pilot apskritis gave the opportunity to the local authorities to interact with the central level and members of the international health community, and participate in reform efforts. Finally, on project monitoring, Lithuania is one of the few countries where the PCU has continued in the MOH after project completion and has become the coordination unit for a different project. (c) Other Unintended Outcomes and Impacts (positive or negative)

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes)

22

Beneficiary Surveys Two satisfaction surveys were carried out during project implementation: a baseline survey in 2001 and an evaluation survey in 2005. The surveys provided information on patients’ satisfaction with PHC institutions in the pilot counties. Two types of population groups were used; individuals who visited PHC facilities that participated in the project (project group) and individuals who visited PHC facilities that did not participate in the project (control group). Evaluation of patient satisfaction took into account issues such as time spent before seeing the physician, features of the physician visited, duration of physician’s visit, contact with medical nurse, and suitability of premises. For details on survey results see Annex 5. Stakeholder Workshops Various stakeholder workshops were organized under Component A. These workshops formed part of the deliverables of the two main TA contracts for consultant services. The first contract included TA on the areas of Ambulance Service Development, Hospital Care Planning (Master-planning) and Capacity Building. Results on these three areas were presented in a national conference and three workshops. The second contract included TA on Resource Allocation Formula Development, Provider Reimbursement and Contracting, Investment Allocation and Public Information. Results on these areas were presented in one workshop, two training courses and one seminar. Furthermore, two study tours were organized by consultants. The first study tour aimed at familiarizing Lithuanian specialists with the Swedish ambulance system, while the second study tour aimed at getting Lithuania experts acquainted with the Irish health information system. Participants in these tours contributed to the preparation of the Ambulance Service Strategy and HMIS. For more details see Annex 6.

4. Assessment of Risk to Development Outcome Rating: Moderate The risk to development outcome is estimated to be moderate. The Government appears committed to build on the achievements of the completed project especially in the areas of HSR and HMIS. On HSR, the Government approved in 2006 the second stage of the “Strategy for the Restructuring of Health Care Institutions” and it plans to intensify its efforts in this area in all counties. Reductions in ALOS and BOR at a national level provided some early evidence of improvements in the way public resource on health are used, but the Government needs to deepen its efforts in HSR, Provider Reimbursement and a narrower definition of the BBP in order to address the issue of indebtedness of the CHIF towards personal health care providers, pharmacies and other health care institutions (actual expenditure on services and medicine exceed planned expenditure). On HMIS, as already discussed, the Government is currently using EU structural funds to finance the implementation of the remaining phases of the system. The EU funds will finance the development of the remaining eight functions of the HMIS and roll-out of the system in three regions. The Government has also maintained the LHP PCU to administer the implementation process under the new EU project. In terms of public spending, Lithuania spent approximately 5% of its GDP on health in 2005, a share which remained relatively stable during the period 2000-2005 (Table 5). Public expenditure on health represents the majority (75%) of total expenditure on health and increased on average and in real terms at a similar pace as the GDP during the period 2000-2005. This, as well as the fact that the level of increase of public expenditure on health (PEH) is higher than the level of increase of total government expenditure provides more evidence of the importance that the GOL places on the health sector. A sustained and increasing stream of public funds needs to be directed to the health sector to maintain investments carried out in project and non-project sites

23

and to expand the reforms to the rest of the country. Nevertheless, as already discussed adequate funding still needs to be combined with structural reforms in order to achieve a more efficient use of resources and address the issue of debts. The Government needs to pay attention to stakeholder groups such as health professionals or county populations with vested interests in the health sector that might attempt to postpone or stop reform efforts. In order to mitigate the social risk related to these groups, the MOH could make use of its communications unit to support, guide and train health care system stakeholders and the general public. Table 5. Public expenditure on health, 2000-2005 (million Litai)

2000 2001 2002 2003 2004 2005 PEH 2,071 2,223 2,509 2,821 3,045 3,345 GGE 14,194 14,620 17,720 19,178 19,264 23,930 GDP 45,848 48,563 51,948 56,772 62,440 71,084 Real Growth PEH 0.55 7.88 12.64 13.69 4.96 3.72 GGE -15.98 3.52 20.96 9.44 -2.33 17.28 GDP 3.94 6.45 6.76 10.51 6.95 7.48 PEH % GDP 4.52 4.58 4.83 4.97 4.88 4.71 PEH % GGE 14.59 15.21 14.16 14.71 15.81 13.98 Note: GGE denotes General Government Expenditure. Source: WHO data.

5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately Satisfactory The Bank's performance during project preparation was moderately satisfactory. The project was based on a sound and detailed background analysis that profited from a long preparation period and a demand driven approach for competitive selection of pilot counties. Nevertheless, although the PAD acknowledged that health sector reforms in the ECA Region were a lengthy and politicized process and projects per se were complex, the Bank team did not apply these lessons when designing the LHP. The LHP was a complex project with three components and a total of 17 sub-components; 13 sub-components under Component A and four sub-components under Component B. The inclusion of multiple reform issues under the Policy Development sub-component in an environment of weak institutional capacity, frequent political changes and a relatively low level of political commitment resulted in extensive institutional discussions, but few political decisions. A simpler project structure could have benefited project implementation, as well as monitoring and evaluation. The high number of sub-components created the need for a high number of key performance indicators. A QAG assessement carried out in October 2002 also identified problems in project design (e.g., “overly ambitious policy and institutional reform agenda”, “Yet, the design of this project did not adequately incorporate these (the PAD) lessons.”) and the quality of performance indicators (e.g., “The quality of performance indicators could have been improved – to reduce the number of key indicators…”).

24

(b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Satisfactory The Bank's performance during project implementation was moderately satisfactory. The project was monitored through frequent implementation visits. Aide-memoires and follow-up letters were prepared in a timely manner and served as the institutional memory for the project and for highlighting pending or important issues to be followed up. Implementation teams frequently visited sites in pilot counties to become better acquainted with project interventions. The Bank team worked closely and effectively with the PCU and SIDA representatives who in a number of cases joined Bank implementation reviews. Implementation could have benefited from a more systematic M&E of key performance indicators. As mentioned in section 2.3, efforts to collect data for M&E purposes were carried out during the project period. However, the documenting of data was not completed. This was partly due to the large number of key performance indicators and the difficult political environment with frequent changes of ministers and a constant need for securing the commitment of the MOH’s leadership on project activities. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The Bank's overall performance during project preparation, appraisal and supervision is rated moderately satisfactory.

5.2 Borrower Performance (a) Government Performance Rating: Satisfactory

The government's performance is rated satisfactory. Both the central and local governments in pilot counties ensured quality of project preparation and implementation and complied with covenants and agreements, towards the achievement of development outcomes. A high level of commitment was demonstrated in the area of HSR which took place not only in pilot counties but in the rest of the country as well. In the area of Policy Development, although there is still ample room for further policy reforms, the Government introduced a revised Resource Allocation Formula and health-specific investment guidelines, and opened the dialogue on issues such as Provider Reimbursement and Contracting, Ambulance Service Development and the content of the BBP. The Government also provided timely counterpart financing for the project.

(b) Implementing Agency or Agencies Performance Rating: Satisfactory The performance of the MOH is rated moderately satisfactory. The MOH actively participated in project preparation and provided its support to project activities during implementation. However, fewer changes in the ministry’s leadership and more active collaboration by the Chair of the Steering Committee might have contributed to a smoother and faster project implementation, especially during the first three years of the project. Furthermore, a stronger level of commitment to project activities related to policy development might have a led to additional policy decisions with respect to key reform areas such as provider reimbursement and contracting, the content of the BBP, ambulance service development, etc.

25

The performance of the PCU is rated satisfactory. The PCU did a good job in implementing the project in a volatile political environment. The PCU repeatedly secured ministerial support following changes in the leadership of the MOH. The PCU was well organized and effective in dealing with procurement, disbursement, progress reports, and in maintaining proper records of the project. The PCU was responsive and attentive to Bank requests and suggestions. The PCU should have paid even greater attention in systematically monitoring the project and documenting this process. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory The borrower's overall performance is rated satisfactory.

6. Lessons Learned (both project-specific and of wide general application) The following lessons can be drawn from the implementation of the Lithuania Health Project: 1. Systematic and documented monitoring of project performance indicators allows for better

project evaluation and increases considerably the demonstration impact that a project can have both in a national and international context.

2. Government ownership not only leads to better implemented projects, but also increases the

probability of continuing with the reform efforts after the completion of the project. In the area of HSR, strong government commitment led to the expansion of the restructuring process to non-pilot counties and the adoption of a second stage of the “Strategy for the Restructuring of Health Care Institutions.”

3. The complexity of projects should be adjusted to the country’s context and circumstances. 4. Frequent changes of key persons in the government and the sectoral ministries have an

adverse impact on the speed and the level of project implementation. 5. Bank management should assure continuity of staff during project implementation. 6. In projects that involve HSR, the existence of some type of conditionality between the project

activities and the availability of funds (i.e., in the Lithuania case, funds became available at the county level only after PIAs were signed) increases the probability that HSR is successfully implemented. WB experience in other ECA countries and in similar projects demonstrate few changes in the number of hospital facilities, departments and beds when there is no conditionality between the restructuring efforts and the availability of funds.

7. Availability of medication and equipment in PHC facilities and better premises increase

patient satisfaction with the provision of services. As the satisfaction surveys carried out under the LHP showed, individuals visiting project facilities referred to better premises and the possibility of having all necessary exams at the facility among the main factors of increased satisfaction.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies

26

(b) Cofinanciers SIDA informed the Bank that it would not be able to provide comments or a contribution to this ICR due to the fact that SIDA employees who participated in the supervision of the LHP were not available or had left the organization. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

27

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

A1. POLICY DEVELOPMENT 1.16 0.98 84.48

A2. STRENGTHENING CAPACITY OF NATIONAL HEALTH INSTITUTIONS

1.47 0.32 21.77

A3. INFORMATION MANAGEMENT

5.83 5.74 98.46

B1. ALYTUS PILOT PROJECT 5.26 4.89 92.96

B2. KAUNAS PILOT PROJECT 7.01 6.88 98.15

B3. UTENA PILOT PROJECT 4.38 3.82 87.21

B4. VILINUS PILOT PROJECT 7.48 6.05 80.75

PROJECT IMPLEMENTATION AND MANAGEMENT

1.35 1.13 83.70

Total Baseline Cost 33.94 29.81 87.80

Physical Contingencies - - - Price Contingencies - - -

Total Project Costs 33.94 29.81 87.80 Project Preparation Fund - - - Front-end fee IBRD 0.21 0.21 100.00

Total Financing Required 34.17 30.02 87.85

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 9.22 6.46 70.06 EC: TECH. ASSISTANCE CIS 0.86 0.14 16.28 International Bank for Reconstruction 21.23 21.18 99.76

28

and Development SWEDEN: Swedish Intl. Dev. Cooperation Agency (SIDA) 2.40 2.24

93.33

Bilateral Agencies (unidentified) 0.50

29

Annex 2. Outputs by Component

This section aims at providing brief evaluations and ratings of achievements by components. Information on project outputs that are closely related to key project indicators and commented in the main body of the ICR is not repeated. The section rather presents a summary of what has been achieved and how it is related to project objectives.

Component A. Support to Health Reform (actual total cost US$7.04 million): Moderately Satisfactory. Sub-component A1. Policy Development (actual total cost US$0.92 million): Moderately Satisfactory. Under the sub-component of Policy Development, TA was provided for six reform areas: geographic resource allocation system, investment funding, provider reimbursement and contracting, needs assessment and service planning, development of the BBP, and health care delivery system restructuring. TA was organized under two main contracts for consultant services. The first contract included TA on the areas of Ambulance Service Development and Hospital Care Planning (alias health care delivery system restructuring) and Capacity Building in these two areas. The second contract included TA on Resource Allocation Formula Development, Provider Reimbursement and Contracting, Investment Allocation and provision of Public Information in these areas. Despite the fact that contract deliverables were produced as stipulated in the TORs, policy decisions were taken regarding three areas: the resource allocation formula, investment funding, and the health care service restructuring (both in pilot and non-pilot counties). Project outputs related to needs assessment and service planning were considered integrated in the health service restructuring strategy developed by the Government. On provider reimbursement and contracting, despite various changes during the project period on contract mechanisms applied by the SPF, no final decisions have been taken yet. On BBP, efforts concentrated on the development of guidelines and treatment protocols and although approximately 20 set of standard protocols were prepared, the content of the BBP has not changed considerably. Sub-component A2. Strengthening Capacity of National Health institutions (actual total cost US$0.32 million): Satisfactory. Outputs related to the strengthening of the Communications Unit of the MOH, the Medical Library and the LHIC were generated according to LHP requirements. These activities greatly contributed to the strengthening of the institutional capacity of these institutions to implement policy reform. Institutional strengthening included staff skill development, provision of information, tools, and methodologies, and building capacity to effectively communicate with the consumers and main stakeholders of the health care system. Funds destined for study tours for the National Health Board were reallocated by the government to the financing of TA under sub-component A1, the financing requirements of which were under-estimated at project preparation. Sub-component A3. Information Management (actual total cost US$5.74 million): Satisfactory. Implementation of activities under the Health Management Information (HMIS) component started approximately two years after project effectiveness. There were various reasons for this delay: i) there was lack of clarity in the Government and the health sector in general regarding the content of the HMIS, ii) non-technical issues such as issues of confidentiality and legal issues needed to be addressed in advance of any project activities, iii) there were various institutions that demanded the control of primary data and iv) there was lack of institutional capacity to fully

30

develop the HMIS (or E-Health). Following intense discussions between the MOH, SPF, LHIC, University hospitals and health care providers, a formal working group was created in 2002 including representatives of all the above-mentioned institutions. The working group, with assistance from Swedish consultants, elaborated a strategy which was complemented by a feasibility study. In a following stage, the Concept (Core system) of the E-Health project was prepared and four functions –the basic electronic patient record, patient registration with PHC providers, referrals to secondary and tertiary level services and laboratories, and records of results- were developed. Relatively recently, in 2006, the Government also took the decision that the owner of the new system will be the MOH and the initial administrator will be SPF where the system will be physically located. The SPF will be responsible for distributing data to the users, e.g., MOH, LHIC, etc. The revolutionary element of E-Health was changing the focus from information collected having as a starting point the episode of care (e.g., no. of PHC visits, hospitalization records, etc.) to a personalized health care record which included clinical information related to the person. Further development of the HMIS (the remaining 11 functions and roll-out of the system in three regions) is currently financed through EU Structural Funds. Component B: Strengthening Capacity of National Health Institutions (actual total cost US$21.64 million): Highly Satisfactory. Activities under this component were implemented in a highly satisfactory way in all four counties (Alytus, Kaunas, Utena and Vilnius) once the PIAs were signed. This has been supported both by the excellent quality of civil works in health care facilities and the achievement of most performance targets in the PIAs. The varying level of time and difficulty for the signing of the PIAS was due to local elections in 2000 and the fact that the restructuring programs in some pilot counties were more challenging involving the merging of hospital facilities and the closing of hospital departments. Furthermore, this component had a significant demonstration effect as HSR took place in the rest of the country as well. Component C: Project Management (actual total cost US$1.13 million): Highly Satisfactory. The LPH benefited by the presence of a strong PCU, both in the MOH and the pilot regions. The PCU worked in a professional and dedicated manner in order to guarantee smooth implementation of a complex project in a volatile political environment. This had a highly positive impact on project implementation and it was acknowledged, unanimously, by Bank staff and project beneficiaries (e.g. health care providers involved in the project, employees of national health institutions, etc.) in project documents and interviews carried out during the preparation of this ICR. Project management could benefit by a more systematic and better documented monitoring of project performance indicators. Nevertheless, it needs to be acknowledged that without the valuable help of the PCU, the collection of data for the monitoring indicators ex post might not have been possible.

31

Annex 3. Economic and Financial Analysis (including assumptions in the analysis) The LHP was subject to an economic and financial analysis at preparation. Economic Analysis. The costs of the project and the expected benefits were estimated in three main areas: (i) policy development and support to national health institutions; (ii) HSR in the four apskritis that support the PHC reform and hospital capacity optimization; and (iii) total costs and benefits of the project including synergistic effects and indirect financial benefits. The costs and benefits of the restructuring component were related to: (i) changes in referral rates to hospitals and specialists; (ii) estimated benefits from expected changes in skills mix and structure of medical staff, costs and benefits of pilot PHC practices with a special focus on the shift of treatments from hospitals to PHC (including indirect benefits like reduced time-costs and travel costs); (iii) savings due to expanding day care surgery and concentration of some care and avoided duplications, benefits in terms of shorter ALOS and cuts in hospital beds; (iv) estimated changes in direct costs and benefits for restructuring mental health by the expansion of out-patient treatment; and (v) estimated costs and productivity gains due to investment in the ambulance sector (including analyzing the volume of different services provided and organizational changes). The summary project costs included also the cost for project management as well as an additional benefit from the demonstration effect and synergistic effect of coordination and implementation process (equal to 1% of public expenditure on health starting from midterm of the project). Data for the analysis was derived from domestic reports, and from ad hoc data collection and research (questionnaires, telephone interviews, etc.). The economic analysis concluded that the project as a whole would yield an estimated internal rate of return (IRR) of 41% if the indirect financial benefits were included. The indirect benefits represented 14% of total project benefits, driven primarily by shorter length-of stay in hospitals due to expansion of day-care surgery and ambulatory care. The total benefits over ten years were nearly US$94 million and the net present value (NPV) over US$49 million. Financial Analysis. The financial project analysis estimated recurrent costs linked to investments, and analyzed how this would affect total public expenditure on health. The fiscal effects were to a large extent equal to the economic costs and benefits, excluding indirect financial benefits. The financial analysis concluded that the project would yield a financial rate of return (FRR) equal to 32% and a NPV of US$34.7 million. A re-estimation of the IRR, FRR and NPV for the economic and financial analysis was not possible due to the unavailability of the original project files. As stated in the PAD, details of the economic and financial evaluations were available in project files. However, during ICR preparation, it was not possible to locate these files either in the IRIS system or in the personal files of the TTL at preparation time and the local consultants responsible for the collection of data for the analysis. Some assumptions and data were included in the PAD, but they were not sufficient for the re-estimation of the IRR and FRR and NPV. Despite the fact that re-estimation of the project’s IRR, FRR and NPV was not possible, the achievement of performance indicators indicates that project benefits identified during preparation were realized First, there was an increase in the utilization of PHC services as

32

compared to ambulatory specialist services and hospitalizations; this implied a reduction in the cost of treatment for patients who were treated by GPs and otherwise would have been treated by a specialist or at the hospital. Second, the creation of day surgery centers led to an increase in the number of day surgeries; saving resources that otherwise would have been used to cover full hospitalizations. Third, project activities resulted in a decrease in ALOS and an increase in BOR, both clear measures of efficiency. Fourth, the project reduced the cost of utilities of health care facilities by reducing excess building capacity. Finally, as demonstrated through the baseline and evaluation surveys, there were improvements in patient satisfaction in project health care facilities which were greater than the ones observed in control facilities.

33

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending

Supervision/ICR Mukesh Chawla Lead Economist AFTH3 PTL Anna Goodman Program Assistant ECSHD Team member Armin H. Fidler Sector Manager ECSHD Sector manager Dominic S. Haazen Sr Health Specialist ECSHD PTL Loraine Hawkins Sr Health Financing Specialist ECSHD PTL Vilija Kostelnickiene Sr Operations Officer ECSPS Team member Marzena Kulis Health Specialist ECSHD TTL John C. Langenbrunner Sr Economist (Health) MNSHD PTL Annie Milanzi Program Assistant MNSHD Team member Toomas Palu Senior Health Specialist ECSHD TTL Panagiota Panopoulou Economist (Health) ECSPS ICR author Pia Helene Schneider Economist (Health) AFTH3 TTL Anna L Wielogorska Sr Procurement Spec. ECSPS Team member

(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)

Stage of Project Cycle No. of staff weeks USD Thousands (including

travel and consultant costs)Lending

FY96 75.02 FY97 48.17 FY98 89.76 FY99 203.08 FY00 34 68.47 FY01 0.00 FY02 0.00 FY03 0.00 FY04 0.00 FY05 0.00 FY06 0.00 FY07 0.00

Total: 34 484.50 Supervision/ICR

FY96 0.00 FY97 0.00 FY98 0.00

34

FY99 0.81 FY00 10 50.18 FY01 31 79.48 FY02 21 34.89 FY03 31 72.31 FY04 26 62.33 FY05 23 51.18 FY06 23 52.88 FY07 13 43.47

Total: 178 447.53

35

Annex 5. Beneficiary Survey Results (if any) Two satisfaction surveys were carried out during project implementation: a baseline survey in 2001 and an evaluation survey in 2005. The surveys provided information on patients’ satisfaction with PHC institutions in the pilot counties. Two types of population groups were used; individuals who visited PHC facilities that participated in the project (project group) and individuals who visited PHC facilities that did not participate in the project (control group). Evaluation of patient satisfaction took into account issues such as time spent before seeing the physician, features of the physician visited, duration of physician’s visit, contact with medical nurse and suitability of premises. Based on the survey results, patient satisfaction with service provision in PHC ambulatories increased by 9.5% between the baseline and evaluation periods (Table 5.1). However, no changes in the level of patient satisfaction were observed for services provided in polyclinics between the two periods. When taking into account both ambulatories and polyclinics, patient satisfaction increased by 1% for the project group (individuals who visited facilities participating in the project). An increase in patient satisfaction for services provided in PHC ambulatories was also registered for the control group but it was lower compared to the one of the main group (8.7%). In the case of polyclinics, the control group registered a decrease of 5% in patient satisfaction. The main reasons for patient dissatisfaction in both project and control groups were long queues, doctors with low level of qualifications and inability to carry all tests in the facility. Table 5.1. Satisfaction with PHC services, 2001 and 2005. Level of satisfaction 2001 2005 % change Project group Polyclinics 3.73 3.67 -1.6 Ambulatories 4.01 4.39 9.5 All 3.82 3.85 0.8 Control group Polyclinics 3.65 3.46 -5.2 Ambulatories 3.8 4.13 8.7 All 3.7 3.64 -1.6 Note: 1. Respondents were asked to qualify services using a scale from 1 (absolutely dissatisfied) to 5 (very satisfied). 2. All changes are statistically significant with the exception of the project group for polyclinics. Source: Satisfaction Surveys, 2001 and 2005. Below, we provide a brief overview of various survey outcomes. This list is not exhaustive.

Increase in the number of GPs working in ambulatories, as compared to the number of Therapists and Pediatricians between the two surveys.

Increase in the frequency that patients visited PHC survey institutions (ambulatories and polyclinics) between the two surveys.

Easy access to the telephone center of PHC institutions. The situation did not changed between the baseline and evaluation surveys for both project and control groups.

Improved satisfaction with the behavior and characteristics of the personnel working in the registration counter of PHC institutions for the project group between the baseline and evaluation surveys. There was no change in the control group.

36

On average, patients waited 33 minutes to see a physician in 2005. The corresponding time in 2001 was slightly lower at 30 minutes.

Improved satisfaction with the behavior and characteristics of the physician for the project group between the two surveys; physicians in ambulatories achieved higher scores than physicians in polyclinics. There was no change in patient satisfaction with the behavior and characteristics of the physician for the control group.

Among different specialists, there was an increase in the level of satisfaction experienced when consulting a GP as compared to a Therapist or a Pediatrician between the two survey periods.

Urgent cases in the control group were referred more often to secondary specialists as compared to urgent cases in the project group.

Increase in the share of patients communicating with nurses working together with the physician, especially in PHC institutions of the project group, between the two surveys.

Increase in the level of satisfaction with the environment (interior of premises, adaptation of premises for the specific nature of the institution, clarity of directions and signs) both for the project and control groups between the two surveys.

37

Annex 6. Stakeholder Workshop Report and Results (if any) Stakeholder Workshops Various stakeholder workshops were organized under Component A. These workshops formed part of the deliverables of the two main TA contracts for consultant services. The first contract included TA on the areas of Ambulance Service Development, Hospital Care Planning (Master-planning) and Capacity Building. Results on these three areas were presented in a national conference and three workshops. The second contract included TA on Resource Allocation Formula Development, Provider Reimbursement and Contracting, Investment Allocation and Public Information. The TA also included the organization the following workshops, training courses, and seminars:

Workshop for SPF/TPFs on capacity building in modeling and statistical techniques to evaluate TPFs theoretical budgets after the application of the new formula;

Training course for SPF/TPFs and managers/economists in major providers on modernization of contracting between purchasers and providers;

Training course on how to conduct rational planning for investment and disinvestment in facilities and equipment at national and regional level, linked with service plans and norms;

Seminar for SPF/TPFs staff on organization of public information campaigns. Furthermore, two study tours were organized by consultants. The first study tour aimed at familiarizing Lithuanian specialists with the Swedish ambulance system, while the second study tour aimed at getting Lithuania experts acquainted with the Irish health information system. Participants in these tours contributed to the preparation of the Ambulance Service Strategy and HMIS.

38

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

The following report of the Government was provided to the Bank by the PCU on March 13, 2006.

EVALUATION REPORT OF LITHUANIA - HEALTH PROJECT (World Bank/Ministry of Health 2000-2006)

Following Resolution No. 193 of the Government of the Republic of Lithuania dated 22 February 2000 “Regarding borrowing 21.24 million US dollars from the International Bank for Reconstruction and Development on behalf of the State”, on 28 February 2000 a Loan Agreement was signed between the Republic of Lithuania, represented by the Ministry of Finance, and the International Bank for Reconstruction and Development (hereinafter referred to as “the Bank”) regarding a loan of 21.24 million US dollars and issuing a grant of 20 million Swedish kronas by the Government of Sweden and a grant of 149 thousand US dollars by the Government of Japan to the Republic of Lithuania for the implementation of Health Project. 1. AIM OF THE PROJECT – to improve efficiency, equity and accessibly of health care system in Lithuania. It is expected to achieve these aims by providing consultations on health reform, strengthening capacity of health care institutions, improving management of health care resources and implementing restructuring programmes for primary health care institutions and hospitals in four pilot counties (Alytus, Kaunas, Utena and Vilnius counties). 2. KEY INDICATORS OF PROJECT RESULTS

The key indicators of project results are as follows: a) more equitable distribution of health care resources among the counties; b) cost control by creating an effective system for concluding agreements between the State Patient Fund and health care service providers; c) established system for the planning and restructuring of services; d) in the four pilot counties services of qualified general practitioners are provided to at least 50 % of patients; e) in the four pilot counties the number of referrals to secondary and tertiary health care specialists as well as to hospitals cut by 20%; f) patients from the pilot counties are more satisfied with the primary health care services provided (data obtained as a result of patient interviews).

3. PROJECT COMPONENTS I. Support to health care reform;

II. Restructuring of health care services in the four counties: Alytus, Kaunas, Utena and Vilnius;

III. Management of project implementation.

Expected results of implementation of component I “Support to health care reform”: a) more effective distribution of health care resources based on territorial demographic indicators; b) clear principles for the evaluation of investment proposals and utilisation of economic resources developed; c) better cost control resulting from the introduction of an effective system of concluding agreements between patient funds and health care service providers; d) the model of needs-based service planning introduced; e) staff representing different institutions is trained on the issues of management, information management and public health policy; f) introduction and management of the information system on the level of health care suppliers upgraded.

39

Implementation of activities planned under this component will be basically funded from grants. Expected results of implementation of component II “Restructuring of health care services”: Two programmes implemented in Alytus county:

1. Primary health care development programme. 2. Hospital restructuring programme.

Total investment planned for Alytus county is 5,263.6 thousand US dollars (including 3,394.5 thousand US dollars from the World Bank loan). Three programmes implemented in Utena county:

1. Primary health care development programme. 2. Hospital restructuring programme. 3. Programme for the development of ambulance services.

Total investment planned for Utena county is 4,383.2 thousand US dollars (including 2,673.4 thousand US dollars from the World Bank loan). Five programmes implemented in Kaunas county:

1. Health education programme. 2. Primary health care development programme. 3. Hospital restructuring programme. 4. Community mental health programme. 5. Programme for the development of ambulance services.

Total investment planned for Kaunas county is 7,013.8 thousand US dollars (including 5,086.8 thousand US dollars from the World Bank loan). Two programmes implemented in Vilnius county:

1. Primary health care development programme. 2. Hospital restructuring programme.

Total investment planned for Vilnius county is 7,477.7 thousand US dollars (including 5,407.5 thousand US dollars from the World Bank loan). Total sum planned for component II is 24,138.3 thousand US dollars. Component III: “Management of project implementation” This component is aimed at strengthening the ability of the Ministry of Health to coordinate the project activities effectively, perform the monitoring of project implementation, administer procurement processes intended under the project and manage the project resources. On the national level responsibility for the project implementation would go to public institution “Health Project Coordination Unit” established under the Ministry of Health of the Republic of Lithuania. Total sum planned for this component is 1,350.6 thousand US dollars. 4. FINANCING OF THE PROJECT

• Project duration: planned - 4.5 years, actual – 6.5 years. • Anticipated general project costs – 34,172.6 thousand US dollars. • Project financing plan:

40

a) Government of Lithuania: 27% (9.2 million US dollars, allocated largely for covering VAT and other taxes related to the acquisition of goods, works and services).

b) World Bank loan: 62.1% or 21,234.9 thousand US dollars. c) Grant from the Government of Sweden: 7% or 2,397.2 thousand US dollars. d) EU-PHARE: 2.5% or 861.5 thousand US dollars. e) Other donors: 1.3% or 459.8 thousand US dollars.

• Actual financing of the project by 31 August 2006 (see table attached). 5. PROJECT-RELATED ACHIEVEMENTS

• Expanded network of primary health care services resulting from the establishment of new GP offices in rural areas (by replacing former medical points or building the offices anew) or by renovating the old ones in towns; suitable working conditions ensured (premises renovated, medical and other equipment purchased).

• Access to the services provided by qualified GPs ensured to 55% of county inhabitants. • Establishment of GP offices in Alytus policlinic and concentration of outpatient

services at Alytus hospital allowed for the consolidation of the concept of general practice in line with which GPs provide services to families without separating children from adults.

• Renovation of GP offices allowed saving about 20-30% of resources allocated for the heating of premises.

• Day surgery centres established and equipped at Kaunas 2nd clinical hospital and Alytus and Kaunas county hospitals by adjusting the existing premises or operating theatres. After the day surgery centres started functioning, the number of in-patients decreased by 20-30%, the average length of stay has significantly decreased.

• In Utena county specialized health care services falling under the main health care profiles were concentrated in three county hospitals: Utena, Anyksciai and Visaginas hospitals. Utena and Visaginas hospitals received investments for renovation, Moletai hospital – for renovation and equipment, Anyksciai hospital – only for equipment. Medical equipment in these hospitals has been concentrated in resuscitation departments and intensive care units.

• In Kaunas and Utena counties plans for the development of ambulance services were prepared and approved by regional development councils. These plans conform to the new concept of the Ministry of Health to replace 40 ambulance cars (30 in Kaunas and 10 in Utena), while fuel costs per one kilometre were cut by 10 %.

• In Kaunas county a programme for the prevention of cardio-vascular diseases was implemented. According to this programme, risk factors of the above diseases are controlled by the newly established prevention rooms equipped with the necessary medical equipment and computers.

• In all municipalities of Kaunas county mental health services were established, 11 community mental health teams were formed and provided with renovated and accommodated premises and supplied with necessary equipment.

• Vilkpede hospital (former Railway) located in Vilnius city was reconstructed and transformed from a general profile hospital into a nursing and palliative care hospital.

• Having restructured health care institutions and started using the existing premises more rationally, several buildings were abandoned and transferred from health care sector for other needs.

6. USEFUL OBSERVATIONS IN RELATION TO PROJECT IMPLEMENTATION

41

• One of the aims of the project was to provide consultations of foreign experts to the

Government of Lithuania on the reorganization of the ineffectively functioning part of health care system. The acquisition of technical assistance was to be financed from the grants issued by the governments of Sweden and Japan, however, the Government of Lithuania was supposed to choose only Swedish consultants. Therefore, the suggested models for the reorganization of health care system came from Scandinavian countries where the financing of health care system as well as traditions significantly differ from those in Lithuania. The Government of Lithuania was not provided with an opportunity to gain experience from a broader scope of countries.

• The frequent change of the World Bank’s staff (TTLs, PTLs, procurement specialists), which has been working with Lithuanian Health Project team, as well as change of Ministers of Health and executives of the Ministry impeded the implementation of the Project.

• Continuity of aspiration to project aims was safeguarded by: 1) Concluding project implementation agreements between the founders of health

care institutions (municipal councils and county manager’s administrations) and the Ministry of Health setting the conditions for programme funding based on the fulfilment of obligations undertaken by the founders with regard to the reorganization of health care institutions. For example, in case of failure to fulfil obligations, i.e. not having restructured health care institutions as provided for in the agreement or not having prepared and approved concept strategies for the implementation of the programme that would be acceptable to the World Bank, financing for the relevant programme would not be allocated, and vice versa.

Project implementation agreements also ensured continuity of implementation of Lithuania-Health Project despite the frequent change of Ministers of Health and executives of the Ministry.

2) Ensuring active involvement of final beneficiaries into project implementation and delegating responsibility by closely cooperating in preparing specifications, assessing tender proposals, accepting goods or works, carrying technical maintenance of construction – renovation works, appointing authorized representatives to workgroups with decision right, etc.

3) Managing and administrating project implementation. A separate institution authorized to act on behalf of the Ministry of Health and staffed with properly trained specialists in project implementation was able to organize project implementation and ensure monitoring of results much more effectively.

42

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders SIDA informed the Bank that it would not be able to provide comments or a contribution to this ICR due to the fact that SIDA employees who participated in the supervision of the LHP were not available or had left the organization.

43

Annex 9. List of Supporting Documents Project documents Agreement on the Implementation of the Lithuanian Health Project in the Alytus Country,

October 12, 2000, Vilnius. Agreement on the Implementation of the Lithuanian Health Project in the Utena Country,

November 2, 2000, Vilnius. Agreement on the Implementation of the Lithuanian Health Project in the Vilnius Country,

December 1, 2000, Vilnius. Agreement on the Implementation of the Lithuanian Health Project in the Kaunas Country, March

27, 2001, Kaunas. Ministry of Health. 2002. Mid-term Evaluation Progress Report of Project Management Unit,

May 2002. ──. 2001. Terms of Reference. Contract for Consultants’ Services between the MOH of the

Republic of Lithuania and AF-International/AF Swedish Management Group, July 31, 2001. ──. 2001. Terms of Reference. Contract for Consultants’ Services between the MOH of the

Republic of Lithuania and Scandinavian Care Consultant Services AB, June 28, 2001. Baltic Surveys. 2005. “Quality of Services of Primary Health Care Institutions: Opinion of

Patients Visiting Doctors. Second Wave.” Final report prepared for the LHP. ──. 2001. “Quality of Services of Primary Health Care Institutions: Opinion of Patients Visiting

Doctors.” Final report prepared for the LHP. World Bank. 2005. Lithuania Health Care Technical Assistance and Training SIDA Grant.

Implementation Completion Memorandum. September 28, 2005. Washington, DC. ──. 2002. Quality of Supervision Assessment. Lithuania Health Project. October 15, 2002.

Washington, DC. ──. 1999. Lithuania Health Project LIL. Project Appraisal Document. Report No: 19855-LT.

World Bank: Washington, DC. ──. 1999. Lithuania Health Project. Loan Agreement. World Bank: Washington, DC. Aide Memoires and Project Status Reports/Implementation Status Results and Report Project Status Report #1. December 27, 1999. Project Status Report #2. June 4, 2000. Project Status Report #3. December 22, 2000. Project Status Report #4. June 25, 2001. Project Status Report #5. December 12, 2001. Project Status Report #6. June 27, 2002. Project Status Report #7. December 27, 2002. Project Status Report #8. June 25, 2003. Project Status Report #9. December 4, 2003. Project Status Report #10. June 28, 2004. Project Status Report #11. December 22, 2004. Implementation Status and Results Report #12. May 5, 2005. Implementation Status and Results Report #13. July 27, 2006. Aide Memoire. November 27-December 1, 2000. Supervision Mission. Aide Memoire. June 22-27, 2001. Supervision Mission. Aide Memoire. January 7-11, 2002. Supervision Mission. Aide Memoire. June 3-14, 2002. Mid-Term Review Mission. Aide Memoire. February 3-7, 2003. Implementation Support Mission. Aide Memoire. March 3-5, 2003. Follow-up visit. Aide Memoire. May 21-23, 2003. Implementation Support Visit. Aide Memoire. October 22-29, 2003. Implementation Support Visit.

44

Aide Memoire. September 15-18, 2004. Supervision Mission. Aide Memoire. March 2-4, 2005. Supervision Mission. Aide Memoire. October 3-5, 2005. Supervision Mission. Aide Memoire. December 12-13, 2005. Supervision Mission. Aide Memoire. June 5-9, 2006. LHP Implementation Review and Workshop on Private-Public

Collaboration in Health. Aide Memoire. October 10-12 and October 17-20, 2006. LHP Implementation Review. Economic and Sector Work (ESW) Ministry of Health. 2006. Implementation Strategy of Health Care Reform’s Aims and Objectives.

Ministry of Health: Vilnius, Lithuania. World Bank. 2005. Managing Health Expenditures in New EU Member States: Options for

Reforms. Draft Report. June 29, 2005. World Bank: Washington, DC. ──. 2004. Lithuania Country Partnership Strategy FY05-07. Report No. 28782-LT. World Bank:

Washington, DC. ──. 2002. Lithuania Country Economic Memorandum. Report No. 25005-LT. World Bank:

Washington, DC. ──. 1999. Lithuania Country Assistance Strategy FY00-02. Report No. 19135-LT. World Bank:

Washington, DC.

45

Annex 10. List of Individuals Interviewed

1. Ms. Janina Kumpiene, Vice Minister, MOH 2. Dr. Mindaugas Plieskis, State Secretary, MOH 3. Ms. Rima Vaikiene, Secretary, MOH 4. Ms. Vita Siniciene, Health, Information Technology Division, MOH 5. Mr. Evaldas Dobravolskas, Chief Specialist, Information Technology Division, MOH 6. Ms. Jurate Sabaliene, Head, Department of International Affairs and EU Integration,

State Patient Fund 7. Mr. Vytautas Kriauza, Deputy Director for Management, State Patient Fund 8. Ms. Jolanta Izdoniene, Chief of Statistics and Planning division, State Patient Fund 9. Dr. Aldona Gaizauskiene, Director, Lithuanian Health Information Centre 10. Ms. Raimonda Janoniene, Adviser to the President, Economic and Social Policy Group,

Office of the President of the Republic of Lithuania 11. Ms. Liubove Murauskiene, Director, Training Research and Development Centre 12. Dr. Zita Ramanauskiene, Director, Vilnius Vilkpedes Hospital 13. Dr. Jonas Kairys, Director, Seskines Polyclinic, Vilniaus Municipality and Consultant to

the Prime Minister, Government of Republic of Lithuania 14. Dr. Kestutis Staras, Director, Vilniaus City Centro Polyclinic and Naujamiescio Branch 15. Dr. Audrone Rackauskiene, Deputy Director, Vilniaus City Centro Polyclinic and

Naujamiescio Branch 16. Mr. Irenijus Puotkalis, Head of IT Division , Vilniaus City Centro Polyclinic and

Naujamiescio Branch 17. Dr. Audrone Zaliene, Deputy Director for Nursing, Vilniaus City Centro Polyclinic and

Naujamiescio Branch 18. Dr. Nijole Dimsiene, Chief Doctor, Sirvintos District PHC Centre 19. Dr. Rimas Mackevicius, Director, Vilnius City Naujosios Vilnios Polyclinic 20. Dr. Julija Pliaukstiene, Director, Vilniaus Municipality Lazdynu Polyclinic 21. Dr. Dileta Marcinkeviciene, Deputy Director, Vilniaus Municipality Lazdynu Polyclinic 22. Dr. Vilija Kristina Zidoniene, Director, Vilniaus Municipality Naujininku Polyclinic 23. Dr. Kazimieras V. Asoklis, Director, Vilniaus Municipality Karoliniskiu Polyclinic 24. Dr. Antonina Cekaiskiene, Deputy Director, Vilniaus Municipality Karoliniskiu

Polyclinic 25. Dr. Daiva Pentiokiniene, Director, PIU, MOH 26. Ms. Ingrida Dudoniene, Administrator, PIU, MOH 27. Ms. Grazvyda Smailyte, Chief Accountant, PIU, MOH 28. Dr. Toomas Palu, Senior Health Specialist, EASHD, World Bank (TTL for preparation

period and implementation period December 1999-December 2000) 29. Ms. Marzena Kulis, Customer Director, Governmental Affairs and Market Access, Pfizer

Poland (TTL for the period June 2002-May 2005). 30. Ms. Pia Helene Schneider, Economist (Health), ECSHD, World Bank (TTL for the

period June 2005-September 2006). 31. Ms. Vilija Kostenickiene, Senior Operations Officer, ECSPS, World Bank (Operations

Officer during the preparation period and implementation period December 1999-January 2006).