public disclosure authorized - world bankdocuments.worldbank.org/curated/en/... · anc ante-natal...

Document of The World Bank

Report No: ICR00003502

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-77170)

ON A

LOAN

IN THE AMOUNT OF EURO 56.10 MILLION (US$ 75.12 MILLION EQUIVALENT)

TO THE REPUBLIC OF TURKEY

FOR A

PROJECT IN SUPPORT OF RESTRUCTURING OF HEALTH SECTOR IN SUPPORT OF THE SECOND PHASE OF THE HEALTH TRANSFORMATION

PROGRAM (APL 2)

January 28, 2016

Health, Nutrition, and Population Global Practice Turkey Country Unit Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective September 30, 2015)

Currency Unit = Turkish Lira US$ 1.12 = 1.00 EUR TRY 3.40 = 1.00 EUR

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

ANC Ante-Natal Care MoD Ministry of Development APL Adaptable Program Lending MoH Ministry of Health CHC Community Health Centres MoH-

GDPA Ministry of Finance General Directorate of Public Accounts` Office at Ministry of Health

CPS Country Partnership Strategy MTR Mid-Term Review CRD Chronic Respiratory Diseases NCD Non-Communicable Diseases CSDPL Competitiveness and Savings Development Policy Loan NICE National Institute for Health and Clinical Excellence CVD Cardiovascular Disease NNMR Neonatal Mortality Rate DEA Data Envelopment Analysis OBF Output-Based Financing DRGs Diagnostic Related Groups PAD Project Appraisal Document EU European Union PDO Project Development Objective FM Family Medicine P4P Pay for Performance FMIS Family Medicine Information System PHC Primary Healthcare GATS Global Adult Tobacco Survey PHeI Public Health Institution GDHR General Directorate of Health Research PHoI Public Hospital Institution GoT Government of Turkey PMDA Pharmaceutical and Medical Devices Agency HIS Health Information System PMSU Project Management and Support Unit HT Hypertension PPDPL Programmatic Public Sector Development Policy

Loan HTA Health Technology Assessment PPP Public-Private Partnership Program Health Transformation Program PSRHS Project In Support of Restructuring of Health Sector HTP Health Transition Project QER Quality Enhancement Review HTSSRP The Health Transformation and Social Security Reform

Project SII Social Insurance Institution

IASC Inter-Agency Steering Committee SPO State Planning Organization IMR Infant Mortality Rate SSI Social Security Institution IOI Intermediate Outcome Indicator SuTP Syrians Under Temporary Protection ISDS Integrated Safeguards Data Sheet TTL Task Team Leader ISR Implementation Status and Results Report TUIK Turkish Statistical Institute LA Loan Agreement TUSAK Turkish Institute for Health M&E Monitoring and Evaluation TUSEB Turkish Institutes of Health Sciences MCH Mother and Child Health U5MR Under-Five Mortality Rate MDG Millennium Development Goal UHI Universal Health Insurance MEDULA Integrated Claims and Utilisation Management System UNICEF United Nations Children's Fund MICs Middle Income Countries WHO World Health Organization MMR Maternal Mortality Ratio

Vice President: Cyril E. Muller Senior Global Practice Director: Timothy Grant Evans

Country Director: Johannes Zutt Practice Manager: Enis Barış

ICR Task Team Leader & Author: Ahmet Levent Yener

TURKEY Project In Support Of Restructuring of Health Sector

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ................................................... 12. Key Factors Affecting Implementation and Outcomes .................................................. 73. Assessment of Outcomes .............................................................................................. 134. Assessment of Risk to Development Outcome............................................................. 205. Assessment of Bank and Borrower Performance ......................................................... 216. Lessons Learned............................................................................................................ 237. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 25Annex 1. Project Costs and Financing .............................................................................. 26 Annex 2. Outputs by Component...................................................................................... 27 Annex 3. Economic and Financial Analysis ..................................................................... 32 Annex 4. Bank Lending and Implementation Support/Supervision Processes................. 38 Annex 5. Beneficiary Survey Results ............................................................................... 40 Annex 6. Stakeholder Workshop Report and Results....................................................... 41 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 42 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 54 Annex 9. List of Supporting Documents .......................................................................... 55 Annex 10. Assessment of Project Achievement towards Outcomes ................................ 56 Annex 11. Strategic Map for Turkish Health System ....................................................... 72

MAP IBRD 75

Data sheet

A. Basic Information

Country: Turkey Project Name: Project in Support of Restructuring of Health Sector

Project ID: P102172 L/C/TF Number(s): IBRD-77170 ICR Date: 01/28/2016 ICR Type: Core ICR

Lending Instrument: APL Borrower: MINISTRY OF FINANCE / TREASURY

Original Total Commitment:

USD 75.12M equivalent

Disbursed Amount: USD 66.06M equivalent1

Revised Amount: USD 71.81M equivalent

Environmental Category: C Implementing Agencies: Ministry of Health, Social Security Institution Cofinanciers and Other External Partners: - B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 05/08/2008 Effectiveness: 09/29/2009 09/29/2009

Appraisal: 01/13/2009 Restructuring(s): 03/26/2013 01/07/2014 05/28/2015

Approval: 06/11/2009 Mid-term Review: 10/24/2011 Closing: 07/31/2013 09/30/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Unsatisfactory

1 Includes EUR 106,000.43 paid after September 30, 2015

i

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Unsatisfactory

Implementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance:


Overall Borrower Performance:


C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Compulsory health finance 5 1 Health 17 24 Public administration- Health 78 75

Theme Code (as % of total Bank financing) Administrative and civil service reform 12 Health system performance 64 75 Injuries and non-communicable diseases 13 12 Other communicable diseases 8 7 Public expenditure, financial management and procurement

3 6

E. Bank Staff

Positions At ICR At Approval Vice President: Cyril E Muller Shigeo Katsu Country Director: Johannes C.M. Zutt Ulrich Zachau Practice Manager/Manager:

Enis Barış Tamar Manuelyan Atınç

ii

Project Team Leader: Claudia Rokx Sarbani Chakraborty ICR Team Leader: Ahmet Levent Yener ICR Primary Author: Ahmet Levent Yener F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The specific objectives of the Project were (i) increasing the effectiveness of the Ministry of Health (MoH) and the Social Security Institute (SSI) in formulating and implementing reforms in provider payments and health systems performance; and (ii) piloting output-based financing for non-communicable diseases (NCD) prevention and control. Revised Project Development Objectives (as approved by original approving authority) The specific objective of the Project is increasing the effectiveness of the Ministry of Health (MoH) in formulating and implementing reforms in provider payments and health systems performance. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Increase in the utilization of primary health care Value quantitative or Qualitative)

2 visits per capita 4 visits per capita

Date achieved 05/31/2009 07/31/2013

Comments (incl. % achievement)

Replaced with Per capita annual visits to family medicine physicians with the restructuring in March 2013 to address the need to evaluate the utilization of primary care services which are provided by family medicine physicians only. Also, as stated above, the lack of the mandatory referral system makes it difficult to collect relevant data for the original indicator.

Indicator 2 : Increase in the percentage of first-time visits to specialists that are referred by family medicine practices

Value quantitative or Qualitative)

TBD TBD

Date achieved 05/31/2009 07/31/2013


Dropped with the restructuring in March 2013 as MoH could not provide the breakdown of the first time visits to specialists. The lack of a mandatory referral system nationwide and the disconnection between the Family Medicine Information System (FMIS) and the Health-Net made the monitoring of this indicator difficult.

Indicator 3 : Increased client satisfaction with health services Value quantitative or 57% 76%

iii

Qualitative) Date achieved 05/31/2009 07/31/2013


Replaced with Client satisfaction with health services with the restructuring in March 2013. As the annual changes in the client satisfaction might vary, it became difficult to track the trend by comparing year to year increases. This PDO indicator was revised to track the overall trend in the client satisfaction.

Indicator 4 : Smoking prevalence among 18-29 age group in Output-Based Financing pilot provinces decreases more relative to non-pilot provinces


Tobacco use knowledge, attitude and practices (KAP) survey to be implemented in 2009

10% decrease in pilot provinces

Date achieved 05/31/2009 07/31/2013 Comments (incl. % achievement)

Dropped with the cancellation of component B with the restructuring in January 2014

Indicator 5 : High blood pressure in population in Output-Based-Financing pilot provinces decreases more as compared to non-pilot provinces re


Baseline to be carried out in 2009 15% change


Dropped with the cancellation of component B with the restructuring in January 2014

Indicator 6 : Decrease in SSI expenditures on pharmaceuticals and outpatient health care


40% expenditures on pharmaceuticals 50% of treatment cost on outpatient

30% 40%


Dropped with the cancellation of component C with the restructuring in January 2014

Indicator 7 : Per-capita annual visits to family medicine physicians (Number, Custom) Value quantitative or Qualitative)

2.00 4.00 2.76

Date achieved 05/31/2009 05/31/2015 09/30/2015


This indicator was introduced as an alternative to indicators that was dropped under the restructuring in March 2013 as it is more measurable and a better indication of the effectiveness of the reforms. It was not achieved. This could be attributed to inadequate increase on the supply side, which could not keep up with the increase from the general demand for care. Since 2013, there has not been a significant increase in the number of family physicians which also exacerbated the ever increasing workload of family medicine.

Indicator 8 : Share of family medicine visits in total visits (Percentage, Custom) Value 35 40 33.35

iv

quantitative or Qualitative) Date achieved 05/31/2009 05/31/2015 09/30/2015


This indicator was added during the restructuring in March 2013. It was not achieved. Though the share of FM visits in total visits increased between 2009 and 2011, and the end-line target was almost met in 2011, the trend turned downwards starting from 2012. Survey findings suggest that more patients prefer public hospitals than family medicine centers since: (1) they are more satisfied with services provided in public hospitals; (2) visiting public hospitals is a necessity for them and (3) co-payments are low.

Indicator 9 : Client satisfaction with health services (Percentage, Custom) Value quantitative or Qualitative)

63.4 76 71.2

Date achieved 05/31/2009 05/31/2015 09/30/2015


This indicator was revised during the restructuring in March 2013. It was not achieved. Turkish Statistical Institute's Life Satisfaction Survey of 2014 indicated the main problems in the health service provision as perceived by the patients are: (1) the existence of co-payments for medical examinations; (2) the existence of medical test and laboratory fees; (3) insufficient number of physicians and health personnel; (4) medicine prices; and (5) waiting time for medical examination and medical tests.

Indicator 10 : Ministry of Health reorganized and focus exclusively on the health sector stewardship function (Text, Custom)


MOH is a provider of health services

MOH is restructured and no longer a provider of health services

MOH is restructured and no longer a provider of health services

Date achieved 05/31/2009 05/31/2015 09/30/2015


Added with the restructuring in March 2013. It was achieved. With the Decree Law on the Organization and Duties of the Ministry of Health and Its Affiliates dated November 3, 2011 the legal base was set for the restructuring and the central Ministry administration was established. The establishment of the affiliated agencies was completed within a year following the enactment of the Decree Law.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of family medicine practices that have completed second level training as compared with baseline

Value (quantitative or Qualitative)

4% 70% 70% 87%

Date achieved 05/31/2009 07/31/2013 05/31/2015 09/30/2015

v


Original indicator maintained. It was achieved. The second level training of 17,887 out of 20,656 family medicine physicians was completed in 2015.

Indicator 2 : Annual performance reports on implementation of MoH Strategic Plan using information from MOH Monitoring and Evaluation system disseminated by MOH to Parliament and published on MOH website


No program Annual report submitted Yes Yes

Date achieved 05/31/2009 07/31/2013 05/31/2015 09/30/2015


Revised as MOH Annual performance program prepared and submitted to the National Assembly for clarification during the restructuring in March 2013. It was achieved. Annual performance program reports were being submitted to the Parliament in November of each year.

Indicator 3 : Percentage of acute care public hospitals using revised clinical guidelines for secondary care as requirement of performance payment system


TBD TBD


Dropped during the restructuring in March 2013. No explanation was provided in the restructuring paper.

Indicator 4 : Dissemination of bi-annual Program Tracking reports by M&E Directorate of MOH


Not available 1 bi-annual report



Indicator 5 : Percentage of family physicians using rational drug prescription guidelines Value (quantitative or Qualitative)

Not available 30% increase



Indicator 6 : Percentage of MoH Implementing Units for Project have formed Project Offices and connected to PMSU-Net for Project monitoring


0 100%

Date achieved 05/31/2009 07/31/2013 Comments (incl. %


vi

achievement)

Indicator 7 : Number of provinces implementing family medicine scheme (Number, Custom)


31 81 81

Date achieved 05/31/2009 07/31/2013 09/30/2015 Comments (incl. % achievement)

Added during the restructuring in March 2013. It was achieved, with nationwide implementation of family medicine scheme.

Indicator 8 : Establishment of Public Health Institution responsible for health promotion, disease prevention and family medicine


Public Health Institution not in existence

Public Health Institution fully operational



Added during the restructuring in March 2013. It was achieved. Public Health Institution was established with the Decree Law No. 663.

Indicator 9 : Share of public hospitals organized in public hospital unions with performance contracts for managers and global budgets


0% 100% 100%


Added during the restructuring in March 2013. It was achieved. The organizational arrangements in public hospital unions were completed in line with the Decree Law No.663.

Indicator 10 : Adoption of National Action Plans for (i) Health Promotion, (ii) Healthy Aging, (iii) Diabetes, (iv) Obesity, (v) Cardiovascular Diseases (CVD), (vi) Chronic Respiratory Diseases (CRD)


4 6 6

Date achieved 05/31/2009 05/31/2015 09/30/2015


Added during the restructuring in March 2013. The indicator is introduced to monitor linkages between activities such as consultancies and final influence on policies. Wording was changed in January 2014, the word adoption has been replaced by preparation and submission which seemed more realistic in the given timespan. It was achieved. Actions for obesity control are incorporated within the action plan for diabetes.

Indicator 11 : Development of the Clinical Guideline Preparation Manual


No Clinical Guideline Preparation Manual exists

Clinical Guideline Preparation Manual prepared and disseminated

Clinical Guideline Preparation Manual prepared and disseminated

Date achieved 05/31/2009 05/31/2015 09/30/2015 Comments Added during the restructuring in March 2013. It was achieved.

vii

(incl. % achievement) Indicator 12 : Establishment of the Pharmaceuticals and Medical Devices Agency of Turkey


No Pharmaceuticals and Medical Devices Agency of Turkey in existence

Agency established and responsible for licensing and regulation of drugs and medical devices

Agency established and responsible for licensing and regulation of drugs and medical devices


Added during the restructuring in March 2013. It was achieved. The Pharmaceuticals and Medical Devices Agency of Turkey was established with the Decree Law No.663.

Indicator 13 : Number of international conferences to share Turkey's health reform experience (Number, Custom)


0 2 3


Added during the restructuring in March 2013. It was surpassed. Three international conferences on health reform were financed in June 2013, February 2015, and May 2015.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 08/25/2009 Satisfactory Satisfactory 0.00 2 04/19/2010 Satisfactory Satisfactory 5.92 3 11/27/2010 Satisfactory Satisfactory 12.93 4 07/05/2011 Satisfactory Satisfactory 19.63 5 01/16/2012 Satisfactory Satisfactory 25.24 6 10/15/2012 Satisfactory Moderately Satisfactory 30.77

7 03/31/2013 Moderately Satisfactory Moderately Unsatisfactory 36.94

8 12/04/2013 Moderately Satisfactory Moderately Unsatisfactory 46.66

9 06/28/2014 Moderately Satisfactory Moderately Satisfactory 53.63 10 12/24/2014 Moderately Satisfactory Moderately Satisfactory 60.09 11 06/23/2015 Moderately Satisfactory Moderately Satisfactory 64.12

12 09/30/2015 Moderately Unsatisfactory Moderately Satisfactory 65.94

viii

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at Restructuring

in USD millions

Reason for Restructuring & Key Changes Made DO IP

03/26/2013 N S MS 36.94

Level 2 restructuring: (i) project components A.3 and A.4 were revised to reflect the changes in the organizational structure of the Ministry of Health, and a new sub-component A.9 was included to share Turley’s experiences in the health sector; (ii) revision of the Results Framework (PDO and Intermediate Outcome indicators as explained in Section F); (iii) extension of the Loan Closing Date by 22 months to May 31, 2015; and (iv) change of the Project name from Health Transformation and Social Security Reform Project (HTSSRP) to Project in Support of Restructuring of Health Sector (PSRHS).

01/07/2014 Y MS MU 49.12

Level 1 restructuring: (i) revision of the PDO in line with the cancellation of Component B and partial cancellation of Component C; (ii) revision of the Results Framework (dropped indicators as explained in Section F); (iii) cancellation of Component B; (iv) partial cancellation of Component C (Capacity Building of the Social Security Institution); (v) reallocation of the funds from Component B to Component A and partial cancellation of the funds of Component C; and (vi) reallocation of funds between disbursement categories.

05/28/2015 N MS MS 63.05

Level 2 restructuring: (i) extension of the Loan Closing Date by 4 months from May 31, 2015, to September 30, 2015 in

ix

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at Restructuring

in USD millions

Reason for Restructuring & Key Changes Made

DO IP

order to complete all activities planned under Component A, maintain critical fiduciary and operational capacity and institutional memory until the expected effectiveness of the new Health Systems Strengthening and Support Project

If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Unsatisfactory Against Formally Revised PDO/Targets Moderately Unsatisfactory Overall (weighted) rating Moderately Unsatisfactory

I. Disbursement Profile

x

1. Project Context, Development Objectives and Design

1. The Health Transformation and Social Security Reform Project (HTSSRP), which wasrenamed in March 2013 as the Project in Support of Restructuring of Health Sector (PSRHS) was an integral part of the Government of Turkey’s (GoT) implementation of the Health Transformation Program (Program) covering the period 2003 to 2013. 2 The Program’s overall objective was to improve the governance, efficiency, user and provider satisfaction and long-term fiscal sustainability of the Turkish health care system.

2. The original loan was approved on June 11, 2009 in the amount of Euro 56.10 million(USD 75.12 million equivalent), and the Loan Agreement (LA) became effective on September 29, 2009. The Project was the second phase of an Adaptable Program Lending (APL) of the World Bank’s support to the Program. The first phase (APL 1) which was implemented between 2004 and 2009 through the Health Transition Project (HTP - P074053), supported the introduction of the family medicine model, and helped build capacity of the Ministry of Health (MoH) and the Social Security Institution3 (SSI) to expand health insurance coverage and to improve family medicine service delivery. The total amount for the APL 1 was Euro 49.40 million (US$60.60 million equivalent). The HTP was restructured following the mid-term review (MTR) in September 2006 and was closed on June 30, 2009.4 The second phase continued under the Project to support broad reforms in the sector aimed at increasing hospital autonomy, expanding family medicine services, and further strengthening performance management and pay-for-performance initiatives.

1.1 Context at Appraisal

3. Country and Sector background. Turkey was adversely affected by the globalfinancial crisis in 2008. However, the structural progress observed in the financial and real sectors since 2001 and prevailing economic and fiscal policies helped the country to appropriately address the downturn. Social security and universal health insurance reforms were helpful in protecting the vulnerable population during this economic downturn, while maintaining fiscal sustainability.

4. Main Health Sector Issues. By the time of appraisal in 2009, while key healthoutcomes (such as life expectancy at birth, maternal mortality ratio, and infant mortality rate) were showing steady progress and Turkey was performing better than the upper middle income countries such as Mexico and Brazil, Turkey continued to lag behind in international comparisons with OECD. Communicable diseases were the major contributor

2 The Program was a ten-year health reform program aimed at addressing the major challenges in the Turkish health sector. The Program was included in the GoT’s Ninth Development Plan and formed the basis for the Ministry of Health’s Strategic Plan for 2010-2014 with an accompanying performance-based budget. 3 The UHI Law of 2008 unified the previously fragmented social insurance schemes (active and retired civil servants, blue- and white-collar workers in the public and private sectors, and the self-employed, as well as green card holders) under a single institution, the Social Security Institution (SSI). 4 Implementation Completion and Results Report, Report No. ICR780

1

to Turkey’s burden of disease, but Turkey started facing the challenges of the epidemiological transition, with non-communicable diseases (NCD) increasingly taking centre stage. The health reforms targeted institutional and organizational changes to eliminate fragmentation and duplication in the health financing and delivery systems and to assure universal access to health insurance and health services. 5. Country Partnership Strategy and Rationale for Bank involvement. The Bank was and remains an important source of support for policy and system reform in the Turkey health sector. At appraisal, the Project5 was consistent with the Turkey Country Partnership Strategy (CPS).6 The CPS development objective for the health sector fully supported the GoT’s Ninth Development Plan (2007-2013) whose main health sector-related objective was to “make the health sector effective.” The Project was supportive of Pillar II of the CPS (Equitable human and social development). One of the expected results under Pillar II was “The health care system has improved governance, efficiency, user and provider satisfaction and long-term sustainability”. Support to social security reforms and continuation of support to the Government’s Health Transformation Program were identified in the CPS as key priorities during the CPS period. The operation was expected to directly contribute to achieving these priorities.

6. From the GoT’s perspective, the Bank’s global knowledge and technical experience in health reforms and institutional development made it an important partner for the implementation of the Program. Monitoring and evaluation of these reforms, with strong emphasis on results, was considered as instrumental for achieving sustainability. The Bank was uniquely positioned to contribute to this and the continued implementation of the Program.

1.2 Original Project Development Objectives (PDO) and Key Indicators 7. The Project Appraisal Document (PAD) and the LA listed the original PDO as: (i) increasing the effectiveness of the MoH and the SSI in formulating and implementing reforms in provider payments and health systems performance; and (ii) piloting output-based financing for NCD prevention and control. 8. Key indicators were as follows: (i) increase in the utilization of primary health care; (ii) increase in the percentage of first-time visits to specialists that are referred by family medicine practices; (iii) increase in client satisfaction with health services; (iv) decrease in SSI expenditures on pharmaceuticals and outpatient health care; (v) more decrease in smoking prevalence among 18-29 age group in Output-Based Financing pilot provinces relative to non-pilot provinces; and (vi) more decrease in high blood pressure in population in Output-Based-Financing pilot provinces as compared to non-pilot provinces.

5 Project Appraisal Document, Report No. 46115-TR 6 Country Partnership Strategy, Report No. 42026-TR

2

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 9. A Level 1 Project Restructuring, approved by the Board in January 2014,7 revised the PDO as follows: “to increase the effectiveness of the MoH in formulating and implementing reforms in provider payments and health systems performance”. In the original Project design, the output-based financing scheme was envisaged as a pilot and highlighted as such in the original PDO. In accordance with the MoH’s decision not to pilot this option using Project funds but to implement it nationwide, this part of the PDO was no longer relevant and, therefore, removed. References to SSI were also removed in the PDO as the Borrower had decided to carry out activities originally planned under Component C (Capacity building of the Social Security Institution) using its own funds. 10. While the PDO was revised in January 2014, the Results Framework was revised twice: once in March 2013 with a Level 2 Project Restructuring,8 and at the time of the Level 1 Project Restructuring in January 2014. In March 2013, one PDO indicator was dropped, two were revised, and two new indicators were added. With the March 2013 restructur ing four Intermediate Outcome Indicator (IOI) were dropped, one was revised and seven new IOIs were added. March 2013 restructuring covers only the changes related with Component A. In January 2014, three PDO indicators and seven IOIs that were related with Component B and C were dropped. Annex 10 presents an assessment of Project achievements towards outcomes, including information on the Project’s original and revised Key Performance Indicators (KPIs), the rationale for the revisions, and targets value at Closing.

1.4 Main Beneficiaries 11. Relative to the APL 1, the APL 2 had a wider scope of beneficiaries on both the demand and supply side (in addition to the MoH and SSI). The main beneficiary was the Turkish population for whom the Project aimed to improve access to health care and their health. Women and children were of special focus as part of the performance-based activities of the family medicine scheme. Among the general population, people with low socio-economic status and living in geographically remote areas remained as a priority to benefit from increased Primary Healthcare (PHC) coverage through the scaling up of family medicine. On the supply side, health providers and the MoH staff were the beneficiaries of capacity building interventions (training, knowledge sharing activities, seminars etc.) to improve the quality and efficiency of health care delivery and strengthen the MoH’s capacity in effective policy formulation and implementation.

1.5 Original Components 12. The Project had three main components as described below:

7 Restructuring Paper (Report No:RES12001) Board approval: January 7, 2014 8 Restructuring Paper (Report No:RES9770) dated March 26, 2013

3

Component A: Support for MoH’s Strategic Plan for 2010-2014 (EUR 46.17 million): to build the capacity of the MoH to become an effective steward of the health sector and complete service delivery reforms. This component included eight sub-components as follows: A.1. Strengthening preventive and primary health care services (EUR 13.12 million): to scale-up the family medicine program, and support the MoH in implementing preventive health care services for the control of communicable and non-communicable diseases. A.2. Implementing public hospital reforms (EUR 2.48 million): to support the implementation of hospital reforms, to pilot and implement nation-wide the Public Hospital Union model, which would transform MOH hospitals in Turkey into public enterprises. A.3. Improving the quality of health services (EUR 1.58 million): to build the capacity of the MoH’s Performance Management Department and enhancing its role in the implementation of pay-for-performance (P4P) systems in Turkey and regulate health providers with the objective of ensuring good quality health services. A4. Institutional restructuring and capacity building (EUR 10.46 million): to support the restructuring of the MoH, and to support the strengthening of strategic management capacity to implement strategic financing methods in the field of performance-based budgeting and ensuring standardization, continuity and coordination in specialty and in-service training programs. A.5. Managing pharmaceutical and medical devices regulation (EUR 0.59 million): to assist the MoH in implementing the National Medicine Policy, which aimed at ensuring rational drug use, increasing the quality of medicine, vaccines and biological products, implementing institutional arrangements for assuring Good Clinical Practices and Good Manufacturing Practices, and increasing capacity to implement market surveillance and inspection. A.6. Improving health information systems (EUR 8.63 million): to support the expansion of the MoH National Health Information System (HIS) in order to (i) improve health informatics standards, confidentiality, security and privacy principles of personal and institutional health records, (ii) establish a data warehouse within the scope of decision support system; and (iii) initiate data mining practices to establish tele-medicine and tele-health systems. A.7. Supporting inter-sectoral health responsibility implementation of the Government’s Health Transformation Program and aligning health policies with world standards (EUR 5.50 million): to: (i) assist the MoH implement inter-sectoral cooperation and aligning health policies with international standards, especially European Union (EU) guidelines and standards; (ii) strengthen the capacity of the Project Management and Support Unit (PMSU) to efficiently coordinate the use of international financing/donor resources in the health sector; and (iii) build capacity within the “Public-Private Partnerships (PPP) Unit of MoH”.

4

A.8. Establishing a National Public Health Agency (EUR 3.81 million): to convert the Refik Saydam Hygiene Center Presidency into the National Public Health Agency responsible for all key public health functions including: disease surveillance, outbreak management, development of policies and plans related to public health, and public health regulation. Component B: Piloting Output-Based Financing for Non-communicable Diseases (NCD) Prevention and Control (EUR 6.84 million): to pilot test and evaluate a model for implementing preventive health care interventions for NCD at the provincial level using output-based financing mechanisms. This component consisted of two sub-components: B.1. Output-based financing for family medicine practices (EUR 2.99 million): to pilot implementation and an impact evaluation of an output-based financing mechanism for family medicine practices in two provinces of Turkey. B.2. Capacity-building for provincial and central levels implementation of output-based financing pilot: (EUR 3.85 million): to build capacity in relevant institutions at the provincial and central MoH levels to make the output-based financing for family medicine practices a success. Component C: Capacity building of the Social Security Institution (EUR 3.09 million): to enhance the Social Security Institution’s (SSI) capacities for the implementation and promotion of UHI so that SSI becomes a strategic purchaser in the health sector and makes evidence-based decisions vis-à-vis the implementation of provider payment reforms, public and private provider contracts and continuous update of the UHI benefits package. This component consisted of two sub-components. C.1 SSI capacity-building for the implementation and promotion of UHI (EUR 2.77 million): technical assistance on: (i) provider payment systems, enhanced contracting mechanisms and pay-for-performance systems; (ii) implementing the Integrated Claims and Utilization Management System (MEDULA) and strengthening the relationship between availability of claims data and using the information to guide policy and program decisions under UHI; (iii) development of capacity in producing evidence-based information for the SSI Reimbursement Commission on clinical guidelines, cost-effective health, pharmaceutical and medical technology interventions such as through the development of Health Technology Assessment (HTA) capacity in SSI. C.2. SSI Project management (EUR 0.32 million): to finance the Project Implementa t io n Unit (PIU) for effective Project management and implementation coordination.

5

1.6 Revised Components 13. The Project was restructured three times, in March 2013 (Level 2), January 2014 (Level 1), and May 20159 (Level 2). Project components were revised through the Level 2 Project Restructuring of March 2013: (i) sub-components A.3 and A.4 were revised in line with the organizational restructuring according to the Decree Law no. 663; and (ii) sub-component A.9 was introduced to strengthen collaboration with other countries and international institutions in health policy development and to share Turkey’s experiences. Component B was cancelled with the January 2104 restructuring as a result of the MoH’s decision to roll out the model nationwide hence making the pilot no longer relevant. The MoH later indicated that the job description of family physicians already included NCD prevention and control activities. No funds were spent under this component. As the output-based financing (OBF) was being rolled out, some funds originally planned for support to implementation and monitoring of the activities were reallocated to Component A. In addition, with the January 2014 restructuring the undisbursed funds under sub-component A.5 were reallocated within Component A as the newly established Pharmaceutical and Medical Devices Agency will not be able to disburse funds due to its new budget structure. Component C was partially cancelled as some funds were already spent during implementation; the undisbursed funds were cancelled from the loan. Details on the revisions made as part of these restructurings are presented in Annex 10.

1.7 Other significant changes 14. In addition to changes above, the following changes were also made under the three restructurings: 1st Restructuring: (i) extension of the Loan Closing Date by 22 months from July 31, 2013 to May 31, 2015; and (ii) change of the Project name from “Health Transformation and Social Security Reform Project (HTSSRP)” to “Project in Support of Restructuring of Health Sector (PSRHS)”. 2nd Restructuring: reallocation of funds between disbursement categories. 3rd Restructuring201510: (i) a second and final extension of the Loan Closing Date by 4 months from May 31, 2015, to September 30, 2015 in order to complete all activit ies planned under Component A, and maintain critical fiduciary as well as operational capacity and institutional memory until the expected effectiveness of the new Health Systems Strengthening and Support Project (Board approval September 21, 2015).11

9 Restructuring Paper (Report No:RES19050-TR) dated May 28, 2015 11 Health Systems Strengthening and Support Project (P152799) became effective on November 26, 2015.

6

Table 1: Reallocation of Loan Proceeds (January 2014) Category Original Amount of the

Loan Allocated (Expres s ed in EUR)

Revised Amount of the Loan Allocated (Expres s ed in EUR)

(1) Goods and Technical Services under Part A of the Project

8,810,000 2,406,000

(2) Consultants’ Services (a) under Part A (b) under Part C

23,430,000

2,410,000

29,601,000

398,922 (3) Training and Workshops (a) Under Part A (b) Under Part C

17,010,000

600,000

20,346,000

212,896 (4) Capitation Payments under Part B.1 (Part B was cancelled)

2,990,000 0

(5) Operating Costs (a) Under Part A (b) Under Part C

770,000 80,000

657,000

2,591 Amount cancelled as of 09/25/2012* 2,475,591 Total Amount 56,100,000 56,100,000

* The cancellation date was recorded as September 25, 2012 in line with the request of the Borrower to match the official cancellation date with the internal records of the implementing agency. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 15. Soundness of background analysis was satisfactory. In addition to the information from global best practices in health reforms, the Project design duly incorporated the findings of studies conducted prior to Project start. These included the Burden of Disease Study (2006), the Joint Portfolio Performance Review for Turkey (2007), OECD-World Bank Review of the Health Sector in Turkey, and Program (2008).12 The Project also benefitted from in-depth background analysis conducted under the Program, which evaluated many key aspects of Turkey’s health system 13 , including health system resources, health financing, health service delivery and organizational framework. 16. Assessment of the Project design. The Project’s design was adequate to support ongoing health reforms in Turkey. Considering the long term nature of the health reforms, the choice of a two-phase APL was also adequate. Project components were designed to support continued health reform, mainly along the lines of strengthening PHC (scale-up of the family medicine model and addressing NCD), continuing public hospital reforms, strengthening Health Management Information System (HMIS), and building institutiona l capacity. While it was not a novelty for the country, the output-based financing for NCD in primary care could be considered as an important effort to expand results-based

12 http://www.oecd-ilibrary.org/social-issues-migration-health/oecd-reviews-of-health-systems-turkey-2008_9789264051096-en http://ekutuphane.sagem.gov.tr/kitaplar/turkey_burden_of_disease_study.pdf http://siteresources.worldbank.org/INTTURKEY/Resources/2007_JPPR-Treasuary_Presentation.pdf 13 World Bank, 2003, Turkey: Reforming the Health Sector for Improved Access and Efficiency, Report # 24358-TU.

7

http://www.oecd-ilibrary.org/social-issues-migration-health/oecd-reviews-of-health-systems-turkey-2008_9789264051096-en

http://www.oecd-ilibrary.org/social-issues-migration-health/oecd-reviews-of-health-systems-turkey-2008_9789264051096-en

http://ekutuphane.sagem.gov.tr/kitaplar/turkey_burden_of_disease_study.pdf

http://siteresources.worldbank.org/INTTURKEY/Resources/2007_JPPR-Treasuary_Presentation.pdf

financing (RBF). While the PDO and the envisaged components are deemed to be highly relevant and internally coherent to further advance the implementation of the on-going HTP, a certain degree of disconnect remains between thee components and the activit ies on one hand, and the PDO and the indicators on the other hand, which became all the more prominent after the restructuring (of the PDO) and elimination of some of its indicators. More specifically, the project PDO was all about strengthening MOH’s capacity to become a truly effective steward of the health system, yet many of the activities undertaken under the Component A were mostly in relation to strengthening quality of service delivery at the primary care level, as reflected in the set of PDO and IOI focusing on either inputs (training), outputs (the number of visits) or outcomes (patient satisfaction) in conjunction with service delivery.. Most of the activities under the project were training programs or consultancies, having indirect effect on the broader outcomes (i.e. utilization, client satisfaction). The relation between project activities and project objectives from a health system functions perspective has been elaborated in detail in Annex 11. 17. The QER of October 2008 indicated the need for PAD to provide more detail on readiness in terms of how technical inputs would be identified to work with the client. QER also noted to incorporate these under implementation arrangements which PAD did, describing how each component will be implemented by the responsible institution. 18. Adequacy of the Government’s commitment. Strong alignment of project design with the Program, higher MoH policy documents and strategic plans contributed to the continuation of MoH’s ownership. The Program formed the basis for the MoH’s Strategic Plan and the accompanying performance-based budget. The Project was developed in a way to duplicate the successful design of the previous Project that supported the Program to ensure continuation of the GoT’s ownership and support. 19. Assessment of risks. The discussion on the risk identification and mitigation measures included the risk pertaining to the legal amendments necessary to implement OBF component as this was rated as substantial. The mitigation measures indicated possible legal changes at different levels with the probability of occurrence and finally gave a moderate risk rating. This rating was partly influenced by the Program’s successful implementation and the GoT’s strong ownership at the time of Project preparation. It was not very easy to foresee any political or governance related risks (such as changes in the upper management of the Ministry, the Project Unit, and the implementing units). Still, the likelihood of delays or problems in the legislative amendments could have been elaborated in more detail in the risk mitigation section as critical prerequisites for a successful start and implementation of the component. This would then keep the risk for the OBF component as substantial. 2.2 Implementation 20. The Project became effective on September 29, 2009, and closed on September 30, 2015, 26 months later than the original Closing Date of July 31, 2013, as a result of two extensions. The Mid-term Review (MTR) took place as part of Bank visits that took place

8

during the period October-December 2011. Discussions were held on the revisions of the PDO and the Results Framework and of a proposed merger of Components A and B.

21. Progress towards achievement of the original PDO and the overall Implementa t ion Progress (IP) were rated satisfactory in the first years of Project implementation. The first downgrading happened in October 15, 2012 setting the IP rating to Moderately Satisfactory due to delays in decisions regarding the implementation of Component C related to SSI. This was followed with the downgrading of the DO rating to Moderately Satisfactory, and IP rating to Moderately Unsatisfactory on March 31, 2013. The reasons for the downgrading were as follows: (i) the second PDO: piloting output-based financ ing for NCD prevention and control was unlikely to be met in its current formulation; (ii) finalization of the restructuring was delayed due to extended discussions between implemented agencies; (iii) disbursement was slow due to changes in the MoH management and MoH’s reorganization; and (iv) implementation capacity at the PMSU was weakened due to turnover of key staff. After the first restructuring of March 2013, implementation capacity and quality started improving with the recruitment of consultants. Though the PDO indicators (reflecting the previous year’s performance) started to worsen after 2013, in line with the positive recovery in the implementation capacity and gradual increase in the pace of disbursement the IP rating was upgraded to Moderately Satisfactory on June 28, 2014, while the DO rating was kept as Moderately Satisfactory. The PDO rating was downgraded to Moderately Unsatisfactory just before the Closing Date on September 30, 2015, as three out of four PDO indicators were below the closing date targets. 22. Factors outside the control of the GoT. None. 23. Factors within the control of the GoT. The necessary change in legislation to implement the OBF pilot for component B was approved two years after the project became effective. Even after the passing of the legislation, uncertainties on the part of the MoH remained on the implementation details of the OBF model. Although a high level Inter-Agency Steering Committee (IASC) was planned to be established, with representatives of the Under-secretariat of Treasury, Ministry of Development (MoD),14 MoH and SSI to provide inter-agency coordination, oversight and policy guidance for overall implementation of the Project, this has never materialized.

24. The project restructuring on the PDO and Intermediate Outcome Indicators took almost two years to complete with the March 2013 and January 2014 restructurings due to changing decisions of the implementing agencies on major issues. The SSI had informed the Bank team of its decision in early 2011 to cancel the undisbursed portion of the loan under Component C (Capacity Building of the Social Security Institution) but the new SSI management decided not to cancel the unused loan proceeds until late 2011. As the SSI could not get an agreement with the MoD on the activities to be implemented under the Project, the remaining funds under Component C could be cancelled only in 2013. There were long series of exchanges with the MoH in order to finalize the restructuring document

14 MoD was called the Undersecretariat of State Planning Organization before 2011.

9

of December 2013. The process took longer than expected due to changing demands of the MoH upper management.

25. Factors within the control of the implementing agencies. Delays in the decision making processes (specifically about the use of Project funds) and frequent departure of Project-related staff capacity (especially on procurement and financial management) caused delays in implementation thus adversely affecting timely disbursements. 26. The original disbursement lag was 21.69 percent or 24 months at the time of closing. The Project has suffered from delays in disbursement throughout its implementation. At the time of the Level 1 restructuring of January 2014, US$ 49.12 million equivalent (65.4 percent) were disbursed out of the total loan amount of US$ 75.12 million. Disbursements followed a similar trend after the restructuring and reached 93.5 percent of the loan at the time of closing.15 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 27. Design at approval. The PAD included an M&E framework with six PDO indicators and ten IOIs with baselines set and target values indicated. While original PDO indicators and intermediate indicators captured the essence of the PDO, they were not all adequate, as only one of the PDO indicators has any face validity vis-a-vis the stewardship funct ion; and some of the indicators were either not relevant, or could actually be measured with the available data.. Selected PDO indicators reflected the health system overall utilizat ion, intermediate outcomes and to some extent, final health outcomes. The outcome indicator linked to component A (on patient satisfaction) and those linked to components B and C (on selected NCD conditions’ prevalence, coverage, behavior change and strategic purchasing) were appropriate in measuring critical areas of health system performance. While the indicators were ambitious, the significantly positive progress of the reform and its implementation, as well as the GoT’s strong ownership and engagement in the reform justify their selection. However, the main issue with the M&E design was that the indicators were selected on the basis of their predictive validity to gauge the implementation of the overall reform program, rather than of the activities under the Project. 28. Implementation. While the majority of the indicators were tracked regularly for the purpose of the ISRs, there had been challenges in data collection and consistency at times. Issues related to data collection and reporting can be summarized as follows:

• Data collection for intermediate outcome indicators related to component B was contingent upon the actual start of the OBF pilot. However, the pilot did not materia lize and, therefore, necessary legal changes did not take place. • Different data sources for some of the intermediate outcome indicators were used after the MoH reorganization, which led to data inconsistencies as noted in ISRs (e.g.,

15 New withdrawal applications were processed after the loan closing (US$ 389,969) and unused balance of the designated account was transferred to the Loan Account on February 19, 2016 (US$ 457,471.42) .

10

data on patient satisfaction varied from 89 percent (in September 2012 and February 2013 ISRs) to 73 percent (in September 2013 ISR). • New reporting methodologies were used by the MoH after its reorganization. For instance, data on annual visits to family physicians were collected from the MoH statistics yearbook starting in December 2014, as recorded by ISRs. • Timely monitoring of some indicators was affected by the transfer of the TUSAK to the General Directorate of Health Research (GDHR) as part of the MoH reorganiza t ion of 2011. • As the downturn in the PDO indicators after 2013 was not effectively monitored, corrective action could not be realized until the last stages of project implementation.

29. Revisions as part of the restructuring. Both the PDO and intermediate outcome indicators were revised in March 2013 and January 2014 as explained in detail under Section 1.3. The restructuring papers were not clear enough to reflect the details of the revised M&E framework including information on reporting frequency, data sources and responsible units and justification for revision, cancellation, progress to date and likelihood of achievement. The restructuring mainly focused on addressing changes related with the PDO indicators that were linked with the dropped or discontinued components. For the remaining component, the restructuring was opportunistic (rather than corrective) as the PDO indicators were performing satisfactorily at the time of restructuring, and the disconnection between the project activities and their effect on the PDO indicators was not taken into consideration sufficiently. 30. Utilization. The following were observed with regards to utilization:

• Findings of surveys and studies conducted by MoH implementing units on NCD (diabetes, etc.) were used for strategy development. Survey results proved to be important in defining priorities for adopting new strategies and health care programs and developing prevention and promotion programs. • The PMSU involvement in M&E responsibility was limited to monitor ing intermediate outcome indicators. • Many of the intermediate outcome indicators reflected a specific action being accomplished—process indicators. Hence, these indicators had little value for decision making or resource allocation. • Indicators which would have had an impact on decision making and resource allocation were dropped due to the cancellation of components B and C. These indicators could have measured results on critical interventions, such as coverage, behavioral change, intermediate health outcomes and improvements in health financ ing system (cost control and strategic purchasing).

2.4 Safeguard and Fiduciary Compliance 31. Safeguard. The original Project and the restructurings were rated appropriately as environmental Category C with no adverse impacts expected. During the preparation phase of the Project none of the safeguard policies was triggered as Project activities were going to be implemented within physical footprint for any type of construction –including minor civil works. During restructurings, there were no amendments to the Integrated Safeguards

11

Data Sheet of the Project since the content of the restructurings did not have any implications related to environmental and/or social safeguard policies of the Bank. Since the safeguard policies were not triggered there was no environmental or social assessment prepared for the Project and none were prepared during Project implementation. Safeguard-related implementation support visits were not needed, therefore not conducted by the Bank during implementation. 32. Procurement. Procurement was rated moderately satisfactory over the life of the Project. Though the Project implementation did not suffer from procurement-related delays, the rating was given to reflect the weak capacity of the implementing agency in this area, especially around the time of the Level 1 restructuring. The Bank worked closely with the MoH to address this aspect, through focused implementation support and capacity building efforts in the area of procurement. The rating was kept moderately satisfactory until closing mainly due to the continuing need for capacity building (due to lack of experienced civil servants or consultants), as well as the inefficient coordination between the MoH procurement units and the PMSU.

33. Financial Management. The Project had initially two implementing entities (the MoH and SSI). As the Project implementing entity, in charge of Components A and B, the MoH has maintained satisfactory financial management arrangements for the Project all through implementation. The SSI maintained satisfactory arrangements until Component C was cancelled. The PMSU which was responsible for providing financial management support to all MoH implementing units also kept the accounting entries for the Project. The accounting entries were used in following the Project`s financial position against foreign currency and in generating Project financial statements. The basis of the accounting entries prepared by the PMSU was the payment confirmations of the Central Bank of Turkey. In order to improve the information flow from the general directorates, the PMSU has developed an integrated system which all general directorates were required to utilize at all stages of the procurement processes. Through this system the PMSU was able to ensure that the internal control procedures were applied by all general directorates and the required supporting documentation for accounting entries were completed and flowed on a timely basis to the PMSU. This has contributed significantly to the satisfactory rating of the financial management arrangements throughout Project implementation. 2.5 Post-completion Operation/Next Phase 34. The revised PDO continued to be relevant. Increasing the effectiveness of MoH in implementing the health reforms and in health system performance has been embedded in the new Health Systems Strengthening Support Project (P152799, approved on September 21, 2015), where the particular focus is on strengthening the family medicine system and to effectively address the prominent epidemiological changes in the country (NCDs and aging population, in particular). The new Project’s PDO is: to improve primary and secondary prevention of selected NCDs, increase the efficiency of hospital management, and enhance the capacity of the MoH for evidence-based policy making.” Results through improvements in those areas are expected to positively contribute to the Turkish health system performance.

12

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 35. Relevance of Objectives, Design and Implementation is rated Substantial under the original project and Modest after Level 1 restructuring. 36. Relevance of Objectives (rating - PDO 1: Substantial/PDO2: Substantial). The Project’s objectives before and after restructuring were responding to the health sector challenges and identified government priorities. The implementation of a transformed health financing, delivery and regulatory framework is still a key element of achieving the National Health Sector Program goals. The Program reform, which the Project supported, was instrumental in achieving universal health coverage to enhance equity substantia l ly and lead to quantifiable and beneficial effects on all health system goals, including an improved level and distribution of health outcomes, enhanced fairness in financing and better financial protection, and increased user satisfaction.16 The objectives are also in line with the higher level objectives of the GoT (Ninth and Tenth Development Plans, and the MoH’s medium-term strategic plans), and of the Bank’s CPSs.17 The Project’s support to strengthen preventive and primary care through scaling-up the family medicine program is still relevant today as family medicine system is in its early stages in Turkey and needs further improvements to become more effective and fully settled. Support to preventive care for the control of NCDs is also particularly relevant today as the disease burden of NCDs has become prominent and will continue to be substantial in the medium and long term. Original PDO on increasing the effectiveness of the MoH and SSI in formulating and implementing reforms in provider payments and health systems performance is relevant since provider payment systems and health system performance are not one-time-only agenda items and are subject to continuous improvement. 37. Relevance of Design and Implementation (rating - PDO 1: Substantial/PDO2: Modest). the results chain under the original project had a sound underlying logic overall where the Project components were reasonably expected to conclude in a number of critical outputs (or intermediate outcomes): such as scaled-up family medicine program, piloted Public Hospital Union model, developed pay-for-performance (P4P) system to control health providers with the objective of ensuring increased utilization of primary health care, and increased client satisfaction of health services. The results chain revised with the first Level 2 and Level 1 restructurings, on the other hand, focused only on the PDO indicators that were about service delivery, though the restructured Project was mainly about stewardship and governance. The original PDO indicators on patient satisfaction, selected NCD prevalence, coverage, behavior change and strategic purchasing were complementing each other from a health system performance perspective. When components B and C were dropped or discontinued, the integrity of the preliminary design was affected.

16 Atun, R. et al, 2013, Universal Health Coverage in Turkey: Enhancement of Equity, Lancet 382 (9886): 65–99 17 Report No. 42026-TR, and 75520

13

38. Institutional and implementation arrangements at the entry were appropriate. First, at the operational level and with regards to the Project management itself, the decision to use existing MoH technical implementing units reduced significantly the need for additiona l consultants and management capacity. Second, the project envisaged the assessment of progress and achievements of the project each year in line with the MoH’s annual implementation and procurement plans. And third, periodical audit arrangements intended for Component B, for family physicians’ reporting on NCD related activities, were beneficial to ensure continuity and quality of follow up of these interventions. However, the design after Level 1 restructuring failed to build an efficient M&E framework on NCD related activities.

39. The interventions mostly comprised of the capacity building efforts and they were relevant to support the reform elements of the Program rather than attaining Project objectives. Capacity building efforts under component A focused particularly on health system stewardship, primary health care strengthening, health information system and NCDs in particular. Interventions designed under component B were relevant as they intended to introduce, through pilots, the output-based payment model for the preventions and control of targeted NCDs and to assess the effects of the model on health outcomes related with the targeted NCDs. Component C capacity building efforts focused mainly on strategic purchasing which is an important element of UHC deepening/sustainability. When Component B and C were dropped/discontinued the interventions under the project failed to serve the puposes of PDO indicators in general. 3.2 Achievement of Project Development Objectives 40. Efficacy (rating - PDO 1: Modest/PDO2: Modest) is rated Modest under origina l Project and Level 1 restructuring. This rating is based on the analyses of the accomplishment of project PDO indicators and intermediate outcome indicators as well as the contribution of project activities to the achievement of the PDOs. Two PDO indicators were achieved and one partially achieved out of the six original PDO indicators at the time of the Level 1 restructuring. Only one PDO indicator out of four was achieved at the time of Project closing according to administrative and survey data (Statistical Yearbooks) from the MoH, and the Turkish Statistical Institute (TUIK)’s Life Satisfaction Surveys. See Annex 10 for details. PDO achievements before restructuring 41. Achievements of PDO 1: increase the effectiveness of the SSI and the MoH in formulating and implementing reforms in provider payments and health systems performance. At the time of restructuring the project was on track to successful achievement of the two key outcome indicators’ then-targets, associated with this project objective. Therefore PDO was substantial. Annex 11 includes the details on indicators. 42. The utilization of primary health care increased. Indicator target partially achieved. With increased access to family medicine providers, utilization of primary care services increased to 3.3 outpatient visits per capita in 2011 compared with 2.1 in 2008. Though the

14

utilization rate started falling in 2012 (3.1 outpatient visits per capita), the target was on track at the time of restructuring.

43. The percentage of first-time visits to specialists that are referred by family medicine practices increased. Indicator target achieved. The target of 1.5 was met in 2010 as reported by the MoH based on the HMIS data.

44. Client satisfaction with health services increased. Indicator target achieved its then-target at the time of restructuring. The target of 70 percent was met in 2012, as the patient satisfaction with health care services increased from 63.4 percent in 2008 (baseline) to 89.8 percent.

45. SSI expenditures on pharmaceuticals and outpatient health care decreased. The indicator was neither achieved nor shown progress. Though the rate was declining in 2011 (38.7 percent), the expenditures on pharmaceuticals and outpatient health care reached 41.8 percent at the time of the restructuring, which was even higher than the baseline figure (38.7 percent) 46. Achievements of PDO 2: Piloting output-based financing for non-communicable diseases (NCD) prevention and control. Output-based financing component of the Project was never launched and was subsequently dropped with the Level 1 restructur ing as the government decided to implement the OBF nationwide under its own funding. At the time of the restructuring, there was no activity related with the following related key indicators: (i) smoking prevalence among 18-29 age group in OBF pilot provinces decreases more relative to non-pilot provinces; and (ii) high blood pressure in population in OBF pilot provinces decreases more as compared to non-pilot provinces. Therefore, the ICR team rates progress towards these indicators as negligible. Information on the current status of smoking and hypertension prevalence is presented in Annex 10. PDO achievements after Level 1 restructuring of January 2014 47. Achievements of the revised PDO: Increase the effectiveness of the MoH in formulating and implementing reforms in provider payments and health systems performance. With project support, the family medicine (FM) model has been an important reform implemented by the government in the health sector, aiming at increasing access to health services and making the sector ready to deal in a cost effective manner with core health issues (such as growing burden of NCD). The project led to the successful achievement of one key outcome indicator associated with this project objective, but well ahead of the restructuring (Reorganization of the MoH with exclusive focus on health sector stewardship function). 48. Per capita annual visits to family medicine physicians and Share of family medicine visits in total visits: Indicator targets not achieved. These two indicators related to family medicine scheme and its utilization were not met. This could be attributed to inadequate increase on the supply side, which could not keep up with the increase from the general demand for care.

15

49. Human resources in health, both in terms of family physicians and family medicine staff (specifically nurses and midwives), were insufficient. Since 2013, there has not been a significant increase in the number of family physicians which also exacerbated the ever increasing workload of family medicine. Number of registered patients to each family physician was high when compared to international standards18. High volume of patients reduces the time allocated to each patient thus narrowing the scope of a thorough examination. High workload of family physicians could also be considered among the main reasons for reduced patient satisfaction with regards to family medicine leading to a reduction in per capita visits to family physicians. 50. Additionally high workload of family physicians does not allow implementing a formal/mandatory referral system, which was piloted during the early phases of the Program, but did not work due to this high workload. The public has not yet reached an adequate level of awareness about the crucial role family medicine as most of the patients see family medicine centers as a place to go for prescription or obtain referrals for hospital care. TUIK’s 2014 Life Satisfaction Survey clearly highlights the tendency of population for provider choice at higher levels of care rather than the primary level: Irrespective of having a social insurance, patients prefer public hospitals as the first contact point of health care; Almost half of the population with social insurance and around two thirds of those without social insurance seek care from public hospitals. Only around one third of both groups seek care at primary level, through family medicine centers (Figure 4 in Annex 10). Survey findings suggest that more patients prefer public hospitals than family medicine centers since: (1) they are more satisfied with services provided in public hospitals; (2) visiting public hospitals is a necessity for them and (3) co-payments are low. Close location of the family medicine centers is the only factor which affects the provider choice towards family medicine (Figure 5 in Annex 10). 51. Client Satisfaction with health services: Indicator targets not achieved. The rate concerning client satisfaction with health services was not met and presented a decreasing trend since 2011. TUIK's Life Satisfaction Survey of 201419 indicated the main problems in the health service provision as perceived by the patients are: (1) the existence of co-payments for medical examinations (60.9 percent of the population); (2) the existence of medical test and laboratory fees (53.6 percent of the population); (3) insufficient number of physicians and health personnel (51.1 percent of the population); (4) medicine prices (48.7 percent of the population); and (5) waiting time for medical examination and medical tests (35.3 percent of the population). 52. Ministry of Health reorganized and focuses exclusively on the health sector stewardship function: Indicator target achieved. The PDO indicator concerning the stewardship function of the MoH was met with the approval of the Decree Law on the

18 For comparative countries and regions, WHO Health For All 2013 data suggests general practitioners (or family doctors) per 100.000 population as follows: Turkey: 54.1, Spain: 75.2, Germany: 66.7, France: 160.1, Eur-A area: 85.6, European Region: 64.6 (http://data.euro.who.int/hfadb/tables/tableB.php?w=1277&h=718 accessed on 08.01.2016) 19 http://www.tuik.gov.tr/Kitap.do?metod=KitapDetay&KT_ID=11&KITAP_ID=15

16

http://data.euro.who.int/hfadb/tables/tableB.php?w=1277&h=718

http://www.tuik.gov.tr/Kitap.do?metod=KitapDetay&KT_ID=11&KITAP_ID=15

Organization and Duties of the Ministry of Health and Its Affiliates on November 3, 2011. The establishment of the PHeI, the PHoI and the Pharmaceutical and Medical Devices Agency20 (PMDA) had been completed within a year following the enactment of the Decree Law. The Cancer Institute is expected to be established under the newly established Turkish Institutes of Health Sciences21 (TUSEB) which aims to promote research and development on health sciences and technology. General discussion on the achievement of project development objectives 53. With the restructuring and cancellation of the two components and with the revision of the PDO, the project concentrated on component A. In line with the revised PDO of increasing the effectiveness of the MoH in formulating and implementing reforms in provider payments and health systems performance, component A focused on supporting MoH for health system performance improvement. To this end, project activit ies addressed different dimensions (functions) of the health system (stewardship, resource generation, service delivery-utilization and financing), which in themselves, produced dimensional/functional results contributing to the overall system performance. Annex 2 and Annex 11 (1) elaborate project’s inputs-activities and outputs/outcomes and (2) provide a visual mapping of these activities with the health system dimensions.22 Though, the interventions under the project provided timely resources, flexibility, and mobility to the MoH to attain Program higher objectives, this success could not be rightly/fa ir ly reflected to the Project rating. As a result, the project rating was adversely affected despite the full achievement of IOIs and significant attainments made throughout the reform such as the success on the areas of MCH and ANC. 54. A broader health system level observation reveals that increased health sector spending is a major driver of the achievements: TURKSTAT data suggests that health expenditure as % of GDP increased from 4.8% in 1999 to as high as 6.1% in 2008. Figures for the last few years are around 5.3%. The achievements for the ante natal care (ANC) and mother and child healthcare (MCH) services are worthy of note. In fact, ANC and MCH care constituted an important strategic priority for Turkish health system and had direct relevance with the project’s role in expanding family medicine. Improvements on these areas are not only limited with improved historical trends, results also highlight improved equity among east and west (thus improved regional and socio-economic status equity).

55. Health Outcomes: According to interagency estimates, between 1990 and 2010 Maternal Mortality Ratio (MMR) was reduced by 70%. Turkey exceeded the Millennium Development Goal (MDG) target set for ¾ MMR reduction. While regional differences do still exist, they seem to have become less prominent. Infant Mortality Rate (IMR) also

20 With the 2011 MoH organizational restructuring, PHeI, PHoI, and PMDA became affiliated agencies of the MoH 21 http://www.resmigazete.gov.tr/eskiler/2014/11/20141126-3.htm http://saglik.gov.tr/TR/dosya/1-94578/h/turkiye-saglik-enstituleri-baskanligi.pdf 22 Health system dimensions are represented as Turkish Health System Strategic Map, which constitute the basis of the Health System Performance Assessment Study of 2012. Same strategic map is also included in the MoH’s 2013-2017 Strategic Plan.

17

http://www.resmigazete.gov.tr/eskiler/2014/11/20141126-3.htm

http://saglik.gov.tr/TR/dosya/1-94578/h/turkiye-saglik-enstituleri-baskanligi.pdf

improved during the Program. MoH Statistical Yearbooks suggest that between 2009 and 2013 IMR decreased by 40% allowing Turkey to also exceed the MDG target for 2/3 reduction in IMR. Sub-national inequalities in IMR have also been reduced.

56. Coverage: Immunization improved both in terms of coverage and depth. Number of antigens increased from seven in 2002 to 13 in 2012. Turkish Demographic Health Surveys suggest that coverage increased both nationwide and on regional level and regional differences decreased. Ratio between best covered and worst covered regions decreased from 1.6 in 1993 to 1.1 in 2013.

57. Coverage and regional equity for ANC also improved. Between 1993 and 2013 the share of women having at least one ANC visit to a health care provider increased by 35.8%. Interregional differences have also declined as the ratio between the best and worst covered regions for ANC visits decreased from 2.61 to 1.06. Annex 3 provides a more detailed analysis on the achievements for MCH and ANC. 3.3 Efficiency 58. Efficiency (rating - PDO 1: Substantial/PDO2: Substantial) is rated Substantial under the original Project and Level 1 restructuring. 59. Design efficiency. The Project supported the strengthening of preventive and primary health care through scaling up family medicine program and implementing preventive care. This way, the Project aspired to contribute to improving efficiency, especially when addressing NCDs, through: (1) reducing the work load on secondary care provision; (2) increasing access to care; (3) producing better health outcomes in the longer run; and (4) allowing for more cost effective interventions (in many countries preventive NCD interventions are significantly cost effective than clinical treatment of such conditions). 60. Implementation efficiency. With regards to the interventions and Project related activities, the hospital management reform, the unification of the health insurance schemes, the family medicine program, as well as the restructuring of the MoH function to stewardship all contribute to increasing efficiency in the health sector. 61. Cost Effectiveness. As per justification in the original PAD, the economic analysis for the Project consists of an evaluation of the Program from a public finance angle. However, a traditional economic analysis with rates of return analysis has not been attempted. Additionally, as the Project is the second phase of the original APL, there is no economic analysis providing quantitative estimates of the economic rate of return based on a cost-benefit analysis. In this respect the discussion on the economic impacts of overall health reform interventions supported by the Project are based on the criteria of Musgrove framework for public sector involvement in the health sector as presented in the PAD and on the evidence from the literature. A broader discussion is presented in Annex 3.

18

3.4 Justification of Overall Outcome Rating 62. The project’s overall outcome rating is Moderately Unsatisfactory. Table 1 below presents a summary of detailed ratings in order to arrive at an overall rating of outcome for the original and revised PDO. While relevance and efficiency criteria were substantia l, efficacy was modest under the original Project and after the Level 1 restructuring. The overall rating reflects significant shortcomings in the operation’s achievement of its objectives as explained under section 3.2 above. The weight represents percentage of loan disbursed over the total loan amount of 65 percent at the time of the Level 1 restructur ing (Table 2). Table 1 Original PDO Revised PDO Relevance Substantial Modest Objectives Substantial Substantial Design Substantial Modest Implementation Substantial Modest Efficacy Modest Modest PDO 1 Substantial Modest PDO 2 Negligible Dropped Efficiency Substantial Substantial Overall rating MU MU

Table 2 Original PDO Revised PDO Overall Rating MU MU Rating value 3 3 Weight 65% 35% 100% Weighted value 1.95 1.05 3.00 Final rating MU

HS: 6; S: 5; MS: 4; MU: 3; U: 2; HU: 1. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 63. Scaling up of family medicine increased access and quality of care in poorer and more rural areas implying improved equity. As has been elaborated in Annex 10, performance-based activities of family physicians on MCH and ante-natal care (ANC) led to highly positive health outcomes for women and children.

19

(b) Institutional Change/Strengthening 64. Several of the Project’s components positively contributed to MoH’s strengthening where involved units used Project resources to bring international experience and knowledge to build institutional capacity for the continuing reforms. Among those efforts, the General Directorate of Health Research (GDHR) promoted in-depth research for health system which would provide valuable and critical input for evidence-based decision and policy making. The GDHIS established and maintain critical capacity to further develop the MoH’s information systems. All the implementing units related with preventive and primary care were moved under the PHeI. The PHoI assumed the service provision responsibility. International expertise was also instrumental in supporting public hospitals in their effort to improve and measure quality of care. (c) Other Unintended Outcomes and Impacts (positive or negative) 65. No unintended outcomes or impacts have been observed. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 66. Not applicable. 4. Assessment of Risk to Development Outcome Rating: Moderate 67. The ICR team rates the risk that project development outcomes will not be mainta ined as Moderate. Key reforms are currently preserved in a solid legal framework by law and in the GoT’s strategic planning (10th Development Plan, the MoH Strategic Plan), and therefore unlikely to be reversed. 68. The Project mainly funded capacity building, training and institutional strengthening within the MoH and closely involved stakeholders such as family physicians. Since the MoH staff and family physicians are direct beneficiaries of the Project capacity building efforts, medium to long term effects are considered sustainable. 69. The declining trends of patient satisfaction can be considered as early indication for further fine tuning and planning within the health system. Increasing workload and scope of family physicians despite stagnating pay structures, disproportionate rights of patients and physicians in favor of the former, lack of formal mandatory referral system in the absence of sufficient health workforce at the primary care level are among the main areas that need to be addressed. While the primary care/family medicine service delivery model has been successfully scaled up nation-wide, a shift towards public/private partnership through inpatient based service organization and delivery model in large urban settings is likely to undermine the further progress in maintaining and safeguarding hierarchy across levels of healthcare.

20

5. Assessment of Bank and Borrower Performance 5.1 Bank Performance a. Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 70. Main Strengths. The main strengths in ensuring quality at entry were (i) extensive background work and analysis; (ii) the APL design that supported Turkey’s health system reform; and (iii) a clear rationale for Bank’s involvement. Planned interventions were poverty focused and gender-sensitive. The Project focused on improving the coverage and accessibility of health services in favor of the poor population and emphasizing the health and well-being of women and their children. Preparation activities included the assessment of fiduciary-related risks, the articulation of a sound set of actions and other measures to reduce these risks and the establishment of structures within the implementing units for assuming the full range of fiduciary responsibilities. A Quality Enhancement Review (QER) conducted on November 13, 2008 made recommendations that were incorporated into the Project design: (1) definition of the PDO to clearly distinguish between the government’s Program and the objectives of the Project; (2) reduction in the number of indicators to the most critical ones from a preliminary long-list; (3) inclusion of performance indicators to reflect the Project goal of cost containment for financ ia l sustainability; and (4) inclusion of an explicit objective of strengthening Turkey’s capacity to measure and use health metrics in decision making as reflected in two IOIs linked to components A and C. 71. Shortcomings: There was a certain degree of disconnect between the PDO, components and the activities which became more noticeable after the restructuring of the PDO and elimination of some of its indicators. The primary development objective of the Project was strengthening the stewardship function of the MOH, yet the components, activities and most of the indicators were about service delivery function. The activit ies envisaged within the scope of the Project were only indirectly linked to the expected outcomes of the results framework, especially after restructuring. While the definition of each sub-component was detailed and had a connection with the results, the activit ies planned under the Project did not have direct effects on the outcomes. Though data collection instruments were selected carefully, and responsible units for data collectio n were defined prudently, the M&E design had few shortcomings during the preparatory phases (See Section 2.3). Risk assessment and risk mitigation plans were sound overall, with the exception of the legal risks related to Component B. The legal changes were only approved in November 2011 (a delay of almost two years).

b. Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Unsatisfactory

21

72. Bank’s performance during Project implementation is rated Moderately Unsatisfactory. 73. The Bank teams engaged with the MoH and SSI on a wide variety of health and social security policy and technical issues at all stages of the Project, and provided technical support. Three Bank staff assumed TTL responsibility during Project implementation with handover missions. Though the supervision activities were performed regularly, and Aide-memoires and ISRs were prepared periodically, the Bank teams could not effectively rate the performance of implementing units. Although the Bank teams provided timely and precise support to the implementing units in an environment where the decision making processes of the MoH and SSI were slowed down due to organizational restructuring and frequent changes in managerial levels, the Bank teams did not note the badly performing PDO indicators and did not make timely changes on the DO ratings. This was critical as the restructuring failed to adjust the results framework with the project activities. 74. Some shortcomings on M&E supervision are noted after the Project restructuring. As there was inconsistent reporting on indicators, the Bank team wanted to work directly with the implementing units to collect most recent and relevant information on expected outcomes and to regularly report on outcome indicators, but could not succeed in doing so. Instead, the Bank team continued working closely with the PMSU in line with the communication protocol agreed with the MoH. As a result, the flow of information as reflected in ISRs post-restructuring was broken. After restructuring, the Bank overlooked to ensure/supervise consistency on indicator-related content in the ISRs. Indicators which were dropped at restructuring continued to appear in the ISRs system rather than being deleted or simply notified as dropped. This created some level of confusion for the ICR team while reviewing the ISRs. c. Justification of Rating for Overall Bank Performance 75. The overall Bank performance is judged to be Moderately Unsatisfactory on the basis of a rating of moderately satisfactory for quality at entry and moderately unsatisfactory for supervision. 5.2 Borrower Performance a. Government Performance Rating: Moderately Satisfactory 76. With regards to the institutional framework, the GoT had extensive experience in dealing with the World Bank, including under the framework of Project lending. The Government provided a strong enabling environment, especially politically, as several important legislations were passed such as the MoH Restructuring Decree Law, and relevant regulations for the establishment of affiliated institutions of the MoH (includ ing the PHeI, and PHoI). The performance of the Treasury (Borrower) and the Ministry of Development (MoD) – in charge of the development plans and annual programs - were

22

notable in providing timely support and guidance to the Project and overall program management. However, the planned high level Inter-Agency Steering Committee (IASC) with representatives of the Treasury, MoD, 23 MoH and SSI to provide inter-agency coordination, oversight and policy guidance for overall implementation of the Project, the IASC was never functional. b. Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory 77. The performance of the MoH and the SSI is rated moderately unsatisfactory. Overall, the MoH’s commitment to the health reform and ownership of the Project remained high throughout the life of the Project even though there were pitfalls related to MoH’s reorganization, restructuring of the Project, and the changes in upper management –including the Minister. The SSI has also continued strengthening its institutional capacity. However, due to changing decisions of the implementing agencies on major issues, the restructuring process took almost two years. Additionally, the delays in decision making had resulted in negative consequences in implementation, and not full disbursement of the loan proceeds. While the role of the Bank in sharing knowledge and policy advice was generally recognized in some areas, the implementing agencies could not establish a direct link between the reforms and the Project as well. c. Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory 78. Given the challenges presented above, the overall borrower performance is rated as moderately unsatisfactory. 6. Lessons Learned 79. Providing a broad support to the strategic plan development and implementa t ion through disintegrated micro activities creates problems in monitoring and evaluating progress on the main objectives of the Project. Project activities should be interdependent with each other to mutually reinforce each other’s contribution towards the broader development objective rather than leading to discrete results –even though they support a part of the broader Strategic Plan. 80. Capacity building and technical assistance by bringing in international expertise and knowledge is appreciated by the borrower who also realizes the importance of using robust approaches in health system reforms and improvements. It would be of high value added for Projects supporting sector reforms to introduce a well-focused and customized approach to bring in international expertise and knowledge. Striking examples such as the following good practices supported by the project are (1) providing training programs and

23 Before 2011, MoD was established as the State Planning Organization.

23

consultancy for Public Hospitals and PHC system to increase the quality of care; (2) providing advisory activities for research framework and priority agenda to set the direction for the continuing reforms and to evaluate the achievements of Program; and (3) developing in-house capacity in the General Directorate of Health Investments in legal, financial, operational, and structural aspects of contract management of PPP investments. 81. Proper risk assessment of possible uncertainties likely to emerge during implementation and having a flexible design allowing to mitigate such risks is a necessity. There should be an in-depth exploration of the legislative background for any Project component/element as well. Component B of the Project suffered from the uncertaint ies concerning the legislation. While the risk analysis in the PAD included a discussion on the likelihood of different scenarios, there was no discussion on a scenario that no legisla t ive arrangements take place to allow the effective start of the component, and how to address it. A detailed analysis of the legislation (or the lack of legislation) could have revealed potential obstacles for proper implementation. This would then necessitate a more detailed risk analysis considering a number of scenarios, including the worst case one. 82. In order to minimize or avoid delays and challenges in Project implementation, both the Bank and the MoH can think of effective means and mechanisms to try to retain critical human resources to the extent possible (civil servants and the consultants) within the MoH Project Management Unit. Development of handover arrangements and training would also help ensuring a smooth transition. In the long term, these are also important for institutiona l memory. Frequent loss and high turnover of such human capacity caused many delays and created challenges throughout the Project, especially in the areas of project management, financial management and procurement.

83. Throughout all phases of the project (with or without restructuring) design, monitor ing and supervision of the results framework necessitates close, regular and timely attention by the Bank to make sure that the project objectives and PDO indicators are in line and consistent to reflect the outcomes of the project. Failure to overlook any gaps between project development objectives and key indicators might result in unrealistic outcome ratings as it was the case for this project. Considering the success of the Program, the Project could have got a higher rating if the M&E activities could focus on PDO indicators that were more correlated with the substance of the Program.

84. For projects supporting broader reforms, failure to timely monitor and to detect the possible deterioration on the reform trends might lead to misinterpret the outcomes in the medium and long run and to misread the sustainability. In the case of this project and for the family medicine system in particular, decreased patient satisfaction as well as decreased utilization can be the early warnings on reforms changing direction. Such changes in the reform focus risk losing the achievements acquired and they threaten the sustainability of the reforms.

24

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners a. Borrower/implementing agencies 85. The Borrower’s contribution is received on March 24, 2016 in response to the Bank’s letter of September 30, 2015. b. Cofinanciers 86. Not applicable. c. Other partners and stakeholders 87. Not applicable.

25

Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ Million equivalent)

Components Appraisal Estimate

Restructured Estimate

Actual/Latest Estimate

Percentage of


Percentage of appraisal

A-Support for MOH’s Strategic Plan for 2010-2014 61.82 70.42 70.5524 100% 114%

B-Piloting Output-Based Financing for Non-communicable diseases (NCD) prevention and control

9.16 0 0 NA NA

C-Capacity building of the Social Security Institution 4.14 0.81 0.81 100% 20%

Total Baseline Cost 75.12 71.23 71.81 100% 95% Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 75.12 71.23 71.81 Front-end fee IBRD 0.18 0.18 0.18 Total Financing Required 75.30 71.48 72.99 (b) Financing (US$ Million)

Source of Funds Appraisal Estimate


Actual/Latest Estimate

Percentage of Restructured Estimate

Percentage of Appraisal

Borrower 0.00 0.00 0.00 0 0 International Bank for Reconstruction and Development

75.12 71.23 71.81 100% 100%

Total 75.12 71.23 71.81

24 After the restructuring of categories and reallocation of funds in January 2014, only the funds under Component A were disbursed.

26

Annex 2. Outputs by Component

Sub-Component Planned outputs at Appraisal Actual outputs/outcomes at ICR A Support for MOH’s Strategic Plan for 2010-2014: The objective of this component was to build the capacity of the MOH to become an effective steward

of the health sector and complete service delivery reforms. The component financed goods, technical assistance, services and training required for eight out of ten programs of the MOH’s Strategic Plan for the Health Sector for 2010-14.

A.1 Strengthening preventive and primary health care services

Financing goods, technical assistance, services, and training for the MoH to scale-up the family medicine program, expanding from the current 33 provinces to all 81 provinces of Turkey, as well as support the MOH implement preventive health care services for the control of communicable and non-communicable diseases.

The capacity building focused on the family physicians training, community health center staff training, M&E for FM implementations, design and planning for the NCDs and risk factors and had a significant share (79%) within total A1 budget. 575 family physicians, and 3173 CHC staff were trained. A workload analysis was conducted to standardize work procedures for more effective service delivery and better quality of care by family physicians. Job descriptions of FM personnel were revised in line with the results of the analysis. Various research studies conducted on areas including workload of FM physicians, patient and provider satisfaction, women’s health, infant and under-five mortalities, reimbursement and pricing policies on pharmaceuticals. Training outputs/outcomes: • A Distant Health Learning System (USES) established. USES has a total of 31 training modules (15 modules on public health, 4 modules on personal development and 16 modules on health management). There are 27,215 health care personnel registered to USES. Each registered participant completed three programs on average (PHeI January 2015 data) • Evidence-based clinical practice guidelines for family physicians are developed and published on the web. Outputs/outcomes related with NCDs: • Chronic Pulmonary Disease Control Program developed • Cardiovascular Disease Control Program developed • Kidney Disease Control Program developed • Chronic Diseases and Risk Factors Study conducted • National Physical Activity Guidelines developed

27

Sub-Component Planned outputs at Appraisal Actual outputs/outcomes at ICR • Childhood Obesity Surveillance Initiative study conducted • European Action Salt Network meeting organized to increase awareness on reducing salt consumption • Healthy diet and obesity counseling units are established within Community Health Centers. • Outreach materials were developed and disseminated on obesity, diabetes, and physical exercise • School Canteen Regulation (for healthy food consumption in primary and secondary schools) was issued • Training of trainer activities organized to improve the capacity of health care personnel on disease management for diabetes. • Vital registration system (on deaths) is established.

A.2 Implementing Public Hospital Reforms

Financing technical assistance and training in support of the implementation of hospital reforms. The main aim of hospital reforms under the Program is to pilot and eventually implement nation-wide the Public Hospital Union model.

Proof-based diagnosis and treatment manual for the Secondary Health Care Services was developed Training provided to over 300 MoH personnel and social workers on increasing access to treatment and quality of services in the clinics of the alcohol and substance addicts treatment and training center (AMATEM)

A.3 Improving the quality of health services

Financing technical assistance and training for the MOH to build the capacity of the Performance Management Department and enhance its role in the implementation of pay-for-performance (P4P) systems in Turkey and regulate health providers with the objective of ensuring good quality health services.

Training provided to key MoH staff on quality standards, and assurance. Training provided to key MoH staff on ICD 10 AM DRG system. 4 international congresses were held on performance and quality systems in health sector. The 5th version of the Health Quality Standards and accreditation standards in health were developed. Number of DRG clinical codes increased. The clinical quality development system was developed to keep the health data of every patient in xml format.

A.4 Institutional restructuring and capacity building

Financing technical assistance and services, training and goods in support of the restructuring of the Ministry of Health (including establishment of the National Medicine and Medical Devices Agency, the National Cancer Institution, an internal inspection structure as well as

Health System Performance Assessment Stakeholder Meetings Health System Performance Assessment Capacity Building Training Research Techniques in Healthcare Services Training Basic Cancer Registry Training

28

Sub-Component Planned outputs at Appraisal Actual outputs/outcomes at ICR institutionalization of the Monitoring and Evaluation functions of the MoH). Supporting the strengthening of strategic management capacity to implement strategic financing methods in the field of performance based budgeting and ensure standardization, continuity and coordination in specialty and in-service training programs.

Cancer screening training for Family Physicians KETEM Early Cancer Diagnosis and Screening and Training Skills Training KETEM Mammography Techniques and Communication Skills Training Immediate outcomes: • Cancer registry infrastructure is strengthened, • Active cancer registry system ongoing nationwide (the active system is expanded from 14 provinces to all 81 provinces) • National and international standards are developed on cancer screening, cancer registry and reporting. Accounting systems of Provincial Health Directorates were aligned with international accounting standards.

A.5 Managing pharmaceutical and medical devices regulation

Financing technical assistance and training with the objective of assisting the MoH implement the National Medicine Policy. The National Medicine Policy aims at ensuring rational drug use, increasing the quality of medicine, vaccines and biological products, implementing institutional arrangements for assuring Good Clinical Practices and Good Manufacturing Practices, and increasing capacity to implement market surveillance and inspection.

No output, as disbursements under this subcomponent discontinued after the organizational restructuring of the MoH.

A.6 Improving health information systems

Financing technical assistance, training and goods in support of the expansion of the MoH National Health Information System (HIS) to improve health informatics standards, confidentiality, security and privacy principles of personal and institutional health records, establish a data warehouse within the scope of decision support system and to initiate data mining practices, to establish tele-medicine and tele-health systems.

Consultancy was given to MoH staff for HMIS, and MoH staff trained on various aspects of HMIS. Electronic Health Record (EHR) system is fully in line with the general data privacy and security as well as privacy of the personal health records Drug Tracking System is in place to support the efforts on rational drug use National Health Data Dictionary (NHDD) developed and updated to introduce national and international standards for the collection, storage and analysis of data Telemedicine and tele-radiology implementations are in place

A.7 Supporting inter-sectoral health responsibility

Financing technical assistance and services, goods and training with the objective of (i) assisting the MoH implement inter-sectoral cooperation and align health

Staffing of the Project Management Support Unit and PPP Unit under General Directorate of Health Investments (GDHI). Institutional capacity of the Department of Public-Private

29

Sub-Component Planned outputs at Appraisal Actual outputs/outcomes at ICR implementation of the Government’s Program and aligning health policies with world standards

policies with international standards, especially EU guidelines and standards, (ii) strengthen the capacity of the Project Management and Support Unit (PMSU) to efficiently coordinate the use of international financing/donor resources in the health sector, (iii) build the capacity of the “Public-Private Partnerships (PPP) Unit for the Health Sector”

Partnership was strengthened to support the ongoing city hospitals investment program. In-house capacity in legal, financial, operational, and structural aspects of contract management was developed within GDHI.

A.8 Establishing a National Public Health Agency

Financing technical assistance, goods and training for the Refik Saydam Hygiene Center Presidency to convert itself into the National Public Health Agency, responsible for all key public health functions for the Republic of Turkey including: disease surveillance, outbreak management, development of policies and plans related to public health, and public health regulation.

Hospital Infections Surveillance Program was developed Laboratory Diagnosis Tests were standardized Institutional capacity of the Refik Saydam Hygiene Center Presidency was strengthened during its transformation into the PHeI.

A.9 Sharing Turkey's experiences

Financing provision of equipment, technical assistance, technical services and training to MOH to strengthen its collaboration with other countries and international institutions in health policy development, and to share country experiences and the financing of MOH’s operating costs

Three international conferences on health reform were financed. 1) the evaluation of 10th year of the Health Transformation Program: Turkey Universal Health Insurance Ministerial Conference (Istanbul, June 2013) and (2) – Organization of Islamic Cooperation Strategic Health Action Plan Implementation (2014-2023) Second Leader Countries Meeting (February 2015), and (3) Turkey-Germany Health Symposium (May 2015)

B Piloting Output-Based Financing for Non-communicable diseases (NCD) prevention and control: The objective of this component is to pilot test and evaluate a model for implementing preventive health care interventions for non-communicable diseases (NCD) at the provincial level using output-based financing mechanisms. The design of this component takes into account the experience of designing and implementing output-based financing in World Bank Projects in the health sector in Latin America (namely Argentina and Uruguay)

B.1 Output-based financing for family medicine practices

Supporting pilot implementation and an impact evaluation of an output-based financing mechanism for family medicine practices in two provinces of Turkey.

No output

B.2 Capacity-building for Provincial and Central level Implementation of Output-Based Financing Pilot

Financing technical assistance, goods, training and incremental operating costs with the objective of capacity building of relevant institutions at the provincial and central MOH levels.

No output.

30

Sub-Component Planned outputs at Appraisal Actual outputs/outcomes at ICR C Capacity building of the Social Security Institution: The objective of this component is to enhance the Social Security Institution’s capacities for the

implementation and promotion of UHI. The component finances technical assistance and training for SSI with the objective of strengthening SSI capacities in becoming a strategic purchaser in the health sector and making evidence-based decisions vis-a-vis the implementation of provider payment reforms, contracting with public and private providers and continuous update of the UHI benefits package.

C.1 SSI capacity-building for the implementation and promotion of UHI

Supporting: (i) technical assistance and training for the UHI General Directorate on provider payment systems, implementation of enhanced contracting mechanisms and pay-for-performance systems, (ii) technical assistance and training for the SSI unit implementing MEDULA and the Strategic Planning Department of SSI to strengthen the relationship between availability of claims data and using the information to guide policy and program decisions under UHI, including strengthening expenditure tracking and updating UHI actuarial estimates, (iii) development of SSI capacity in producing evidence-based information for the SSI Reimbursement Commission on clinical guidelines, cost-effective health, pharmaceutical and medical technology interventions, for example through the development of Health Technology Assessment (HTA) capacity in SSI.

SSI personnel was trained for the purpose of increasing the effectiveness of the UHI implementation. Existing IT infrastructure was assessed and improved to cope up with future internet traffic requirements, including providing support to the necessary software development works of the relevant departments of the SSI. SSI’s medium and long-term programs, main principles and policies, objectives and priorities and their conformity to international social rights norms and criteria were evaluated to support ongoing and planned Projects.

Annex 11 presents the Strategic Map for Turkish Health System. The map is useful to track the connection of the sub-components of the project with the health sector functions and outcomes.

31

Annex 3. Economic and Financial Analysis No formal economic and financial analysis was included in the PAD. No net present value (NPV), economic rate of return, cost effectiveness, or cost-benefit analysis was calculated, and thus no baseline is available for comparison. As with most health reform Projects, no formal economic and financial analysis is conducted here because of the difficulties in attribution to a policy-oriented Project. Instead of a traditional economic analysis, this section elaborates the outcomes of the health reform from a public finance angle, according to Musgrove’s three criteria and provides an overview of impact of improved health outcomes on economic growth from the literature. Musgrove framework for public sector involvement in the health sector State intervention in the health care markets is justified on the basis of (1) optimal production of public goods (2) subsidization of poor population who cannot afford the health insurance or the inexpensive health care that the non-poor can finance out-of-pocket and (3) correction or offset of failures in the health insurance markets. Overall, meeting those criteria contributes to the health system performance results as (1) good health outcomes (2) cost control (3) patient and provider satisfaction and (4) medical and financ ia l equity. State-based health interventions create a significant share of the impact of health spending on health improvements thus forming the basis for public health expenditure to be somewhat more effective than private expenditure for improved health. The Project’s PAD takes reference of three criteria of Musgrove framework: Criteria 1: Public goods specifically apply for the implementation of family medicine and its scope on MCH activities which are highly relevant from a public health implementa t ion and disease control perspective of health public goods. In terms of results, immuniza t ion and ANC efforts helped to improve maternal and child health outcomes (infant mortality rates and maternal mortality ratio (MMR)). Additionally, regulation updates and developments, implementation of mandates, provision of information and health research can be considered as other important public goods produced within the scope of Program. Significant improvements have been observed for Mother and Child related health outcomes where maternal mortality ratio (MMR) and infant mortality rates (IMR) are largely reduced. According to interagency estimates, MMR is reduced from 67 deaths per 100,000 live births in 1990 to 20 deaths in 2010 per 100.000 live births (Figure 1). MoH’s 2013 statistical yearbook suggests MMR as 15.9 per 100.000 live births. 25 Turkey exceeded the MDG for MMR target set for ¾ MMR reduction. Regional differences do still exist, however, they seem to be less prominent. MMR improved in the all regions with the exceptions of West and Central Anatolia. Program period was also successful to

25 http://sbu.saglik.gov.tr/Ekutuphane/kitaplar/sapercentC4percent9FlpercentC4percentB1kpercent20istatistikpercent20ypercentC4percentB1llpercentC4percentB1percentC4percent9FpercentC4percentB1percent202013.pdf (accessed on 07.12.2015)

32

http://sbu.saglik.gov.tr/Ekutuphane/kitaplar/sa%C4%9Fl%C4%B1k%20istatistik%20y%C4%B1ll%C4%B1%C4%9F%C4%B1%202013.pdf

http://sbu.saglik.gov.tr/Ekutuphane/kitaplar/sa%C4%9Fl%C4%B1k%20istatistik%20y%C4%B1ll%C4%B1%C4%9F%C4%B1%202013.pdf

significantly reduce MMR in some less developed regions (North East Anatolia and East Black Sea) (Figure 2, MoH Statistical yearbooks). Figure 1 Figure 226

Infant Mortality Rate (IMR) also improved during Program. MoH Statistical Yearbooks suggest that IMR decreased by 40% from 13.1 per 1.000 live births in 2009 to 7.8 per 1.000 live births in 2013 (Figure 3) allowing Turkey to also exceed the MDG target for 2/3 reduction in IMR.. Sub-national inequalities in IMR have become less prominent, however, the gap between the best and worst performing regions is still almost twice as much (Figure 4). Figure 3 Figure 4

Turkish Demographic Health Surveys also suggest remarkable improvements for infant mortality. The decade prior to the Program shows improvement in the mortality ratios, but the decrease for all infant mortalities was higher during Program period (Figures 5 and 6). Regional results also suggest improvement both in terms of overall decline and reduced gaps among regions (Figures 7 and 8). Criteria 2: Poverty and equity: Equity in terms of access to health care improved during Program. Considering access to maternal and child health services is a good proxy for access to basic health care services., One can conclude that the access to basic health care significantly improved especially through the performance based incentives on mother and child health activities within the scope of family medicine scheme. Differences between east and west with large socioeconomic disparities have been largely reduced mainly for

26 The NUTS classification (Nomenclature of territorial units for statistics) is a system used by EU to divide the economic territory for the purpose of the collection, development and harmonization of statistics and to make socio-economic analyses of the regions. NUTS1 is a classification for major socio-economic regions under which Turkey has 12 regions.

33

immunization, ante-natal care and mother and child health. Turkish Demographic Health Surveys highlight the improvements in mother and child health efforts (one of the highest priority areas of the Program): Figure 5 Figure 6

Figure 7 Figure 8

Immunization uptake improved both in terms of coverage and depth. Number of antigens increased from seven in 2002 (BCG, combined diphtheria–pertussis–tetanus, oral polio, measles, and hepatitis B) to 13 in 2012 (with the addition of Haemophilus influenzae type B, rubella, mumps, pneumococcal conjugate vaccine, varicella, and hepatitis A). Figures 9 and 10 below show the increased coverage for measles immunization nationwide and on regional level. Increase in coverage also came with increased equity where striking regional differences decreased. On the regional level, highest coverage was 93 percent and lowest coverage was 58 percent in 1993. By 2013, highest coverage for regions reached to 97 percent and lowest coverage rose to 88 percent (Figure 10).

Figure 9 Figure 10

Coverage and regional equity for ANC also improved. While the share of women having at least one ANC visit to a health care provider was 62.3 percent in 1993 nationwide, this figure rose to 97 percent in 2013 (Figure 11). Equity improvement was substantial too: In

34

1993 on the regional level, highest coverage was 86 percent and lowest coverage was 33 percent. By 2013, highest coverage for regions reached to 99 percent and lowest coverage rose to 93 percent (Figure 12). Figure 11 Figure 12

Similarly, share of deliveries at a health facility and deliveries attended by skilled health personnel increased nationwide (Figure 13) and in all regions implying accelerated equity improvement: In 1993 on the regional level, share of delivery at a health facility was 80 percent for the highest region and lowest figure was 30 percent. In 2008, highest coverage was 98.3 percent and lowest coverage was 72.3 percent. By 2013, highest regional coverage reached to 99.8 percent and lowest figure rose to 91.7 percent (Figure 14). Figure 13 Figure 14

Criteria 3: Failings peculiar to insurance markets for health (efficiency and equity): Unification of insurance schemes increased breadth and depth of insurance coverage for the overall population but the increase for lower income population was more significant : Increase in obligatory insurance coverage was prominent for the first three poorest expenditure deciles (Figure 15). Content of the benefits package evolved with Program. The breadth of the insurance coverage increased for the overall population but the increase for lower income population was more significant; increase in obligatory insurance coverage was significant for the poorest expenditure deciles. For the out-of-pocket expenditures, percentage of individuals whose health care cost exceeds different thresholds of their total non-food expenditures decreased significantly during Program period. Impoverishment effects of out-of-pocket payments have also been reduced.27 (i) For 15 percent budget threshold the decrease (percent change) between 2003 and 2011 is 42 percent, (ii) for 25 percent budget threshold percent change is 45.5 percent and (iii) for 40 percent budget threshold percent change is 66.7 percent (Figure 16).

27 Aran and Hentschel, 2012.

35

Figure 15 Figure 16

Figure 17

Impoverishment effects of out-of-pocket payments have also been reduced: percent change (decrease) in the share of population who are non-poor before health expenditures but become poor after health expenditures (using poverty line with no health expenditures for bottom 3 deciles) is 62.2 percent, whereas percent change (decrease) in the poverty headcount with impoverishing effect of health

expenditures is 39.7percent for the period between 2003 and 2008 (figure 17).28 Impact of improved health outcomes on economic growth: An overview from the literature A recent study by Amiria et al (2013)29 indicates that the improvements in the health outcome indicators concerning mother and child health have an effect on GDP growth and that the causal effect of maternal and child mortality on GDP is generally stronger in high-income countries and upper middle-income countries. Considering the fact that Turkey is an upper middle income country and has witnessed significant improvements on MCH health outcomes (especially during the Program through the performance based approaches adopted within the family medicine scheme), one might conclude that these improved health outcomes positively affected the GDP growth. Same study presents efficiency rates calculated for each country using data envelopment analysis (DEA) in a Barro framework where the efficiency rate for any given country demonstrates the magnitude of the impact of MCH outcomes on GDP. For Turkey, the efficiency rate is given as 80.8 percent for the period 2001-2010. The interpretation of this efficiency rate is that if under-five mortality rate is reduced by 1percent, and the average GDP increase for this period is 4.28 percent,30 then GDP in Turkey would increase by 3.46 percent (0.808*4.28 percent). Another study

28 http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2012/08/22/000158349_20120822081258/Rendered/PDF/wps6178.pdf (accessed on 07.12.2015) 29 http://www.who.int/pmnch/topics/part_publications/201303_Econ_benefits_econometric_study.pdf 30 Average GDP growth for 2001-2010, based on World Bank data

36

http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2012/08/22/000158349_20120822081258/Rendered/PDF/wps6178.pdf



http://www.who.int/pmnch/topics/part_publications/201303_Econ_benefits_econometric_study.pdf

by Bloom et al (2008) 31 indicates that several studies find health outcomes to be a significant predictor of economic growth and gives specific reference to a study by Bhargava et al. (2001) which highlights that the effect of health on economic growth is larger in developing countries than in developed ones.

31 http://siteresources.worldbank.org/EXTPREMNET/Resources/489960-1338997241035/Growth_Commission_Working_Paper_24_Population_Health_Economic_Growth.pdf

37

http://siteresources.worldbank.org/EXTPREMNET/Resources/489960-1338997241035/Growth_Commission_Working_Paper_24_Population_Health_Economic_Growth.pdf

http://siteresources.worldbank.org/EXTPREMNET/Resources/489960-1338997241035/Growth_Commission_Working_Paper_24_Population_Health_Economic_Growth.pdf

Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending Furuzan Bilir Operations Officer ECCU6 Operations Loraine Hawkins Country Sector Coordinator EASHD HIS John A. Innes Lead Operations Officer ECSH3 HIS Jennifer Manghinang Temporary EAPVP Administration Elif Yonca Yukseker Program Assistant ECCU6 Administration Supervision/ICR Rekha Menon Practice Manager GHN06 Task Team Leader Claudia Rokx Lead Health Specialist GHN04 Task Team Leader

Ayse Seda Aroymak Senior Financial Management Specialist GGODR Financial

Management Furuzan Bilir Operations Officer ECCU6 Operations Ibrahim Sirer Senior Procurement Specialist ECSO2 HIS Salih Kemal Kalyoncu Senior Procurement Specialist GGODR Procurement Salih B. Erdurmus Procurement Specialist GGO03 Procurement Jennifer Manghinang Temporary EAPVP Administration Ethan Yeh Economist ECSH1 HIS

Ahmet Levent Yener Senior Human Development Specialist GSPDR ICR Task Team

Leader & Author

Safir Sumer Consultant GHNDR Consultant, ICR Author

Norosoa Andrianaivo Senior Program Assistant GHN03 Administration Elif Yonca Yukseker Program Assistant ECCU6 Administration Gozde Yilmazturk Team Assistant ECCU6 Administration

Jesko Hentschel Country Director LCC7C Country Sector Coordinator

Cristobal Ridao-Cano Program Leader EACIF Program Leader William D. Wiseman Program Leader LCC1C Program Leader Son Nam Nguyen Lead Health Specialist GHN01 Health

38

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY07 6.36 39.30 FY08 16.88 147.62 FY09 50.58 286.64 Total: 73.82 473.56 Supervision/ICR FY10 33.86 178.25 FY11 30.79 123.27 FY12 12.60 80.41 FY13 26.87 132.19 FY14 36.31 181.97 FY15 26.93 119.91 FY16 13.44 45.13 Total: 180.80 861.31

39

Annex 5. Beneficiary Survey Results No beneficiary survey was conducted.

40

Annex 6. Stakeholder Workshop Report and Results No stakeholder workshop was conducted.

41

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Comments by the Undersecretariat of Treasury (Borrower) on the ICR 1. Project Restructurings: The Borrower notes the several restructurings the project has gone through and would therefore like to underline the importance of accurately calibrating the demand and supply sides of the project. The goodness of fit between project activities and project objectives should have been more strongly established in order to avoid delays and complicat ions during implementation. The borrower notes that several indicators remain unachieved after restructuring. Several original project indicators were dropped after restructuring due to difficulties in data collection and disaggregation (page iii, indicator 1, indicator 2). The Borrower therefore, opines that a forward looking and realistic analysis of the capacity of the implementing entity, in particular in data collection, should be conducted during the selection of PDO indicators. 2. Closing Date Extensions: Similarly, revisions in the project closing date suggest that project design must be carefully crafted during preparation stage, reflecting realistic expectations from implementing agencies, in line with their respective capacities. Also, the Borrower notes that the closing date of the project was further extended in order to preserve “critical fiduciary as well as operational capacity and institutional capacity until the expected effectiveness of the new Health System Strengthening and Support Project”. Taking this opportunity, the Borrower would like to highlight the importance of locking in capacity building in a sustainable way in MoH as opposed to temporary and ad-hoc solutions to capacity challenges as they arise. 3. Supervision of Projects: While the legislation to implement the Output Based Financing pilot for component B has taken more time than foreseen (paragraph 23), better communication with relevant institutions, in order to reassess client demands, is imperative for making timely interventions in order to prevent a larger scale project derailing at a later stage. In that respect, project supervision by the Bank must be strengthened. To support the Borrower in taking timely action on the issues accentuated in successive aide-mémoires, the Bank and the Borrower should also explore solutions on the senior management level. The disbursement profile of the project can be raised through improved performance of the implementing entities and close monitoring and supervision of that performance by the Bank. Thereby, additional financial costs for the Borrower can be avoided within the scope of the project. These points are particularly salient in connection with the satisfactory

42

implementation of the ongoing health project, Health System Strengthening and Support Project. Comments by the Ministry of Health on the ICR 1. Project Components A. Support for MoH's Strategic Plan A.1. Strengthening Preventive and Primary Health Care Services Implementation: • The capacity building efforts focused on family physicians training, community health center staff training, monitoring and evaluation for family medicine practices, design and planning for the NCDs and risk factors. • A workload analysis was conducted to standardize work procedures for more effective service delivery and better quality of care by family physicians. Job descriptions of family medicine personnel were revised in line with the results of the analysis. Outcome evaluation: • 575 family physicians and 3173 CHC staff were trained. • A National Prevention and Control Program on Chronic Airway Diseases was published. • A training-of-trainers manual on asthma and COPD was prepared for family physicians. • A guideline for Home Care Service Delivery in Chest Diseases was published. • Air Quality and Health book was published. • "A Diagnosis and Treatment Guideline for Elderly Health" as well as "Elderly Health Modules - Training-for-Trainers Manual" for Family Medicine practices was published. • "A Basic Information Guideline on Accessibility for Disabled Individuals in Health Facilities" was published. • Trainer teams were formed by providing training-for-trainers on obesity and diabetes for physicians, dietitians and nurses in 81 provinces. • A total of 1446 physicians were trained on Emergency Obstetric Care. • A National Action Plan and Implementation Program on Healthy Ageing was prepared for improving elderly health services and geriatrics. • Meetings on obesity with international participation were held. • Workshops and evaluation meetings were held with the participation of universit ies, non-governmental organizations and other stakeholders at the preparation stage of National Tobacco Control Program Action Plan. Achievements: • The family medicine program was scaled up by expanding from the current 33 provinces to all 81 provinces of Turkey, as well as supporting the MoH in implementing preventive health care services for the control of communicable and non-communicab le diseases.

43

• Public awareness for non-communicable diseases and risk factors was increased, and the capacity of health staff was strengthened. • "Childhood Obesity Surveillance Initiative-3 (COSI-TUR)" is being carried out in 21 countries by the World Health Organization. Turkey participated in the 3rd round of the survey in 2012-2013 school-year, and as a result of the completion of the survey significant information on childhood obesity was obtained and the results were shared with national and international stakeholders. • Field practices of CHC staff in preventive and primary health care services were strengthened. • The need for training-related devices and equipment was met in Urla National and International Emergency Disaster Training and Simulation Center in which the health staff working in the field of prehospital health care services are trained, and it is ensured that the training are provided in a more effective manner and in high-level standards by utilizing advanced technological programs and based on the world standards. • Highly skilled human resources that cannot be recruited from the public resources were employed as "consultants" and thus the Ministry's performance was enhanced. • A capacity (labor, equipment) was built to measure, evaluate and improve the performance of the Air Ambulance system. • An increased awareness for the overall Ministry and provinces were achieved through the effective use of Emergency Health Automation System in 81 provinces, the efficiency of operational processes were increased, and the unity in implementation was ensured. A2: Implementing Public Hospital Reforms Implementation: • It was aimed to ensure that all sorts of preventive, diagnostic, therapeutic and rehabilitative health services are rendered in the health facilities affiliated to the Public Hospital Institution of Turkey, to monitor and evaluate their activities, to make performance evaluation, to prepare reports, and to establish all sorts of infrastructure for evaluation system; and the targeted activities were achieved through trainings/meetings and workshops as well as consultancy service procurements under the project. Outcome evaluation: • It is ensured that activities for effective, efficient and quality service delivery in public hospitals were implemented in line with the restructuring of the Institution. • Training was provided to 105 MoH personnel on increasing access to treatment of neuromuscular diseases and improving service quality. • Training was provided to 176 social workers on medical social service implementations. Achievements: • The MoH ensured patient and staff satisfaction at a maximum level as well as effective budget use as a result of minimizing its available resources by equally increasing performance and quality service delivery, human resources arrangements, and administrative structures in the Public Hospitals.

44

A3: Improving the Quality of Health Services Implementation: • It was aimed to carry out standardization and accreditation activities for traditional and complementary medicine practices in a proper manner by receiving consultancy support regarding legislative works for the said practices. Outcome evaluation: • Monthly reports were prepared for the activities with respect to traditional and complementary medicine practices. • Training was provided to key MoH staff on quality standards and assurance. • The 5th version of the Health Quality Standards and accreditation standards in health were developed. Achievements: • An on-site examination was made in international organizations, and contribut ions were made to standardization and international accreditation activities in training. A4: Institutional Restructuring and Capacity Building Implementation: • Family Physician trainings were provided and integrated into cancer screenings. • The current situation of the PHoI was analyzed, and its vision, mission, institutiona l values and principles were defined. • Studies were carried out on topics and activities such as human resources, project, budget, and efficiency all of which were needed within the administrative structuring of the PHoI. Outcome evaluation: • It is ensured that restructuring and capacity building activities for PHoI were implemented. Achievements: • The Cancer Institute of Turkey was established. • Cancer registry infrastructure was strengthened. Active cancer registry was initiated in the provinces where passive cancer registry had been implemented and thus active cancer registry was expanded to 81 provinces. • Regional studies and evaluations were carried out to collect and analyze cancer data in a healthy and quality manner. • Cancer policies were developed in Turkey by participating in international cancer surveys and exchanging information about cancer trends at the international level. • The PHoI acquired strategic planning skills with the restructuring. Contributions were made for the quick adaptation of the Institution's decision-makers and implementing units to the new structure. • A health system evaluation capacity was built in the Ministry of Health. • A capacity was built for scientific researches, analyses and evaluations.

45

• Evidence-Based Medicine Guidelines were developed and up-to-date versions were made available to target audience. • A national awareness about Health Technology Assessment was provided, a capacity was built, and collaborations were provided. • Studies were carried out to ensure standardization of health statistics and a capacity was built. • It is ensured that the national health statistics were collected in line with internationa l standards and shared with the relevant institutions and organizations in the process of international health statistics. In this context, our country's data was sent to WHO European Region "WHO-Europe Health for All", "OECD/Eurostat/WHO-Europe Joint Questionnaire on Non-Monetary Health Care Statistics" and "OECD Health Data" databases and thus Health Data of Turkey was updated. Additionally, launching meetings were held for checking Turkey's health data included in the reports/publications that were planned to be published by the OECD Health at a Glance and/or organizations such as WHO, EUROSTAT and OECD. • A capacity was built for software, technical infrastructure, electronic content, and distance learning management for the first time between public institutions in the Ministry of Health. • A capacity was built by developing software to determine training needs, and an Analysis Report for Health Personnel Training Needs was prepared using data mining methods. • An electronic training material capacity (as written and e-learning materials) was built by securing copyrights by themselves: 25 for Public Health topics, 18 for Health and Hospital Management topics, and 4 for Health Communication Management topics (47 in total). • A capacity was built for international article publication in which training studies are collected. • The process of Health Technology Assessment in evidence-base healthcare policy making and implementation was created at the national level. A6: Improving Health Information Systems Implementation: • It is aimed to improve the health information systems. Achievements: • Improving the health information standards, confidentiality, security and privacy principles of personal and institutional health records; establishing a data warehouse and initiating data mining practices within the scope of decision support system; and expanding the coverage of MoH National Health Information System (HIS) to establish tele-medic ine and tele-health systems were bolstered. • Human resources and information technologies capacity was built in order to measure, evaluate and improve the performance of health system. A7: Supporting Intersectoral Health Responsibility and Aligning Health Policies with World Standards

46

Implementation: • It is aimed to assist the Government Program implement intersectoral health responsibility and to align health policies with the world standards. Outcome Evaluation: • Project culture within the Ministry of Health was further strengthened through trainings and studies conducted. • A capacity (workforce, equipment) was built within the Ministry of Health to reach the objectives set under the Project. • National and international standards on health have started to be developed. • Measures were taken to ensure the sustainability of reforms realized under the Project. • New international practices were adopted thanks to the increased internationa l relations; in addition, practices specific to our country has begun to be shared with the other countries. A8: Functionalizing a National Public Health Agency Implementation: • It is aimed to build labor a force capacity and to strengthen the management information systems within the scope of arrangements realized within PHeI. Outcome Evaluation: • Evidence-based clinical practice guidelines for family physicians were developed and published on the web. • Various research studies conducted on areas including workload of FM physicians, patient and provider satisfaction, women’s health, infant and under-five mortalit ies, reimbursement and pricing policies on pharmaceuticals. • Official website of PHeI was prepared and maintained. • 34 websites were designed and maintained within PHeI. • Trainings were organized to monitor the application results of "Public Health Institution of Turkey Strategic Plan" and to build a performance program conforming to the plan. Achievements: • The capacity on standardization of health statistics was built. • New ongoing Burden of Disease Study as well as Chronic Diseases and Risk Factors Study is of importance in terms of providing evidence for challenges related to NCDs. • Health System Performance Assessment Study and Exclusive Essay of Lancet on Turkish Health reforms are two striking examples of extensive evaluation of Turkish Health System experiencing critical health reforms. • Researches on family medicine practice (e.g. workload of family physicians, patient and provider satisfaction studies) are the studies which help better shaping of family medicine practice for efficient and effective service provision. • Functionalization of Public Health Institution of Turkey was bolstered. • PHeI 2014-2017 Strategic Plan was prepared.

47

• Corporate culture of Public Health Institution of Turkey on project preparation, delivery and management was developed. • PHeI Performance Program was prepared. A9: Strengthening Collaboration with Other Countries and International Institutions in Health Policy Development, and Sharing Country Experiences Implementation: • It is aimed to contribute to global health through collaboration and development assistance. Achievements: • Collaboration with other counties and international institutions was strengthened and county experiences were shared through the activities conducted under the Project. A common Declaration of Intent, which was approved by the Ministries of Foreign Affairs of both countries, was signed in order to develop collaboration in the field of health through Turkish-German Health Symposium. This agreement is the first agreement signed between the Ministries of Health of both countries. 2. Achievements of the MoH Although the Project was rated as unsuccessful in ICR report in terms of project development objectives, it brought along critical achievements for MoH. It has started with Health Transition Project Phase 1 and 2, and then it has changed and made an important progress in health system through PSRHS, and this change has assumed a structural identity through Decree Law No. 663, which is based on four primary justifications. Turkey has realized critical improvements in service supply and demand thanks to a great health reform namely Health Transition Project (HTP) over the past decade. These improvements have reflected positively on health outcomes, trends in health financing and utilization of health services. This Project has bolstered the sustainability of all these positive developments and achievements. In addition; it facilitated the increase of employment, improvements of public services, achievement of objectives included in MoH 2013-2017 Strategic Plan, development and implementation of policies and programs set in the field of health. The performance of Project Management capacity built within the Ministry was mainta ined by being supported with trainings and human resources under this project. In the next period; the present situation, which was realized by revealing the benefits of this systems and placing the project management culture within the organization systematically, will be carried to higher levels. Other achievements of the project are as follows: • "General Directorate of Primary Health Care, Refik Saydam Hygiene Center Presidency, Mother and Child Health, Tuberculosis Control, Malaria Control, Cancer Control Units" were incorporated and Public Health Institution of Turkey was established when the Decree Law on the Organization and Duties of Ministry of Health and its

48

Affiliates No. 663 was published on the Official Gazette numbered 28103 (bis) and dated 02.11.2011. • Cancer Institute was established under the roof of Turkish Institutes of Health Sciences. • Thanks to the daily trainings given to 40.000 persons on Project Management, a project management and project planning infrastructure was built within MoH. Standard documents on project management were prepared, and project preparation standards and methods were determined during the preparation process of the project. • A capacity (workforce, equipment) was built within the MoH to reach the objectives set under the Project. • Highly skilled human resources that cannot be recruited from the public resources were employed as "consultants" and thus the Ministry's performance was enhanced. • A capacity was built in order to measure, evaluate and improve the performance of health system. • National and international standards on health have started to be developed. • Measures were taken to ensure the sustainability of reforms realized under the Project. • Family medicine practice was initiated throughout the county. • A capacity was built in order to measure and evaluate the healthcare services. • New international practices were adopted thanks to the increased internationa l relations; in addition, practices specific to our country has begun to be shared with the other countries. 3. Lessons Learned Fundamental changes taken place in the organizational structure of the Ministry of Health during the preparation process of the project and incorporation of General Directorate of Primary Health Care, Refik Saydam Hygiene Center Presidency, Mother and Child Health, Tuberculosis Control, Malaria Control, Cancer Control Units within the Public Health Institution of Turkey and the new structure emerged as a result of these changes resulted in serious problems in terms of management and execution of the project. These problems also had a substantial impact on coordination works. The subtitles of the new project and the new executives were defined within the context of reorganization, and all stages were expressed to the units, which did not take part in the process earlier, from the very beginning. Subsequent involvement of implementing units in the project phase resulted in difficulties in execution and management stages of the project at times. Problems encountered during the implementation of Project had also determined the potential development opportunities. Efficient maintenance of the coordination process is important in terms of getting the expected results from the projects. Comments by the Social Security Institution on the ICR 1. Background The agreement for "Health Transformation and Social Security Reform Project" to be carried out by the Ministry of Health and the Presidency of the Social Security Institut io n (SSI) was signed on 30 June 2009 between the Republic of Turkey and the Internationa l Bank of Reconstruction and Development. €56.10 million was allocated for the Project that

49

was to be carried out by the Ministry of Health (MoH) and the Social Security Institut ion (SSI). €3.090 million of said allocation was set aside for use in funding the activities carried out by the Social Security Institution. The loan granted to SSI was allocated for realization of the actions included under Component C of the “HTSSRP Loan Agreement”. In the original version of the loan agreement, Component C titled “Capacity Building of the SSI” reads as follows:

• C.1. Provision of technical assistance and training for SSI, with the objective of enhancing SSI’s capacity for the implementation of universal health insurance, in the following areas: (a) implementation and promotion of universal health insurance regulations, health provider contract management, invoice control, benefits package, monitoring and evaluation; and (b) expenditure management in SSI, includ ing expenditure tracking and reporting and strengthening of relevant expenditure databases, supporting the Reimbursement Commission of SSI in improving their capacity to make payment decision based on available evidence-based criteria. • C.2. Provision of support to the SSI’s Project Implementation Unit for effective Project management.

In this framework, no loan portions were disbursed for any of the activities in 2009. During the 2010-2012 activity periods of the project, activities carried out were funded under the budget lines for consultancy services, training and project overheads. In SSI's request filed with the Undersecretariat of Treasury on 31/01/2011, it was stated that a significant portion of the granted loan could not be used due to high operational costs and since the need for consultancy and training services, which constitute the loan components, has already been met, and it was requested that the loan be cancelled after setting aside an appropriate amount for the "Activity to Develop Asynchronous Training Monitors to Enable On-line Delivery of SSI Trainings" along with €220,559.76 corresponding to one-year remuneration for the Training consultant. However, it was later decided that the loan be utilized by the Directorate General of Universal Health Insurance in order for the realization of the actions included under Component C of the "HTSSRP Loan Agreement", which the Undersecretariat of Treasury had been requested to cancel; hence, the Undersecretariat of Treasury was asked to disregard our letter dated 26/12/2011 and numbered 23509416 on the cancellation of the loan. With the amendment made in the Loan Agreement, Component C of the Project was redefined as follows:

• Provision of technical assistance and training for building the capacity of the SSI: Establishment of a distance learning system for universal health insurance applications. • Provision of support to the SSI Project Implementation Unit for effective project management including the financing of operational costs. • This sub-component will finance the Project Coordination Unit for effective management of the project.

50

In line with the objective of said component, for the purpose of providing technical support and capacity building through better management of the policy and programme decisions included under the scope of the Universal Health Insurance, the expenditure components and outputs of the 3 (three) projects of the Directorate General of Universal health Insurance were sent to the Ministry of Development with our letter dated 16/04/2012 and numbered 7174484 as required under Footnote 60: "Spending shall be enabled only after the project expenditure components and outputs are defined and approved by the Ministry of Development" in the 2012 Investment Program. The reply letter of the Ministry of Development stated, in brief, that all the newly proposed projects cover capacity-building actions and are characteristic training-consultancy projects, that the projects included in the 2012 investment program and the new projects mostly consist of institutional capacity-building components of the same characterist ics, that the work processes planned within the project scope cannot possibly yield the expected results fully in the one-year period remaining for project completion, and that the actions that have significance for the Institution should be enabled by making a revision in the 2012 investment program, consequently informing that spending within the scope of 2009K130150 Health Transformation and Social Security Reform Project was not found appropriate. In this context, the Undersecretariat of Treasury was asked to take the necessary actions for concluding the cancellation transactions for the loan granted to our Institution with the letter dated 27/07/2012 and numbered 585562. 2. Health Transformation and Social Security Reform Project (7717-Tu) Implementa t io n Results No loan portions were used in 2009 for the Health Transformation and Social Security Reform Project. The actions implemented in 2010-2012 and the loan portions used within the scope of each action are summarized below. 2.1.Consultancy Services Increasing the Effectiveness of the UHI: A training consultant was employed to train the personnel and the institutional development consultants for the purpose of increasing the effectiveness of the universal health insurance. The consultants were paid €141,622.03 (TL 280,993.01) for 2010; €42,742.63 (TL 97,523.16) for 2011 and €2,994.07 (TL 7,064.80) for 2012. Improving the Existing Computer Network: A consultant was hired to identify the Information and Communication Technologies (ICT) requirements, assess the existing network and improve the network in accordance with future internet traffic requirements, including providing support to the necessary software development works of the relevant departments of the SSI and carrying out the activities related to information technologies. The consultant was paid €4,780.32 (TL 9,786.75) in 2010.

51

Monitoring & Evaluation: A consultant was hired for monitoring and evaluation of SSI’s medium and long-term programs, main principles and policies, objectives and priorit ies and their conformity to international social rights norms and criteria, and for supporting the projects and programs prepared to this end. The consultant was paid: €34,048.35 (TL 67,193.78) in 2010 and €3,079.24 (TL 6,226.22) in 2011. Personnel Expenses: Project Director, a Contract Manager, a Financial Expert, an Accountant, and a Translator were hired to work at the Project Coordination Unit The personnel was paid €156,135.21 (TL 308,395.96) in 2010 and €13,519.70 (TL 27,336.82) in 2011. 2.2.Training Services Printing the Information Materials: The "activity to print the training materials aimed at informing all the segments served by our Institution, within the scope of the Social Security Week that will be held on 10-16 May 2010" was carried out for the purpose of providing technical support and training within the scope of building SSI's capacity. Tender proceedings were carried out for contracting the training activities to be covered under the Training item of the Loan for Health Transformation and Social Security Reform Project no.7717-TU, with the "Shopping Procedures" in accordance with Article 3.5 of Chapter III of the Guidelines "Procurement Under IBRD Loans and IDA Credits" of the World Bank. Contract value is TL 146,598.00.-+ VAT (€90,016.98-). Action to Develop Asynchronous Training Monitors to Enable On-line Delivery of the Trainings of the Social Security Institution: In order to enable on-line delivery of the legislation trainings of the staff of the central and local organization of our Institution via asynchronous distance learning method, the tender proceedings for the "Activity to Develop Asynchronous Training Monitors to Enable On-line Delivery of the Trainings of the Social Security Institution" were carried out in line with the request filed by our Human Resources Department dated 03/11/2010 and numbered 16803420. With said activity, it was aimed to ensure more efficient and more effective delivery of SSI services by using asynchronous modes of training to meet miscellaneous training needs of the staff with regard to Universal Health Insurance and Social Insurances transactions, with the ultimate purpose of supporting the Social Security Reform, improving Institutional capacity, and ensuring that the central and local staff receive training on Social Security Reforms in a timely manner. Tender proceedings were carried out for contracting the training activit ies to be covered under the Training item of the Loan for Health Transformation and Social Security Reform Project no.7717-TU, with the "Shopping Procedures" in accordance with Article 3.5 of Chapter III of the Guidelines "Procurement Under IBRD Loans and IDA Credits" of the World Bank. Contract was signed with the winner on 16.02.2011. Contract value is TL 286,500.00.-+ VAT. The activity was completed on 10/04/2012. For development of asynchronous training monitors, the following payments were made: TL 28,650 (€11,736.03-) in 2011 and €111,142.99 (TL 257,850.00) in 2012, making a total sum of €122,879.02 (TL 286,500.00).

52

2.3.Project Overheads Project operational expenses consist of the advertisement costs for the procurement tender announcements, and vehicle and gas costs for the motor vehicles of the directorate. For these expenses, €1,958.49- (TL 3,998.65) was paid in 2010 and €632.89- (TL 1,279.71) in 2011. There were no project overheads for 2012. 3. Financial Status of the Project Our Institution was allocated €3.09 million with the “Health Transformation and Social Security Reform Project”. As of 03/12/2012, the loan portions utilized for the Project as per SSI’s activity categories are as follows. Table 1. Project Loan Portions Utilized as per Activity Categories (Euro)

Republic of Turkey Social Security Institution Project Coordination Unit Health Transformation and Social Security Reform Project 7717 TU / DFN-39259

(EURO)

Category Allocated Used Remaining Amount

Loan Use Percentage

A. Consultancy Services 2,410,000.00 398,921.55 2,011,078.45 16.55% B. Training 600,000.00 212,896.00 387,104.00 35.48% C. Project Overheads 80,000.00 2,591.38 77,408.62 3.23% Total 3,090,000.00 614,408.93 2,475,591.07 19.88% Table 2. Cumulative Investments per Years and Activity Categories (Euro)

Republic of Turkey Social Security Institution Project Coordination Unit Health Transformation and Social Security Reform Project 7717 TU / DFN-39259

(EURO)

Project Activities Realized

2010 2011 2012 TOTAL

A. Consultancy Services 336,585.91 59,341.57 2,994.07 398,921.55 Individual Consultants and Support Personnel for

Increasing the Effectiveness of the UHI 141,622.03 42,742.63 2,994.07 187,358.73

Improving the Existing Computer Network 4,780.32 0.00 0.00 4,780.32

Monitoring & Evaluation Consultancy 34,048.35 3,079.24 0.00 37,127.59

Personnel Expenses 156,135.21 13,519.70 0.00 169,654.91

B. Training 90,016.98 11,736.03 111,142.99 212,896.00 Developing, Printing and Distributing Training

Materials 90,016.98 0.00 0.00 90,016.98

Distance Learning Program 0.00 11,736.03 111,142.99 122,879.02

C. Project Overheads 1,958.49 632.89 0.00 2,591.38

Operational Expenses 1,958.49 632.89 0.00 2,591.38

Total 428,561.38 71,710.49 114,137.06 614,408.93

53

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable.

54

Annex 9. List of Supporting Documents

1. Project Appraisal Document: Turkey Health Transition Project, dated April 20, 2004, (Report No: 27717-TR);

2. Turkey Health Transition Project, dated July 14, 2010, ICR Report No. ICR780; 3. Project Appraisal Document: Turkey Project In Support of Restructuring of Health

Sector, dated May 18, 2009 (Report No: 46115-TR); 4. Aide Memoires, Back-to-office Reports, Implementation Status Reports and

Project Implementation Plan; 5. Turkey Country Partnership Strategy (CPS) for the period FY08-FY11, dated

January 25, 2008, Report No. 42026-TR; 6. Restructuring Papers on Proposed Project Restructurings of the Project In Support

of Restructuring of Health Sector, Loan Number 7717-TR, Report No: RES9770-TR, dated March 26, 2013; Report No: RES12001, Board approval dated January 7, 2014; Report No: RES19050-TR, dated May 28, 2015;

7. Project progress reports, as well as key Technical Assistance reports.

55

Annex 10: Assessment of Project Achievement towards Outcomes This annex provides complementary discussion on assessment of project outcomes. (A) Status of PDO and Intermediate Outcome Indicators Before and After Restructuring Tables 1 and 2 summarize the PDO indicators and Intermediate Outcome Indicators (IOIs) before and after restructuring and provide the reasons for revision and cancellation at the time of the March 2013 and January 2014 restructurings. In this section, two restructur ings changing the PDO indicators and IOIs were considered as one ultimate restructuring. There were 6 original PDO indicators and 13 original IOIs. After restructuring, there were a total of 4 PDO indicators and 9 IOIs.32 Table 1: Original PDO indicators and reasons for change at restructuring

Original PDO Indicator New PDO Indicator Reasons for change Restructuring Increase in the utilization of primary health care

Per capita annual visits to family medicine physicians

The revision is made to address the need to evaluate the utilization of primary care services which are provided by family medicine physicians only.

March 2013, Level 2

Increase in the percentage of first-time visits to specialists that are referred by family medicine practices

Dropped as MoH could not provide the breakdown of the first time visits to specialists. The lack of a mandatory referral system nationwide and the disconnection between the Family Medicine Information System (FMIS) and the Health-Net made the monitoring of this indicator difficult.

March 2013, Level 2

Share of FM visits over total visits. Added. March 2013, Level 2 Increased client satisfaction with health services

Client satisfaction with health services

As the annual changes in the client satisfaction might vary, it makes it difficult to track the trend by comparing year to year increases; this PDO indicator was revised to track the overall trend in the client satisfaction.

March 2013, Level 2

32 With the level 1 restructuring on January 7, 2014, 4 IRIs related with Component A, which were added during the first restructuring on March 26, 2013, were dropped as they have already been achieved. However, the restructuring paper noted that these IRIs would be in ICR. Therefore, the Bank teams monitored the progress of the dropped IRIs.

56

Original PDO Indicator New PDO Indicator Reasons for change Restructuring MoH reorganized and focus

exclusively on health sector stewardship function:

Redefining the role of the MoH as a steward (focusing on policy formulation, regulation and monitoring and evaluation) rather than a provider for the health sector was an important objective of the Project, this indicator was added during the restructuring of January 2014.

March 2013, Level 2

Smoking prevalence among 18-29 age group in Output-Based Financing pilot provinces decreases more relative to non-pilot provinces

Dropped with the cancellation of component B January 2014, Level 1

High blood pressure in population in Output-Based-Financing pilot provinces decreases more as compared to non-pilot provinces

Dropped with the cancellation of component B January 2014, Level 1

Decrease in SSI expenditures on pharmaceuticals and outpatient health care

Dropped with the cancellation of component C January 2014, Level 1

Table 2: Original Intermediate Outcome Indicators and reasons for change at restructuring

Original Intermediate Outcome Indicators New Intermediate Outcome Indicators Reasons for change Component A Percentage of family medicine practices that have completed second level training as compared with baseline

Annual performance reports on implementation of MoH Strategic Plan using information from MOH Monitoring and Evaluation system disseminated by MOH to Parliament and published on MOH website

MoH Annual performance program prepared and submitted to the National Assembly

Revised for clarification during the restructuring in March 2013.

Percentage of acute care public hospitals using revised clinical guidelines for secondary care as requirement of performance payment system

Dropped during the restructuring in March 2013. No reason given in the Bank’s restructuring paper

Dissemination of bi-annual Program Tracking reports by M&E Directorate of MOH (in PAD)


Percentage of family physicians using rational drug prescription guidelines


57

Original Intermediate Outcome Indicators New Intermediate Outcome Indicators Reasons for change Percentage of MoH Implementing Units for Project have formed Project Offices and connected to PMSU-Net for Project monitoring (in PAD)


Number of provinces implementing family medicine scheme

Added during the restructuring in March 2013. The indicator is dropped during January 2014 restructuring as the nationwide roll-out of the family medicine scheme is completed. Still, the Bank teams monitored the progress of the dropped IRIIOI as the restructuring paper noted that these IRI IOIs would be included in the ICR.

Establishment of Public Health Institution 33 responsible for health promotion, disease prevention and family medicine


Share of public hospitals organized in public hospital unions with performance contracts for managers and global budgets


Adoption of National Action Plans for (i) Health Promotion; (ii) Healthy Aging; (iii) Diabetes; (iv) Obesity; (v) Cardiovascular Diseases (CVD); (vi) Chronic Respiratory Diseases (CRD)

Added during the restructuring in March 2013. The indicator is introduced to monitor linkages between activities such as consultancies and final influence on policies. Wording changed in January 2014, the word adoption has been replaced by preparation and submission which seemed more realistic in the given timespan.

33 With the 2011 MoH reorganization after the enactment of the Decree Law which, the Public Health Institution became an affiliated agency of the MoH.

58

Original Intermediate Outcome Indicators New Intermediate Outcome Indicators Reasons for change Development of the Clinical Guideline

Preparation Manual

Establishment of the Pharmaceuticals and Medical Devices Agency of Turkey

New indicator (March 2013) Dropped in January 2014 The indicator is dropped since the Pharmaceuticals and Medical Devices Agency of Turkey is established and the target was reached, but the Bank teams monitored the progress of the dropped IOI as the restructuring paper noted that these IOIs would be included in the ICR.

Number of international conferences to share Turkey’s health reform experience

New indicator (March 2013) The indicator is added to reflect the new sub-component’s (A.9) target of strengthening collaboration with other countries and sharing country experiences

Component B Percentage of population of pilot provinces screened for NCD risk factors

Dropped in January 2014, with the cancellation of Comp B but is included in the ICR.

Higher increase in appropriate knowledge and attitudes regarding global risk factors for NCD in pilot provinces as compared with non-pilot provinces


Percentage of cases in pilot provinces diagnosed and under follow-up by family group practices for the following NCD: hypertension, diabetes, obesity/overweight

Dropped in January 2014, with the cancellation of Comp B, no pilot was conducted.

100% compliance with reporting on implementation of screening activities as well as patient records


Component C Actuarial analysis of UHI updated for 2010 and 2012 based on updated macro and micro data

Dropped in January 2014, with the partial cancellation of Comp C but is included in the ICR.

Costly interventions under Benefits Package for UHI revised using cost-effectiveness criteria and guidelines developed by SSI

Dropped in January 2014, with the partial cancellation of Comp C but is included in the ICR.

Prospective payment systems based on international evidence (capitation, global budgets,

Dropped in January 2014 but is included in the ICR Dropped with the cancellation of Comp C.

59

Original Intermediate Outcome Indicators New Intermediate Outcome Indicators Reasons for change DRGs) or outpatient and acute care adopted for University, private and MOH hospitals

(B) Indicators achievements before restructuring Before restructuring, out of 6 original PDO indicators, 2 were achieved, 1 was not achieved, 1 was identified to be on track, and 2 were not materialized (Table 3). Although progress was observed in Component A, none of the indicators associated to Components B and C materialized or were achieved. Table 3: PDO indicators achievements before restructuring

PDO Indicators Baseline (2009) Target for July 2013 Actual as of March 2013 Assessment Component A Increase in the utilization of primary health care

2.00 4.00 3.30 On track with the increasing trend

Increase in the percent of first-time visits to specialists that are referrals by FM practices

1.2 1.5 1.5 Achieved

Client satisfaction with health services

63.4 70 89.8 Achieved

Component B Smoking prevalence among 18-29 age group in pilot provinces relative to non-pilot provinces

Baseline value was supposed to be identified by NCD survey to be conducted in 2011

The OBF component to be deleted as a part of the Project restructuring.

Not materialized

High blood pressure in population in pilot provinces decreases more as compared to non-pilot provinces

Baseline value was supposed to be identified by NCD survey to be conducted in 2011

The OBF component to be deleted as a part of the Project restructuring.

Not materialized

Component C Decrease in SSI expenditures on pharmaceuticals and outpatient health care

40.00 (40% of expenditures on pharmaceuticals 50% of treatment costs on inpatient)

33.00 (33% for pharma exp, 42% for inpatient exp)

41.8 (38.7% in the end of 2011 - SSI expenditures on pharmaceuticals as a percent of SSI spending)

Not achieved

60

Intermediate Outcome indicators for Component A presented a diverse picture: Out of six IOIs, one was achieved, two were showing progress and three were subject to revision. IOIs for component B did not materialize since this component never started and was cancelled. Similarly, IOIs for component C were not measured since activities related with these indicators were not conducted under the Project (Table 4). Table 4: IOIs achievements before restructuring

Intermediate Outcome Indicators Baseline (2009) Target for July 2013 Actual as of March 2013 Assessment COMPONENT A

Percentage of family medicine practices that have completed second level training as compared with baseline

0 80.00 21.00 Showed progress

Annual performance reports on implementation of MoH Strategic Plan using information from MoH Monitoring and Evaluation system disseminated by MoH to Parliament and published on MoH website

Non-existent Third report published 2011 performance program published

Showed progress

Percentage of acute care public hospitals using revised clinical guidelines for secondary care as requirement of performance payment system

Baseline will be collected in 2000

TBD Was subject to revision

Dissemination of bi-annual Program Tracking reports by M&E Directorate of MoH

Not available 1 bi-annual report HSPA report published Achieved

Percentage of family physicians using rational drug prescription guidelines

Not available 30.00 0.00 Was subject to revision

Percentage of MoH Implementing Units for Project have formed Project Offices and connected to PMSU-Net for Project monitoring

0 100 0 Was subject to revision

61

Intermediate Outcome Indicators Baseline (2009) Target for July 2013 Actual as of March 2013 Assessment COMPONENT B

Percentage of population of pilot provinces screened for NCD risk factors

0 70 0 Not materialized

Increase in appropriate knowledge and attitudes regarding global risk factors for NCDs in pilot provinces as compared with non-pilot provinces


Percentage of cases in pilot provinces diagnosed and under follow-up by family group practices for the following NCDs: hypertension, diabetes, obesity/overweight


100percent compliance with reporting on implementation of screening activities as well as patient records


COMPONENT C

Actuarial analysis of UHI updated for 2010 and 2012 based on updated macro and micro data

1st actuarial analysis completed

2 actuarial analysis of UHI conducted in 2010 and 2012 respectively

Not conducted under the Project

Not measured


Review of the benefits package and identification of interventions and baseline expenditures

Decrease in the spending on the interventions as compared with baseline


Not measured

Prospective payment systems based on international evidence (capitation, global budgets, DRGs) or outpatient and acute care adopted for University, private and MOH hospitals

Stock taking and development of action plan for implementation of prospective payment systems

Relevant prospective payment systems implemented


Not measured

62

(C) Indicator achievements after restructuring Out of four PDO indicators, only one indicator was achieved. Table 5: PDO indicator achievements after restructuring

Indicator Baseline (2009) Target (2015) Actual (2015) Assessment Component A Per-capita annual visits to family medicine physicians

2 4 2.76 Not Achieved

Share of family medicine visits in total visits 35 40 33.35 Not Achieved Client satisfaction with health services 63.4 76 71.2 Not Achieved Ministry of Health reorganized and focus exclusively on the health sector stewardship function

MOH is a provider of health services

MOH is restructured and no longer a provider of health services.

Achieved. MOH is restructured and no longer considered as a provider of health services

All of the five intermediate outcome indicators were achieved. Other intermediate outcome indicators were dropped during the restructuring (Table 6). Table 6: IOI achievements after restructuring

Indicator Baseline (2009) Target (2015) Actual (2015) Assessment Establishment of Public Health Institution responsible for health promotion, disease prevention and family medicine



Public Health Institution fully operational Achieved

MoH Annual performance program prepared and submitted to the Parliament

0 Yes Annual Performance Programs of 2014 and 2015 were submitted to the Parliament

Achieved

Adoption of National Action Plans for (i) Health Promotion (ii) Healthy aging (iii) Diabetes (iv) Obesity (v) CVD (vi) CRD

4 6 6 Achieved

Development of the Clinical Guideline Preparation Manual

No Clinical Guideline Preparation Manual exists

Guideline prepared and disseminated

Guideline prepared and disseminated Achieved

Number of international conferences to share Turkey’s health reform experience

0 2 3 Achieved and target surpassed

63

(D) Achievements of PDO indicators (D) (1) Client Satisfaction with health services: Data from TUIK life satisfaction surveys of 2011 and 2014 show that the perception of the population on the problem areas of the health care system deteriorated implying that more people think that co-payments, test fees, number of health care staff, medicine prices, waiting time and medical examination are problematic areas of the health care system. In addition, perception of the population on the problem areas of the health care system changed for the worse at the family medicine centers as opposed to public hospitals: percent change in people using family medicine centers and perceiving co-payments, test fees, number of staff, medicine prices, waiting time and medical examination as a problem is greater than the percent change in people using public hospitals and perceiving co-payments, test fees, number of staff, medicine prices as a problem in public hospitals. Perception related to waiting time and medical examination at public hospitals shows as improving (Figure 1). Figure 1 Figure 2

Source: TUIK Life Satisfaction Surveys 2011 and 2014 Source: MoH Statistical Yearbook, 2014 Health care utilization data show a clear indication of decline in patient satisfaction as well. Figure 2 shows a decline in per capita visits at the primary care level since 2011 whereas per capita visits to MoH hospitals had a steady increase. As elaborated in the discussion on family medicine utilization indicators (visits to family physicians), the decline in patient satisfaction concerning waiting time and medical examination may be highly related with the insufficient number of family physicians and family medicine staff. High patient workload is a factor that adversely affects the medical examination time and quality. Supply side increases for primary care are not

64

promising either. Figure 3 below suggests that specialization is still the priority choice for medical school graduates and increase in the number of specialist physicians is higher than the increase in the number of general practitioners during the period 2010-2014. Figure 3

Source: MoH Statistical Yearbook, 2014 (D) (2) and (D) (3) Per capita annual visits to family medicine physicians and Share of family medicine visits in total visits These two utilization indicators were not achieved. Health human resources for family medicine are insufficient and the number of registered patients to each family physician is high; reducing the time allocated to each patient. This can negatively influence patient satisfaction and reducing per capita visits for family physicians. Since 2013, there has not been a significant increase in the number of family physicians (Table 7). Table 7: Family Medicine Department of the PHeI data

2011 2012 2013 2014 2015 Number of family physicians 20.216 20.060 20.497 20.724 20.760 Number of registered population per family physician 3.538 3.582 3.565 3.602 3.606

Source: Family Medicine Department of the PHeI The absence of a mandatory referral system and the fact that the public see family medicine centers as a place to go for prescription or to obtain referral for hospitals care are other reasons for reduced per capita visits. Almost half of the population with social insurance and around two thirds of those without social insurance go to public hospitals. Only around one third of both groups seek care at the

65

primary level, through family medicine centers (Figure 4). Additionally, more patients prefer public hospitals than family medicine centers for perceived better services, perceived necessity to visit hospitals and low co-payments (Figure 5). Figure 4 Figure 5

Source: TUIK Life Satisfaction Survey 2014

(D) (4) Ministry of Health reorganized and focus exclusively on the health sector stewardship function: The PDO indicator concerning the stewardship function of the MoH was achieved with the approval of the Decree Law of 2011. The establishment of the PHeI, PHoI, and PMDA was completed within a year following the enactment of the Decree Law. The Cancer Institute is expected to be established Turkish Institutes of Health Sciences34 (TUSEB), which aims to promote research and development in health sciences and technology. (E) Recent update on dropped and Achieved PDO indicators and Intermediate Outcome Indicators

34 http://www.resmigazete.gov.tr/eskiler/2014/11/20141126-3.htm http://saglik.gov.tr/TR/dosya/1-94578/h/turkiye-saglik-enstituleri-baskanligi.pdf

66

http://www.resmigazete.gov.tr/eskiler/2014/11/20141126-3.htm

http://saglik.gov.tr/TR/dosya/1-94578/h/turkiye-saglik-enstituleri-baskanligi.pdf

Table 8: Dropped PDO indicators

Indicator Baseline (2009) Target (2015)

Actual (2015) Status

Smoking prevalence among 18-29 age group in pilot provinces relative to non-pilot provinces

GATS 2008 31.20% WHO data (2012) (M) 42.7% (F) 13.5%

- GATS 2012 27.10 % WHO data (2015) (M) 39.5% (F) 12.4%

Dropped during Level 1 restructuring (January 2014) together with the deletion of component B on output-based financing.

High blood pressure in population in pilot provinces decreases more as compared to non-pilot provinces

HT prevalence according to Turkish Society of Hypertension and Renal Disease study of 2003: Total: 31.8% (M) 27.5% (F) 36.1%

- HT prevalence according to Turkish Society of Hypertension and Renal Disease study of 2012: Total: 30.3% (M) 28.4% (F) 32.3%

Dropped during Level 1 restructuring (January 2014) together with the deletion of component B on output-based financing

Smoking prevalence baseline and actual data values provided in the last ISR of September 2015 were taken from GATS survey of 2008 and 2012 in Turkey. The smoking prevalence rates take into account the overall population as opposed to 18-29 age group as defined for the output-based financing component indicator. Global Adult Tobacco Surveys (GATS) of 2008 and 201235 suggest a decrease in the smoking prevalence among adults from 31.2% in 2008 to 27.1% in 2012, representing a 13.1% relative decline. Smoking prevalence among males decreased from 47.9% in 2008 to 41.5% in 2012 implying a relative decline of 13.4%. Smoking prevalence among females decreased from 15.2% in 2008 to 13.1% in 2012 implying a relative decline of 13.8%. More recent data from WHO36 also suggests decline in the smoking prevalence for both gender; smoking prevalence among males has decreased from 42.7% in 2012 to 39.5% in 2015 implying a relative decline of 7.5%. Smoking prevalence among females has declined from 13.5% in 2012 to 12.4% in 2015 implying a relative decline of 8.1%. Best buy interventions on tobacco control were implemented succesfully by MoH and contributed

35 http://www.who.int/tobacco/surveillance/survey/gats/gats_turkey_2008v2012_comparison_fact_sheet.pdf?ua=1 36 http://apps.who.int/gho/data/node.main.271?lang=en

67

http://www.who.int/tobacco/surveillance/survey/gats/gats_turkey_2008v2012_comparison_fact_sheet.pdf?ua=1

http://apps.who.int/gho/data/node.main.271?lang=en

to the decrease in the smoking prevalence. The interventions included tax increases,37 smoke-free indoor workplaces and public places,38 health information and warnings39 and bans on tobacco advertising, promotion and sponsorship.40 Hypertension prevalence: Two studies conducted by Turkish Society of Hypertension and Renal Diseases41 in 2003 and 2012 indicate a slight decline in hypertension prevalence among the general population from 31.8% to 30.3% implying a change of 4.7%. Decline for female hypertension prevalence was significant from 36.1% to 32.3% with a relative decline of 10.5%. Male hypertension prevalence however increased from 27.5% to 28.4% with a relative increase of 3.3%. Same studies indicate an increased awareness among population for hypertension: from 40% in 2003 to 54.7% in 2012. Antihypertensive drug use among overall population has remarkably increased during this period, from 31% in 2003 to 47.5% in 2012. To conclude, while the change in the HT prevalence is not very significant, the significant improvement in population awareness and behavior change as reflected to antihypertensive drug use, might lead to improved disease burden in the medium term. The preliminary findings for the 2013 Burden of Disease study42 indicate that both smoking and hypertension are among the most important risk factors of NCDs. Table 10: Intermediate Outcome Indicators Achieved at the Time of Restructuring

Indicator Baseline (2009) Target (2015) Actual (2015) Remarks Number of provinces implementing family medicine scheme

31 81 81 Family medicine system started in 2005 and rolled out throughout Turkey in phases (started in one pilot province in 2005, 6 provinces joined in 2006, 7 provinces in 2007, 17 provinces in 2008, 4 provinces in 2009 and finally 46 provinces in 2010).

Establishment of Public Health Institution responsible for health promotion, disease prevention and family medicine




Public Health Institution was established in November 2011 with the Decree Law No. 663

37 Taxes were first increased in 2003, and since then various tax measures have been implemented. In 2002, the total tax rate on tobacco was 64%. In 2011, it was 80%. This level is in compliance with WHO recommendations. 38 Tobacco control law 5727 in 2008 (revised version of 1996 Law – 4207) - introduced a complete ban on smoking in closed public places 39 Tobacco control law 5727 necessitates the inclusion of health information and warning on tobacco products 40 Tobacco control law 5727 prohibited mass media advertising, promotion and sponsorship of tobacco products 41 Turkey Hypertension Prevalence Studies PatenT and PatenT2 in 2003 and 2012 42 www.tusaf.org/EN/dosya/2.../tusafnazan-yardim.pptx (the study has not yet been published and the results are taken from N. Yardim’s presentation)

68

http://www.tusaf.org/EN/dosya/2.../tusafnazan-yardim.pptx

Indicator Baseline (2009) Target (2015) Actual (2015) Remarks Share of public hospitals organized in public hospital unions with performance contracts for managers and global budgets

0.00 100.00 100.00 Public Hospitals Institution was established in 2012 and supervises public hospitals’ financial and administrative compliance through its local organizations, Public Hospital Unions. All public hospitals are run by Public Hospital Unions. The Global Budget has been introduced in 2006 for public hospitals and basically provides flexibility to MoH on reimbursements against a capped budget.

Establishment of the Pharmaceuticals and Medical Devices Agency of Turkey

No Pharmaceuticals and Medical Devices Agency of Turkey in existence

Agency Established and responsible for licensing and regulation of drugs and medical devices

Agency Established and responsible for licensing and regulation of drugs and medical devices

The Agency was established in November 2011 by the approval of the Decree Law on the Organization and Duties of the Ministry of Health and Its Affiliates

Percentage of population of pilot provinces screened for NCD risk factors

0 - 0 General Remarks for Turkey: TUIK’ 2014 Health Survey 43 suggests that 37.6 % of the 15+ age population had blood sugar examination within the last 12 months and 34.4 % of this population had blood cholesterol examination within the last 12 months. While these values do not indicate formal screening rates, they can nevertheless be used as rough proxies.

Increase in appropriate knowledge and attitudes regarding global risk factors for NCDs in pilot provinces as compared with non-pilot provinces

0 - 0 General remarks for the awareness in Turkey: Hypertension Prevalence Studies PatenT and PatenT2 in 2003 and 2012 suggest an overall increase in awareness against hypertension (HT): Overall population HT awareness increased from 40

43 http://www.tuik.gov.tr/PreHaberBultenleri.do?id=18854

69

Indicator Baseline (2009) Target (2015) Actual (2015) Remarks % to 54.7 %, for males HT awareness increased from 28 % to 40.6 % and for females HT awareness increased from 48 % to 66.9 %. TURDEP I (2002) and TURDEP II (2010) studies 44 suggest a decrease in the diabetes awareness: overall population awareness decreased from 67.7 % in 2002 to 54.5 % in 2010.

Percentage of cases in pilot provinces diagnosed and under follow-up by family group practices for the following NCDs: hypertension, diabetes, obesity/overweight

- - 0 General Remarks for Turkey: Recently published MoH Statistical Yearbook 2014 indicates that 33.7 % of the population is overweight and 19.9 % of the population is obese. Preliminary findings of the 2013 Burden of disease study 45 suggest an increase of 4 % for the NCD disease burden between 2002 and 2012. While top five causes of the disease burden are all NCDs, diabetes is the 4th top cause (3.8 %). Same study also highlights smoking, hypertension and obesity as the most important risk factors for NCDs.

100% compliance with reporting on implementation of screening activities as well as patient records (through family medicine information system)

0 - 0 No data could be provided by MoH on the reporting compliance for screening activities and patient records, Rate of submission of reports for online protocols to Heath.NET has been reported as 90 % for 2014 and 89.3 % for the first half.

44 http://diyabet.gov.tr/content/files/bilimsel_arastirmalar/turdep_1_turdep_2.pdf 45 www.tusaf.org/EN/dosya/2.../tusafnazan-yardim.pptx (the study has not yet been published and the results are taken from N. Yardim’s presentation)

70

http://diyabet.gov.tr/content/files/bilimsel_arastirmalar/turdep_1_turdep_2.pdf

http://www.tusaf.org/EN/dosya/2.../tusafnazan-yardim.pptx

Indicator Baseline (2009) Target (2015) Actual (2015) Remarks Actuarial analysis of UHI updated for 2010 and 2012 based on updated macro and micro data

1st actuarial analysis completed

- - No update could be obtained from SSI.


Review of Benefits Package and identification of interventions and baseline expenditures


Prospective payment systems based on international evidence (capitation, global budgets, DRGs) or outpatient and acute care adopted for University, private and MOH hospitals

Stock-taking and development of action plan for implementation of prospective payment systems


71

Strategic Map for Turkish Health System

Sub-component

Activities/outputs and Health system dimension addressed (dimensions addressed by project activities are shown in blue on the map)

A1 Training and establishment of training systems for primary health care physicians and workers (Resources: Improve and maintain distribution, competence and motivation of health human resources)

Development of NCD control programs (Services: Strengthen health promotion and disease prevention)

NCD surveys (Stewardship: Create intelligence, monitor and evaluate) Establishment of vital registration system on deaths (Stewardship:

Create intelligence, monitor and evaluate and Resources: Enhance Management Information Systems and upgrade information technology)

A2 Training of MoH personnel and social workers (Services: Improve quality and safety of health care services and pharmaceuticals)

A3 Training of MoH staff on quality standards, and assurance, DRG system (Services: Improve quality and safety of health care services and pharmaceuticals)

Development/update of health quality standards and accreditation standards (Services: Improve quality and safety of health care services and pharmaceuticals and Stewardship: Increase MoH capacity for planning, regulation and supervision)

72

A4 Health System Performance Assessment Study, training on research techniques, cancer registry training, strengthening of cancer registry infrastructure and registry standards (Stewardship: Create intelligence, monitor and evaluate and Increase MoH capacity for planning, regulation and supervision)

Various trainings on cancer screening (Services: Improve quality and safety of health care services and pharmaceuticals)

A5 A6 Capacity building on HMIS, improvements on HMIS in terms of

standards, health records, drug records, telemedicine implementations (Resources: Enhance Management Information Systems and upgrade information technology and Improve and maintain capacity, quality and distribution of health system infrastructure)

A7 Capacity building for GDHI (Stewardship: Increase MoH capacity for planning, regulation and supervision and Resources: Improve and maintain capacity, quality and distribution of health system infrastructure)

A8 Development of evidence-based clinical practice guidelines for family physicians (Services: Improve quality and safety of health care services and pharmaceuticals)

Capacity building on standardization of health statistics, various research efforts (Stewardship: Create intelligence, monitor and evaluate)

A9 Organization of international conferences (Stewardship: Foster international cooperation and intersectorial action)

C1 Training of SSI staff for UHC implementation and improvements in SSI IT infrastructure (Financing: Sustain UHC)

73

MAP SECTION

74