1

Assessment of health progress and performance,

mainland Tanzania: Analytical report

Introduction and outline

Dar es Salaam, 4 February 2013

Objectives of the analytical report

• To produce a comprehensive assessment of health progress and performance in mainland Tanzania in order to inform the MTR of HSSP III and other strategies/plans as relevant

• Other strategies / plans may include:– National Roadmap strategic plan to accelerate reduction of maternal, newborn

and child deaths 2008-2015– Primary health services development programme 2007-2017 (MMAM)– MDG progress reviews, Global Fund and other development partner related

monitoring– National strategies: Vision 2025, MKUKUTA

• To strengthen capacity for health progress and performance assessment

2

Indicators & analysis

• Focus on the key indicators and targets of HSSP III (32)– Results framework: inputs-outputs-outcomes-impact– Consideration of "current" situation and 2015 targets

• Also take into account additional indicators in the same programme areas if these can help assess progress or are included in related strategies/plans– E.g. life expectancy (Vision 2025), RMNCH indicators

• Emphasis on disaggregation (equity) if possible, especially subnational (mostly region if possible district)

• Take into account any contextual factors, such as economic and social indicators at subnational and national level

3

Inputs & processes Outputs Outcomes Impact

32 Indicators for HSSP III 2009-2015, MOHSW, Tanzania

Service access•EMOC facilities

Medicines and medical products stockouts in health facilities

Quality•Malaria lab confirmation•TB treatment success•Leprosy treatment success

Immunization•Measles, DPT/penta3Vitamin A in childrenAntenatal and delivery care•ANC At least 4 times•ANC started before 16 wks•TT2•Skilled birth attendanceContraceptive prevalenceHIV•ART for PMTCT•ART coverageMalaria•IPT2•ITN among children and pregnant womenCholera CFR among treatedHypertension prevalence 25-64

Neonatal, infant and child mortality ratesMaternal mortality ratioTotal fertility rate

HIV prevalence•15-24 years•Pregnant women 15-24Malaria parasitemia in childrenTB notification rateLeprosy notification rate

Cholera incidence

Orphanhood prevalence

Financing•THE per capita•Enrollment in CHF

Human resources•Density by region: MO and AMO; nurse midwives; pharmacists; health officers; lab•Training institutions with accreditation

Child growth: severe underweight, severe stunting

Cross cutting issues in HSSP III

1. Equity: geographic, vulnerable groups

2. Gender sensitivity

3. Quality of services, management etc.4. Community ownership, including

healthy life styles, care in the family, health service interface

5. Coherence in health services planning and implementation

6. Complementarity in governance: management, PPP

Equity analyses

Improved service coverage

Improved intervention quality, uptake and impact

Implementation areas in HSSP III

1. District health services

2. Referral hospital services

3. Central level support

4. Human resources

5. Health care financing

6. Public private partnership

7. Maternal newborn and child health

8. Disease control: HIV/AIDS, TB & leprosy, NTD and epidemic prone diseases, NCD, environmental health

9. Emergency preparedness

10. Social welfare

11. M&E

12. Other important issues: capital investments, ICT

156 indicators, mostly for management purposes, limited use in this report

Some could be included in the analytical progress report

156 indicators, mostly for management purposes, limited use in this report

Some could be included in the analytical progress report

Main data sourcesInput indicators

• Financial tracking; resources, expenditure– National Health Accounts: 2005, latest– Public Expenditure Reviews– Other sources

• Health workforce– HR data bases: quality, other sources than MoH– Recent special studies

• Policy changes– All relevant policies and policy changes since 2009 from

qualitative review

Main data sourcesOutput indicators

• Availability and readiness of health services: tracer medicines and medical products– National data bases (by regional and district): facilities, specific type of services (ARV

therapy, PMTCT, EMOC)– Health facility data (HMIS)– Facility surveys

• 2008/09: 15 districts, NIMR & WHO• 2012: 27 districts, Ifakara

• Quality of services– Health facility data (HMIS):

• Lab confirmation rates for malaria• TB treatment outcome (success rate)• Research studies

• Outpatient utilization rates– HMIS, economic survey

Main data sourcesOutcome indicators

• Coverage of interventions (with equity)– HMIS: ANC, PMTCT, postnatal care, delivery, CS rates, immunization,

vitamin A, ART coverage, TB treatment and notification rates, FP– TDHS 2004/05 and 2010/11– THMIS 2011 – National panel survey– Research studies

• Risk factors– TDHS 2004/05 and 2010/11– STEPS 2012 if available– Research studies

Main data sourcesImpact indicators

• Mortality and fertility– TDHS 2004/05 and 2010/11– Census– HMIS (causes of death, case fatality rates)– Health and Demographic Surveillance Studies (Ifakara, NIMR)

• Morbidity– HMIS– Surveillance system (HIV, cholera)– Surveys: THMIS

• Financial protection

Analytical approachesWhat can be done?

• Data quality assessment: completeness accuracy• Target and trend analysis: rate of progress• Putting data from different sources together (to obtain best

estimate and assess data quality)• Equity analysis: geographic, individual characteristics• Stepwise analysis using the results framework • Efficiency analysis: comparing results with inputs• Estimates: use of statistical modelling • Comparative analysis: internal and external• Lives saved computation (LiST)

End

12

Putting data from different sources together

• Health facility and survey data available for the same indicators

• Coverage of interventions: immunization, ANC, SBA/institutional delivery, etc.

• Assess biases, make adjustments

13

ComparisonDelivery rates: HMIS fairly consistent with DHS 2010

HMIS is higherRukwa

Kigoma

TDHS 2010 is higher

Disaggregation (equity)

• Health facility reports– age– subnational data: district

• Health surveys– sex, age– Education, wealth quintile– Place of residence: urban rural, province/region

15

An example from a survey

2012 Countdown Report

17

Stepwise analysis

Evaluating health system strengthening and reforms A stepwise approach

Improved health outcomes

& equity

Financial risk protection

Responsiveness

Fina

ncin

g

Infrastructure / ICT

Health workforce

Supply chain

Information

Interventionaccess & services

readiness

Interventionquality, safety and efficiency

Coverage of interventions

Prevalence risk behaviours & factorsG

over

nanc

e

Inputs & processes Outputs Outcomes Impact

Have finances been disbursed?Have policies been changed?

Is the process of implementation happening as planned?

Has access to services improved?Did the quality of services improve?Has utilization improved?

Did intervention coverage improve? Have risk behaviours improved?

Have health outcomes and equity improved? Are services responsive to the needs? Are people protected against financial risks?

Contextual changesNon health system determinants

Contextual changesNon health system determinants

18

Efficiency – comparing inputs and results

0

50

100

150

200

250

0 200 400 600 800 1000

Total Health Expenditure per capita (US$)

Un

der

-5 m

ort

alit

y ra

te Underperformers – higher mortality than expected on the basis of money for health

Good performers

Regional performanceCoverage of deliveries by health worker

density*

Better than average performers

Poorer than average performers

* Dar es Salaam and Kilimanjaro have more than one-third of health workers in Mainland and are excluded

Pwani Iringa

Morogoro

Tanga

ArushaMwanza

Mbeya

Shinyanga

Use of estimates

• An estimate is based on statistical modeling with transparent assumptions to obtain the best picture of the real situation

• Done for many indicators– Mortality; child, maternal, adult, life expectancy,

causes of death– Coverage: immunization, water and sanitation

• Often done by global agencies to obtain comparable data– Same year, same method for all countries

20

Child mortality – IGME estimateswww.childmortality.org

21

22

Benchmarking

• Comparison– Who to compare with: Regional average / peer

countries– How: average, median or best performers, compared

to international targets (e.g. MDG, Abuja 15%)

• Statistical measures– Absolute or relative progress – Ranking (e.g. 14th out 42 countries)– Percentile (e.g. 67th percentile out of 42 countries)– Position compared to country or regional

mean/median

• Benchmarking– Comparison with top performers (e.g. best 10%)

23

Performance assessmentPutting it all together

• Identifying contextual changes– Demographic, economic, social and political factors

• Progress assessment– Compared to targets– Compared to peers– Putting together data from different sources

• Equity analysis– Trends in equity gaps by key stratifiers

• Efficiency analysis– Results by inputs; use of summary measures

• Performance = Summarizing and interpreting the results

How well is the analysis in the annual review report done?

1. Data quality assessment included?

2. Target and trend analysis done?

3. Stepwise systematic analysis of progress and performance?

4. Data from different sources is put together?

5. Equity receiving attention?

6. Efficiency analysis done at subnational level?

7. Comparative analysis within country done?

8. Comparative analysis with peer countries done ?

24

25

Benchmarking of progress (spreadsheet)

• Spreadsheet - data from World Health Statistics 2010

• Selected set of indicators with data over time– Total health expenditure per capita– General government expenditure on health as percent of total

expenditure– DPT3 coverage– Child stunting– Child mortality (under five) rate

• Use different measures to examine the data and position over time

• How well has your country done: Subjectively interpret your country's results and draw your conclusion for your country

26

Components of the progress and performance review

1. Progress: the extent to which health system goals in terms of levels of health and financial risk protection have been attained

2. Equity: the progress in term of distribution of the health system goals

3. Efficiency: the extent to which the resources used by the health system have produced the maximum possible benefit to society

Perfo

rman

cePo

licie

s, s

trat

egie

s, re

sour

ce a

lloca

tion

Context

Context