assessing impact of health aid: how can public health sciences contribute to mutual accountability?...
TRANSCRIPT
Assessing impact of health aid: how can public health
sciences contribute to mutual accountability?
Moving from evaluating the impact of programmes and interventions to
health systems and capacity strengthening
Phyllida Travis, 6 February 2012
IHP+ experience with
1. Issues and challenges in communicating results of health aid
2. Some examples of linking aid effectiveness to 'results'
3. Of strengthening mutual accountability for results
4. Approaches to making health aid for system strengthening more effective / efficient
Who is IHP+?
2007 2012
Developing countries: 8 30
Bilateral donors: 8 13
Int'l agencies and foundations*: 11 12
TOTAL 27 55
*African Development Bank, Bill and Melinda Gates Foundation, European Commission, GAVI Alliance, Global Fund, International Labour Organization, UNAIDS, UNICEF, UNDP, WHO, UNFPA, World Bank
1. Issues, challenges in communicating results
Want results that are memorable, persuasive, credible, useful, but
Politicians / public v public health experts Country policymakers/managers v global perspective Stories v numbers Progress v impact Impact of aid per se v impact of way give aid Joint v individual results Plausible v causal links 'Quick wins' v sustained gains Improved messaging now v in 5 years, w new methods
Communicating benefits, results
IHP+ country teams meeting 2010: Clear progress on putting aid effectiveness principles into practice Need to better document results / outcomes, find ways of
conveying better. Need mix of good evidence, good stories, moderate claims
In 2011 IHP+Results annual report on signatories' progress - visual; what
not why OECD TTHATS report for Busan - sound; not sexy Accumulation of narrative country examples - what, why, how;
distillation needed Commission on Information and Accountability - new catalyst
May 2011 IHP+ Results report Individual scorecards
And comparisons across partners
2. Some country specific examples linking aid effectiveness and results
Ethiopia, Nepal, Mali (all of which have positive trends in service delivery and health outcomes) plus DRC, Benin
Asking 3 questions To what extent have aid effectiveness principles been
put into practice? Has this contributed to better results?
Has health aid actually become more effective? Have health systems been strengthened? Have health services improved?
What was critical in achieving these results? Where were the barriers and how have they been overcome?
Aid effectiveness and results…different perspectives
Can trace plausible links from 'putting Paris into practice' via stronger systems to improved services and then …more tenuously….health outcomes
All note 'two speed implementation' of Paris agenda Is also an argument that goes
Outcomes are important Proving link between AE and results (outcomes) is more
of a global than country concern Risks excluding other important, common sense benefits to
countries of working to 'one plan, one budget, one report'
Need frame the debate to reflect wider range of benefits/results
Links between effective aid and results: examples Nepal: scale up of free maternal health care from a few districts
to nation-wide over 2-3 years was possible because government and donors acted collectively – no one could have done it alone. Institutional deliveries risen from18% 2006 to 28% 2011.
Ethiopia: Health extension worker programme went nation-wide over 5 years, faster than many thought possible, because FMOH provided a clear vision; credible plan and indicated where support was needed from DPs
DRC: A new MOH single donor coordination unit led to threefold reduction in management costs for donor funds, from 28% - 9%. Liberated funds for other uses. Message: better coordination of resources can deliver greater value for money.
DRC: Reform of health financing – some results MOH wanted to see
The reform benefits from the support, both technical and financial, of all partners even if its utilisation is gradual
AE principles in practice? Other types of results one MOH is
interested inIndicators Target for 2009/10
(Compact)Achievement 2009/10
% DPs providing confirmation of long term commitments
95% 63%
(no baseline available)
% DPs which don't ask Government for a separate plan
95% 58%
(baseline 23%)
% DPs' activities and budgets reflected in the government's plan
100% 76%
(baseline 44%)
% of Funds provided through Government
preferred modalities
60%
19%
(baseline 24%)
3. Strengthening mutual accountability
MA important route towards greater aid effectiveness, but slow progress
Mutual accountability efforts in health Countries
MOH and EDPs: Compacts; some MOHs incorporating indicators of DP aid behaviour into JAR agendas eg Mozambique, Ethiopia, Nepal…
MOH and parliaments: politicians getting involved eg Nigeria MOH and CSOs, research institutes – mixed; efforts to increase;
Uganda Global / regional level
3rd round of monitoring by IHP+Results – 19 countries in 2012 (10 last time); 15 DPs
Upcoming meeting of southern health CSOs – will discuss scorecards as one tool for MA
Note: mutual v joint accountability / responsibility crops up a lot in-country
4. Making health aid more effective in health system and capacity strengthening: Nepal
Existing instruments 5 year health sector programme (NHSP2), results framework;
partnership agreements; joint annual review (JAR) 2012 JAR: systems problems prominent
E.g. only 15% of facilities had no drug stock outs (target 70%) Getting TA better aligned with NHSP-2 priority problems a major theme
New actions to harmonize/align TA with priority problems TA matrix against NHSP2 objectives identifies gaps / overlaps Joint technical co-operation agreement agreed To match TA to priorities in Annual Work Plan and Budget
Ie not just about systems knowledge/ research but practical instruments
Factors enabling aid to effectively contribute to
health systems and capacity strengthening
Critical enabling factors seem to be Government ownership Leadership from MOH and at least some DPs Strategic plans and review processes established and taken
seriously Joint financing arrangements that are adhered to (including
more predictable funding), complemented by more joined up TA?
Some home-grown form of mutual accountability
Common barriers DP own organizational incentives/adaptability Willingness to synchronize work Trust
spares
Moving from impact evaluation to health systems and capacity
strengthening An aid effectiveness perspective…joint responsibility Context
post Busan; Old chestnuts
One does not follow the other – move in parallel what politicians and the public listen to – and when Impact has to be more than health Benefits of 'more effective aid' often on more proximal
aspects of system performance Existing approaches – 2 country examples New developments
Increased emphasis Utility of compacts – what they contain
Common monitoring framework of health progress and performance
Data collection
Indicatordomains
Analysis & synthesis
Communication & use
Administrative sourcesFinancial tracking system; NHADatabases and records: HR, infrastructure, medicines etc.Policy data
Facility assessments Service readiness
Population-based surveysCoverage, health status, equity, risk protection, responsiveness
Clinical reporting systemsquality, coverage, health status
Civil registration
Data quality assessment; Synthesis of progress and performance; Evaluation
Regular country health sector review processes; Global reporting
Improved health outcomes
& equity
Social and financial risk protection
Responsiveness
Efficiency
Fina
ncin
gInfrastructure;
ICT
Health workforce
Supply chain
Information
Interventionaccess & services
readiness
Interventionquality, safety
Coverage of interventions
Prevalence risk behaviours &
factorsGov
erna
nce
Inputs & processes Outputs Outcomes Impact
International funding remains unpredictable
Donor commitments for health as % of total expenditure on health
0
10
20
30
40
50
60
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008
Mali
Mauritania
Tanzania
Eritrea
Donor commitments for health as % of total expenditure on health
0
10
20
30
40
50
60
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008
Benin
Burundi
Guinea
Liberia
Growing numbers of players in health: Chad
Nations Unies
UE
France
Suisse
Allemagne
Egypte
USAIDChine
Cuba
ONUSIDA
UNFPA
UNICEF
PAM
OMS
Membres de l'UE
Libye UNHCR
PNUD
OCHA
FAO
Initiatives mondiales
Fonds mondial contre le Sida, la tuberculose et le
paludisme
GAVI
ONG nationales et internationales
MSFs CCSIDED
Croix Rouge
COOPI
MDM
MENTHOR
BASE
UNAD EMEET
CSAI
ACF
ECHO
CICR
Ordre
de Malte
BM
BADInstit.
fin.
int.
issues
Focus on (quick) results does not need to compromise HSS / CB efforts, but needs to be managed
Results need to be expressed in different ways
Traditional approaches to HS and capacity strengthening mixed record
AE p's apply to this as
Links back to impact evaluations / public health research
One of the 9 standard JAR background documents is a research report, summarising major health related studies in Nepal in last year.
In 2011 research report covered findings from Major periodic surveys eg DHS, NLSS, service tracking survey Financing studies commissioned as part of strategy
development Studies on effectiveness of new policies eg reducing barriers
to maternal health services Studies on aid effectiveness and results
Improving background documents is an iterative process Some research topics identified in JAR eg on
procurement
Snapshot of Mali
Progress with putting AE principles into practice Coherent planning framework (PRODESS+, MTEF);
stronger MOH ownership; more alignment with strategies; less with systems; greater management for results; some greater donor harmonization, MA
Results 1: more effective aid Mixed: more predictable pledges; disbursement worse;
some transaction costs seen as investment (but many still negative); TA more focused on systemic priorities
Result 2: health system strengthening
27 October 2011 Mali Case Study 25
• Important progress in implementing aid effectiveness principles practice and behavior changes highly probable contribution to HSS and sector results
• BUT “two-speed implementation” of the principles of the Paris Declaration and IHP+: – Only 13/50 donors signed the Compact and 3 do SBS– Sharp decrease in executed ext. financing since
2008– Still many earmarked funding (thematic./geograph.)– Specific procedures and PIUs– Aid not predictable– Individual missions / audits in addition to joint ones
Expectations must be realistic
Mali: Conclusion and perspectives