key tools for effective scale-up cost modeling and data...
TRANSCRIPT
Cost Modeling and Data Dashboards: Key Tools for Effective Scale-up
USAID Global Health Mini-UniversitySeptember 14, 2017
Helping countries reach sustainable impact at scale
Jim Ricca, MD, MPHMCSP Learning and Implementation Science Team
Leader/Jhpiego
“Deliberate efforts to increase the impact of health service innovations successfully tested in pilot or experimental projects so as to benefit more people and to foster policy and program development on a lasting basis.”
- WHO/ExpandNet
Hence the twin goals to achieve sustainable impact at scale:
• Service expansion (high effective coverage of the population in need)• Institutionalization in country systems (so that gains are sustainable)
Definition of Scaling Up
3
… predominantly an organizational, managerial, policy and political task, and not just a technical task
Translation: Achieving sustainable impact at scale is not just a larger work plan, to train more providers, equip more health facilities and cover more people.
Scaling up is …
4
✓ Do• Develop a scalable model in one or
more district (“the scalable unit”) that can then be replicated in others.
• Develop systematic scale plan (with costs), addressing health systems bottlenecks
• Scale up in phases
• Have a plan BUT be flexible; monitor frequently (monitoring dashboard)
• Have a multi-stakeholder team managing the scale up process
X Don’t• Have “project mentality” only when
starting
• Think that scale up just means training more people
• Scale up all at once
• Don’t track because “there are no measures in the health information system”
• Think that the process is too difficult to manage because there are so many partners involved
Avoiding “Empty Scale up”: Scale Do’s and Don’ts
5
How is planning to achieve sustainable impact at scale NOT like project planning?
• Reaching full impact at scale takes time• More than one organization contributes to progress• We need a plan that allows us to learn, be flexible, and adapt
as needed• We need to think early about how to mobilize resources over
the long term (i.e., beyond the time horizon of any one project)
6
Essential Inputs
National scale up Plan (with costs & benchmarks)
Initial Financing
Key policies
Human Resources Sufficient health workers trained and well
supervised
Supply of productSufficient products procured & distributed
Health Information System & Data Use Key information recorded, reported, and
used
Serv
ice a
vaila
bilit
y#
facil
ities
, dist
ricts
, cou
ntie
s with
serv
ice in
pl
ace
Demand – adequate and appropriateKnowledge and barriers identified and
addressed
Implementation strategies across health systems
Output Outcome Impact
Cove
rage
of p
opul
atio
n in
nee
d #
rece
ivin
g se
rvice
/ #
in n
eed
of se
rvice
Mor
talit
y ra
te
Leadership / Management / Partnership
Taking a systems approach: an illustrative logic model
7
The drivers of effective scale up
• Policy development, advocacy and implementation• Creating a (district) scalable model that can be replicated• Leadership, management and partnership for the scale up process• System oriented planning for scale up, both strategic and
operational• Costing of plans to aid in ongoing resource mobilization • Data use for action, with emphasis on data that can be frequently
tracked for management and continuous learning
8
Costing for Scale-up: Principles and Analysis for Helping Babies Breathe
(HBB) Scale-up in Rwanda
Ben Picillo, MPHMCSP Health Systems Strengthening and Equity
Team/R4D
Outline
1. Basics of Costing 2. Costing for Scale-up3. Analysis for Rwanda HBB/ENC 4. Q&A
10
Basics of Costing• Costing focuses on estimating financial (and sometimes non-financial)
costs of implementing interventions/services:• Financial costing identifies the required resources (i.e., money) • Economic costing also factors in the opportunity cost of time
• Costing is not budgeting, which focuses on future planning for expenses
• Normative costing determines costs based on what something “should” cost, while actual/current costing does not use norms to determine costs• Normative: ANC requires 15 minute consult with nurse• Actual: ANC nurse saw 10 ANC patients in a day; labor costs for ANC
are based on nurse’s daily rate divided by 10
11
Principles for Costing for Scale-up
• Clearly define scalable unit and intervention package• Define final audience for analysis outputs and ensure
perspective is appropriate • Ensure analysis shows costs by standard cost categories, cost
drivers, and one-time versus recurring costs • Develop flexible cost models that show cost under multiple
scenarios• Present costs in relation to outputs (if possible) or relate to
budget/economic context • Create clear dissemination and advocacy plans using analysis
outputs
12
Helping Babies Breathe Intervention
• Clinical intervention for newborn resuscitation for children born not breathing; estimated 47% reduction in mortality (AAP)
• Rwanda reported fully scaling up HBB, but MCSP at start-up found challenges in its implementation
Photo: AAP
13
Methodology
• Retroactive activity-based costing based on MCSP approach to strengthen HBB from November 2015-October 2016
• Quantification of level of effort and other cost inputs (e.g., transport) based on approach that MCSP-supported districts used
• Modeling of costs using quantities of inputs used and MOH standard pay (non-mentors), transport, and per diem rates
• Creation of model with capability of scenario analysis• Allows flexibility of scale-up sequence, intensity and duration of
approach 14
Costed Activities from HBB Improvement Package
Strategy ActivityActivity
TypeDescription
Preparation
Provider skill assessment Start-up Situational analysis in 10 districts; assume 25% focused on HBB/ENC
ENC/HBB situational analysis Start-up Dedicated situational assessment in initial 4 districts
Orientation district stakeholder Start-up District stakeholder orientation for each district in prep for scale-up process
Initial kick-off workshop Start-up Initial small workshop held to commence annual start-up process
Equipment procurement Start-up Resuscitation mannequin and two penguin suction devices per facility
ENC refresher training Start-up Training for lead trainers and mentors for the training of trainers session
ENC/HBB post-training follow-up Start-up Dedicated follow-up with mentors post-refresher training in 4 districtsTraining of
TrainersENC/HBB ToT
Start-up Training of trainers for LDHF methods for mentors
LDHF LDHF training Start-up LDHF approach led by mentor trained in provides in 4 districts
Mentorship
On-site mentorship Recurrent Monthly mentorship visits in health facilities
On-site mentorship oversight Recurrent Quarterly mentor oversight visits to district-based mentors in facilities
ENC/HBB mentors meeting Recurrent Mentors meeting (among themselves) from each district
ENC/HBB progress meeting Start up Meeting of leader mentors from each district (4)
ENC/HBB mid-year workshop Recurrent Meeting of stakeholders from each district (4) and MOH
QIQI oversight
RecurrentQuarterly oversight visits from QI Advisor; Other QI activity are covered under Mentorship activities
15
Overview of Costing Model
16
Select mentor, start-year, and provider
training parameters
Overview of Costing Model
17
Select key basic parameters
Define details of practice improvement
package
Transport and per diem comprise the majority of total costs by cost category
18
MaintenanceScale-up
Mentorship is the largest cost driver for overall implementation
19Note: “Start-up” refers to costs associated with district receiving implementation package for the first-time that year; “Maintenance” refers to districts that have received package in previous year(s).
MaintenanceScale-up
Average annual costs per district range from US$20-25k in scale-up years, but drop to US$15k after full scale-up
20
MaintenanceScale-up
Projected total cumulative costs for 24 districts (without mentor salaries)
MaintenanceScale-up
21
Estimated costs of package suggest it is a relatively low-cost intervention with potential for high-impact
• Average total annual district cost: • US$25,000 in first year of introduction• US$15,000 in subsequent years
• Compared to average district health budgets, total annual district costs represent less than 1% of total budget*
• Estimated US$350-450k total cost per year for full scale-up in Y3-Y6 represents less than 1% of total government expenditures on health**
• Annual cost per new birth: $1.40 per birth per year***
22
*Source: Musanze Integrated Health Plan (Joint Finance), 2016**Source: Rwanda National Health Accounts 2014; Does not factor in costs for PIH approach*** Based on 360,000 birth cohort for 2016
A Quick Quiz…
23
Question 1
True or False:
Costing and budgeting are the same process of collecting costs and projecting into future.
24
Question 2
How could this analysis be improved? Jacky has been asked by the Government to conduct a costing analysis to inform the state by state scale-up of chlorhexidine, which comprises four different activities. Jacky does the analysis and only reports the total costs by standard cost categories (e.g., salary, travel) for all of the activities aggregated at the national level.
25
Question 3
How could the presentation of this analysis be improved?Ernest has been asked to estimate the costs of scaling up advanced distribution of misoprostol for self-administration (ADMSA). He estimates that it will cost $30,000 per district to maintain the intervention. He only presents this aggregate district cost estimate to the Ministry of Health.
26
Best Practices in Designing Data Dashboards for Scale-Up
Vikas DwivediMCSP Measurement, Monitoring Evaluation and Learning
Team/JSI
Defining dashboards
28
Scaling-up interventions to prevent Post-partum Hemorrhage (PPH)
29
• 2012 WHO guidelines:•All women receive a uterotonic during
the third stage of labor for PPH prevention•Oxytocin recommended, but requires a
skilled birth attendant (SBA)•At home births without an SBA,
administration of misoprostol by CHWs is recommended
Source: Khan et al., 2006
▪ PPH accounts for 34% of maternal deaths in Africa
ANC by Health Worker(40.3-17.0%)
Delivery at Health Facility(11.5%)
Deliver at Home(88.5%)
Home Health Promoters
TBA/RelativesFrom pregnancy to delivery
30
Strategies to reduce PPH
Counseling and Misoprostol Distribution at home by HHP
• Home visit• Education Session on PPH and
Misoprostol• Misoprostol Distribution
Counseling and Misoprostol Distribution at ANC
• ANC Visit• Education Session on PPH and
Misoprostol• Misoprostol Distribution
Reduce PPH
AMTSL + Oxytocin at health facility
Community Health Facility
31
Program Information
Expected Pregnant women 900
ANC coverage 30%
No. of CHW Trained 142
No. of ANC providers Trained 44
Period Jan – June, 2013
Deliveries
Health Facility 300
Home 580
Stock-outs
Oxytocin at Health Facility 2
Misoprostol at CHW level 0
Home Deliveries that took Misoprostol
Uterotonic Coverage
Women Counselled and Provided Misoprostol by Place of Counseling (Estimated Pregnant Women)
Uterotonic Coverage by Place of Birth52.7%
Monitoring dashboard for PPH prevention
32
Period Jan – June, 2013
Women Counselled and Provided Misoprostol by Place of Counselling (Estimated Pregnant Women)
First 3 months of intervention First 6 months of intervention
Country Dashboard
33
Ask some Questions
• What is our target?• Are we on track?• Have we trained enough CHWs and HWs?• Do we have enough stock of Misoprostol?• Are there geographical challenges?
34
Period Jan – June, 2013
Women Counselled and Provided Misoprostol by Place of Counseling (Estimated Pregnant Women)
Area X
Area Y
Population, geo and inputs are the same in the two areas
• CHWs in Area Y are not actively identifying pregnant women
• Due to rainy season it has been difficult to gather reports from Area Y
Country Dashboard
35
Effective visualizations
help stakeholders
understand information
and support
decision-making.36
Data shows us diversity.
37
What is a data dashboard?
• A way of visually conveying information – often quantitative in nature – in an accurate, compelling format.
• Usually makes relationships more apparent (e.g. by clustering, color coding and by putting items in scale).
• Can be static or interactive
38
The Visualization Process
Who is your audience?
Building the chart and gather feedback
What story do we want to tell?
Identify a review process and use!
1Who is your audience?
40
Different stakeholders have different data needs.
41
On the most common visualization mistakes
“Time isn’t adequately spent
on just what is the question
that you’re trying to answer,
and what does your audience
need
to know?” Cole NaussbaumerStorytellingwithData.com
From: http://cxcafe.maritzcx.com/storytelling-with-data-dashboarding-with-cole-nussbaumer/
Turning data into useful
information
Using data for
decision-making
So far…Now…
43
Consider your stakeholders’ literacy, numeric literacy, and what data they need to make decisions.
44
Who are the users of data on Scale-up?
• What is the level of health system?• Is there are technical working group?• Are there members from Ministry of Health
and external partners?
45
2Identify the story you want to tell & consider additional available data.
46
The most important question
in visualization:
Read more at: http://stephanieevergreen.com/most-important-question/
Stephanie EvergreenEvergreen Data
What’s your point?
48
Select Indicators for Scale-up
Start with the data you’ve collected.
49
Then, identify additional data available that would help you tell your story better visually.
50
Edit your data as necessary to tell your story.
e.g. create percentages
from raw numbers,
check quality, etc.
51
3Design your chart or graph
52
Consider the kind of data story you have.
Distribution Part to Whole Correlation
Time Series Compare Categories Ranking
Image credit: Column Five Media’s Visage Data Visualization 101
53
And consider consulting your colleagues
M&E AdvisorGraphic Designer
Technical Expert
Communications Expert
54
4Plan for how you’ll share your visualization when it’s complete.
55
“You can’t fatten a cow by weighing it”
How to use the information for scale-up
• Help define what is important • Set goals and expectations for specific individuals or groups• Encourage specific actions in a timely manner• Highlight exceptions and provide alerts when problems
occur• Communicate progress and success
57
Title & labelsBe clear and succinct (6-12 words), in telling your reader the key takeaway & including labels that are essential
Recommendation of 6-12 words from the Data Visualization Checklist http://stephanieevergreen.com/dataviz-checklist/
Some examples…
District 4 was the only district where coverage declined during the five year project.
Graph from https://www.urbanreproductivehealth.org/sites/mle/files/issu_mid-term_research_brief_english.pdf
Use of implants and injectables has increased among women using family planning.
The proportion of family planning users choosing implants & injectables has increased.
66
OSCE Skill Score (target > 80)
HF1 HF2 HF3 HF4 HF5 HF6…
Jan 45 0 30 25 0 40
Feb 50 0 50 30 0 50
Mar 70 0 80 40 0 65
Apr 80 0 85 45 0 75
May 85 0 90 50 0 80
Jun 95 0 80 50 0 85
Facility readiness to provide newborn resuscitation
HF1 HF2 HF3 HF4 HF5 HF6
Trained/mentored on ENC_HBB 4 0 3 1 0 3
Bag and mask in place2 0 2 1 0 2
People & their blogs:Cole Naussbaumer | Storytelling with DataAlberto Cairo | The Functional ArtStephanie Evergreen | Evergreen DataJon Schwabish | Policy VizAnn K. Emery | Emery EvaluationAmanda Makulec | Amanda Makulec
Favorite posts & resources from these experts:Decluttered Excel Templates from ColeExcel Tutorials (video!) from AnnRemakes of Visualizations from Jon
Communities around data viz:Data Viz for Development at DataVizHub.coHelpMeViz to crowdsource data viz expertise
Two online viz tools I recommend:Piktochart (great for making icon matrices)Visage
Books worth buying:
On basic visualization principles:The Wall Street Journal Guide to Information Graphics | Dona M. WongShow Me the Numbers | Stephen Few
On designing presentations with visual content (including graphs & charts):Slideology | Nancy Duarte
A few resources for building your data viz skills
67
Consult with great resources for
choosing the right chart type
The Graphic ContinuumJon Schwabish & Severino Rebecca policyviz.com/presentations/graphic-continuum-video
Chart ChooserJuice Analyticslabs.juiceanalytics.com/chartchooser/index.html
Data Visualization 101Visage | visage.co/data-visualization-101-design-charts-graphs/l
68
69
70
For more information, please visitwww.mcsprogram.org
This presentation was made possible by the generous support of the American people through the United States Agency for International Development (USAID), under the terms of the Cooperative Agreement AID-OAA-A-14-00028. The contents are the responsibility of the authors and do not
necessarily reflect the views of USAID or the United States Government.
facebook.com/MCSPglobal twitter.com/MCSPglobal