welcome to i-tech hiv/aids clinical seminar series an introduction to operations research or how can...
TRANSCRIPT
Welcome to I-TECH HIV/AIDS Clinical Seminar Series
An Introduction to Operations Research
or
How can I make my health program better?
Dr. Mark Micek, MD, MPH
December 4, 2008
The IssueThe Issue
It is difficult to…It is difficult to………effectively deliver scientifically proven health effectively deliver scientifically proven health
interventions in the “real world”interventions in the “real world”……translate research into health program settingstranslate research into health program settings
Why?Why? Research generally occurs in controlled settingsResearch generally occurs in controlled settings
Homogenous sample, controlled settingHomogenous sample, controlled setting Health programs exist in a complex settingHealth programs exist in a complex setting
Heterogeneous clients, multiple settingsHeterogeneous clients, multiple settings Dependent on external context of care-delivery systemDependent on external context of care-delivery system
i.e. policy, resource availability ($ and personnel), i.e. policy, resource availability ($ and personnel), community perceptionscommunity perceptions
Example of difficulty translating Example of difficulty translating proven treatment into practiceproven treatment into practice
HAART reduces mortality among patients with HIVHAART reduces mortality among patients with HIV BUTBUT many eligible HIV-positive people don’t start HAART many eligible HIV-positive people don’t start HAART
Monthly flow through the HIV care system in Beira and Chimoio, Mozambique, Jun 04 - Sept 05
HIV+
Undergo CD4 testing (78%)
Enroll at HIV clinic (59%)
Eligible for HAART (48%)
Start HAART (46%)
0
100
200
300
400
500
600
700
Ave
rag
e p
atie
nts
per
mo
nth
Example of difficulty translating Example of difficulty translating proven treatment into practiceproven treatment into practice
Short-course Short-course ARV therapy can ARV therapy can decrease decrease mother-to-child mother-to-child HIV transmissionHIV transmission BUTBUT few HIV+ few HIV+
women receive women receive treatmenttreatment
Data from: Evaluation of United Nations-Supported Pilot Projects for the Prevention of Mother-to-Child Transmission Data from: Evaluation of United Nations-Supported Pilot Projects for the Prevention of Mother-to-Child Transmission of HIV: Overview of Findings. UNICEF, New York, 2003.of HIV: Overview of Findings. UNICEF, New York, 2003.
Women tested for HIV and receiving ARV prophylaxis in 11 pilot pMTCT sites, Jan00-Jun02
Attend ANC at pMTCT site
Counseled for HIV (71%)
Tested for HIV (70%)
HIV-positive (9%) Received ARVs
(49%)0
100000
200000
300000
400000
500000
600000
Nu
mb
er o
f p
reg
nan
t w
om
en
How can we improve the How can we improve the performance of our performance of our
programs in an evidence-programs in an evidence-based way?based way?
Use the principles of Operations Use the principles of Operations ResearchResearch
What is OR?What is OR?Generic definitionGeneric definition
““Use of systematic research techniques for program Use of systematic research techniques for program decision-making to achieve a specified outcome.” decision-making to achieve a specified outcome.”
[Population Council, 2006][Population Council, 2006]
Goal = find a best possible solution to improve Goal = find a best possible solution to improve performance of the organizationperformance of the organization Early examples in military, businessEarly examples in military, business Use data, statistics, mathematical modelingUse data, statistics, mathematical modeling
Goal (health care) = “to increase the efficiency, Goal (health care) = “to increase the efficiency, effectiveness, and quality of services delivered by effectiveness, and quality of services delivered by providers, and the availability, accessibility, and providers, and the availability, accessibility, and acceptability of services desired by users”acceptability of services desired by users”
[Population Council, 2002][Population Council, 2002]
3 Core Principles of OR3 Core Principles of OR
1.1. Study health programsStudy health programs
2.2. Actively try to make the program betterActively try to make the program better
3.3. Use results to improve the programUse results to improve the program
Corollary: requires collaboration between Corollary: requires collaboration between managers and researchersmanagers and researchers
Core principle of OR #1:Core principle of OR #1:Study health programsStudy health programs
Health program is key to health care deliveryHealth program is key to health care delivery OR usually focuses on existing programOR usually focuses on existing program
Research problem = Research problem = programprogram problem problem Research intervention = Research intervention = programprogram solution solution
Feasible within context of entire systemFeasible within context of entire system Assures problems and solutions are defined Assures problems and solutions are defined
by realities of the health program systemby realities of the health program system
Without involvement of a health program, it’s Without involvement of a health program, it’s not ORnot OR
Health program as systemHealth program as system
Outputs
Raw Materials:
•Finances
•Costs
•Staff
•Facilities
Activities:
•Trainings
•Supervision
•Logistics
•Reporting
•Record keeping
•Patient flow strategies
Products of program activities:
•# condoms distributed
•# people tested for HIV
•# enrolling for HIV care
Effect on knowledge or behavior:
•# youth using condoms
•# women using pMTCT services
•# starting ART
•% adhering to ART
Outcomes Impact
Change in health/well-being
•HIV/STD incidence
•# deaths
Health ProgramUnder managers’ control
Inputs Process
Effects of Health Program
IntegratedIntegrated HealthHealth NetworkNetwork ModelModel
Information, Education and Communication at all levels
VCT Centers
1-2 / 100,000 Inhabitants
National TB Program
Maternity(Niverapine)
Internal MedicinePediatrics
STD Consults
Blood Bank
•Home based care•Community DOT•PLWHA groups•Nutrition support
Day Hospital1-2 doctors, PA’s,
Counselors•OI treatment•CD4 testing•HAART•Adherence support
Health Centers
TARV
Mental Health
MTCT
Health programs are complex systemsHealth programs are complex systems
Adhere to HAART
Test for HIV
Enroll in HIV clinic
CD4 testing
Start HAART (if eligible)
Schedule appt with clinician
Return for appt with clinician
Time and drop-off
Time and drop-off
Time and drop-off
Pt returns to get blood
drawn
Clinician orders CD4
Time and drop-off
Return for appointment with clinician
Time and drop-off
Blood drawn
Schedule appt for results
Time and drop-off
Enroll in HIV clinic
Core principle of OR #2:Core principle of OR #2:Actively try to make the program Actively try to make the program betterbetter
Better “understanding” of situation is not enoughBetter “understanding” of situation is not enough Better can mean…Better can mean…
Improve access to servicesImprove access to services Improve qualityImprove quality Limit costs (find cost-effective strategies)Limit costs (find cost-effective strategies) Improve healthImprove health
Without actively trying to improve the program, Without actively trying to improve the program, it’s not ORit’s not OR
Core principle of OR #3:Core principle of OR #3:Use results to improve programUse results to improve program
Using results can mean…Using results can mean… Implement new strategy on local / national scaleImplement new strategy on local / national scale Influence national / international policyInfluence national / international policy Dissemination of results, develop “best practices”Dissemination of results, develop “best practices”
OR successful only if results used to improve OR successful only if results used to improve the programthe program Published papers are NOT a valid indicator of OR Published papers are NOT a valid indicator of OR
successsuccess
Corollary: Corollary: OR requires collaboration between OR requires collaboration between
managers and researchersmanagers and researchers
Program managers/policy-makersProgram managers/policy-makers Should be involved in ALL ASPECTS of research process Should be involved in ALL ASPECTS of research process
Understand health care systemUnderstand health care system Help ensure problem is important, solution is feasibleHelp ensure problem is important, solution is feasible Help ensure results will be implementedHelp ensure results will be implemented
ResearchersResearchers Understand of research methodologyUnderstand of research methodology Responsible of recommending and implementing Responsible of recommending and implementing
appropriate research techniquesappropriate research techniques
Can be the same personCan be the same person
Broad methodologies of ORBroad methodologies of OR
Modeling (classic)Modeling (classic) Develop mathematical model to mimic health care Develop mathematical model to mimic health care
systemsystem Manipulate to find the best possible “solution”Manipulate to find the best possible “solution”
Optimize efficiencyOptimize efficiency Maximize Y given constraints XMaximize Y given constraints X
Intervention-based (Population Council)Intervention-based (Population Council) Design/test best way to deliver servicesDesign/test best way to deliver services Similar to quality improvement (IHI/WHO)Similar to quality improvement (IHI/WHO)
Intervention-based ORIntervention-based OR
Population CouncilPopulation Council IHI CollaborativeIHI Collaborative
LinearLinear CyclicalCyclical
1. Identify program problem1. Identify program problem 1. 1. PPlanlan
2. Generate program solution2. Generate program solution 2. 2. DDoo
3. Test program solution3. Test program solution 3. 3. SStudytudy
4. Use/disseminate results4. Use/disseminate results 4. 4. AActct
Act Plan
DoStudy
Steps in intervention-based ORSteps in intervention-based OR
1.1. Identify program problemIdentify program problem• Usually determined in ongoing programUsually determined in ongoing program• Routine data (M&E, surveillance) vs. program evaluationRoutine data (M&E, surveillance) vs. program evaluation• Under control of program managerUnder control of program manager
2.2. Generate program solutionGenerate program solution• Review workflow, talk to staff/clientsReview workflow, talk to staff/clients• Consider exploratory study if causes/solution unknownConsider exploratory study if causes/solution unknown• Must be feasibleMust be feasible
3.3. Test program solutionTest program solution• Level of intervention: facility vs. individualLevel of intervention: facility vs. individual• Data measurement: routine vs. added proceduresData measurement: routine vs. added procedures• Allocation: non-randomized (quasi-experimental) vs. randomized Allocation: non-randomized (quasi-experimental) vs. randomized
(experimental)(experimental)
4.4. Use/disseminate resultsUse/disseminate results• Continue/expand successful interventionsContinue/expand successful interventions• Influence national/international policyInfluence national/international policy• OR not typically “generalizable” but can be relevant for similar programs OR not typically “generalizable” but can be relevant for similar programs
(“best practices”)(“best practices”)
Common OR study designsCommon OR study designs(Experimental/quasi-experimental)(Experimental/quasi-experimental)
Pre-post control /non-equivalent control groupPre-post control /non-equivalent control group
Simple time-series (some control for time)Simple time-series (some control for time)
Step-wedge time-series (better control for time)Step-wedge time-series (better control for time)
Time
Exp group O O O X O O O
Time
Exp group 1 O X O O O
RA/non-RA Exp group 2 O O X O O
Exp group 3 O O O X O
Time
Exp group O1 X O2RA/non-RA
Control group O3 O4
Example 1: HIV testing in pMTCT Example 1: HIV testing in pMTCT program in Rwandaprogram in Rwanda
Program problemProgram problem: : • 18% of women tested for HIV in a pMTCT program in 18% of women tested for HIV in a pMTCT program in
Rwanda did not return for resultsRwanda did not return for results• System has 2-3 day delay in getting HIV test resultsSystem has 2-3 day delay in getting HIV test results
What is a potential solution?What is a potential solution?
How could we test this solution?How could we test this solution?
Source: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. WHO. Geneva, 2004.Source: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. WHO. Geneva, 2004.
Example 1: HIV testing in pMTCT Example 1: HIV testing in pMTCT program in Rwandaprogram in Rwanda
ProblemProblem: 2-3 day : 2-3 day delay in getting HIV delay in getting HIV test results test results 18% 18% did not return for did not return for resultsresults
Potential solutionPotential solution: : Same-day resultsSame-day results
Test of solutionTest of solution: Time : Time series, dropout ~0 series, dropout ~0 post-interventionpost-intervention
Source: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. WHO. Geneva, 2004.Source: An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. WHO. Geneva, 2004.
Example 2Example 2: Low number of people : Low number of people starting ART per monthstarting ART per month
0
20
40
60
80
100
120
Q1 2004 Q2 2004 Q3 2004 Q4 2004 Q1 2005 Q2 2005 Q3 2005
Beira
Chimoio
National ARV Program
Example 2Example 2: Low number of people : Low number of people starting ART per monthstarting ART per month
What is the program’s problem?What is the program’s problem? Do we need more information to determine Do we need more information to determine
what the cause of the problem is?what the cause of the problem is? What is the program’s solution?What is the program’s solution?
Do we know enough to develop a solution?Do we know enough to develop a solution?
What should our next step be?What should our next step be?
Identify steps required to start ARTIdentify steps required to start ART
HIV testing centers VCT
Home-based Care
Pregnant
Day Hospital Clinical evaluation (CD4)
Start HAART in
eligible patients
Adherence to ARV
Treatment Adherence to
Care
Youth
Community
TB patients
Ill/Hospitalized Hospital
Youth VCT
pMTCT
STEP 1 HIV Testing
STEP 2 Arrival to
Day Hospital
STEP 3 CD4
Testing
STEP 4 Start
HAART
Using programmatic data:Using programmatic data:Where are patients lost?Where are patients lost?
Monthly flow through the HIV care system in Beira and Chimoio, Mozambique, Jun 04 - Sept 05
HIV+
Undergo CD4 testing (78%)
Enroll at HIV clinic (59%)
Eligible for HAART (48%)
Start HAART (46%)
0
100
200
300
400
500
600
700
Ave
rag
e p
atie
nts
per
mo
nth
Step 1
Step 2
Step 3
Step 4
Using programmatic data:Using programmatic data:What are priorities to address?What are priorities to address?
Step 1: Tested for HIV Step 1: Tested for HIV1229 931
HIV+ HIV+440 23036% 25%
Step 2: Enroll in clinic Extra if fixed Step 2: Enroll in clinic Extra if fixed240 to 100% 150 to 100%54% 30 65% 15
Adults Adults230 14496% 96%
Step 3: Obtain CD4 Extra if fixed Step 3: Obtain CD4 Extra if fixed191 to 100% 102 to 100%83% 7 71% 12
Eligible Eligible86 55
45% 54%
Step 4: Start HAART Extra if fixed Step 4: Start HAART Extra if fixed36 to 100% 29 to 100%
42% 50 52% 27
Beira Chimoio
By City
Why do HAART-eligible patients Why do HAART-eligible patients not start ARVs (step 4)?not start ARVs (step 4)?
Poor follow-up also reported as reason for not starting HAART in other studiesPoor follow-up also reported as reason for not starting HAART in other studies Giordano TP et al, Factors Associated with the Use of Highly Active Antiretroviral Therapy in Patients Newly Entering Care in an Urban Giordano TP et al, Factors Associated with the Use of Highly Active Antiretroviral Therapy in Patients Newly Entering Care in an Urban
Clinic. JAIDS, 32:399-405.Clinic. JAIDS, 32:399-405.
Follow-up of HAART eligible patients (CD4<200) that do and do not start HAART
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
FU>0d FU>30d FU>60d FU>90d
Pro
po
rtio
n w
ith
fo
llow
-up
bey
on
d t
imep
oin
t
HAART
No HAART
Improving rates of starting ARVs Improving rates of starting ARVs in HAART-eligible patientsin HAART-eligible patients
Reasons for poor Reasons for poor follow-upfollow-up Pre-HAART procedure Pre-HAART procedure
too cumbersome too cumbersome Dissatisfaction with Dissatisfaction with
servicesservices Trouble paying Trouble paying
transportation coststransportation costs Poor understanding of Poor understanding of
clinic proceduresclinic procedures Stigma of going to HIV Stigma of going to HIV
clinicclinic DeathDeath
Potential solutionsPotential solutions Change workflow around Change workflow around
HAART-eligible patientsHAART-eligible patients Improve counselingImprove counseling Improve relationship Improve relationship
between patients and between patients and health care workershealth care workers
Decentralize ARV Decentralize ARV servicesservices
Number of HIV+ pregnant women enrolled at Number of HIV+ pregnant women enrolled at ART site <30 days after HIV testingART site <30 days after HIV testing
On-site ART vs. Off-site ART clinic: OR 7.2 (CI 5.9-8.8, p<0.001)
0
500
1000
1500
2000
2500
Beira Chimoio Nhamatanda Catandica
Off-site ART clinic On-site ART clinic
Total HIV +
Enrolled
34%
22% 76% 74%
30% 75%
ART-eligible starting ART (Total and ART-eligible starting ART (Total and <<90 90 days), Sofala and Manica, 2004-2007days), Sofala and Manica, 2004-2007
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Beira
Nham
atan
da
Caia
Ponta
Gea
Mar
rom
eu
Chimoio
Catan
dica
Man
ica
Espun
gabe
ra
Gondo
la
1o d
e M
aio
Chitob
e
ART TotalART <90days
Vertical VerticalIntegrated Integrated
Total stated on ART:
Vertical 50% vs. Integrated 65% (p<0.001)
ART <90 days:
Vertical 37% vs. Integrated 59% (p<0.001)
N=9,193
Other examples of OROther examples of OR
Program problem Potential OR studies
(Pre-post control, time series, step-wedge)
Poor adherence to HAART Study strategies to improve adherence (DOT, counseling, community-based treatment partners,
food/transport subsidies)
Low rates of HIV treatment among TB patients, inpatients
Study integration of HIV services into TB programs or hospital services
Underdiagnosis of TB among HIV+ patients Study strategies to increase TB screening/diagnosis (training/mentorship, standardized care algorithms,
screen at VCT)
Low rates of HIV treatment among pregnant women
Study strategies to improve retention into care (decentralize HIV services, improve counseling,
involvement of peer counselors)
Poor provider adherence to HIV care protocols Study strategies to improve adherence to protocols (training/mentorship, performance reviews,
involvement of lower-level HCWs)
Low level of prevention behaviors among HIV+ people
Study strategies to increase prevention (couples counseling/care, intensify prevention counseling in
testing and care centers)
OR resourcesOR resources
Designing HIV/AIDS Intervention Studies: An Operations Research Designing HIV/AIDS Intervention Studies: An Operations Research Handbook. Andrew A. Fisher and James Foreit. The Population Council, Handbook. Andrew A. Fisher and James Foreit. The Population Council, New York, 2002. Available at: New York, 2002. Available at: http://www.popcouncil.org/pdfs/horizons/orhivaidshndbk.pdfhttp://www.popcouncil.org/pdfs/horizons/orhivaidshndbk.pdf
An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an An Approach to Rapid Scale-up: Using HIV/AIDS Treatment and Care as an Example. World Health Organization, Geneva, 2004. Available at: Example. World Health Organization, Geneva, 2004. Available at: http://www.who.int/entity/hiv/pub/prev_care/en/rapidscale_up.pdfhttp://www.who.int/entity/hiv/pub/prev_care/en/rapidscale_up.pdf
The Breakthrough Series: IHI’s Collaborative Model for Achieving The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Institute for Health Care Improvement. Breakthrough Improvement. Institute for Health Care Improvement. Cambridge, MA, 2003. Available at Cambridge, MA, 2003. Available at http://www.ihi.org/NR/rdonlyres/BCA88D8F-35EE-4251-BB93-E2252619A0http://www.ihi.org/NR/rdonlyres/BCA88D8F-35EE-4251-BB93-E2252619A06D/0/BreakthroughSeriesWhitePaper2003.pdf6D/0/BreakthroughSeriesWhitePaper2003.pdf
Population Council / Horizons program on HIV/AIDS OR: Population Council / Horizons program on HIV/AIDS OR: http://www.popcouncil.org/horizons/http://www.popcouncil.org/horizons/
Welcome to I-TECH HIV/AIDS Clinical Seminar Series
Next session: December 18th, 2008Robert Harrington
Opportunistic Infections