extreme makeover, data edition inside the box 2007 city match conference – skills building session...
TRANSCRIPT
![Page 1: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/1.jpg)
Extreme Makeover, Data EditionInside the Box
2007 City MatCH Conference – Skills Building Session
Juan M. Acuña M.D., M.Sc.
MCH Medical Epidemiologist
MCHEPI Program Team Leader, CDC
![Page 2: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/2.jpg)
Objectives of the presentation
1. Review validity and the EBPH process
2. Review EPI principles guiding evidence application as a MCH decision driver
3. Review on current examples based on programmatic information in MCH
![Page 3: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/3.jpg)
Relevant questions based on the previous presentation
• Is there a need for EBPH?
• What do we call evidence?
• How do we assess evidence?
• How do we provide recommendations based on the best evidence?
• How do we implement an action plan?
• How do we evaluate impact?
![Page 4: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/4.jpg)
Changes or new programs are based on recommendations
A. Good evidence to support decisions
B. Fair evidence to support decisions
C. Poor evidence that does not provide direction to do or don’t do
D. Fair evidence to support don’t do
E. Good evidence to support don’t do
![Page 5: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/5.jpg)
Let’s change the paradigm…
• MCH covers maternal, child, health
• Other close fields (injury, infections, death, behaviors, etc”
• MCH belongs to public health
• We are temporary MCH practitioners
• Let’s talk about public health
![Page 6: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/6.jpg)
Basic concepts about EBPH
WORKING DEFINITION (PH):
Promoting healthy people in
healthy communities
![Page 7: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/7.jpg)
Public Health• Prevents epidemics and spread of disease
• Protects against environmental hazards
• Prevents injuries
• Promotes and encourages healthy behaviors
• Responds to disasters and assists communities in recovery
• Assures the quality and access to services
APHA
![Page 8: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/8.jpg)
Services
Monitor health status YesDiagnose, investigate public hazards YesInform, educate, empower NoMobilize community partners NoDevelop policies and plans NoEnforce laws and regulations NoLink people to health services NoAssure expert workforce NoEvaluate effectiveness, access, quality YesResearch new insights and solutions yes
Services data-based
![Page 9: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/9.jpg)
Public Health data-action loop:
Case recollection
Population information
Risk factor data (PRAMS)
ANALYSIS
Programs and policies
RATES AND PROPORTIONS
1. absolute risk
2. population “mapping”
3. tendencies
Complex Analyses
• Cause
• risk factors
• costs
• morbidity
Program evaluation
INFORMATION
Inside the box
outside t
he box
![Page 10: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/10.jpg)
Sources of evidence in PH
• “soft” information: review processes, personal information, “gut” feelings
• “adequate” information: routinely collected information, case review programs, passive systems
• “strong” information: active surveillance, some clinical studies
• “very strong”: randomized clinical trials
![Page 11: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/11.jpg)
2. EPI components in public health problems
Examples
• Referral distance is associated to mortality
• Implications of fetal intervention (clinical and public health) to pediatrics and PH
![Page 12: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/12.jpg)
Perinatal Mortality Rate by Distance from Shreveport, Region 7, 1995-9
57.4
18.324.3
61.9
16.5
30.3
9
34.444
010203040506070
0 20 40 60 80 100
Distance from Shreveport
Mo
rtal
ity
Rat
e
p:0.38
r:0.48
Example 1
![Page 13: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/13.jpg)
So the main issue here is Validity
Valid truth
• Unbiased (bias needs to be avoided)• True and tolerable level of chance (chance
needs to be measured)• Took into account the presence of
characteristics that change the relationship of interest (controlled for confounding)
![Page 14: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/14.jpg)
LBW - SGA LAPRAMS data 1998-1999
Population at risk
LA 1998-1999:
130,294 pregnancies
Smoking OR: 3.5
Wt-Gain OR: 3
Counseling OR: 1.7
Prevalence:
LBW: 7% (9,120)
VLBW: 2% (2,605)
SGA: 15% (19,544)
AFp:
LBW: 9% (820)(+?)
VLBW: 2% (52) (+?)
SGA: 2% (390)(+?)
![Page 15: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/15.jpg)
Why the concern?
• Knowledge is rapidly expanding
• The use of “EB decision-making” is common
• Large amount of published (scientific) literature
• Larger amounts of (unused) stored data
• Lack of guidelines for the EBPH process
• Large degree of uncertainty about change
![Page 16: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/16.jpg)
Critical appraisal of the evidence
• The most difficult step (for PH officials)
• Needs technical evaluation (epi)
• Needs “special” skills
• Less than 10% of active personnel has the skills required. Less than 1% has a method.
• Time-consuming step
• Source of “biased” decisions
![Page 17: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/17.jpg)
Critical appraisal of the evidence• Needs assessment,
surveillance (diagnosis):– Prevalence
– Sensitivity
– Specificity
– PPV / NPV
– LRP / LRN
– Pretest odds / postT odds
– post-test probability
– Receiver-operator curve
• Prognosis:– time variables
– survival curves
– time series
– prognosis estimates
– precision
– relationship with screening and diagnosis
![Page 18: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/18.jpg)
Critical appraisal of the evidence
• Intervention:– Blindness (evaluation)
– Randomization (evaluation)
– Control event rate
– Relative Risk reduction
– Absolute Risk reduction
– Number needed to treat
• Risk:– Odds ratio
– Risk ratio
– Incidence
– Prevalence
– Exposure
– inference
– probability distribution
![Page 19: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/19.jpg)
Critical appraisal of the evidence
• Other more “general” terms:– Bias– confounders– probability distribution and chance– p value– confidence intervals– logistic regression, multivariate analysis– univariate analysis
![Page 20: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/20.jpg)
Surveillance Systems
Epidemiological Studies
Prevention Programs
Risk factors
Protective factors
Public concerns
Prevention strategies
Public policy
Education
Prevalence rates
Registry of cases for study or referral
Monitor prevention
Example # 3:
![Page 21: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/21.jpg)
Birth CertificatesPredictive Value Positive 76%Sensitivity 28%
Hospital Discharge DataPredictive Value Positive 85-95%Sensitivity 70-90%
Example # 3:
Evaluation of Data Studies
![Page 22: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/22.jpg)
Conflict in PH
To do things right
To do the right things (right)
DRIVING FORCE: best evidence for the best practice
PROBLEMS: How is this done? How to do it always? How to do it always the same?
![Page 23: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/23.jpg)
Best Evidence
• Makes sense (relevant)
• Unbiased
• Available
• Statistically significant (chance)
• Significant to Public Health
• Leads to correct decisions
![Page 24: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/24.jpg)
Best EvidenceUnbiased: well designed (bias cannot be measured)
random error
average: 10 (very precise) average: 4
![Page 25: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/25.jpg)
Best EvidenceUnbiased: well designed (bias cannot be measured)
Systematic error random error
average: 4 average: 4
![Page 26: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/26.jpg)
Best Evidence
Available:
• Published (strength of evidence)
• Surveillance systems
• Routinely collected information
• Peer information
• Smart opinion
• Other
![Page 27: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/27.jpg)
Evidence
I - Evidence from RCT
II-1 - Well designed non-randomized trials
II-2 - Cohort, Case Control analysis
II-3 - Comparisons of places, time, interventions, better more than 1 center
III - Opinion of authorities, descriptive studies, expert peer groups or committees
![Page 28: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/28.jpg)
Best Evidence
Statistically significant:
Chance (randomness):
•p values
•Confidence intervals
95% p
no association
no difference
Measurable!
![Page 29: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/29.jpg)
Evidence
Statistical significance
Meaningful to Public Health
BOTH
good best fair
We have been taught to accept statistical significance. If large samples (as in many cases), we are bound to have it, even if it is not meaningful.
![Page 30: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/30.jpg)
Best EvidenceLeads to correct decisions: MSAFP vs. Folic Acid for NTDs
• UK and Northern Ireland: one of the highest prevalences in the world
• Prenatal screening for more than 35 years
• Prenatal termination of the affected pregnancies
0
5
10
15
20
25
UK
Ireland
MSAFP
![Page 31: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/31.jpg)
Folic Acid Fortification
Fortificationlevel
Diet + %population> 0.4mg
Casesprevented
% casesprevented
Cases tooccur
0 0 0.6 0 0 2000140 100 2.2 405 20 1595350 250 19.8 919 46 1081420 300 30.6 1082 54 918360 400 48.3 1324 66 676700 500 61.3 1499 75 501
![Page 32: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/32.jpg)
4. Change PH practices
Public Health is about:
• Research
• Advocacy
• Community Services
• Education
• Wisely invest as little money as possible to make the biggest and better change possible
![Page 33: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/33.jpg)
Changes are based on recommendations
A. Good evidence to support decisions
B. Fair evidence to support decisions
C. Poor evidence that does not provide direction to do or don’t do
D. Fair evidence to support don’t do
E. Good evidence to support don’t do
![Page 34: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/34.jpg)
Evidence
I - Evidence from RCT
II-1 - Well designed non-randomized trials
II-2 - Cohort, Case Control analysis
II-3 - Comparisons of places, time, interventions, better more than 1 center
III - Opinion of authorities, descriptive studies, expert peer groups or committees
![Page 35: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/35.jpg)
Framework for evaluation• Do I want (have) to evaluate the study?
• Outline the study
• Is the study believable?
• What is the Public Health relevant finding?– Are the variables of interest included?
– Are findings explainable by chance, bias or confounders?
– Is the finding believable within our knowledge?
• Will the study help me with my population?– Is my population similar?
– Is my problem similar?
– Will the findings benefit my programs and policies?
![Page 36: Extreme Makeover, Data Edition Inside the Box 2007 City MatCH Conference – Skills Building Session Juan M. Acuña M.D., M.Sc. MCH Medical Epidemiologist](https://reader036.vdocuments.net/reader036/viewer/2022062322/56649f295503460f94c4289c/html5/thumbnails/36.jpg)
Inside the Box Workshop Summary
We have:
1. Reviewed concept and situation of “evidence based” practice of public health
2. Reviewed relevant basic EPI concepts
3. Reviewed EPI aspects of public health
4. Provided element to build a framework to evaluate evidence
5. Evaluated the role of evidence in PH decisions