applicability and transferability role of review authors
DESCRIPTION
Applicability and Transferability Role of review authors. Belinda Burford Rebecca Armstrong Emma Tavender. Some definitions. applicability and transferability are two very important and related concepts (often used interchangeably) Two sides of the one coin: - PowerPoint PPT PresentationTRANSCRIPT
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Applicability and Transferability
Role of review authors
Belinda BurfordRebecca ArmstrongEmma Tavender
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Some definitions
• applicability and transferability are two very important and related concepts (often used interchangeably)
• Two sides of the one coin:– Applicability: whether or not it would be feasible to implement this
intervention in a particular setting • OR “Is it possible?”
– Transferability: whether we can expect to see similar results after implementing this intervention in a particular setting
• OR “Even if it’s possible, will it work?”
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Some challenges for complex reviews
• Heterogeneity, heterogeneity, heterogeneity!• Population
– Interactions between intervention, population characteristics and individual preferences/attitudes
– Groups rather than individuals• Intervention
– Multiple components• ‘active’ components?; interactions between components?
– Fidelity/integrity difficult to measure• Comparisons
– Poorly described• Outcomes
– Proxy/surrogate outcomes• Setting
– Important! Often poorly described
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Imag
e fr
om:
http:
//w
ww
.har
lem
-sch
ool.c
om/1
0TH/
mat
h_p
df/m
ath.
htm
l
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Organising the information
What are some approaches you have taken in your reviews to help the reader consider applicability/transferability?
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
What do end-users need from our reviews?
• Who are your end-users?– Differing needs
• What do they need to know1:– Could it work in my jurisdiction?– Will it work?– What would it take to make it work?– Is it worth it?
• Many resources for end-users for assessing applicability of systematic reviews:– SUPPORT tools to support policy-making– Applicability/transferability checklist2
– Many, many more…1 Lavis et al. Use of research to inform public policymaking Lancet 2004; 364: 1615–21 Lancet 20052 Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83.
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Starting from the end-user perspective: Applicability
End-user question1 What can the review provide?2
Does the political environment of the local society allow for this intervention to be implemented? • Study setting/context
Is there any political barrier to implementing this intervention? • Study setting/contextWould the general public and the targeted (sub) population accept this intervention? Does any aspect of the intervention go against local social norms? Is it ethically acceptable?
• Characteristics of study populationStudy setting/context
Can the contents of the intervention be tailored to suit the local culture? • Intervention components
Are the essential resources for implementing this intervention available in the local setting? • Resources required
Does the target population in the local setting have a sufficient educational level to comprehend the contents of the intervention?
• Characteristics of study population
Which organisation will be responsible for the provision of the intervention in the local setting? • Organisational context
Is there any possible barrier to implementing this intervention due to the structure of that organisation?
• Barriers to implementation identified in studies
Does the provider of the intervention in the local setting have the skill to deliver this intervention? If not will training be available?
• Who delivered the intervention? Was training provided?
1 Questions from: Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83.2 Table from: Burford et al. Forthcoming
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
End-user question1 What can the review provide?2
What is the baseline prevalence of the health problem of interest in the local setting? What was the difference in prevalence between the study setting and the local setting?
• Baseline prevalence in study populations
Are the characteristics of the target population comparable between the study setting and the local setting? Is it possible that the characteristics of the target population, such as ethnicity, socioeconomic status, educational level etc will have an impact on the effectiveness of the intervention?
• Characteristics of study population• Outcomes analysed by relevant
socio-demographic factors
Is the capacity to implement the intervention comparable between the study setting in such matters as political environment, social acceptability, resources, organisational structure and the skills of the local providers?
• Study setting/context• Resources required• Intervention deliverers
Starting from the end-user perspective: Transferability
1 Questions from: Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83.2 Table from: Burford et al. Forthcoming
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Organising the information
Place Population• Geographical• Setting• Scale• plus…• consider factors important for
your review…
• Age• Gender• Race/Ethnicity• Income-level• Education• plus…
Intervention Characteristics Intervention Components• Types of roles involved• Interaction between roles• Delivery• Flexibility/tailoring• Component within a broader
system/program• plus…
• Number of components• Type of components• Frequency• plus…
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Approaches for dealing with this in your review
• Extrinsic Approach– Give context-dependent guidance for applying
the review findings– 1-2 paragraphs in the discussion– Implementation tables
• Intrinsic Approach– Build into the review planned subgroup
analyses• Good to do both
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
EXAMPLE: Interventions for preventing obesity in children1
• Data extraction– “implementation factors” based on “practice-
relevant tool”• Extrinsic approach
– Described these across studies in results section and summarised in the discussion
• Intrinsic approach– Meta analyses to explore heterogeneity
1 Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Extrinsic Approach - Results
• Effectiveness– Maintenance / Sustainability of effects– Equity
• PROGRESS-Plus– Harm-adverse/unintended effects
• Implementation– Intervention design and theoretical basis– Process evaluation (including: intensity/dose, acceptability,
barriers/facilitators)– Resources needed (including: who delivered, funding, hours of
implementation, other?)– Strategies to address disadvantage/diversity
Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Extrinsic approach: Discussion
Applicability of the evidenceThis review included studies from high income countries as well as lower-middle- and upper-middle-income countries, with five studies conducted in countries within the latter two groupings (Thailand, Brazil, Chile and Mexico). This means that, while predominantly conducted within high-income settings, the findings from this review may be generalisable to a number of settings. A total of nineteen studies specifically reported incorporating strategies to target socio-economic and/or cultural diversity or disadvantage. One such study was conducted outside of the high-income country setting, in Chile, an upper-middle-income country. Of the remaining eighteen studies, seven studies conducted in the USA were of interventions targeting African American children and their communities and another two studies targeted Native American communities. Other studies targeted participants of low socio-economic status, or were implemented in areas of social disadvantage. By far the most common setting for interventions included in this review were schools (43 studies). Other interventions were (or included) home-based (14 studies), community-based (six studies), or were set in a health service (two studies) or care setting (two studies). Eleven studies incorporated interventions across multiple settings. Most interventions took a combined dietary and physical activity approach to obesity prevention (31 studies). As a single strategy, targeting physical activity alone was more popular (17 studies) than targeting diet alone (seven studies). The predominant theoretical basis for interventions in this review was behaviour change theory. Other theories represented include environmental change strategies, the socio-ecological framework, social learning theory, health promotion theory, transtheoretical models, and youth development and resiliency based approaches. The theoretical basis for interventions was explicitly reported in approximately half of the included studies.
Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Intrinsic approach: Synthesis
• Meta analyses to explore heterogeneity:– Age group of participants– Setting:
• Education only• Education plus other settings• Non-education
– Intervention type:• PA• Diet• PA/diet combined
– Intervention duration:• <=12 months• > 12 months
Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
The next update?
• Intrinsic– meta-regression: investigate multiple factors simultaneously
(continuous and categorical) for their possible influence on the size of the intervention effect• Caution:
– Generally not considered when <10 studies in meta-analysis– Characteristics should be pre-specified (supported by scientific rationale)
[ours will be post-hoc]– Select a small number of characteristics– Careful interpretation (Ch 9 of Cochrane Handbook)
• Extrinsic– Implementation tables
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Using qualitative synthesis to explore heterogeneity of complex interventions
2 reviews:• Qualitative review on HIV/AIDS patients perspectives on adherence• Cochrane Review of intervention trials to improve adherence to therapy in HIV/AIDS patients
Tabulated whether the interventions in the CR corresponded with the patients’ perspectives on how to help them
Candy et al. BMC Medical Research Methodology 2011, 11:124
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Exploring temporal study design: Handwashing behaviour in LMIC
Vind
igni
et a
l. Sy
stem
atic
revi
ew: h
andw
ashi
ng b
ehav
iour
in lo
w- t
o-m
iddl
e-in
com
e co
untr
ies:
ou
tcom
e m
easu
res a
nd b
ehav
iour
mai
nten
ance
. Tro
pica
l Med
icin
e an
d In
tern
ation
al H
ealth
. 20
11;1
6(4)
:466
-477
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Handwashing: Temporal study design
• Community settings (n=16 studies)– Formative research: average = 4.8 months– Intervention: average = 17.2 months– Evaluation: average = 9.6 months
• Schools (n=4 studies)– Formative research: 1 study– Intervention: average = 4.8 months– Evaluation: average = 5 months
• Health care settings (n=7 studies)– Formative research: 1 study– Intervention: average = 2.5 months– Evaluation: average = 3.2 months
Vindigni et al. Systematic review: handwashing behaviour in low-tomiddle-income countries: outcome measures and behaviour maintenance. Tropical Medicine and International Health. 2011;16(4):466-477
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Example from the EPOC Group
Population typeStudies
satisfying EPOC Design Criteria
Comparative studies not satisfying EPOC
Design Criteria
Descriptive studies only Total
Urban 7 12 16 35
Urban Disadvantaged 0 1 6 7
Rural 1 4 14 19
Rural Disadvantaged 1 0 11 12
Total 9 17 47 73
Are there different amounts of relevant evidence?
Eg. Specialist outreach clinics in primary care and rural hospital settings
The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community WellbeingMelbourne School of Population Health
Specialist Outreach review